According to Apple Support, my old iMac can be used in Target Display Mode if it’s running High Sierra or earlier. However, I had upgraded it to Catalina. Getting it back down to High Sierra proved to be quite the challenge.
Part 1: Easy-to-find stuff, cataloged for posterity
I was able to find the installers via Internet search, but after downloading it to the iMac, it wouldn’t run. It claimed that Catalina was too old, which seems like a bad error message for a two-sided version check (Catalina is actually too new, but they didn’t expect they would need to say that.)
However, once again, the Internet came to the rescue: I could use the command- line script bundled with the installer to write it to a USB stick. It seems that Apple has this answer as well; in my case, it was this:
sudo "/Applications/Install macOS High Sierra.app/Contents/Resources/createinstallmedia" --volume /Volumes/BIT_BUCKET
(Any new-lines in the above command should be entered as spaces at the Terminal prompt.)
The command took about five minutes, but during one dead-end, I found it takes 30 minutes to copy to a USB 2.0 stick.
Part 2: The other roadblocks
The High Sierra installer acted like it couldn’t handle the Catalina disks, and did not quite know how to error out properly. The installer would ask for the password twice, then… would not unlock the disk. Clicking “Unlock…” to attempt to proceed would then do nothing. Trying to mount the volumes first, using Disk Utility, didn’t improve the situation.
I tried to erase the volumes using the installer’s Disk Utility, but it hung on “deleting volume.” Going to choose a startup disk when the volume was hypothetically deleted didn’t show it, but rebooting would prove that the volume was there. Incidentally, quitting the startup disk selection would remove the menu bar and leave a featureless gray screen. Oops.
I ran First Aid, turned off FileVault, and even reinstalled Catalina along the way, but none of that seemed to do anything.
What ended up working was to boot into Catalina’s recovery environment, select “show all devices” in its Disk Utility, then erase the entire APFS container and replace it with a “macOS Extended (Journaled)” disk. After that point, booting the iMac would bring it into a new menu that looked like “choose startup disk” with no disks.
I had to unplug and re-plug the USB stick for it to show up, and then I could boot from it and install High Sierra in the usual manner. Success!
Part 3: The cable failure
(Section added 2022-09-03.)
I missed the information that Target Display Mode specifically requires a Thunderbolt 2 cable. The Thunderbolt 3 to mini DisplayPort cable I actually bought fits physically, but does not function.
Moreover, the standard was completely revamped for version 3, so it requires an active adapter for Thunderbolt 2, and a specific Thunderbolt 2 cable. Between them, the current price is about $140, while a new monitor can be had for $200.
I have decided to declare the experiment a failure, and abandon the effort to have an external display for the laptop.
The future may eventually hold a 5K display, shared between the work laptop and my personal desktop, once that requires an upgrade. My current CPU (the third PCIe GPU died years ago, so I quit putting them in) is rated for 2K output over a port that the motherboard doesn't have, or 1920x1200 otherwise.
My USB stick was named BIT_BUCKET because, once in a while, the stick would occasionally forget it had data, or even a filesystem. But, it managed to survive long enough to get the macOS installer running.