Jackdmp keeps crashing -- Aggregate device with HDMI display

[keybounce]

My system output device is an aggregate device, containing the internal speakers on my mac, an HDMI display/speaker (Macbook pro, HDMI connection on built-in video card), and a USB headset (mic and speaker)

If I attempt to use Jack with the headset not connected, Jack wants to treat the system audio as the built-in speaker and microphone, and won't use the aggregate device with the two speaker outputs.

If I use Jack with everything, it works, until the TV is turned on. There are no complaints from turning the TV off. But if I turn it on, then I get a message saying that the audio settings have changed, Jackdmp will now quit; followed by jack server has quit, so jack pilot will now quit.

The whole "nothing plays correctly unless jack server was started first" means that I have shut down and restart everything in the morning when I turn my external monitor back on.
The need to reset everything's routing means that this is painful.
And the inability to save and restore routing unless there is a perfect match of things seen by jack -- including programs that only show up when active and go away when not -- means that this is all manual and error prone.

Basically, I want Jack to be rock-solid (not crash), any more than the existing aggregate device does not crash, and work with things started earlier than it.