I even got it to show up like this, after deleting all tasks.json files I created:

This was intentional so that the quick pick could be much faster and not actually go and fetch the tasks. However, I can make this better for tasks that were in a tasks.json (it won't help for purely contributed tasks though).

I can make this better for tasks that were in a tasks.json

Still seeing old names from tasks.json appear - what should I be expecting?

Since you're changing an npm task we'd need to go the the npm extension to determine if there is still a npm: watch-typecheck task. In your case you're seeing the expected behavior because the goal here is not to go the the extensions so that we can keep the picker fast.

To verify, try changing the label of a shell or process task instead. That's where I added some extra checking since those are only user defined.