The idea behind that is to have panel where all urls patterns are visible and clickable in one place to make debugging and plain development easier, especially when you use application_namespace and instance_namespace.
This is an interesting suggestion. However, it doesn't quite match the primary purpose of the Debug Toolbar, which is to provide debug information for the current page.
Fortunately, you can develop this feature independently, as the Debug Toolbar makes it easy to integrate third party panels. If you do, you can send a pull request to add it to the list of third party panels in the docs.