Microsoft debugging symbols caching proxy using FastEndpoints and MongoDB.
When working frequently with WinDbg to analyze kernel memory dumps, symbol (down)loading is a very common occurrence. What's also quite common is the abysmal performance of the official Microsoft Symbol Servers, taking easily between two to three seconds for every request! Being fed up with wasting precious lifetime I made this little caching proxy you can put in between your WinDbg instances and use it in conjunction with an offline caching directory. The proxy server will store a copy of the requested symbol in a MongoDB database, including information if the symbol even exists upstream. This information gets cached for a week, so you won't waste any more time twiddling thumbs while the Microsoft servers take their sweet time to return a 404.
Let's see if my efforts paid off, shall we! I've opened a kernel memory dump which requested 1090 files at a total size of 186 Megabytes with three different strategies outlined below. Let's go!
12 minutes, 17 seconds. Abysmal and shameful! 😒
1 minute, 20 seconds. Whoa, we're getting somewhere! 😃
48 seconds. Heck yeah! 🥳
I'm hosting an instance of this project at https://symbols.nefarius.at/ which you can consume and help growing by adjusting your symbol source settings like so:
.sympath cache*D:\symbols;srv*https://symbols.nefarius.at/download/symbols
This means my local symbol cache resides at D:\symbols, adjust to your own preferences.
Like any other of my public web services it only logs an absolute minimal amount required for debugging purposes and I'll never sell your data out to the big bois 😉
If you like this idea and want to keep my public instance happy, up and running consider making a donation 💸
- Caching! Pretty much the main purpose 😁
/download/symbols/{Symbol}/{SignaturePlusAge}/{File}- Serves the typical download request the debugger invokes.
- Badges! Embed server statistics anywhere via generated SVGs 🖼️
/api/badges/cachedSymbolsTotal- Gets total amount of cached records.
/api/badges/cachedSymbolsFound- Gets the amount of records found upstream (these actually contain the data).
/api/badges/cachedSymbolsNotFound- Gets the amount of records flagged as "not found" upstream.
A non-exhaustive list of notes about features that I may implement some day:
- Add support to provide a custom directory where the server can pick up custom/private symbols from and cache them with the same methods.
- Add support for PDB files
- Add support for binaries (
.sys,.dll,.exe)
- Make retry and cache invalidation parameters configurable e.g. via
appsettings.json - Maybe add custom "admin" endpoints to upload custom symbols, delete existing entries and invalidate cache on command?
- Add timer to periodically check if 404ed symbols are now available
- Add timer to automate some tasks, maybe log last symbol access and scrub some that never got requested after a few months or so
dotnet publish -p:PublishProfile=Properties\PublishProfiles\release-win-x64.pubxml -c:Release .\WinDbgSymbolsCachingProxy.slnDocker build
docker build -t nefarius.azurecr.io/wdscp:latest . ; docker push nefarius.azurecr.io/wdscp:latestThis application benefits from these awesome projects ❤ (appearance in no special order):
