Thanks...that is a side effect I'd forgotten about. I specified "localhost" intentionally because I wanted to force hostname validation (I also get exceptions.ValueError: Invalid DNS-ID using just the loopback address). My main goal was to keep the benchmark as "real" as possible, but I'm beginning to think it might be more useful to have several, one to do the full path, one that only does cert validation, etc. If nothing else they might be useful in doing a coarse assessment of changes from release to release (assuming the hardware running the benchmark stays the same).