Point validation is really expensive and apparently unnecessary when
we already trust the keys in our known_hosts file.
With my current known_hosts file of 657 keys, this reduces the time
taken to complete a remote task in fabric from 24 seconds to 7 seconds.
This requires python-ecdsa >= 0.11.