lib/tx.py: fix wtxid and vsize calculation

[SomberNight]

• the wtxid calculation has been incorrect since it was added in 9c123a7
  • this was not exposed externally anywhere (nothing uses wtxids, yet)
• the vsize calculation has been incorrect since it was added in e5b4f5f
  • it was off by ~7.5 vbytes for segwit txs, and ~6 vbytes for non-segwit txs
  • this was externally observable via the mempool.get_fee_histogram RPC
• added test cases