Documentation around `validate` performance

[dhirschfeld]

Hi @mayeut! 👋
Thanks for making pybase64 available - it looks like an awesome library!

I'm just trying to determine whether or not to use validate=True but am finding I don't have enough context to make the decision.

The readme says:

To get the fastest decoding, it is recommended to use the pybase64.b64decode and validate=True when possible.

...but I'm struggling to understand how decoding + validation can be faster than just decoding on it's own?

[dhirschfeld]

Poking about it seems validate=True does builtin_decode(s, altchars, validate=False) plus a bunch of extra work that validate=False doesn't have to do.

The only thing I can see that could make validate=True faster is that it errors out early if len(s) % 4 != 0. Is the performance benefit just due to failing loudly rather than taking a slow code-path?

[mayeut]

Please do not look too much into the fallback code as it's unlikely you'll be using this in the end.
In the case of the fallback code, both are equivalent in terms of speed in nominal cases.

decoding + validation can be faster than just decoding on it's own?

Validation is always done somehow. In the case of validation=False, the invalid input chars are just skipped.

When using the C extension (most likely), decoding/validation has a single pass implementation in SIMD which is not the case when validation=False (c.f. #4).

[dhirschfeld]

Thanks for the explanation!