Support Unpickling CachedQueryResults Created By Py2

[phil-lopreiato]

This is a rabbit hole, as pickling across python versions is a terrible idea.

However, we can hack around a few of the major issues that come up. Start by adding a repro unit test that tries to load+depickle a value I got from the datastore admin tool for something written by the py2 app.

Next, we need to fix:

• we changed the import path of models, so we'll need to rewrite those on the fly. We can do this with a custom Unpickler. See this answer.
• bytestrings are handled totally differently between python 2 and 3, so we need to make sure that the py3 reader will unpickle using encoding="bytes". We can do this in our custom unpickler too. Plus there's this whole thing about unpickling datetimes. See this answer.
• Finally, there's a ndb incompatibility. The legacy app uses a custom pickling format (a serialized protobuf), which it does by defining __getstate__ and __setstate on the base Model class. Since the py3 compatible ndb library does not do this, unpickling them will fail. I filed Can't Unpickle Models Pickled by Legacy Library googleapis/python-ndb#587 in hope of working with upstream to fix this, but if not, we can potentially hack around it ourselves by manually deserializing the protobuf and setting the field ourselves.

WARNING: After this, we should be able to read data written by the legacy app, but we can not yet write data that the legacy app can read. So after this, when we re-enable CachedQueryResult, we'll have to make sure that we don't write anything to the datastore.

[codecov]

Codecov Report

Merging #3234 (c168a9f) into py3 (cd50970) will increase coverage by 0.06%.
The diff coverage is 92.30%.

@@            Coverage Diff             @@
##              py3    #3234      +/-   ##
==========================================
+ Coverage   95.41%   95.48%   +0.06%     
==========================================
  Files         361      367       +6     
  Lines       15865    16328     +463     
==========================================
+ Hits        15137    15590     +453     
- Misses        728      738      +10     

Impacted Files	Coverage Δ
src/backend/common/models/cached_model.py	88.42% <86.25%> (-11.58%)	⬇️
src/backend/common/models/cached_query_result.py	100.00% <100.00%> (ø)
...es/tests/cached_query_result_compatibility_test.py	100.00% <100.00%> (ø)
src/backend/api/main.py	100.00% <0.00%> (ø)
src/backend/api/handlers/team.py	100.00% <0.00%> (ø)
src/backend/api/handlers/error.py	100.00% <0.00%> (ø)
src/backend/common/queries/types.py	100.00% <0.00%> (ø)
src/backend/common/queries/team_query.py	100.00% <0.00%> (ø)
src/backend/common/queries/award_query.py	100.00% <0.00%> (ø)
src/backend/common/queries/event_query.py	100.00% <0.00%> (ø)
... and 31 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd50970...c168a9f. Read the comment docs.