NDB: Cannot Query/fetch using ancestors stored in default namespace from another namespace

[Nathanmalnoury]

We are currently migrating our application to the appengine new runtime using python3.8, and we have identified a significant difference in the way namespaces are handled in queries using ancestors, compared to the python 2.7 datastore library.

In 2.7, an ancestor query simply used the namespace of the ancestor, no matter the current namespace (as defined by namespace_manager). It was, of course, forbidden to specify an ancestor and a namespace argument that would not match.

In the Python 3 python-ndb library however, querying using ancestors stored in the default namespace fails if the current namespace is not the default one.

In our application, most interactions with ndb are done using a user-specific namespace as the current namespace rather than the default one. In this namespace, we sometimes have to query an entity stored in the default namespace using an ancestor which raises an InvalidArgumentError. So far the best workaround we've found to this issue is to temporarily set the current namespace to the default one, to do our query, and then to set the current namespace back to it's initial value.

Are we missing something ? If not is there a better way of handling this issue ?

Environment details

1. API : Datastore (Cloud Firestore in Datastore mode)
2. OS: Ubuntu 20.10
3. Python version and virtual environment information: Python 3.8.6
4. google-cloud-ndb version:

Name: google-cloud-ndb
Version: 1.7.1
Summary: NDB library for Google Cloud Datastore
Home-page: https://github.com/googleapis/python-ndb
Author: Google LLC
Author-email: googleapis-packages@google.com
License: Apache 2.0
Location: /home/nathan/Projects/v3-poc/venv/lib/python3.8/site-packages
Requires: redis, google-cloud-datastore, pymemcache
Required-by: 

Code example

import os
from google.cloud import ndb

class Dummy(ndb.Model):
    foo = ndb.StringProperty(default='')


with ndb.Client().context():
    entity1 = Dummy(foo='bar', namespace='')
    entity1.put()
    key_entity1 = entity1.key


    entity2 = Dummy(foo='child', parent=key_entity1)
    entity2.put()

with ndb.Client().context(namespace='other_context'):
    l = Dummy.query( ancestor=key_entity1, namespace='').fetch()

Note: It seems to be an issue with the fetch command only, as the query itself set the correct (default) namespace, as we were able to assess with some print in the source code. The only way to make that snippet work was to modify the fetch call to fetch(namespace='') which seems to be a rather dirty way of getting things done.
It might be worth noticing that the issue does not appear if the namespace of entity1 is anything else but ''.

Stack trace

Traceback (most recent call last):
  File "/.../scratch.py", line 23, in <module>
    l = Dummy.query(namespace='', ancestor=key_entity1).fetch()
  File "/.../venv/lib/python3.8/site-packages/google/cloud/ndb/utils.py", line 114, in wrapper
    return wrapped(*args, **new_kwargs)
  File "/.../venv/lib/python3.8/site-packages/google/cloud/ndb/model.py", line 5273, in _query
    query = query_module.Query(
  File "/.../venv/lib/python3.8/site-packages/google/cloud/ndb/query.py", line 1396, in __init__
    raise TypeError("ancestor/namespace mismatch")
TypeError: ancestor/namespace mismatch