DatastoreIO accepts single namespace.

[chandra-vs]

I have a thousands of namespaces, how can i pass all namespace at once. Where withNamespace() can accept only one.

[dhalperi]

Can you say more about what you specifically want to do? You want to query all entities in any namespace?

[chandra-vs]

I want to process the Entities from all the namespace. For example., I have an Employee entity and i would like to add a new field to the entity and value as a String for all namespaces.

[dhalperi]

I don't think that this is supported by the Datastore API – you cannot read from multiple namespaces at the same time. Since the main purpose of Datastore namespaces is for isolation, they discourage and disallow using multiple namespaces in the same job.

(However, I'm reaching out to them to confirm.)

[dhalperi]

Datastore team has confirmed, modifying entities in multiple namespaces is strongly discouraged.

However, it should be easy to programmatically generate multiple Dataflow pipelines, each of which affects only one namespace.

Please re-open if we can provide more assistance.

Thanks!

[rajesh99]

Request to give an example on multiple dataflow pipelines.
It will be great if you can extend the WordCount datastore example to generate multiple dataflow piplelines, so that datastore entities in mulitple namespaces can be processed.

[chandra-vs]

I would like to reopen. I have more than 5000 namespaces. Not sure, how to do the multiple pipelines.
Any example will be appreciated.