Create Template to bulk import from Datastore/Firestore exported files

[salrashid123]

Cloud Firestore and Cloud Datastore share the same leveldb export format.

Feature request to support bulkd processing of datastore/firestore files.

reference:

• Firestore: import/export
• Datastore: import/export
• Firestore -> BigQuery: https://cloud.google.com/bigquery/docs/loading-data-cloud-firestore

I can confirm , the following python snippet reads and displays the export files.

#!/usr/bin/python
# virtualenv env
# source env/bin/activate

import sys
sys.path.append('/apps/google-cloud-sdk/platform/google_appengine/')
from google.appengine.api.files import records
from google.appengine.datastore import entity_pb
from google.appengine.api import datastore

raw = open('2018-11-05T17_49_44_60804_all_namespaces_all_kinds_output-0', 'r')
reader = records.RecordsReader(raw)
for record in reader:
    entity = datastore.Entity.FromPb(entity_pb.EntityProto(contents=record))
    print entity

• blog post