Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schema generation from a list #58

Closed
ZiggerZZ opened this issue Nov 18, 2020 · 3 comments
Closed

Schema generation from a list #58

ZiggerZZ opened this issue Nov 18, 2020 · 3 comments

Comments

@ZiggerZZ
Copy link
Contributor

Hi,

I think it would be useful (at least for me :)) to add a function to the library that generates a schema from some data, something like

from bigquery_schema_generator import generate_schema
data = [{"first_column": 1, "second_column": "value"}, {"first_column": 2, "second_column": "another value"}]
schema = generate_schema(data)  # returns a list

I came up with this function:

from subprocess import check_output
def generate_schema(data):
    data_string = ""
    for d in data:
        if d:
            data_string = data_string + json.dumps(d) + '\n'
    data_bytes = data_string.encode('utf-8')
    s = check_output(['generate-schema'], input=data_bytes)
    schema = json.loads(s)
    return schema

but I'm sure there's a more efficient way.

@bxparks
Copy link
Owner

bxparks commented Nov 18, 2020

This is a duplicate of #47 which I closed due to lack of activity. I give some pointers there for how this can be implemented. If you want to take a crack at it, I recommend waiting until #57 is merged because that's a substantial refactoring.
[Edit: Replaced #40 with more recent #57]

@ZiggerZZ
Copy link
Contributor Author

Oh, haven't seen this issue.
I'll wait till the PR is merged.

@bxparks
Copy link
Owner

bxparks commented Dec 5, 2020

#40, #57 was replaced with #61 and merged in. Maybe you want to rebase and take another crack at this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants