-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Single-DAG DataStore load flow using CKAN DataStore API #17
Conversation
Note: Opening a separate PR for docs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. Minor query / comments.
tags=['api_load'] | ||
) | ||
|
||
def full_load(csv_input, json_output, **kwargs): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is full_load
? Ah, ok it's a task that does it all in one?
try: | ||
with open(json_output) as f: | ||
records = json.load(f) | ||
return load_csv_via_api( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
load_csv_via_api
should provbably be renamed: load_records_via_api
log.info('Table was created successfuly') | ||
else: | ||
return response.json() | ||
except Exception as e: | ||
return {"success": False, "errors": [e]} | ||
|
||
|
||
# def send_resource_to_datastore(resource, headers, records, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we going to uncomment at some point? Otherwise feel free to drop.
[datapusher-style]; relates to #10