Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Single-DAG DataStore load flow using CKAN DataStore API #17

Merged
merged 1 commit into from
Jun 15, 2020

Conversation

hannelita
Copy link
Contributor

[datapusher-style]; relates to #10

@hannelita
Copy link
Contributor Author

Note: Opening a separate PR for docs

@rufuspollock rufuspollock changed the title Single-DAG load flow using API Single-DAG DataStore load flow using CKAN DataStore API Jun 15, 2020
Copy link
Member

@rufuspollock rufuspollock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Minor query / comments.

tags=['api_load']
)

def full_load(csv_input, json_output, **kwargs):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is full_load? Ah, ok it's a task that does it all in one?

try:
with open(json_output) as f:
records = json.load(f)
return load_csv_via_api(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

load_csv_via_api should provbably be renamed: load_records_via_api

log.info('Table was created successfuly')
else:
return response.json()
except Exception as e:
return {"success": False, "errors": [e]}


# def send_resource_to_datastore(resource, headers, records,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we going to uncomment at some point? Otherwise feel free to drop.

@rufuspollock rufuspollock merged commit 7f82bc5 into master Jun 15, 2020
@hannelita hannelita deleted the feat/api-csv-load branch June 19, 2020 01:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants