Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor - DAG Payload on Aircan Connector #61

Closed
3 of 5 tasks
hannelita opened this issue Jul 16, 2020 · 1 comment
Closed
3 of 5 tasks

Refactor - DAG Payload on Aircan Connector #61

hannelita opened this issue Jul 16, 2020 · 1 comment
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@hannelita
Copy link
Contributor

hannelita commented Jul 16, 2020

Acceptance

  • Using CKAN API, upload a resource and pass enough information to trigger the DAG with:
"conf": {
  resource: {  // a f11s data resource object
    "path": ckan_resource['url'],
    "format": ckan_resource['format'],
    "ckan_resource_id": ckan_resource['id'],
    "schema": ckan_resource['schema']
  }
  "ckan_config": {
    "api_key": ..
  }
}

~Note: Airflow does not support nested JSON; we'll have to send the info in a one-level json ~
Note2: When using JINJA templates, it doesn't. When fetching params from **context, it works, so we're adopting the original proposal

"conf": {
    "path": ckan_resource['url'],
    "format": ckan_resource['format'],
    "ckan_resource_id": ckan_resource['id'],
    "schema": ckan_resource['schema']
    "ckan_api_key": ..
}

Tasks

@hannelita hannelita added the enhancement New feature or request label Jul 16, 2020
@hannelita hannelita added this to the Sprint - 20 July 2020 milestone Jul 16, 2020
@hannelita hannelita self-assigned this Jul 16, 2020
@hannelita hannelita added this to To Do in Project Management via automation Jul 16, 2020
hannelita added a commit that referenced this issue Jul 16, 2020
@hannelita hannelita moved this from To Do to In Review in Project Management Jul 16, 2020
@hannelita hannelita moved this from In Review to Done in Project Management Jul 17, 2020
@hannelita hannelita reopened this Jul 17, 2020
Project Management automation moved this from Done to To Do Jul 17, 2020
@hannelita hannelita moved this from To Do to In progress in Project Management Jul 22, 2020
@hannelita hannelita moved this from In progress to Prioritized in Project Management Jul 22, 2020
@hannelita hannelita moved this from Prioritized to In progress in Project Management Jul 22, 2020
@hannelita
Copy link
Contributor Author

Docs on datopian/ckanext-aircan#13. This PR and #71 fix this issue

@hannelita hannelita moved this from In progress to Done in Project Management Jul 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
No open projects
Development

No branches or pull requests

1 participant