Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

data_refs can describe bind mounts during local docker run #33

Merged
merged 2 commits into from
Aug 23, 2019

Conversation

DXist
Copy link
Contributor

@DXist DXist commented Aug 22, 2019

While experiment code is being developed and debugged locally using docker run it might be handy to bind mount local directories with data.

For local docker run I created following override file polyaxonfile.local.yml:

environment:
  data_refs:
    - "~/image_data:image_data_ref"
    - inference_output_data_ref

The changes in this PR propagate data_ref items to POLYAXON_RUN_DATA_PATHS variable and configure bind mounts for data_ref items formatted as HOST_PATH:DATA_REF_NAME. Other data_ref items can be referred in the experiment code by name.

Additionally polyaxon/sdks#30 allows to do local docker runs without Polyaxon server if POLYAXON_NO_OP env variable is set.

@DXist DXist changed the title WIP: data_refs can describe bind_mounts during local docker run WIP: data_refs can describe bind mounts during local docker run Aug 22, 2019
Copy link
Contributor

@mmourafiq mmourafiq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. I think the logic here is growing, and at some point we need to start adding testing.

@mmourafiq mmourafiq self-requested a review August 22, 2019 16:35
Co-Authored-By: Mourad <mouradmourafiq@users.noreply.github.com>
@DXist
Copy link
Contributor Author

DXist commented Aug 23, 2019

@mouradmourafiq I committed the change you've suggested.

@DXist DXist changed the title WIP: data_refs can describe bind mounts during local docker run data_refs can describe bind mounts during local docker run Aug 23, 2019
@mmourafiq mmourafiq merged commit e38293c into polyaxon:master Aug 23, 2019
mmourafiq pushed a commit that referenced this pull request Oct 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants