Workflow for Interactive Development?

[jayleverett]

Hi There -

I commonly find myself making experimental code changes within a notebook job. The naive workflow for doing this is pretty ugly - it goes something like:

1. Open the relevant .py files in the browser and make the desired edits
2. Test the impact of these edits in the jupyter terminal/notebook
3. Repeat steps 1 and 2 until you're happy
4. Copy what you did (if you can remember) to your local copy of the code and commit

I'm sure there's a better way of doing this with polyaxon's current feature set, and suggestions would be greatly appreciated.

That said, I think the ultimate solution might be a command like polyaxon notebook sync ..., which would inject the local copy of the code into a running notebook job. This would move step (1) into the local editor, thereby eliminating step (4).

Is something like this supported right now? And if not, what would you suggest as a workaround?

[mmourafiq]

This is similar to #262

[jayleverett]

Agree that it's similar, but that issue seems concerned with downloading code from a notebook job. What I would like is the ability to upload code into a running notebook job, so that I can develop in my local editor.

Is this possible, and if not, is it on the roadmap?

[mmourafiq]

The problem with upload code, is how to handle already changed files inside the notebook?
What happened when 2 team members work on the same project, and one is pushing new changes, should it also impact the other's user notebook. There are a couple of edge cases, and some work is in progress.

I mentioned the other issue, because previously Polyaxon allowed to mount the code to notebook, however it did not allow further uploads, what could be done is; alert the user that some notebooks are currently running and will potentially be impacted by uploading a new version of the code, the user can confirm the upload.

[jayleverett]

The problem with upload code, is how to handle already changed files inside the notebook?

Speaking only for myself, I would be comfortable having those files overwritten. If an upload were possible, I wouldn't be editing files in the browser/notebook job (besides the .ipynb file itself). I suppose inadvertently nuking .ipynb files could become an issue, and the user would need to remember to save/move/download those and should receive a warning as you say.

What happened when 2 team members work on the same project, and one is pushing new changes, should it also impact the other's user notebook.

This one confuses me a bit, as I thought it wasn't possible to run two notebook jobs simultaneously for the same polyaxon project. We're still on v0.4.4 right now - perhaps this has changed in v0.5?

[github-actions]

This issue has not seen any recent activity.

[mmourafiq]

This is quite old and several changes have been made since the creation of this issue, notebooks are more customizable and can handle more complex workflows in v1.

Feel free to reopen.