Split up the backend, frontend, and the docprovider

[krassowski]

Closes #269

New structure:

• packages: (npm)
  • @jupyter/collaboration (no changes)
  • @jupyter/collaboration-extension: filebrowser and server-side execution plugin extracted
  • @jupyter/docprovider (no changes)
  • @jupyter/docprovider-extension (new) home for filebrowser/drive and server-side execution plugin (for now)
• projects: (python, new)
  • jupyter-server-ydoc: jupyter-server extension
  • jupyter-docprovider: JupyterLab/Notebook 7+ extension (@jupyter/docprovider-extension)
  • jupyter-collaboration-ui: JupyterLab/Notebook 7+ extension (@jupyter/collaboration-extension)
  • jupyter-collaboration: empty package which installs all the three above

Breaking changes

We should call this a new major version as now it is necessary to import from jupyter_server_ydoc rather than jupyter_collaboration (though I am not sure if anyone was importing anything as there was no public API before?).

[github-actions]

👈 Launch a Binder on branch krassowski/jupyter_collaboration/split-packages

[krassowski]

This is now ready, the release check will be fixed once we make a release. The only question from me is whether it is ok to have linked versioning, or if we want to reset version in the packages which are getting split up and have independent versioning in individual python packages?

[davidbrochart]

Thanks so much Mike, that's quite a lot of changes!

The only question from me is whether it is ok to have linked versioning, or if we want to reset version in the packages which are getting split up and have independent versioning in individual python packages?

I'd rather not have linked versioning, because IIUC a change released in one package will force a release in the other packages even though nothing has changed in them.
To me that kind of breaks SemVer, right? Imagine a breaking change release in a package where actually nothing has changed, that can be considered a lie 😄
I have linked versioning in Jupyverse's plugings, and that also leads to more work with the conda packages.
But what are the drawbacks of independent versioning?

[krassowski]

Can you move jupyter-server-ydoc on PyPI to Project Jupyter org (https://pypi.org/org/jupyter/, same way as jupyter-collaboration already is) and add me as a maintainer?

[davidbrochart]

Failure is unrelated, see jupyter-server/jupyter_server_fileid#75.

[davidbrochart]

Can you move jupyter-server-ydoc on PyPI to Project Jupyter org (https://pypi.org/org/jupyter/, same way as jupyter-collaboration already is) and add me as a maintainer?

It seems that I cannot transfer it because I am not an owner or manager of any organizations, so I'll let you do it.
I sent an invitation to krassowski, let me know if that was not you PyPI user name.

[krassowski]

I am not an owner/manager on Jupyter either. I now uploaded alpha versions of the three packages (excluding jupyter-collaboration itself) and restarted the check release job to ensure it passes.

@jtpio can you help with the transfer of jupyter-server-ydoc, jupyter-docprovider and jupyter-collaboration-ui to Jupyter org on PyPI?

[krassowski]

All green, merging!

[jtpio]

@jtpio can you help with the transfer of jupyter-server-ydoc, jupyter-docprovider and jupyter-collaboration-ui to Jupyter org on PyPI?

Looks like I don't have the rights to do this unfortunately, the jupyter org does not show up in the list:

Maybe @blink1073 or @fcollonval can help with this.

[jtpio]

We should call this a new major version as now it is necessary to import from jupyter_server_ydoc rather than jupyter_collaboration (though I am not sure if anyone was importing anything as there was no public API before?).

Could there be a way to shim the imports otherwise? And add a warning to recommend updating the imports. Wondering if that could help avoid making the major version.

[krassowski]

Could there be a way to shim the imports otherwise?

I did look into that. There is only one thing that was exported from the top-level __init__ which was the YDocExtension. However, it was not really useful unless you also imported from other modules and mirroring the hierarchy to ensure imports work did not seem worth it to me.

I am operating on an assumption that no one used the Python API if you know of production uses though then I am happy to revisit.

[krassowski]

In the public land I do not see any production imports that would be broken:

• from jupyter_collaboration
• import jupyter_collaboration

Except for https://github.com/datalayer/jupyter-rtc-test which would need migration CC @echarles.

[blink1073]

@jtpio I've made you a manager on the org, can you please try again?

[davidbrochart]

Except for https://github.com/datalayer/jupyter-rtc-test which would need migration CC @echarles.

This repo is unmaintained anyway, it's still using the old CRDT stack.

[echarles]

This repo is unmaintained anyway, it's still using the old CRDT stack.

It is maintained, at slow pace, and may continue to support btw y-crdt

[fcollonval]

Thanks @blink1073 for updating the rights on PyPI. Unfortunately only jupyter organization owners may add/remove projects as far as I remember (one of the annoying organization limitation of PyPI).

[krassowski]

@blink1073 I invited you as owner to jupyter-docprovider and jupyter-collaboration-ui on PyPI. Can someone with rights also send an invite for jupyter-server-ydoc?

[davidbrochart]

I just did.

[krassowski]

Thank you all!

[Zsailer]

Hey folks, I'm late to the party here. Thanks for doing the work here! I think this is the "right" move going forward.

However, I think we need to keep the 2.x branch open until at least JupyterLab 4.2 goes "stable" (right now it's in RC). This PR raised the minimum requirement for JLab packages to 4.2 beta, forcing folks to use this with pre release of JupyterLab.

@jzhang20133 and I have been running RTC in production and fixing multiple issues we've observed. We can't depend on an pre-release at this point in time, so we'll need to backport our changes to the 2.x line temporarily and cut releases from there to unblock and continue getting feedback from our users.

In the public land I do not see any production imports that would be broken:

We have (not yet open-sourced) extensions that import from jupyter_collaboration, so this PR also broke our work here. I don't think a shim is necessary—we can tolerate the changes on our end. I think more folks will begin adopting this library though, so we should do this more prudently going forward.

I'll make a 2.x branch, backport our changes, and cut some releases from there.