Add Jupyter Server Architecture diagram #801

andreyvelich · 2022-04-25T16:54:00Z

I added Jupyter Server diagram to the documentation.
As we discussed on the community meeting, I used draw.io tool, so others can easily modify this in the future.

Please take a look at the doc changes.

In the following PRs I will add workflows for the Session creating and deleting.

cc @blink1073 @Zsailer

codecov-commenter · 2022-04-25T17:01:47Z

Codecov Report

Merging #801 (e40591d) into main (c51f5a8) will not change coverage.
The diff coverage is n/a.

❗ Current head e40591d differs from pull request most recent head 4a4fee2. Consider uploading reports for the commit 4a4fee2 to get more accurate results

@@           Coverage Diff           @@
##             main     #801   +/-   ##
=======================================
  Coverage   69.96%   69.96%           
=======================================
  Files          62       62           
  Lines        7368     7368           
  Branches     1223     1223           
=======================================
  Hits         5155     5155           
  Misses       1841     1841           
  Partials      372      372

Impacted Files	Coverage Δ
jupyter_server/serverapp.py	`65.11% <0.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c51f5a8...4a4fee2. Read the comment docs.

blink1073

Excellent, thanks for this! I added some minor suggestions inline.

docs/source/developers/architecture.rst

blink1073 · 2022-04-25T17:32:48Z

docs/source/developers/architecture.rst

+  ServerApp. It starts a new Kernel for a user's Session and generates a
+  new Kernel ID.
+
+- **Kernel Spec Manager** parses file with JSON specification for a Kernel.


Suggested change

- **Kernel Spec Manager** parses file with JSON specification for a Kernel.

- **Kernel Spec Manager** parses files with JSON specification for Kernels, and provides a list of available kernel configurations.

I find the Jupyter Client reference in the diagram to be a little out of place. I wonder if we were to also add Kernel Manger with a relationship between it and MappingKernelManager it might help clarify. If so, I would suggest adding an entry labeled Jupyter Client with sub-entries of Kernel Spec Manager and Kernel Manager, which would then allow some introduction as to the importance the jupyter_client package serves within the Server architecture.

That makes sense, thanks @kevin-bates!

kevin-bates

Thank you @andreyvelich - this will be very helpful to developers! I had a couple of comments to (perhaps) help clarify things a bit further.

kevin-bates · 2022-04-25T19:00:52Z

docs/source/developers/architecture.rst

+- **Kernel Gateway** is the web server that provides access to Jupyter Kernels.
+  There are different ways to create this gateway. If your ServerApp needs to
+  communicate with remote Kernels, you can use
+  `the Enterprise Gateway <https://github.com/jupyter-server/enterprise_gateway>`_,
+  otherwise you can use `the Kernel Gateway <https://github.com/jupyter-server/kernel_gateway>`_.


Because the box states, Kernel Gateway yet both forms of gateway servers are listed here, perhaps using the term Gateway Server may be more clear:

Suggested change

- **Kernel Gateway** is the web server that provides access to Jupyter Kernels.

There are different ways to create this gateway. If your ServerApp needs to

communicate with remote Kernels, you can use

`the Enterprise Gateway <https://github.com/jupyter-server/enterprise_gateway>`_,

otherwise you can use `the Kernel Gateway <https://github.com/jupyter-server/kernel_gateway>`_.

- **Gateway Server** is a web server that, when configured, provides access to Jupyter Kernels running on other hosts.

There are different ways to create a gateway server. If your ServerApp needs to

communicate with remote Kernels residing within resource-managed clusters, you can use

`Enterprise Gateway <https://github.com/jupyter-server/enterprise_gateway>`_,

otherwise, you can use `Kernel Gateway <https://github.com/jupyter-server/kernel_gateway>`_, where Kernels run locally to the gateway server.

I agree with that, I will rename Kernel Gateway to Gateway Server on the diagram also.

kevin-bates · 2022-04-25T19:03:49Z

docs/source/developers/architecture.rst

+- **Mapping Kernel Manager** is responsible to operate multiple Kernels in the
+  ServerApp. It starts a new Kernel for a user's Session and generates a
+  new Kernel ID.


Suggested change

- **Mapping Kernel Manager** is responsible to operate multiple Kernels in the

ServerApp. It starts a new Kernel for a user's Session and generates a

new Kernel ID.

- **Mapping Kernel Manager** is responsible for managing the lifecycles of the Kernels running within the

ServerApp. It starts a new Kernel for a user's Session and facilitates interrupt, restart, and shutdown operations against the kernel.

kevin-bates · 2022-04-25T19:12:14Z

docs/source/developers/architecture.rst

+  ServerApp. It starts a new Kernel for a user's Session and generates a
+  new Kernel ID.
+
+- **Kernel Spec Manager** parses file with JSON specification for a Kernel.


I find the Jupyter Client reference in the diagram to be a little out of place. I wonder if we were to also add Kernel Manger with a relationship between it and MappingKernelManager it might help clarify. If so, I would suggest adding an entry labeled Jupyter Client with sub-entries of Kernel Spec Manager and Kernel Manager, which would then allow some introduction as to the importance the jupyter_client package serves within the Server architecture.

andreyvelich · 2022-04-25T21:19:02Z

Thank you for the review @blink1073 and @kevin-bates.
I modified architecture with Kernel Manager and Gateway Server.
Please let me know if that sounds good.

blink1073

👍🏼

kevin-bates

Great information @andreyvelich - thank you!

for more information, see https://pre-commit.ci

andreyvelich · 2022-04-26T17:44:09Z

Thank you for the feedback!
@Zsailer @blink1073 @kevin-bates Are we ready to merge this PR ?

Zsailer · 2022-04-26T17:50:57Z

Thanks, @andreyvelich. Great work here!

Zsailer · 2022-04-26T17:54:34Z

@andreyvelich, can you share a link to the source document for this diagram? Or maybe you can export the document as XML and we can save it in this repo. That would allow others to edit the diagram in the future using drawIO.

andreyvelich · 2022-04-26T18:02:17Z

@andreyvelich, can you share a link to the source document for this diagram? Or maybe you can export the document as XML and we can save it in this repo. That would allow others to edit the diagram in the future using drawIO.

I believe, https://app.diagrams.net/ allows to import this png file and make some modification.
At least, it works for me and I can edit the diagram.
Did you try that ?

Zsailer · 2022-04-26T18:41:01Z

The link takes me to a blank diagram and a dialogue to choose from my existing diagrams... but I don't have access to this diagram. Is there a share link specifically for this diagram?

andreyvelich · 2022-04-26T21:44:01Z

@Zsailer Are you able to execute this:
File -> Open From -> Device -> Select the png file from your local file system ?
I didn't create shared diagram on the Google Drive.
Do we need this if contributors can just upload png images directly to the draw.io and make changes ?

blink1073 · 2022-04-26T21:47:52Z

I tested by downloading the file and uploading to draw.io when reviewing the PR.

Zsailer · 2022-04-26T22:24:51Z

Woah! I didn't realize draw.io could edit PNG. 🤯 .

Sorry for the my ignorance here, @andreyvelich! This is great!

blink1073 added the documentation label Apr 25, 2022

blink1073 previously approved these changes Apr 25, 2022

View reviewed changes

kevin-bates reviewed Apr 25, 2022

View reviewed changes

andreyvelich dismissed blink1073’s stale review via d938d52 April 25, 2022 21:16

blink1073 previously approved these changes Apr 25, 2022

View reviewed changes

kevin-bates previously approved these changes Apr 26, 2022

View reviewed changes

andreyvelich and others added 4 commits April 26, 2022 18:02

Add Jupyter Server Architecture diagram

6bbd773

[pre-commit.ci] auto fixes from pre-commit.com hooks

9e0b2d3

for more information, see https://pre-commit.ci

Changes after review

744ae7b

Fix Kernel Manager name

4a4fee2

andreyvelich dismissed stale reviews from kevin-bates and blink1073 via 4a4fee2 April 26, 2022 17:03

andreyvelich force-pushed the doc-add-architecture branch from e40591d to 4a4fee2 Compare April 26, 2022 17:03

kevin-bates approved these changes Apr 26, 2022

View reviewed changes

Zsailer approved these changes Apr 26, 2022

View reviewed changes

Zsailer merged commit 241f0f2 into jupyter-server:main Apr 26, 2022

andreyvelich deleted the doc-add-architecture branch April 26, 2022 17:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Jupyter Server Architecture diagram #801

Add Jupyter Server Architecture diagram #801

andreyvelich commented Apr 25, 2022

codecov-commenter commented Apr 25, 2022 •

edited

Loading

blink1073 left a comment

blink1073 Apr 25, 2022

kevin-bates Apr 25, 2022

andreyvelich Apr 25, 2022

kevin-bates left a comment

kevin-bates Apr 25, 2022

andreyvelich Apr 25, 2022

kevin-bates Apr 25, 2022

kevin-bates Apr 25, 2022

andreyvelich commented Apr 25, 2022

blink1073 left a comment

kevin-bates left a comment

andreyvelich commented Apr 26, 2022

Zsailer commented Apr 26, 2022

Zsailer commented Apr 26, 2022

andreyvelich commented Apr 26, 2022 •

edited

Loading

Zsailer commented Apr 26, 2022

andreyvelich commented Apr 26, 2022

blink1073 commented Apr 26, 2022

Zsailer commented Apr 26, 2022

	- Kernel Spec Manager parses file with JSON specification for a Kernel.
	- Kernel Spec Manager parses files with JSON specification for Kernels, and provides a list of available kernel configurations.

Add Jupyter Server Architecture diagram #801

Add Jupyter Server Architecture diagram #801

Conversation

andreyvelich commented Apr 25, 2022

codecov-commenter commented Apr 25, 2022 • edited Loading

Codecov Report

blink1073 left a comment

Choose a reason for hiding this comment

blink1073 Apr 25, 2022

Choose a reason for hiding this comment

kevin-bates Apr 25, 2022

Choose a reason for hiding this comment

andreyvelich Apr 25, 2022

Choose a reason for hiding this comment

kevin-bates left a comment

Choose a reason for hiding this comment

kevin-bates Apr 25, 2022

Choose a reason for hiding this comment

andreyvelich Apr 25, 2022

Choose a reason for hiding this comment

kevin-bates Apr 25, 2022

Choose a reason for hiding this comment

kevin-bates Apr 25, 2022

Choose a reason for hiding this comment

andreyvelich commented Apr 25, 2022

blink1073 left a comment

Choose a reason for hiding this comment

kevin-bates left a comment

Choose a reason for hiding this comment

andreyvelich commented Apr 26, 2022

Zsailer commented Apr 26, 2022

Zsailer commented Apr 26, 2022

andreyvelich commented Apr 26, 2022 • edited Loading

Zsailer commented Apr 26, 2022

andreyvelich commented Apr 26, 2022

blink1073 commented Apr 26, 2022

Zsailer commented Apr 26, 2022

codecov-commenter commented Apr 25, 2022 •

edited

Loading

andreyvelich commented Apr 26, 2022 •

edited

Loading