New Features in VSCode Plugin: Auto server termination after max idle time and default installation of python + jupyter notebook extensions #1977

JasonZhu1313 · 2023-11-19T18:19:28Z

New Features in VSCode Plugin: Auto server termination after max idle time and default installation of python + jupyter notebook extensions

TL;DR

Please replace this text with a description of what this PR accomplishes.

This is a followup PR of VSCode Plugin contribution: #1922 with additional features:

Terminate the VSCode server after reaching the maximum idle time when no activity is detected.
Enabling the installation of VSCode plugins from VSCode plugin marketplace urls, by default python and jupyter notebook plugins.

Type

Bug Fix
Feature
Plugin

Are all requirements met?

Complete description

How did you fix the bug, make the feature etc. Link to any design docs etc

Termination after max idle time
Replace original hard coded server_up_seconds with max_idle_seconds, check the heartbeat file located at ~/.local/share/code-server/heartbeat ever 60 seconds source, if the duration if the time till last connection is longer than max idle seconds, terminate the VSCode server.
Installation of python + jupyter notebook extensions from urls
Expose plugins_remote_paths argument in vscode annotation, so users could choose plugin urls they'd like to install. By default python and Jupyter notebook plugin will be installed.

The urls of plugin follows this pattern:

https://${publisher}.gallery.vsassets.io/_apis/public/gallery/publisher/${publisher}/extension/${extension name}/${version}/assetbyname/Microsoft.VisualStudio.Services.VSIXPackage

We'll be extracting the ${extension name} from url using regex as the extension name on local disk, the extension will be named with .vsix extension.

Ex. Python plugin url

https://ms-python.gallery.vsassets.io/_apis/public/gallery/publisher/ms-python/extension/python/2023.12.0/assetbyname/Microsoft.VisualStudio.Services.VSIXPackage

Will be downloaded on local disk as python.vsix for installation.

Tracking Issue

#1922

Tests

The delta (duration in seconds after last activity detected) is refreshed after connected to vscode server. And server is terminated after delta reaches to max idle time.

{"asctime": "2023-11-17 08:29:41,806", "name": "flytekit", "levelname": "INFO", "message": "heart beat path exist?: True"}
{"asctime": "2023-11-17 08:29:41,806", "name": "flytekit", "levelname": "INFO", "message": "The latest activity on code server is 238.5610237121582 seconds ago."}
{"asctime": "2023-11-17 08:29:41,806", "name": "flytekit", "levelname": "INFO", "message": "metadata"}
{"asctime": "2023-11-17 08:29:41,806", "name": "flytekit", "levelname": "INFO", "message": "238.5610237121582"}
{"asctime": "2023-11-17 08:29:41,806", "name": "flytekit", "levelname": "INFO", "message": "36000"}
{"asctime": "2023-11-17 08:29:41,806", "name": "flytekit", "levelname": "INFO", "message": "False"}
{"asctime": "2023-11-17 08:30:41,867", "name": "flytekit", "levelname": "INFO", "message": "heart beat path is here: /home/flytekit/.local/share/code-server/heartbeat"}
{"asctime": "2023-11-17 08:30:41,885", "name": "flytekit", "levelname": "INFO", "message": "heart beat path exist?: True"}
{"asctime": "2023-11-17 08:30:41,886", "name": "flytekit", "levelname": "INFO", "message": "The latest activity on code server is 14.362635612487793 seconds ago."}
{"asctime": "2023-11-17 08:30:41,886", "name": "flytekit", "levelname": "INFO", "message": "metadata"}
{"asctime": "2023-11-17 08:30:41,887", "name": "flytekit", "levelname": "INFO", "message": "14.362635612487793"}
{"asctime": "2023-11-17 08:30:41,887", "name": "flytekit", "levelname": "INFO", "message": "36000"}
{"asctime": "2023-11-17 08:30:41,888", "name": "flytekit", "levelname": "INFO", "message": "False"}
{"asctime": "2023-11-17 08:31:41,922", "name": "flytekit", "levelname": "INFO", "message": "heart beat path is here: /home/flytekit/.local/share/code-server/heartbeat"}
{"asctime": "2023-11-17 08:31:41,923", "name": "flytekit", "levelname": "INFO", "message": "heart beat path exist?: True"}
{"asctime": "2023-11-17 08:31:41,923", "name": "flytekit", "levelname": "INFO", "message": "The latest activity on code server is 14.39638066291809 seconds ago."}
{"asctime": "2023-11-17 08:31:41,923", "name": "flytekit", "levelname": "INFO", "message": "metadata"}
{"asctime": "2023-11-17 08:31:41,923", "name": "flytekit", "levelname": "INFO", "message": "14.39638066291809"}
{"asctime": "2023-11-17 08:31:41,923", "name": "flytekit", "levelname": "INFO", "message": "36000"}
{"asctime": "2023-11-17 08:31:41,923", "name": "flytekit", "levelname": "INFO", "message": "False"}

Plugins are installed successfully

welcome · 2023-11-19T18:19:32Z

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
Sign off your commits (Reference: DCO Guide).

codecov · 2023-11-19T18:45:30Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (f6ae51c) 62.54% compared to head (f065503) 80.67%.
Report is 7 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #1977       +/-   ##
===========================================
+ Coverage   62.54%   80.67%   +18.12%     
===========================================
  Files         313      173      -140     
  Lines       23252    16943     -6309     
  Branches     3525     2312     -1213     
===========================================
- Hits        14543    13668      -875     
+ Misses       8286     2702     -5584     
- Partials      423      573      +150

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

plugins/flytekit-vscode/flytekitplugins/vscode/decorator.py

ByronHsu · 2023-11-20T00:15:14Z

nit you have to resolve the DCO issue by adding sign-off to your commit. Check details

JasonZhu1313 · 2023-11-20T07:41:16Z

7 expected, 1 pending, and 2 successful checks

plugins/flytekit-vscode/flytekitplugins/vscode/decorator.py

…le seconds and install the python + jupyter notebook extentions Signed-off-by: JasonZhu1313 <jasonchu13@outlook.com>

Signed-off-by: JasonZhu1313 <jasonchu13@outlook.com>

Signed-off-by: byhsu <byhsu@linkedin.com>

ByronHsu · 2023-11-20T22:30:44Z

Thanks jason for the great work! I have validated the following use cases, and they all worked well

from flytekit import task, workflow
from flytekitplugins.vscode import vscode, VscodeConfig
from flytekitplugins.vscode.constants import DEFAULT_CODE_SERVER_EXTENSIONS

@task(
    container_image="localhost:30000/flytekit-vscode:0.0.2",
    environment={"FLYTE_SDK_LOGGING_LEVEL": "20"}
)
@vscode(

)
def t():
    ...



@task(
    container_image="localhost:30000/flytekit-vscode:0.0.2",
    environment={"FLYTE_SDK_LOGGING_LEVEL": "20"}
)
@vscode(
    max_idle_seconds=10,
)
def t_suicide():
    ...



config_with_vim = VscodeConfig(
    extension_remote_paths=DEFAULT_CODE_SERVER_EXTENSIONS+["https://open-vsx.org/api/vscodevim/vim/1.27.0/file/vscodevim.vim-1.27.0.vsix"]
)

@task(
    container_image="localhost:30000/flytekit-vscode:0.0.2",
    environment={"FLYTE_SDK_LOGGING_LEVEL": "20"}
)
@vscode(
    config=config_with_vim
)
def t_vim():
    ...



@workflow
def wf():
    t()
    t_suicide()
    t_vim()

JasonZhu1313 · 2023-11-20T22:32:48Z

Thanks jason for the great work! I have validated the following use cases, and they all worked well

from flytekit import task, workflow
from flytekitplugins.vscode import vscode, VscodeConfig
from flytekitplugins.vscode.constants import DEFAULT_CODE_SERVER_EXTENSIONS

@task(
    container_image="localhost:30000/flytekit-vscode:0.0.2",
    environment={"FLYTE_SDK_LOGGING_LEVEL": "20"}
)
@vscode(

)
def t():
    ...



@task(
    container_image="localhost:30000/flytekit-vscode:0.0.2",
    environment={"FLYTE_SDK_LOGGING_LEVEL": "20"}
)
@vscode(
    max_idle_seconds=10,
)
def t_suicide():
    ...



config_with_vim = VscodeConfig(
    extension_remote_paths=DEFAULT_CODE_SERVER_EXTENSIONS+["https://open-vsx.org/api/vscodevim/vim/1.27.0/file/vscodevim.vim-1.27.0.vsix"]
)

@task(
    container_image="localhost:30000/flytekit-vscode:0.0.2",
    environment={"FLYTE_SDK_LOGGING_LEVEL": "20"}
)
@vscode(
    config=config_with_vim
)
def t_vim():
    ...



@workflow
def wf():
    t()
    t_suicide()
    t_vim()

Thanks! Nice integrate test covers all cases

Signed-off-by: byhsu <byhsu@linkedin.com>

plugins/flytekit-vscode/README.md

Signed-off-by: byhsu <byhsu@linkedin.com>

troychiu · 2023-11-21T05:52:58Z

LGTM. Thank you @JasonZhu1313 and @ByronHsu!

troychiu · 2023-11-21T06:12:11Z

I am wondering if it's possible or even needed to add tests to test exit handler?

welcome · 2023-11-21T06:52:17Z

Congrats on merging your first pull request! 🎉

… time and default installation of python + jupyter notebook extensions (flyteorg#1977) --------- Signed-off-by: JasonZhu1313 <jasonchu13@outlook.com> Signed-off-by: byhsu <byhsu@linkedin.com> Co-authored-by: byhsu <byhsu@linkedin.com>

… time and default installation of python + jupyter notebook extensions (flyteorg#1977) --------- Signed-off-by: JasonZhu1313 <jasonchu13@outlook.com> Signed-off-by: byhsu <byhsu@linkedin.com> Co-authored-by: byhsu <byhsu@linkedin.com> Signed-off-by: Rafael Raposo <rafaelraposo@spotify.com>

JasonZhu1313 requested review from wild-endeavor, kumare3, eapolinario, pingsutw and cosmicBboy as code owners November 19, 2023 18:19

ByronHsu reviewed Nov 20, 2023

View reviewed changes

JasonZhu1313 force-pushed the JasonZhu1313/vscode_plugin branch from 0ed6b19 to 0994c7d Compare November 20, 2023 07:40

JasonZhu1313 closed this Nov 20, 2023

JasonZhu1313 reopened this Nov 20, 2023

JasonZhu1313 force-pushed the JasonZhu1313/vscode_plugin branch from 1364073 to a2989d5 Compare November 20, 2023 08:09

troychiu requested changes Nov 20, 2023

View reviewed changes

JasonZhu1313 added 3 commits November 20, 2023 10:37

Add functionalities to vscode plugin to terminate server after max id…

a706a85

…le seconds and install the python + jupyter notebook extentions Signed-off-by: JasonZhu1313 <jasonchu13@outlook.com>

address comments and refactor code

16271ce

Signed-off-by: JasonZhu1313 <jasonchu13@outlook.com>

address comment

cee300c

Signed-off-by: JasonZhu1313 <jasonchu13@outlook.com>

JasonZhu1313 force-pushed the JasonZhu1313/vscode_plugin branch from ab9f844 to cee300c Compare November 20, 2023 18:37

JasonZhu1313 requested a review from troychiu November 20, 2023 21:37

byhsu added 2 commits November 20, 2023 13:47

minor fix

a3cbc5e

Signed-off-by: byhsu <byhsu@linkedin.com>

add examples to readme

a9f8b7f

Signed-off-by: byhsu <byhsu@linkedin.com>

ByronHsu force-pushed the JasonZhu1313/vscode_plugin branch from 607d270 to a9f8b7f Compare November 20, 2023 22:30

fix lint

571bb08

Signed-off-by: byhsu <byhsu@linkedin.com>

pingsutw reviewed Nov 21, 2023

View reviewed changes

plugins/flytekit-vscode/README.md Show resolved Hide resolved

fix test

f065503

Signed-off-by: byhsu <byhsu@linkedin.com>

troychiu approved these changes Nov 21, 2023

View reviewed changes

pingsutw approved these changes Nov 21, 2023

View reviewed changes

pingsutw merged commit 744c167 into flyteorg:master Nov 21, 2023
74 checks passed

JasonZhu1313 deleted the JasonZhu1313/vscode_plugin branch December 22, 2023 15:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Features in VSCode Plugin: Auto server termination after max idle time and default installation of python + jupyter notebook extensions #1977

New Features in VSCode Plugin: Auto server termination after max idle time and default installation of python + jupyter notebook extensions #1977

JasonZhu1313 commented Nov 19, 2023

welcome bot commented Nov 19, 2023

codecov bot commented Nov 19, 2023 •

edited

ByronHsu commented Nov 20, 2023

JasonZhu1313 commented Nov 20, 2023

ByronHsu commented Nov 20, 2023

JasonZhu1313 commented Nov 20, 2023

troychiu commented Nov 21, 2023

troychiu commented Nov 21, 2023 •

edited

welcome bot commented Nov 21, 2023

New Features in VSCode Plugin: Auto server termination after max idle time and default installation of python + jupyter notebook extensions #1977

New Features in VSCode Plugin: Auto server termination after max idle time and default installation of python + jupyter notebook extensions #1977

Conversation

JasonZhu1313 commented Nov 19, 2023

TL;DR

Type

Are all requirements met?

Complete description

Tracking Issue

Tests

welcome bot commented Nov 19, 2023

codecov bot commented Nov 19, 2023 • edited

Codecov Report

ByronHsu commented Nov 20, 2023

JasonZhu1313 commented Nov 20, 2023

ByronHsu commented Nov 20, 2023

JasonZhu1313 commented Nov 20, 2023

troychiu commented Nov 21, 2023

troychiu commented Nov 21, 2023 • edited

welcome bot commented Nov 21, 2023

codecov bot commented Nov 19, 2023 •

edited

troychiu commented Nov 21, 2023 •

edited