Skip to content

Depend on public jpy#2583

Merged
devinrsmith merged 10 commits intodeephaven:mainfrom
devinrsmith:public-jpy
Jun 28, 2022
Merged

Depend on public jpy#2583
devinrsmith merged 10 commits intodeephaven:mainfrom
devinrsmith:public-jpy

Conversation

@devinrsmith
Copy link
Copy Markdown
Member

No description provided.

@devinrsmith devinrsmith added clean up release blocker A bug/behavior that puts is below the "good enough" threshold to release. jpy labels Jun 24, 2022
@devinrsmith devinrsmith added this to the Jun 2022 milestone Jun 24, 2022
@devinrsmith devinrsmith self-assigned this Jun 24, 2022
@devinrsmith devinrsmith requested a review from niloc132 June 24, 2022 22:03
version=__normalized_version__,
description='Deephaven Embedded Server Python Package',
long_description=README,
long_description_content_type='text/markdown',
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

long_description_content_type is necessary for publication.

test_loader='unittest:TestLoader',
classifiers=[
'Development Status :: 2 - Pre-Alpha',
'Intended Audience :: Developers, Data Scientists',
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Data Scientists is improper classifier.

Comment thread py/server/setup.py
version=__normalized_version__,
description='Deephaven Engine Python Package',
long_description=README,
long_description_content_type='text/markdown',
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

long_description_content_type is necessary for publication

Comment thread py/server/setup.py
test_loader='unittest:TestLoader',
classifiers=[
'Development Status :: 2 - Pre-Alpha',
'Intended Audience :: Developers, Data Scientists',
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Data Scientists is invalid classifier.

Copy link
Copy Markdown
Member Author

@devinrsmith devinrsmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some notes.

@devinrsmith
Copy link
Copy Markdown
Member Author

@devinrsmith devinrsmith marked this pull request as ready for review June 24, 2022 22:08
@devinrsmith devinrsmith requested a review from jcferretti as a code owner June 24, 2022 22:08
@devinrsmith devinrsmith requested a review from jmao-denver June 27, 2022 18:22
jmao-denver
jmao-denver previously approved these changes Jun 27, 2022
repositories {
mavenCentral()

// Uncomment the following if doing local development work that depends on SNAPSHOTs
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably should remove this? or if helpful generally, guard it behind a -P flag, and add mavenLocal for any locally installed builds?

personally i'd prefer it not be there, since it could have surprising other effects, but i can see either way

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the idea of -P guards in buildSrc. Maybe we've already done that elsewhere? That said, I think there are some development workflows we may want to make easier now that jpy is external (and deephaven-csv). I'll remove for now.

google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.46.3
grpcio==1.47.0
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why advance this past the 1.46 that the server (and java client) presently uses?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that said, make sure all py modules get updated accordingly - at a glance i see sphinx (for our docs) is still using 1.46.0, not changed in this diff

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a requirement inherited from tensorflow - the dependencies regularly get updated. As part of update adding jpy as a proper base image dependency, some of the transitive dependencies got updated as well. deephaven/deephaven-base-images#29

Comment thread py/embedded-server/deephaven_server/start_jvm.py
jpy.VerboseExceptions.enabled = True


def _expandLinks(dirname):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these specifically existed in case a caller was specifying wildcard paths to their own extra jars

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: java has the mechanisms accepting wildcards natively - I don't think we should try to do the expanding ourselves if we re-introduce custom classpaths.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that while java does have this mechanism from the command line, it does not work when starting the jvm as we do (at least not in my testing, which IIRC was java 11).

Comment thread py/embedded-server/java-runtime/build.gradle
Comment thread py/jpy-integration/src/pythonToJava/python/test/test_jpy.py
server.run();
checkGlobals(scriptSession.get(), null);
System.out.println("Server started on port " + server.server().getPort());
System.out.println("Server started on: "
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong patch? Also present in conventional server startup, not really part of this diff

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - I'll pull out this for separate consideration.

Comment on lines +75 to +76
# Expand the classpath, so a user can resolve wildcards
expanded_classpath = list(itertools.chain.from_iterable(iglob(e, recursive=True) for e in extra_classpath))
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jmao-denver - can you give this a lookover - I think it replaces the previous functionality. I've tested it out, and it seems to work.

Comment thread py/server/build.gradle Outdated
@@ -3,7 +3,7 @@ plugins {
}

dependencies {
pythonWheel project(path: ':deephaven-jpy', targetConfiguration: 'pythonWheel')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can remove the whole block

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

niloc132
niloc132 previously approved these changes Jun 28, 2022
Copy link
Copy Markdown
Member

@niloc132 niloc132 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you might still be working on the py 3.9 bug - but one final request either in this ticket or in a post-release followup, that we broaden the testing a bit, and get CI to confirm on multiple py versions that we can at least start up the server, run something simple, and shut it down safely.

@devinrsmith
Copy link
Copy Markdown
Member Author

Follow-up: #2593

@devinrsmith
Copy link
Copy Markdown
Member Author

Follow-up: #2592

@devinrsmith devinrsmith merged commit d73ef01 into deephaven:main Jun 28, 2022
@devinrsmith devinrsmith deleted the public-jpy branch June 28, 2022 20:32
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

clean up jpy NoDocumentationNeeded release blocker A bug/behavior that puts is below the "good enough" threshold to release.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants