Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert coursier bootstrap into a flat classpath + filter #984

Merged
merged 21 commits into from Jul 1, 2019

Conversation

Projects
None yet
2 participants
@lihaoyi
Copy link
Owner

commented Jun 30, 2019

We now construct a normal fat jar, but also have Mill build an ammonite-api-whitelist.txt file containing a list of valid classpath entries we would like to expose in the REPL. This list is plumbed through Ammonite as an opaque classPathWhitelist: Set[Seq[String]], to two important places:

  • A WhiteListClasspath, which limits what classes/resources the Scala compiler is able to see at compile time

  • A WhiteListClassLoader, which limits what classes/resources are made available to the REPL at runtime

We perform the classpath filtering at runtime, in the Ammonite code, which lets us fall back to the original simple fat-jar-style launcher. This also gives us the flexibility for embedded versions of Ammonite to inject a different classPathWhitelist callback, e.g. to allow use of application classses while still hiding the Ammonite classpath. The classPathWhiteList is fed into the cache tags to ensure --thin scripts and REPLs have their own caches.

Only static classpath entries are filtered; you can still import $ivy things outside the filtered subset, e.g. if you want to load a different version of uPickle than the one that's filtered out by --thin, you can still do so. To do so, we also plumb through an initialClassLoader/initialClassPath so we can distinguish the static classpath entries from those that are loading by import $ivy in the course of a REPL/script execution

Having the possibility of external (application) classes overriding ammonite dependencies is not yet supported. The easiest thing to do might be to simply take the ammonite fat-jar and shading everything except the classes listed in ammonite-api-whitelist.txt

This should avoid the performance problems we were seeing where coursier's nested-jar arrangement was slowing things down due to repeated unzipping.

We also should no longer need the custom test-runner or test-api modules.

I also took the liberty to make use of the new ./mill version-pinning launcher script, and consolidate the cross-version scala-compiler-related logic into a single CompilerCompatibility.scala file

lihaoyi added some commits Jun 29, 2019

.
.
.
.
.
wip
.
.
.
.
.
- Avoid using `amm[2.12.8].test.run` to run code in development, sinc…
…e the test module has a different `classPathWhiteList` than the non-test module

- Include the `classPathWhiteList` in the cache key of `loadIvy`
.
fix

@lihaoyi lihaoyi merged commit e919d65 into master Jul 1, 2019

0 of 4 checks passed

continuous-integration/appveyor/branch Waiting for AppVeyor build to complete
Details
continuous-integration/appveyor/pr Waiting for AppVeyor build to complete
Details
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
continuous-integration/travis-ci/push The Travis CI build is in progress
Details
@alexarchambault

This comment has been minimized.

Copy link
Collaborator

commented Jul 2, 2019

@lihaoyi Are you sure about renaming interp-api and repl-api to interpApi and replApi? That leads to really weird module names at the end (and weird file paths in the repo too, like amm/replApi/…).

@lihaoyi

This comment has been minimized.

Copy link
Owner Author

commented Jul 2, 2019

Not really sure, maybe we could try converting those subfolders into repl/api/ folders? That's what we do in Mill, and gives nice amm-repl-api module names

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.