New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start to add higher level API, add benchmark module, then small optim #1026

merged 10 commits into from Jan 22, 2019


None yet
1 participant
Copy link

alexarchambault commented Jan 22, 2019

No description provided.

alexarchambault added some commits Jan 22, 2019

Have the methods of coursier.Cache accept sequences of cache policies
Rather than a single policy, that required users to compose calls to
these methods themselves.
Now require CacheLogger-s to be re-usable
One should be able to call init() again after a stopDidPrintSomething()
Rework ArtifactOptions a bit
Move it under coursier.cli.options.shared

Don't require its helpers to be passed arguments it already has in
Move dependency parsing options to a new DependencyOptions
DependencyOptions / DependencyParams should stay in the cli module.

ResolveParams should be moved to the meta module, as part of the new
higher level API.
Keep data in memory when running benchmarks from CLI
Only enabled when passing the --benchmark-cache flag to resolve for now.
Move some option validation logic closer to the actual options
Aiming at moving CacheParams and ResolveParams to the meta module, as
part of the higher level API, without the option validation stuff.

@alexarchambault alexarchambault force-pushed the develop branch 2 times, most recently from d1dc835 to 812e529 Jan 22, 2019

alexarchambault added some commits Jan 22, 2019

Move bits of the resolve command to the meta module
As parts of the new higher level API

This comment has been minimized.

Copy link
Member Author

alexarchambault commented Jan 22, 2019

Output of sbt "benchmark/jmh:run -i 3 -wi 3 -f1 -t1" at e255973:

[info] Benchmark                              Mode  Cnt    Score    Error  Units
[info] ParseTests.parseApacheParent           avgt    3    0,462 ±  0,270  ms/op
[info] ParseTests.parseSparkParent            avgt    3    3,163 ±  0,175  ms/op
[info] ParseTests.parseSparkParentMavenModel  avgt    3    0,776 ±  0,008  ms/op
[info] ParseTests.parseSparkParentXml         avgt    3    2,980 ±  0,632  ms/op
[info] ParseTests.parseSparkParentXmlSaxWip   avgt    3    0,882 ±  0,050  ms/op
[info] ParseTests.parseSparkParentXmlStaxWip  avgt    3    0,739 ±  0,025  ms/op
[info] ProcessingTests.sparkSql               avgt    3    2,193 ±  5,063  ms/op
[info] ResolutionTests.coursierCli            avgt    3    4,078 ±  0,525  ms/op
[info] ResolutionTests.sparkSql               avgt    3  311,430 ± 39,992  ms/op

Output at aad08b5:

[info] Benchmark                              Mode  Cnt    Score    Error  Units
[info] ParseTests.parseApacheParent           avgt    3    0,410 ±  0,084  ms/op
[info] ParseTests.parseSparkParent            avgt    3    2,785 ±  0,239  ms/op
[info] ParseTests.parseSparkParentMavenModel  avgt    3    0,787 ±  0,075  ms/op
[info] ParseTests.parseSparkParentXml         avgt    3    2,341 ±  0,501  ms/op
[info] ParseTests.parseSparkParentXmlSaxWip   avgt    3    0,913 ±  0,064  ms/op
[info] ParseTests.parseSparkParentXmlStaxWip  avgt    3    0,731 ±  0,005  ms/op
[info] ProcessingTests.sparkSql               avgt    3    0,641 ±  0,084  ms/op
[info] ResolutionTests.coursierCli            avgt    3    3,646 ±  1,995  ms/op
[info] ResolutionTests.sparkSql               avgt    3  214,165 ± 16,188  ms/op

(compare 3 last lines in both cases)

aad08b5 mostly replaces some regex-based substitutions by some manual while-loop based stuff, which seems to be a clear win (30% speed-up when resolving org.apache.spark:spark-sql_2.12:2.4.0).

This also shows that a SAX or StAX based parser would likely be more efficient to parse POMs. It should be added in a later PR.

@alexarchambault alexarchambault merged commit 58913a9 into master Jan 22, 2019

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
continuous-integration/travis-ci/pr The Travis CI build passed

@alexarchambault alexarchambault deleted the develop branch Jan 22, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment