New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rsc-compile] use digests from output files and include them in classpath products #6772

Merged
merged 4 commits into from Nov 16, 2018

Conversation

Projects
None yet
3 participants
@baroquebobcat
Contributor

baroquebobcat commented Nov 15, 2018

Zinc compile expects that classpath entries have digests. This ensures that rsc jobs put digest having classpath entries into the classpath.

  • updates readme to include capture snapshot option
  • removes unused workaround tool classpath (Fixes #6520)
  • cleans up logging for desandboxing
  • when scala targets contain java, fall back to just using zinc
  • dump and load digests from files adjecent to output dirs
[rsc-compile] use digests from output files and include them in class…
…path products

Zinc compile expects that classpath entries have digests. This ensures that rsc jobs put digest having classpath entries into the classpath.

- updates readme to include capture snapshot option
- removes unused workaround tool classpath (Fixes #6520)
- cleans up logging for desandboxing
- when scala targets contain java, fall back to just using zinc
- dump and load digests from files adjecent to output dirs
Show resolved Hide resolved src/python/pants/backend/jvm/tasks/jvm_compile/rsc/rsc_compile.py Outdated
Show resolved Hide resolved src/python/pants/backend/jvm/tasks/jvm_compile/rsc/rsc_compile.py Outdated
input_digest = capture_or_read_digest(self.context._scheduler, os.path.dirname(maybe_file))
return ClasspathEntry(maybe_file, input_digest)
else:
return maybe_file

This comment has been minimized.

@illicitonion

illicitonion Nov 15, 2018

Contributor

I'm not in love with this method returning different types... Maybe this should return a ClasspathEntry(maybe_file, directory_digest=None)? Or maybe it should return None, and the caller should skip it?

This comment has been minimized.

@baroquebobcat

baroquebobcat Nov 15, 2018

Contributor

Hm. I think I'll have it return None. Skipping it is probably the best thing if the file doesn't exist yet, since we'll be filling in a new entry after the compile.

This comment has been minimized.

@baroquebobcat

baroquebobcat Nov 15, 2018

Contributor

I rewrote it entirely

@illicitonion

Much cleaner, thanks!

On the .digest approach - what we're doing elsewhere is just re-digesting the files every run (and memoizing them within a single run)... I'm hoping we can v2-ise this stuff soon enough that we shouldn't worry about it too much either way, so .digest files are not a problem, but if this becomes longer-term code as-is, we should think about it a bit :)

@baroquebobcat baroquebobcat merged commit 52c5ae6 into pantsbuild:master Nov 16, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@stuhood stuhood deleted the twitter:nhoward/rsc_next branch Nov 19, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment