Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always keep the CA derivations around #4139

Closed
wants to merge 1 commit into from

Conversation

@regnat
Copy link
Contributor

@regnat regnat commented Oct 12, 2020

Workaround for #4138

@edolstra
Copy link
Member

@edolstra edolstra commented Oct 12, 2020

Does this mean that CA derivations are never garbage-collected?

@regnat
Copy link
Contributor Author

@regnat regnat commented Oct 12, 2020

They aren't until their output-path is (like if keep-derivations is set to true in the config)

@Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Oct 12, 2020

In fact, perhaps they should be kept around even if their output isn't.

Suppose we have:

( resolved-gcc.drv , "out" ) -> foo
( resolved-bash.drv [ foo / "gcc" ] , "out" ) -> bar

If we get rid of the row for foo, then if someone tries to rebuild (unresolved) bash.drv they'll end up rebuilding GCC!

We're going to need a pretty big migration I'm afraid to have the bar row preserve the foo row without also preserving the foo path itself.

Copy link
Member

@Ericson2314 Ericson2314 left a comment

All that said, this PR makes the situation wholly better, and it is an easy enough stop-gap to back out of whenever we do do the sqlite overhaul.

@regnat
Copy link
Contributor Author

@regnat regnat commented Oct 12, 2020

In fact, perhaps they should be kept around even if their output isn't.

Suppose we have:

( resolved-gcc.drv , "out" ) -> foo
( resolved-bash.drv [ foo / "gcc" ] , "out" ) -> bar

If we get rid of the row for foo, then if someone tries to rebuild (unresolved) bash.drv they'll end up rebuilding GCC!

We're going to need a pretty big migration I'm afraid to have the bar row preserve the foo row without also preserving the foo path itself.

I'm not 100% sure, but I think that the DerivationOutputs entries stay there as long as the corresponding derivation is there (even if their output is gc-ed), so in that case the first mapping would stay alive because resolved-gcc.drv would (as it's a dependency of resolve-bash.drv)

@Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Oct 12, 2020

Ah, but resolved-gcc.drv is not a dependency of resolve-bash.drv, because resolved floating CA derivations always have empty inputDrvs. That's the difficulty.

@Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Oct 13, 2020

Ah CI failed because of that annoying spurious failure. Do we have an issue for it?

@regnat
Copy link
Contributor Author

@regnat regnat commented Oct 13, 2020

Ah, but resolved-gcc.drv is not a dependency of resolve-bash.drv, because resolved floating CA derivations always have empty inputDrvs. That's the difficulty.

Oh yes because we only have this mapping for resolved drvs 🤔
I think we need to dig all that a little to properly fix this (but the workaround is worth keeping in the meantime)

@edolstra
Copy link
Member

@edolstra edolstra commented Oct 13, 2020

In fact, perhaps they should be kept around even if their output isn't.

But then derivations will accumulate forever... Or they could be treated like a cache and have some expiration policy.

@regnat regnat closed this Nov 26, 2020
@regnat regnat added this to the ca-derivations-mvp milestone Dec 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants