Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit update checks to once per day #2918

Merged
merged 2 commits into from
Oct 17, 2023
Merged

Limit update checks to once per day #2918

merged 2 commits into from
Oct 17, 2023

Conversation

simonwo
Copy link
Contributor

@simonwo simonwo commented Oct 16, 2023

In #2893 we made more commands check for updates. Strictly not all of these checks are necessary as the code is unlikely to have changed regularly. Instead we should limit the amount we check for updates to (perhaps) once per day.

Every time we perform an update check, we write the current time into update.json. Every time we run a command. if the current time is less than 24h after this value, don't do any checking. If the current time is 24h ahead of this value, do an update check.

Resolves #2894.

@simonwo simonwo self-assigned this Oct 16, 2023
cmd/cli/root.go Outdated
util.Fatal(cmd, fmt.Errorf("failed to initialize bacalhau repo at '%s': %w", repoDir, err), 1)
} else {
ctx = context.WithValue(ctx, util.FSRepoKey, fsRepo)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need an alternative to embedding things in the context. This pattern isn't sustainable, and documentation states it should be avoided. https://pkg.go.dev/context

Copy link
Contributor

@rossjones rossjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use of context is a bit confusing because it looks like a different context references the repo, but I'm assuming there's some hidden pointer refs going on in the background.

versions, err := GetAllVersions(ctx)
if err == nil {
printMessage = &versions.UpdateMessage
err = writeNewLastCheckTime(ctx)
log.Ctx(ctx).Debug().Err(err).Str("LatestVersion", versions.LatestVersion.GitVersion).Msg("Performed update check")
}
}(cmd.Context())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this (cmd.Context()) the ctx with the value attached in root.go?

ctx = cmd.Context()
...
ctx = context.WithValue(ctx, util.FSRepoKey, fsRepo)

It's kinda confusing that this works calling .Context()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there is a cmd.SetContext() call that overlays the new context into the command before the end of the pre-run hook.

@@ -65,23 +69,82 @@ func GetAllVersions(ctx context.Context) (Versions, error) {

var printMessage *string = nil

const updateStatePath string = "update.json"
const updateFrequency time.Duration = 24 * time.Hour

// StartUpdateCheck is a Cobra pre run hook to run an update check in the
// background. There should be no output if the check fails or the context is
// cancelled before the check can complete.
func StartUpdateCheck(cmd *cobra.Command, args []string) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could pass the repo path as an argument to this method and remove the method GetFSRepo which looks in the context.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, if that is not possible, lets un-export the method GetFSRepo and it's internals as the pattern shouldn't be used outside this specific case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The challenge is that this method is implementing the pre/post-run hooks interface, which only accepts string arguments and a context. AFAICS the context is the only way to pass state into hooks, without using a global variable which I think would be worse.

I think un-exporting the method is a sensbile change, and I'll add a comment to the effect of "don't start using this because context is intended to be used sparingly".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It turns out using the config system properly removes the need to do any of this, as it will now set up the absolute path on repo initialisation rather than exposing the repo here. Nice.

Comment on lines 72 to 73
const updateStatePath string = "update.json"
const updateFrequency time.Duration = 24 * time.Hour
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have expected values like these to go in the config pkg and file, but that can always comes later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's a good point and would be in keeping with the other repo values.

Is there any way to provide default values for these? As users upgrading won't have them in their config file ofc. If I just put them in the configenv structs, will they get used if the value is missing from the config file?

Copy link
Member

@frrist frrist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left some suggestions for alternatives.

@wdbaruni
Copy link
Member

We currently have a repo.version file that holds the current version of the repo (not bacalhau):

cat ~/.bacalhau/repo.version
{"Version":1}

We are now introducing a new update.json, which I still agree is the right thing compared to using config.yaml. I am concerned the repo will become a mess as more states need to be persisted. Does it make sense to create a single file that holds both repo.version, latest update check and all future states we care about?

@simonwo
Copy link
Contributor Author

simonwo commented Oct 16, 2023

Does it make sense to create a single file that holds both repo.version, latest update check and all future states we care about?

I'm not sure... this is essentially moving a key-value store implemented on the filesystem into a key-value store implemented in a single file. We'd just be moving "mess on the filesystem" into "mess in a yaml file" and I'm not sure if I think that being conservative on files on the filesystem is all that valuable? What problems do we think this is going to cause as it scales?

In #2893 we made more commands check for updates. Strictly not all of
these checks are necessary as the code is unlikely to have changed
regularly. Instead we should limit the amount we check for updates to
(perhaps) once per day.

Every time we perform an update check, we write the current time into
update.json. Every time we run a command. if the current time is less
than 24h after this value, don't do any checking. If the current time
is 24h ahead of this value, do an update check.
@simonwo
Copy link
Contributor Author

simonwo commented Oct 16, 2023

OK! Thanks for the comments everyone. I have now properly understood the config system and using it properly has actually made most of the issues around contexts disappear. So thanks for the push back @frrist!

Through proper use of the config system, many of the code review
complaints can be avoided entirely.
@simonwo simonwo merged commit 7054515 into main Oct 17, 2023
13 checks passed
@simonwo simonwo deleted the 2893-update-checks branch October 17, 2023 07:37
@wdbaruni
Copy link
Member

I'm not sure if I think that being conservative on files on the filesystem is all that valuable? What problems do we think this is going to cause as it scales?

Just avoiding a bloated repo. No performance or scale issues, just a user experience issue when they see tens of files in their repo not knowing which ones they should or shouldn't touch. Shouldn't be a blocker for this PR, and we can make a more informed decision when we see more of these configs and states in the future. A slight improvement might be to make the file hidden. update.json -> .update.json, or have a directory inside the repo with all these files. Again we can delay this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Limit update checks to once per day
4 participants