Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[batch] Update gcsfuse version to 1.2.0 #13728

Merged
merged 3 commits into from
Oct 25, 2023
Merged

Conversation

jigold
Copy link
Collaborator

@jigold jigold commented Sep 27, 2023

Let's see if this "just works".

@jigold jigold added the WIP label Sep 27, 2023
@danking
Copy link
Contributor

danking commented Sep 28, 2023

We don’t have lab_release in our Ubuntu image. We could add it to hail Ubuntu or just in-line the name. We had bionic here but that’s 18.04 we really should use jammy.

@danking
Copy link
Contributor

danking commented Sep 28, 2023

Also looks like we might need to go to 1.1.0 b/c that’s all that’s been released to Apt https://packages.cloud.google.com/apt/dists/gcsfuse-jammy/main/binary-amd64/Packages

@jmarshall
Copy link
Contributor

jmarshall commented Oct 16, 2023

Looking at the …/Packages URL in the previous comment, 1.2.0 is now available (and 1.1.0 does not appear to be there).

In our recent local hail update deployment, the batch_worker_image job failed repeatedly due to GoogleCloudPlatform/gcsfuse#1424. We worked around this as initially suggested on that issue with populationgenomics/hail@607408b, but later comments on the issue suggest that the better approach would be this PR with an additional change to access the apt repo via https:

-    echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list && \
+    echo "deb https://packages.cloud.google.com/apt $GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list && \

@jmarshall jmarshall mentioned this pull request Oct 16, 2023
@danking
Copy link
Contributor

danking commented Oct 18, 2023

@jigold can I remove the WIP tag? I'd like this to make it into the 0.2.125 release so that AUS can check out a tagged release.

@jigold
Copy link
Collaborator Author

jigold commented Oct 18, 2023

Yes with the caveat that I was going to look at the worker logs in the PR test namespace just to make sure there were no hidden errors that would be problematic. I broke the logging query generator -- fixed in #13813 -- and didn't think this was urgent. I'll take a look at the logs now without that fix in.

@jigold
Copy link
Collaborator Author

jigold commented Oct 18, 2023

I forgot I turned off the syslog so we won't see errors there from while the worker is running.

Up to you on whether you think this change needs more scrutiny.

@jmarshall
Copy link
Contributor

It's not so much so that we can check out a tagged release, as we have already worked around the problem. But I would expect that you and any other installations will also run into the same batch_worker_image failure.

We have been running our production instance using gcsfuse 1.2.0 for about a week now, and I think @illusional will agree with me that we haven't seen any problems from it.

@jigold jigold removed the WIP label Oct 25, 2023
@danking danking merged commit e739a95 into hail-is:main Oct 25, 2023
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants