feat: update vendored xz to 5.6.1 #2
Conversation
Updates the vendored version of `xz` to be `5.6.1`. Also updates the vendor script to support the addition of `SPDX-License-Identifier` headers into some files.
|
Thank you! So if I'm understanding correctly the license of the C files have changed? We'll need to update this repo's readme I guess. |
|
Correct, I realized that after I made this change. I'll update the README |
jaredallard
force-pushed
the
jaredallard/update-vendor-5.6.1
branch
from
76932be to
a3b2f62
Compare
|
I just read that xz 5.6.1 has been compromised: https://news.ycombinator.com/item?id=39865810 I'm sorry, but I find the timing of this PR to be very suspicious. This repo has been dormant for 2/3 years with no PRs. But a compromised xz is released and all of a sudden there's a PR to include it here... |
|
None of the backdoor files ( |
It's not out of the question that a malicious PR author would wait for an approval before updating the PR with malicious contents. I believe GitHub flows allow for that if a repo is configured as such. |
|
I've sent an email to 1Password security
|
Honestly? I would consider 5.6.0 and 5.6.1 source compromised until audited. The attacker did a backdoor in the packaging but they did have repo access, so who knows if anything else is hidden inside. For example, this commit: Is most likely part of hiding the exploit that was in the packaging. |
I would not only consider 5.6.0 and 5.6.1 to be compromised, that GH account signed releases for multiple years. I'm just not a fan of jumping to conclusions on things that are not obviously (but might be) related. |
|
This is serious! While this is mostly incoherent blurb of links and thoughts, this vulnerability "in general" is well planned and timing of 1password personnel pushing this ASAP now sounds suspicious, while obviously it might not be related directly, as 1PW user I obviously certainly hope these are not related... But XZ is backdoored in certain versions and there is no telling yet how deep this rabbit hole goes! List of links relating tho this backdoor:
https://news.ycombinator.com/item?id=39865810 So in any way, DO NOT PUSH THIS VERSION OF XZ TO PRODUCTION!! |
|
The PR author mentions |
https://www.linkedin.com/in/jaredallard/ is a verified LinkedIn account (meaning the account owner has a 1Password/AgileBits email address). To be clear I don't think the 1Password employee is complicit in the supply chain attack. If this PR were a malicious act, it's likely due to this GitHub account having been taken over. |
Good information 👍 However, the takeover must have went unnoticed for multiple days given the timeline of this PR (open + comment 4 days apart). Which is not impossible (account owner on holiday, etc.). |
|
I think it's best to just wait for a comment from @jaredallard. That will help clear things up. |
|
While the immediate results is more or less engineered towards certain "this is getting built, target that", the author has had access to repository over an year.. So "we" or 1PW, needs to be extra careful what has been used and where. |
ghost
mentioned this pull request
|
Hey all, context on this change: I've been working on building a Gentoo binary package server that I maintain on Github. I wanted an easier way to manage Gentoo binary package targets (that is to make managing my current gentoo.rgst.io site easier to operate). As part of this, I needed to support xz archives. I always set BINPKG_COMPRESS When I looked at that repo, I noticed it easily supported updating the version of xz in use and that it hadn't been updated in 3 years. Given that I get a huge kick out of updating things, for example: on ohmyzsh and gentoo (I guess from SRE days?), I updated the submodule, ran the generate code and prayed tests would pass. They did and the functionality worked locally, so I figured I'd be nice and open a PR on that repo since it's basically never gotten any love. That's it. We can see where I used that code out in the open also: https://github.com/jaredallard/binhost/blob/main/go.mod#L5 |
Why not to use zstd? The later one has compression ratio similar to xz, but blazing fast decompression speed. |
Going forward I will now 😄 But, to be honest, I just haven't kept up with the state of the world there. |
|
well that was some really unfortunate timing... |
Perhaps this event will get people to chill to that attitude. |
|
Closing this for... obvious reasons 😄 |
|
I'm still baffled by how github deemed it appropriate to just hide all code and all history of this from public eye in reponse. |
even if this PR is perfectly fine, it helps to build a "web of trust" to xz-5.6. "See, Debian guys, everyone is upgrading to xz-5.6 now, why you are slugs?" |
afaik, lzma (xz) has the same compression ratio as zstd on text files, but ~10% better on many binary files. try e.g. a large executable or a database file, use the same dictionary size and choose maximum compression |
|
@AUTOMATIC1111 I think you assume more than there is. They mention the breach of their terms, which mention malware as forbidden content. Since both maintainers were suspended by GitHub, they are probably just minimizing the damage and chose the easiest solution for the moment. We'll have to wait until there is news from GitHub regarding the incident. |
Obviously the alphabet boys got involved. They're reading this PR too, I'm sure 😉 |
|
We'll just have to wait until the complicit party clarifies that it's not complicit in the cover up. |
its to prevent any inadvertent further deployment |
|
Probably they do not have something more fine-grained. :/ A good solution for the future would be to have a way for their staff to mark a repository as "contains compromised code", which would then add a big disclaimer when viewing it via web, disable unauthenticated git clones, and for authenticated clones require acknowledging (via web) that you really do want to clone a repository containing malicious code. |
I support this idea. GitHub, take notes! |
Protecting history is one of the core ideas of github so scrubing it clean is not the way to go imo. Disabling clones is a better approach |
|
Does anyone here have a mirror of xz that is relatively up-to-date? I'm trying to do security research(i.e. torturing myself for hours by going through EVERY SINGLE COMMIT by the attacker if no one else has done it) and you can't |
|
Thanks! |
Just use the official mirror: https://git.tukaani.org/?p=xz.git;a=summary |
|
Thanks both, someone emailed me with this
<https://github.com/fionn/xz-backdoored> mirror(which probably will be
removed soon :().
…On Sat, Mar 30, 2024 at 8:42 AM Fabian Dellwing ***@***.***> wrote:
Does anyone here have a mirror of xz that is relatively up-to-date? I'm
trying to do security research(i.e. torturing myself for hours by going
through EVERY SINGLE COMMIT by the attacker if no one else has done it) and
you can't git clone from the internet archive.
Just use the official mirror: https://git.tukaani.org/?p=xz.git;a=summary
—
Reply to this email directly, view it on GitHub
<#2 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ATOOUFSKEIDLRLENX3NUQ3LY22XJ5AVCNFSM6AAAAABFEK4ZLKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRYGA3TIMZWGQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
is @inversebrah on github too |
|
only the fake one
|
|
Thank you for your very calm and kind response @jaredallard! I hope my comment calling this PR was suspicious wasn't too offensive. This is a crazy coincidence! I think closing the PR was the right move. In fact, given that the xz attacker was involved in the xz repo for 2 years, I don't see any path for updating the version xz used in for this Go library at all. I am going to note on the repo readme that (a) the repo is essentially frozen and (b) the version of xz in use predates the attack so it's still potentially safe to use, depending on your risk profile. Maybe moving to zstd is the right move though. |
|
Official announcement from XZ team is here: |
Maybe I should send an email to Netcraft in London that their "Security engineer I" Andy Wang is sending emails to employers making baseless claims, 'possibly to shift blame' and to 'investigate if they're a state actor who have an interest in doing so.' (y'know - while we're all making baseless claims here) Witch hunting is a shameful behavior. |
|
This kind of coincidence could happen to anyone who contributes to OSS. It is a shame that some people jumped to conclusions without waiting for an explanation. Might next times we all remember that there are different timezones around the world and that different cultures celebrate different holidays. Thank you very much for the calm and good sense @jamespfennell. I hope no harm was done to @jaredallard employment relationship and that he continues contributing to OSS. |
Informing 1Password about a possible account being compromised is the correct thing to do. While wrong in this case and just a coincidence. The report would, in the case of the account being overtaken, allow for a faster response both from Jared and 1Password, allowing them to limit the damage done to both the company and the person himself. Of course in this case it was not necessary as it was simply a coincidence but if my account was compromised I’d prefer to have a warning as soon as possible (that my account was presenting suspicious behaviour). |
|
CIA agent spotted |
The email to 1Password was a security behavior. An email such as the one you're describing would be harassment/retaliation/any given number of things, and could potentially make you liable for any damages that happen next. |
|
why don't we all just relax with the emailing of employers |
|
Also the attacker included in the 5.6.0 release the support for the long-awaited multi-threading decompression making it very attractive to upgrade to... It was probably a tactic to give a reason to upgrade. It's not always a fault for those who did or tried to do. |
??? |
Updates the vendored version of
xzto be5.6.1. Also updates thevendor script to support the addition of
SPDX-License-Identifierheaders into some files.