cargo crev crate clean not working

[ReactorScram]

If I run cargo crev crate clean and then cargo crev verify, verify always says I have 2 unclean packages. I think they were two crates I was reviewing on an older version of crev, before I un-installed it and later re-installed it.

This is with crev 0.21.4, installed from Cargo. I'm using Arch Linux on x86_64.

Running the clean command under strace shows that it's trying to delete some files that don't exist. It looks like the files in e.g. ~/.cargo/registry/src/github.com-1ecc6299db9ec823/anyhow-1.0.44/ are in a half-deleted state where some files remain (making verify say they're unclean) but clean doesn't know which ones still need deleting.

I'll tar up the unclean folders as a backup, in case there's anything interesting in the presence / absence of files.

[dpc]

After trying it out myself I immediately started getting similar problems. After some investigation I've changed the reported diagnostics and now it looks like this:

4 local crates with digest mismatch detected. Use `cargo crev crate clean [<name>]` to clean any potential unclean local copies. If problem persists, contact the reporter.
Crate aead                 0.4.3           ; local digest: sWHmGSevjVwjDxyT631QUCRIiA0Kq8SpIkkRGx5i9go != sBQJ4TW9c324zYgd5DvWuaz-fdDrJC1fdc8f0JW8RG4 reported by 24YKeuThJDNFSlJyxcl5diSZcKcRbh-0zXM0YxTOFJw (https://github.com/Lucia
noBestia/crev-proofs)                                                                                              
Crate atty                 0.2.14          ; local digest: WJaqS5xu6ORo2px2Rku1ynYYN4TuCdxeS8LR68ngyw4 != 43bdBdz29rGPtdjhLdE0zA-ZudRkwB1ISyTTj0vft-I reported by 24YKeuThJDNFSlJyxcl5diSZcKcRbh-0zXM0YxTOFJw (https://github.com/Lucia
noBestia/crev-proofs)                                                                                                                                                                                                                  
Crate cipher               0.3.0           ; local digest: RWl72FpMPCylFnYYtdCaNUhg_7BBwcocTAgjl_sLQTc != vDRlIx2w0uBLUqrDmno2yli_uUZ4E8-LUexlJLW3Cjo reported by 24YKeuThJDNFSlJyxcl5diSZcKcRbh-0zXM0YxTOFJw (https://github.com/Lucia
noBestia/crev-proofs)                                                                                              
Crate libz-sys             1.1.3           ; local digest: rEGXjh1JwMUUN0Cgcx-24kbPJ2eZUGba3K4TQsgolPo != ZU3ZXFo-2MXtbDQu_s0jFzS-2eFSKSK_jB8HIKEyOAU reported by 24YKeuThJDNFSlJyxcl5diSZcKcRbh-0zXM0YxTOFJw (https://github.com/Lucia
noBestia/crev-proofs)

Any cleaning (even manually) does not seem to help, so I guess the reporter somehow got incorrect digests, which is not what I expected when writing the original code.

Hi @LucianoBestia!

[dpc]

Other than this crate clean [<name>] seems to work just fine.

[bestia-dev]

I Will look into it. I use the function from crev lib to calculate the digest.

[dpc]

@LucianoBestia Oh. In that case I can already tell you, that you probably want to use

cargo-crev/cargo-crev/src/shared.rs

Line 379 in ae2c6ff

pub fn check_package_clean_state(

instead.

For Rust we need to do bunch of sheningans before calculating digest. See comments.

[bestia-dev]

I have this code for the digest:

fn ....
    crate_name: &str,
    crate_version_str: &str,
	
...	
let crate_root = crate::cargo_registry_mod::cargo_registry_src_dir_for_crate(crate_name, crate_version_str)?;
let digest_clean = crev_lib::get_recursive_digest_for_dir(&crate_root, &cargo_min_ignore_list())?;
...



pub fn cargo_min_ignore_list() -> fnv::FnvHashSet<std::path::PathBuf> {
    let mut ignore_list = std::collections::HashSet::default();
    ignore_list.insert(std::path::PathBuf::from(".cargo-ok"));
    ignore_list
}

/// cargo registry src directory for a crate
pub fn cargo_registry_src_dir_for_crate(crate_name: &str, crate_version: &str) -> anyhow::Result<std::path::PathBuf> {
    let crate_dir_name = format!("{}-{}", crate_name, crate_version);
    let crate_src_path = format!(
        "{}/registry/src/github.com-1ecc6299db9ec823/{}",
        home::cargo_home()?.to_str().unwrap(),
        crate_dir_name
    );
    let crate_root = std::path::PathBuf::from_str(&crate_src_path)?;
    // return
    Ok(crate_root)
}

[bestia-dev]

Just for curiosity: Does crates.io have a digest or hash to verify if the downloaded files are not compromised?
If it has, is it the same as the crev digest?

[bestia-dev]

I see now in check_package_clean_state the comments.
All the copying made to avoid unclean state.
Thank you, I will use that.

[dpc]

Just for curiosity: Does crates.io have a digest or hash to verify if the downloaded files are not compromised?

Last time I check it did not, at least not in the unpacked source.

[bestia-dev]

I found this "cksum" field in the json cargo registry files, not in the src directory:
\\wsl$\Debian\home\luciano\.cargo\registry\index\github.com-1ecc6299db9ec823\ca\rg\cargo_auto_lib
Is this perhaps the checksum of the source code folder?
I suppose cargo should have a mechanism to check if the downloaded files are compromised.

{
    "name": "cargo_auto_lib",
    "vers": "0.7.23",
    "deps": [{
...
        }
    ],
    "cksum": "ba680d84f32d0dc2be953b5f7b3f2e6a3fd61fbe7d220b3ad18a6e7bd5d27daa",
    "features": {},
    "yanked": false,
    "links": null
}

[KaiserKarel]

I was also wondering why we are computing digests, instead of using the checksumming mechanism from the Cargo.lock files?

[dpc]

Mostly because cargo is not obligated to stick with these. They are free to repackage all their tar.gz (or whatever is being used) to .xz tomorrow to save on space, and leave cargo-crev hanging. Same with other package managers. That is why crev is using https://github.com/crev-dev/recursive-digest over extracted files to checksum the source code.