Fixed blst version

[aakoshh]

The version of the blst dependency is always fixed to exactly match a specific version, like here.

Is that a true requirement, or was the = left in by accident? It makes it difficult to use libraries built on this one with something else that requires a higher patch version. For example v0.5.0 wants =0.3.9, but another dependency I have wants 0.3.10 of blst. If blstrs was happy with 0.3.9 without = the cargo should be able to pick the higher version.

[dignifiedquire]

this is on purpose, as blst releases sometimes contain breaking changes in patch releases and we have been burnt by this before

[aakoshh]

Thanks. Then I guess I will have to fork this or that dependency and try to patch it. It might work but it's a bit of a maintenance nightmare, and I can miss crucial updates.

Perhaps, since v0.6.0 depends on =0.3.10, it would be possible to also release a newer version v0.5.1 that allows a range you are comfortable with, say, between 0.3.9-0.3.10, so anything that depends on 0.5.0 can be safely upgraded to 0.5.1 and allow cargo to resolve the dependency hell?

[aakoshh]

I forked the projects and made new branches to try until I found a combo that worked (with my current dependencies at least).

I would open PRs but there are no 0.4.x and 0.5.x branches to open them against:

• https://github.com/filecoin-project/blstrs/compare/v0.4.2...aakoshh:v0.4.3?expand=1
• https://github.com/filecoin-project/blstrs/compare/v0.5.0...aakoshh:blstrs:v0.5.1?expand=1

And similarly in bls-signatures:

• filecoin-project/bls-signatures@v0.11.3...aakoshh:bls-signatures:v0.11.4
• filecoin-project/bls-signatures@v0.12.0...aakoshh:bls-signatures:v0.12.1

For me it looks like blstrs:v0.4.3 with bls-signatures:v0.11.4 works against a dependency that needs blst:0.3.10

[vmx]

@aakoshh sorry, I forgot to open filecoin-project/bls-signatures#55. With this PR you should be able to update everything to their latest versions.

[aakoshh]

Thanks @vmx! I checked and at least one more domino that I see you kindly maintain is bellperson, which still depends on blstrs 0.5.0

[aakoshh]

And probably the filecoin-proofs-api as well.

This is the chain I get when I tried to upgrade Forest from 11.0 to 12.0 (which I don't know if it's safe to do as it's a major version change):

error: failed to select a version for `blst`.
    ... required by package `blstrs v0.5.0`
    ... which satisfies dependency `blstrs = "^0.5.0"` of package `bellperson v0.22.0`
    ... which satisfies dependency `bellperson = "^0.22"` of package `filecoin-proofs-api v12.0.0`
    ... which satisfies dependency `filecoin-proofs-api = "^12.0"` of package `forest_fil_types v0.2.2 (/home/aakoshh/projects/consensuslab/forest/types)`
    ... which satisfies dependency `forest_fil_types = "^0.2"` (locked to 0.2.2) of package `forest v0.2.2 (/home/aakoshh/projects/consensuslab/forest/forest)`
versions that meet the requirements `=0.3.9` are: 0.3.9

I can't just patch blstrs and bls-signatures to their latest because cargo can't upgrades something from depending on 0.4 to 0.6 as they are not compatible 🤷

[vmx]

@aakoshh do you need to upgrade to the latest blstrs version? If not, I'd suggest to stay on 0.5.0 for now. Updating the whole dependency chain is a WIP and it will take a while.

[aakoshh]

In my case I couldn't use blstrs 0.5.0 because it required =0.3.9 of blst and something I also wanted to use wanted ^0.3.10. I could have used 0.6.0 but like you said it's a whole dependency chain, and I am unable to use the [patch.crates-io] mechanism to force a dependency like bellperson to go from 0.5.0 to 0.6.0.

Instead what I did was to go back to the earliest blstrs version, which is 0.4.0 required by filecoin-proofs-api 11.0 in use and relax the constraint there to allow 0.3.10, thinking that if blstrs 0.6.0 is happy with it, then it must be all right. And I can use [patch.crates-io] to upgrade 0.4.x to 0.4.3. This is the lightest touch that allows me to do what I need as I'm not a Forest core dev, so I don't want to go updating all kinds of dependencies for them.

But I do wonder if these =0.3.9 style constraints would be better pushed to the applications' root Cargo.toml files rather than in base libraries. I'm also not sure why libraries are not more permissive and just depend on 0.3, allowing any patch version, trusting that the application developer will want to use the newest they can get their hands on with cargo update, or the newest they trust, or the newest another more cautious library like yours allows them to do.

Anyway, I'm unblocked now, although probably this will happen again and again.

[vmx]

Thanks for the explanation! At least it's good to hear that the upgrade to blst 0.3.10 might be worth it, as it sounds like you'll need it. I already though that it would've been better if I wouldn't have upgraded it.

Though usually, things should work out, which is the reason I do breaking releases. Usually all Filecoin related dependencies are taken care of and should end up with the same blst version (once all releases are done). Until then old versions should be used. Though I see how this is a problem that you cannot easily [patch] things, that's really sometimes a pain.

[vmx]

All dependencies should be on blst v0.3.10 by now. I think it is no longer an issue. If it is, please re-open.