Add Solana Program Library as a submodule #90

ruuda · 2021-05-20T13:34:59Z

Solido depends on the SPL stake pool program, which is not present on a local testnet by default, so we need to build it from source. To be able to do that reliably in the tests, pull in the SPL as a submodule here, so we can depend on its stake pool program.

The specific commit adds one patch on top of the upstream repository, which removes the Cargo workspace in that repository, to prevent it from interfering with our workspace.

Aside from helping to test on a localnet, having a pinned version of the SPL stake pool program is also useful if we want to deploy our own version of the stake pool: we can pin it, and upgrade at our own pace. But we can still pull in the latest changes from upstream in the submodule, there is no difficult patch backporting process.

The version of the SPL that we pull in is more recent than the one that we depended on previously. In particular it includes a fix for fee distribution in the stake pool.

ruuda · 2021-05-20T13:37:37Z

This does make one test fail:

---- test_successful_update_balance stdout ----
thread 'test_successful_update_balance' panicked at 'assertion failed: `(left == right)`
  left: `10526315789`,
 right: `20000000000`', program/tests/update_balances.rs:150:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

~~Not sure what’s going on there, it is probably related to using a newer version of the stake pool.~~

Edit: This is definitely caused by the fix for the fee computation. I’ll fix the test.

Edit: Test fixed.

ruuda · 2021-05-20T15:45:23Z

.github/workflows/rust.yml

@@ -4,7 +4,7 @@ on:
  push:
    branches: [ main ]
  pull_request:
-    branches: [ main ]
+    branches: '*'


I noticed CI was not running for this pull request, because I set the base to add-multisig-tests (it depends on #62). This fixes that.

naomijub · 2021-05-21T14:16:25Z

@ruuda the tests seems to be failing still

ruuda · 2021-05-21T15:06:32Z

After running cargo clean I can also reproduce locally. The result makes no sense to me, it receives 20e9 in the fee account, which is equal to the fraction of the rewards in SOL, but they have different units and should have different values ... investigating.

ruuda · 2021-05-21T17:20:28Z

It seems we need to run cargo build-bpf before running cargo test-bpf, because otherwise the test might use outdated artifacts 😕 After running cargo build-bpf the test no longer fails for me. Let's see if that also fixes the problem on CI.

ruuda · 2021-05-21T17:59:49Z

Hmm, so the difference was in running cargo test-bpf from the root of the repository or from the program subdirectory ... if it runs from the program subdirectory, then for some reason the tests use the wrong version of the stake pool program. I have no clue where it comes from though, as all references are path references now. But the build is fixed 🤷‍♀️

enriquefynn · 2021-05-21T19:07:01Z

program/tests/update_balances.rs

@@ -93,6 +96,8 @@ async fn test_successful_update_balance() {
        .await;
    assert!(error.is_none());

+    // Transfer EXTRA_STAKE_AMOUNT Lamports to every validator outside of Lido


Thanks for the comments, I'll be more prolific about it ^^

enriquefynn · 2021-05-21T19:22:36Z

.github/workflows/rust.yml

        sudo apt-get install -y libudev-dev
        sh -c "$(curl -sSfL https://release.solana.com/v1.6.7/install)"
        export PATH="$HOME/.local/share/solana/install/active_release/bin:$PATH"
+        cargo build-bpf


This is going to test also the solana-program-library, is there a way to call only Lido's tests?

cargo test-bpf --manifest-path program/Cargo.toml seems to do the trick, will update that.

enriquefynn

This makes tests to the solana-program-library submodule, I think it would be better to avoid these tests

glottologist · 2021-05-22T07:18:08Z

You would have to mark the tests with #[ignore] in the submodule but I don't think that is viable.

enriquefynn · 2021-05-25T08:37:18Z

Maybe also delete program/tests/fixtures/spl_stake_pool.so so we only use the built one

ruuda · 2021-05-25T08:47:09Z

Maybe also delete program/tests/fixtures/spl_stake_pool.so so we only use the built one

Aaah, that might be where it came from! Good catch, removed.

Solido depends on the SPL stake pool program, which is not present on a local testnet by default, so we need to build it from source. To be able to do that reliably in the tests, pull in the SPL as a submodule here, so we can depend on its stake pool program. The specific commit adds one patch on top of the upstream repository, which removes the Cargo workspace in that repository, to prevent it from interfering with our workspace. Aside from helping to test on a localnet, having a pinned version of the SPL stake pool program is also useful if we want to deploy our own version of the stake pool: we can pin it, and upgrade at our own pace. But we can still pull in the latest changes from upstream in the submodule, there is no difficult patch backporting process.

enriquefynn

LGTM!

ruuda · 2021-05-25T12:34:19Z

I rebased now that #75 has been merged. I had to fix one test that was added, the fix is 36d6902. @enriquefynn if this looks good to you I will go ahead and merge.

After pulling in a new version of the SPL stake pool, which fixes the way it computes rewards, we need to update the test to match, because the number of reward tokens is indeed different.

Without this, if you open a stacked pull request (against the branch of an existing pull request), it will not run CI for that pull request.

For some reason, the "multisig" mod in the crate was conflicting with the "multisig" crate name itself. I'm not sure why this suddenly started happening now, but renaming the mod inside resolved the problem. This updates the submodule to the commit that includes that rename.

The test that depends on the SPL stake pool program fails without first running "cargo build-bpf"; the fee computation is wrong. I'm not sure why this happens, it's as if it uses an older version of the SPL stake pool program, but I don't know where it gets that from. Either way, running "cargo build-bpf" before running the tests makes the tests pass ...

So the previous commit still fails on CI ... what stood out is that the command completed quickly, I think it did not actually build anything. So let's try after a clean.

I think this might be the trigger for "cargo test-bpf" to use the wrong version of the stake pool program? It runs from the wrong directory?

We now pull in the SPL stake pool through a submodule and build it; the artifact should not live in the repository.

ruuda requested a review from naomijub May 20, 2021 13:35

ruuda commented May 20, 2021

View reviewed changes

ruuda mentioned this pull request May 21, 2021

Fee implementation passing all tests #94

Merged

ruuda force-pushed the add-multisig-tests branch 2 times, most recently from 638fef4 to fa81925 Compare May 21, 2021 09:22

Base automatically changed from add-multisig-tests to main May 21, 2021 09:23

ruuda force-pushed the spl-submodule branch 2 times, most recently from 35365ae to 66b5ff2 Compare May 21, 2021 11:44

ruuda mentioned this pull request May 21, 2021

Trusting the SPL stake pool program #95

Closed

ruuda force-pushed the spl-submodule branch from 66b5ff2 to 096eab2 Compare May 21, 2021 17:20

ruuda force-pushed the spl-submodule branch from 61c5a6c to e11c83b Compare May 21, 2021 17:46

enriquefynn approved these changes May 21, 2021

View reviewed changes

enriquefynn reviewed May 21, 2021

View reviewed changes

enriquefynn self-requested a review May 21, 2021 19:29

enriquefynn reviewed May 21, 2021

View reviewed changes

ruuda requested a review from enriquefynn May 25, 2021 07:57

enriquefynn reviewed May 25, 2021

View reviewed changes

ruuda force-pushed the spl-submodule branch from a11d39d to 36d6902 Compare May 25, 2021 12:33

ruuda added 2 commits May 25, 2021 14:55

Fix test for new stake pool rewards

db59a87

After pulling in a new version of the SPL stake pool, which fixes the way it computes rewards, we need to update the test to match, because the number of reward tokens is indeed different.

Run CI for pull requests regardless of branch

5bcad24

Without this, if you open a stacked pull request (against the branch of an existing pull request), it will not run CI for that pull request.

ruuda added 6 commits May 25, 2021 14:55

Try if a clean before build-bpf fixes the build

26e86b6

So the previous commit still fails on CI ... what stood out is that the command completed quickly, I think it did not actually build anything. So let's try after a clean.

Run test-bpf from repository root on CI

213e737

I think this might be the trigger for "cargo test-bpf" to use the wrong version of the stake pool program? It runs from the wrong directory?

Only run our tests on CI, not also the SPL ones

08cb4fb

Delete checked-in spl_stake_pool.so

d943a5b

We now pull in the SPL stake pool through a submodule and build it; the artifact should not live in the repository.

ruuda force-pushed the spl-submodule branch from 4b7ea9f to d943a5b Compare May 25, 2021 12:55

ruuda merged commit 7c0ca1a into main May 25, 2021

ruuda deleted the spl-submodule branch May 25, 2021 13:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Solana Program Library as a submodule #90

Add Solana Program Library as a submodule #90

ruuda commented May 20, 2021 •

edited

Loading

ruuda commented May 20, 2021 •

edited

Loading

ruuda May 20, 2021

naomijub commented May 21, 2021

ruuda commented May 21, 2021

ruuda commented May 21, 2021

ruuda commented May 21, 2021

enriquefynn May 21, 2021

enriquefynn May 21, 2021

ruuda May 25, 2021

enriquefynn left a comment

glottologist commented May 22, 2021

enriquefynn commented May 25, 2021

ruuda commented May 25, 2021

enriquefynn left a comment

ruuda commented May 25, 2021

Add Solana Program Library as a submodule #90

Add Solana Program Library as a submodule #90

Conversation

ruuda commented May 20, 2021 • edited Loading

ruuda commented May 20, 2021 • edited Loading

ruuda May 20, 2021

Choose a reason for hiding this comment

naomijub commented May 21, 2021

ruuda commented May 21, 2021

ruuda commented May 21, 2021

ruuda commented May 21, 2021

enriquefynn May 21, 2021

Choose a reason for hiding this comment

enriquefynn May 21, 2021

Choose a reason for hiding this comment

ruuda May 25, 2021

Choose a reason for hiding this comment

enriquefynn left a comment

Choose a reason for hiding this comment

glottologist commented May 22, 2021

enriquefynn commented May 25, 2021

ruuda commented May 25, 2021

enriquefynn left a comment

Choose a reason for hiding this comment

ruuda commented May 25, 2021

ruuda commented May 20, 2021 •

edited

Loading

ruuda commented May 20, 2021 •

edited

Loading