Skip to content

Drozdziak1/permissioned price accounts #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
May 8, 2023

Conversation

drozdziak1
Copy link
Contributor

@drozdziak1 drozdziak1 commented May 4, 2023

This change prevents the agent from publishing prices that are not permissioned to its publishing keypair, potentially limiting the burn rate of the publishers' production deployments. The invalid attempts are logged to WARN.

How this works

*solana.rs - we open a channel between oracle and exporter, periodically updating the latest known set of permissioned price accounts.

  • oracle.rs - In addition to on-chain state lookup, we filter out permissioned prices and send them over the channel to the exporter. This is done by checking the PriceComp.publisher field against the publish keypair from key store.
  • exporter.rs - Just before building price update Solana transactions, we check for permissioned price set updates on the channel. If no updates are available, pre-existing value from exporter state is used. We filter out unpermissioned symbols using this set and log them to WARN. We continue with publishing the remaining valid symbols.

Testing

  • I verified manually that the debug-level log output corresponds with the desired unpermissioned price warning in a temporary integration test.
  • A new integration test is added which verifies that unpermissioned symbols logging works and does not produce transactions

Review highlights

  • Hangups and channel overflows - by creating a new async channel, we risk bugs related to empty/overflowing channels. I did my best to fall back on empty channel and never let it overflow, but you can never be too sure.

Remaining items

  • Proper Integration Testing - I'm still not 100% sure how to best add this to the integration tests. Because of our choice to use an infallible tx, our only choice seems to be parsing agent or solana tx logs. I don't like this, but if we don't come up with something else, that's probably what I'll do

@drozdziak1 drozdziak1 requested review from jayantk and ali-behjati May 4, 2023 12:31
@drozdziak1 drozdziak1 changed the title Drozdziak1/permissioned price accounts [WIP] Drozdziak1/permissioned price accounts May 4, 2023
@drozdziak1 drozdziak1 changed the title [WIP] Drozdziak1/permissioned price accounts Drozdziak1/permissioned price accounts May 4, 2023
Copy link
Collaborator

@ali-behjati ali-behjati left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! My only big comment is only changing the log level. If you agree with it, please address that before merging ser.

Copy link
Contributor

@jayantk jayantk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this could be simplified a bit. See inline comments

// Update permissioned accounts of this publisher from the
// oracle. The loop ensures that we clear the channel and use
// only the final, latest message; try_recv() is non-blocking.
loop {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to put this in run above? I think you can probably set up the publish_interval timer such that this processing happens between the publish_updates calls and doesn't interfere with them.

(also would be better from a code clarity perspective to put this in a separate function like flush_perm_price_channel() and call it from run())

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh also you may want to look into watch::Receiver thing for NetworkState above, which seems to do exactly the thing you're doing here.

Copy link
Contributor Author

@drozdziak1 drozdziak1 May 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exiled this block to a helper method. I got into some nasty Rust errors when trying to use the watch channel. I explain this in detail in the new update_our_prices method doc, but it boils down to this:

Rustc claims the channel's internal RwLockReadGuard is not Send. This does not sound right, because all generics in tokio::sync::RwLockReadGuard<HashMap<Pubkey, HashSet<Pubkey>>> should meet that requirement. All the enclosing types are Send if the contents type is. Pubkey is Send and happens to be the contents.

I wouldn't put this call inside run, because we would either need to duplicate the keypair request code again, or do it in run. I think the intention with run is to just say "we're doing this one thing periodically".

false
}
})
.collect::<Vec<_>>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't this going to generate a ton of warnings in normal usage? If every publisher has different permissions on mainnet and pythnet, then I think fresh_updates here will contain the union of both. Which means that every time this code runs for solana mainnet, they will get a lot of warnings for the unpermissioned symbols.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

switched to info

perm_price_rx: mpsc::Receiver<HashSet<Pubkey>>,

/// Symbols we are permissioned to publish
perm_price_accounts: HashSet<Pubkey>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe these permissions should go in NetworkState? I think that already contains network-specific information, which this is also.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eh... maybe not. i see why you added this channel, because you want to get the information from the Oracle, which is already getting the other price account information. Maybe this is fine.

Copy link
Contributor

@jayantk jayantk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice

@drozdziak1 drozdziak1 merged commit 85d6b15 into main May 8, 2023
@drozdziak1 drozdziak1 deleted the drozdziak1/permissioned-price-accounts branch May 8, 2023 09:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants