Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prune owned coin idx when inputs are spent #1055

Merged
merged 19 commits into from
Mar 17, 2023

Conversation

Voxelot
Copy link
Member

@Voxelot Voxelot commented Mar 8, 2023

fixes: #1053

Remove owned coin index entries when coins are spent to improve the performance of random-improve based coin selection.

Already released v0.17.4 hotfix from this branch, ready to merge into mainline now.

@Voxelot Voxelot requested a review from a team March 8, 2023 08:06
@Voxelot Voxelot marked this pull request as draft March 8, 2023 08:06
Copy link
Collaborator

@xgreenx xgreenx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also update all internal functions to indicate that we return Unspent coins?
Like unspent_owned_coins_ids instead of owned_coins_ids.

Also, in the code, we will fetch coins and try to filter them based on the status, but now we can do it without filtering.

In the main PR for the next release(0.18.0), we also need to update the endpoint to unspent_coins instead of coins.

P.S. We also need to inform SDK and the application team because it could affect them.

crates/fuel-core/src/executor.rs Outdated Show resolved Hide resolved
Copy link
Collaborator

@xgreenx xgreenx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove the status field from CompressedCoin now=)

crates/fuel-core/src/executor.rs Outdated Show resolved Hide resolved
crates/storage/src/tables.rs Outdated Show resolved Hide resolved
xgreenx
xgreenx previously approved these changes Mar 8, 2023
Copy link
Collaborator

@xgreenx xgreenx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing this field will break the database for beta 3=) Let's remove it in the main PR

xgreenx
xgreenx previously approved these changes Mar 9, 2023
# Conflicts:
#	crates/fuel-core/src/executor.rs
#	crates/types/src/services/executor.rs
@Voxelot Voxelot marked this pull request as ready for review March 11, 2023 03:03
crates/fuel-core/src/database/coin.rs Outdated Show resolved Hide resolved
*asset_id,
(*maturity).into(),
)?;
coin.status = CoinStatus::Spent;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also remove the status fields from CompressedCoin, please?=)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And in some places, we still try to fetch MessageStatus, but we know that all messages from our database are unspent. Could you also update those places, please?

@xgreenx xgreenx requested a review from a team March 13, 2023 16:09
@Voxelot
Copy link
Member Author

Voxelot commented Mar 16, 2023

Removed spent status from messages and coins. I'm hesitant to rename all the apis to unspent_, since in most utxo based chains it's expected that any utxos returned from the state set are unspent.

@xgreenx
Copy link
Collaborator

xgreenx commented Mar 16, 2023

After removing of Spent status setting, we don't need to rename with unspent_ prefix=)

@@ -111,10 +111,6 @@ type Coin {
assetId: AssetId!
maturity: U64!
"""
The spendable status of the coin
"""
status: CoinStatus!
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From an API perspective, we still need to return the status of the coins and messages. Because we plan to move this API to the indexer and they can fetch this information. It is why I said to remove it only from "Compressed" types=)

Copy link
Member Author

@Voxelot Voxelot Mar 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm maybe, although no one was using this part of the API and so it's possible the indexer may not need to track spent coins either. I'd rather rip the bandaid fully off now, and add the status back later if there's an actual requirement for it.

Also the indexer uses a completely different system for managing graphql schemas, so I don't think it'll be a simple migration of existing logic in fuel-core.

@Voxelot Voxelot requested a review from xgreenx March 16, 2023 23:33
@Voxelot Voxelot merged commit 13aa644 into master Mar 17, 2023
@Voxelot Voxelot deleted the Voxelot/beta-3-coin-select-perf branch March 17, 2023 00:53
@xgreenx xgreenx mentioned this pull request Apr 27, 2023
xgreenx added a commit that referenced this pull request Apr 27, 2023
The change bumps the version to `0.18.0` and exposes
`sync_max_get_header` and `sync_max_get_txns` in the helm chart.

# Release 0.18.0

## Overview

A new release brings:
- Optimization for the execution based on the performance from beta 3
and on internal benchmarks. Improved metrics gathering.
- Stabilization and better test coverage of the `fuel-vm`. We removed
almost all unsafe code and added test cases for each opcode. Fixed some
edge cases with memory in the `fuel-vm`.
- Fully integrated Merkle trees and filled all malleable fields in the
transactions.
- Added retryable messages, removed redundant fields from it, and
updated the API to support a new commitment schema.

## What's Changed

### Breaking

- All unsafe functions were replaced with safe analog in the
`fuel-crypto` - FuelLabs/fuel-vm#346
- `$hp` holds the address of the last available byte in a heap, while
previously it was `$hp - 1` -
FuelLabs/fuel-vm#377
- Each variant in the `fuel_tx::Input` enum now has its own type -
FuelLabs/fuel-vm#364
- Message nonce is unified and now `Bytes32` everywhere -
FuelLabs/fuel-vm#394
- Removed the `message_id` field from all places -
FuelLabs/fuel-vm#397,
FuelLabs/fuel-vm#373,
- Unified the block height in all places with the introduction of a new
`BlockHeigh` - FuelLabs/fuel-vm#410
- Make SMO instruction take data ptr as an argument -
FuelLabs/fuel-vm#404
- Now the chain id affects the signature and predicate's owner and
should be passed into the `sign` function -
FuelLabs/fuel-vm#406
- Updated the `produce_blocks` endpoint to accept the start time and the
number of blocks. All new blocks will use the previous timestamp as a
base - #1059
- The `fuel-core` stores only unspent coins and messages, so all API
that previously returned spent coins is affected - Prune owned coin idx
when inputs are spent by @Voxelot in
#1055
- The message proof API has been changed to be compatible with a new
version - #1071
- The `fuel-core` now has retryable messages and coin messages.
Retryable messages can only be consumed during successful transaction
execution. The coin message acts as common coins. `resouces_to_spend`
API was replaced with `coins_to_spend` that returns a new `CoinType`
type. - #1067

## All changes
* adding fuel-core service monitor to helm chart by @rfuelsh in
#1037
* Adding specific app selector by @rfuelsh in
#1039
* use sticky sessions for GQL requests to sentries by @Voxelot in
#1042
* Add ingress service name to helm chart by @rfuelsh in
#1043
* Added test to verify the iteration over owned transactions by @xgreenx
in #1041
* Change sentry lb to use clusterIP by @Voxelot in
#1045
* Fixed Tempfile dependency by @ControlCplusControlV in
#1048
* Write contract code in raw by @freesig in
#994
* attempt to use buildjet runners by @Voxelot in
#1017
* Set tx pointer during block production by @Voxelot in
#1054
* Used `BASE_AMOUNT` for test with bob to pay for fee by @xgreenx in
#1057
* Prepare GraphQL Crate for extraction by @ControlCplusControlV in
#1012
* Support large contract deployments over p2p by @Voxelot in
#1062
* fix yaml and add linting job by @Voxelot in
#1063
* Actualized the ABI for message to be compatible with last version of
the Solidity contracts. by @xgreenx in
#1060
* feat: Contract state and assets merkle data storage by @bvrooman in
#1008
* Take into account the previous block timestamp during block production
by @xgreenx in #1059
* Prune owned coin idx when inputs are spent by @Voxelot in
#1055
* Retrayable messages fuel-core part by @xgreenx in
#1067
* chore: Additional Tests for Contract States and Balances by @bvrooman
in #1073
* Rust 1.68.1 & Sparse Registry by @Voxelot in
#1077
* Nginx tuneup by @Voxelot in
#1080
* chore: Update balance and state Merkleization by @bvrooman in
#1084
* Support sticky session in the client by @xgreenx in
#1079
* Added e2e test to run 1000 `dry_run` by @xgreenx in
#1091
* Use docker.io auth credentials to overcome rate limiting by @Voxelot
in #1092
* honeycomb tracing subscription by @Voxelot in
#1083
* Update withdrawal proof API to support proving from a block header
committed to L1 by @xgreenx in
#1071
* Upstream v0.17.8 by @Voxelot in
#1102
* use chainid for transactions and predicates by @Voxelot in
#1107
* fix the dry-run e2e test to actually perform dry runs by @Voxelot in
#1111
* peer reputation with separate app & gossipsub score by @leviathanbeak
in #1028
* Adding sentry data synchronization by @rfuelsh in
#1115
* Capture metrics for graphql api by @Voxelot in
#1113
* Update cargo audit and add to commit by @freesig in
#1152
* add task params by @leviathanbeak in
#1159
* Took into account that block creation could take some time by @xgreenx
in #1162
* Applying optimizations from `0.17.11` release by @xgreenx in
#1158


**Full Changelog**:
v0.17.3...v0.18.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prune spent coins from OwnedCoins column
2 participants