Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The indexer does not see some transactions #1414

Closed
chlenc opened this issue Oct 14, 2023 · 10 comments
Closed

The indexer does not see some transactions #1414

chlenc opened this issue Oct 14, 2023 · 10 comments
Assignees

Comments

@chlenc
Copy link

chlenc commented Oct 14, 2023

The indexer does not see some transactions

Hi guys, I just faced some mystic in the spark spot market, I found some orders in order books that don't want to be matched with another, the id of that order is 69
Снимок экрана 2023-10-14 в 18 13 11

After that, I made sure this order was active on my indexer service, which means the frontend works well
image

After that, I tried to ask for a contract about that order by id, and in the state of contract this order is completed
request

println!("{:#?}", methods.order_by_id(69).simulate().await.unwrap().value);

response

Order {
    asset_0: Bits256([ 174, 55, 188, 15, 235, 102, 230, 10, 137, 227, 1, 212, 80, 187, 70, 64, 170, 155, 215, 206, 221, 133, 110, 37, 62, 35, 152, 158, 174, 83, 110, 146]),
    amount_0: 300000000000,
    asset_1:Bits256([ 139, 247, 149, 30, 163, 34, 47, 224, 186, 233, 184, 17, 194, 177, 66, 161, 255, 65, 115, 97, 220, 247, 69, 120, 85, 237, 71, 125, 45, 154, 133, 80]),
    amount_1: 1000000000,
    status: Completed,
    fulfilled_0: 300000000000,
    fulfilled_1: 1000000000,
    owner: 0x194c4d5d321ea3bc2e87109f4a86520ad60f924998f67007d487d3cc0acc45d2,
    id: 69,
    timestamp: 4611686020124620025,
    matcher_fee: 1000,
    matcher_fee_used: 1000,
}

I checked my indexer's logs and found only the creation on block 4850467 and the first fulfillment logs on block 4851917

After I checked mather logs I noticed the first order match was crushed with the next error
(For some reason, I have all transactions crash with errors there, but the orders themselves are closed 🤷🏻‍♂️)

❌ c3185ce989f81cff132392d97158f557186888b46f92b1896226c926e4f351ac + 2a788a2c0a72683fd7fd06802a325a04c559b1c5cc80344c68b250253c57e76f: Error: Transaction is not inserted. UTXO is not existing: 0xbebaf54d4c15b15a2067b9afcaa4e849c2537af6ac2512704ff2f3878b7b5d3701: {"response":{"data":null,"errors":[{"message":"Transaction is not inserted. UTXO is not existing: 0xbebaf54d4c15b15a2067b9afcaa4e849c2537af6ac2512704ff2f3878b7b5d3701","locations":[{"line":2,"column":3}],"path":["submit"]}],"status":200,"headers":{}},"request":{"query":"mutation submit($encodedTransaction: HexString!) {\n  submit(tx: $encodedTransaction) {\n    id\n  }\n}","variables":{"encodedTransaction":"0x0000000000000000000000000000000200000000009896800000000000000000000000000000001800000000000000680000000000000002000000000000000c00000000000000010000000000000000000000000000000000000000000000000000000000000000724028a8724428805d451000724828882d41148a2404000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2058c9c621c42bde654dd31560ab3bf68a5b9f517894ee7db0b6f82ea5d3e3a0000000074cffba200000000000028d8000000000000004500000000000000440000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2058c9c621c42bde654dd31560ab3bf68a5b9f517894ee7db0b6f82ea5d3e3a0000000000000000bebaf54d4c15b15a2067b9afcaa4e849c2537af6ac2512704ff2f3878b7b5d3700000000000000015cd7a90ddc6e4c9ba59c08f9fcd5ec59f1ab9998088a28176f395803c7bd4534000000001c8bca8000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000025cd7a90ddc6e4c9ba59c08f9fcd5ec59f1ab9998088a28176f395803c7bd453400000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000407cae272bae756c5a1d25ea5c990cccaa175340085ecd33f5d6787fb0f5c109f29f0ca434e63f002d6d8cf52f892cf2cd08ce527f3276b97b6bf5f7d2963608a7"}}}

and after I have found a few math tries with this error:

❌ c3185ce989f81cff132392d97158f557186888b46f92b1896226c926e4f351ac + 6a339ff6defc6e73cf17dea3ea81f41e8bec092b4ced841a6c10732c0402a727: Error: Transaction is not inserted. UTXO is not existing: 0xe1190ac8b9dead8ec551e5478ec88a7f07b58606e609e9f30634538694da7c0d01: {"response":{"data":null,"errors":[{"message":"Transaction is not inserted. UTXO is not existing: 0xe1190ac8b9dead8ec551e5478ec88a7f07b58606e609e9f30634538694da7c0d01","locations":[{"line":2,"column":3}],"path":["submit"]}],"status":200,"headers":{}},"request":{"query":"mutation submit($encodedTransaction: HexString!) {\n  submit(tx: $encodedTransaction) {\n    id\n  }\n}","variables":{"encodedTransaction":"0x0000000000000000000000000000000200000000009896800000000000000000000000000000001800000000000000680000000000000002000000000000000200000000000000010000000000000000000000000000000000000000000000000000000000000000724028a8724428805d451000724828882d41148a2404000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2058c9c621c42bde654dd31560ab3bf68a5b9f517894ee7db0b6f82ea5d3e3a0000000074cffba200000000000028d8000000000000004500000000000000280000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2058c9c621c42bde654dd31560ab3bf68a5b9f517894ee7db0b6f82ea5d3e3a0000000000000000e1190ac8b9dead8ec551e5478ec88a7f07b58606e609e9f30634538694da7c0d00000000000000015cd7a90ddc6e4c9ba59c08f9fcd5ec59f1ab9998088a28176f395803c7bd4534000000001c88878e00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000025cd7a90ddc6e4c9ba59c08f9fcd5ec59f1ab9998088a28176f395803c7bd45340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000409d9ec149c7e9fbd42f95760d3c0560280ae6bc5dd94924c7c8f15bb6b27dbd9fb585710fd801df46251872396834d0b6763fb9bcd06a4009d435d22a94e6c630"}}}

after that all mather's match attempts fails with contract's error OrderIsNotActive

Can you help me to figure out what it can be?

toolchain

Default host: aarch64-apple-darwin
fuelup home: /Users/alexey/.fuelup

installed toolchains
--------------------
beta-3-aarch64-apple-darwin
beta-4-rc.2-aarch64-apple-darwin
latest-aarch64-apple-darwin (default)
beta-4-aarch64-apple-darwin
hotfix
my-custom-toolchain

active toolchain
-----------------
latest-aarch64-apple-darwin (default)
  forc : 0.46.0
    - forc-client
      - forc-deploy : 0.46.0
      - forc-run : 0.46.0
    - forc-doc : 0.46.0
    - forc-explore : 0.28.1
    - forc-fmt : 0.46.0
    - forc-index : 0.21.0
    - forc-lsp : 0.46.0
    - forc-tx : 0.46.0
    - forc-wallet : 0.3.0
  fuel-core : 0.20.5
  fuel-core-keygen : Error getting version string
  fuel-indexer : 0.21.0

fuels versions
---------------
forc : 0.45
forc-wallet : 0.45

Sources

Spot markets contract: https://github.com/compolabs/spark/blob/beta-4/contracts/spot-market/src/main.sw
Indexer: https://github.com/compolabs/spark/blob/beta-4/services/spark-indexer/src/lib.rs
Mathcer ts-sdk version: 0.62

@chlenc
Copy link
Author

chlenc commented Oct 14, 2023

Could it be because I don't use constructions like that?
https://github.com/compolabs/spark/blob/ra0x3s_improvements/services/spark-indexer/src/lib.rs#L48

        let order0 = match OrderEntity::load(uid(event.order_0_id.to_be_bytes())) {
            Some(order) => order,
            None => {
                error!("Order not found: {}", uid(event.order_0_id.to_be_bytes()));
                panic!("");
            }
        };
        let order1 = match OrderEntity::load(uid(event.order_1_id.to_be_bytes())) {
            Some(order) => order,
            None => {
                error!("Order not found: {}", uid(event.order_1_id.to_be_bytes()));
                panic!("");
            }
        };

@ra0x3
Copy link
Contributor

ra0x3 commented Oct 16, 2023

@chlenc

  • Hm, there's a lot of info here
  • I'm not entirely sure what you're asking, but let's clear up a few things

You can create orders like this:

let order = OrderEntity { foo: "bar", id: 1 };
order.save();
info!("ID is {}", order.id);  // "ID is 1"
  • Here ☝🏼 I created a new OrderEntity and I gave it an id manually. Then I saved it with .save(). So now OrderEntity with id = 1 will be in my database.

You can also create orders like this:

let order = OrderEntity::new("bar");
order.save();
info!("ID is {}", order.id);  // "ID is 256745215632"
  • Here 👆🏼 I created a new OrderEntity, but instead of having the id be manually assigned by me, an id field was automatically created using the contents of the OrderEntity
    • We do this so that users can deterministically take a piece of data and derive it's id

Finally, you can also create orders like this:

let order = OrderEntity::new("bar").get_or_create();
order.save();
info!("ID is {}", order.id);  // "ID is 256745215632"
  • Here ☝🏼 we are creating a new order, same as we did with the last example, but we're using .get_or_create() to either load the entity if the ID already exists OR _create the entity if that ID does not exist.

In summary:

  • There's a slight chance there's something wrong here (I doubt it though)
  • I would be careful when you're creating your entities
    • Look over your indexer to know when you're creating new entities, and when you should be loading pre-existing entities
    • Use ::new() when you want to automatically derive a unique ID for an entity
      • Use .get_or_create() when you want to automatically save or load the entity (depending on whether or not the ID exists in the database)
  • If you want a more relevant answer to your question, you'll have to ask your question a bit more clearly :)

@ra0x3 ra0x3 self-assigned this Oct 16, 2023
@ra0x3
Copy link
Contributor

ra0x3 commented Oct 16, 2023

@chlenc

  • I saw you deleted your comment :) I'll wait to follow up on your question (from the deleted comment)
  • Also feel free to point me to your exact lib.rs file on whatever branch you're using and I can take a look myself
    • You've linked to a few files above ☝🏼 so not sure which one I should be looking at

@chlenc
Copy link
Author

chlenc commented Oct 16, 2023

@chlenc

  • I saw you deleted your comment :) I'll wait to follow up on your question (from the deleted comment)

  • Also feel free to point me to your exact lib.rs file on whatever branch you're using and I can take a look myself

    • You've linked to a few files above ☝🏼 so not sure which one I should be looking at

Hi, I'm a little sick today, I'll double-check everything again and reply tomorrow

@chlenc
Copy link
Author

chlenc commented Oct 17, 2023

@ra0x3 , hi, here is my current lib.rs file, could something not be saved due to an error in the code?
https://github.com/compolabs/spark/blob/beta-4/services/spark-indexer/src/lib.rs

@chlenc
Copy link
Author

chlenc commented Oct 17, 2023

But I think if that match log with order 69 fulfillment wasn't printed that means it can be something wrong with the network, not with the indexer 🤔

        info!("Spark: 💟 Match event \n{:#?}", event);

I'll try to redeploy the spark and check if phantom orders will appear again

@chlenc
Copy link
Author

chlenc commented Oct 18, 2023

So, yesterday I redeployed the contract and after 1day of work we have this list of phantom orders:

Buy orders: 74 | Sell orders: 9 | Total orders: 145

🛸 Phantom order #8:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #1:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #13:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #14:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #15:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #44:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #38:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #47:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #49:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #52:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #56:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #60:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #62:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #69:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #75:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #82:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #88:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #91:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #92:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #98:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #101:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #97:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #105:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #106:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #107:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #108:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #110:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #111:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #113:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #114:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #115:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #121:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #126:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #131:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #132:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #23:
      Status from indexer: Active
      Status from state:   Completed

🛸 Phantom order #137:
      Status from indexer: Active
      Status from state:   Completed

@chlenc
Copy link
Author

chlenc commented Oct 18, 2023

I will redeploy the contract again, do you have any recommendations on how I can investigate this problem?

@ra0x3
Copy link
Contributor

ra0x3 commented Nov 27, 2023

@chlenc Any updates here? :)

@chlenc
Copy link
Author

chlenc commented Dec 19, 2023

we are now actively working on the development of the new market, and this problem is about the spot market, I think we can close this issue, if the problem repeats on new versions, I will start a new issue.

@chlenc chlenc closed this as completed Dec 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants