fstmp files not removed when import failed

[RobQuistNL]

Checklist

• This is not a security-related bug/issue. If it is, please follow please follow the security policy.
• This is not a question or a support request. If you have any lotus related questions, please ask in the lotus forum.
• This is not a new feature request. If it is, please file a feature request instead.
• This is not an enhancement request. If it is, please file a improvement suggestion instead.
• I have searched on the issue tracker and the lotus forum, and there is no existing related issue or discussion.
• I am running the Latest release, or the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
• I did not make any code changes to lotus.

Lotus component

• lotus daemon - chain sync
• lotus miner - mining and block production
• lotus miner/worker - sealing
• lotus miner - proving(WindowPoSt)
• lotus miner/market - storage deal
• lotus miner/market - retrieval deal
• lotus miner/market - data transfer
• lotus client
• lotus JSON-RPC API
• lotus message management (mpool)
• Other

Lotus Version

1.15.2

Describe the Bug

When we import a deal, and the importing fails with;

ERROR: normal shutdown of state machine

after running or 3 minutes - the fstmp file is not removed.

Logging Information

-

Repo Steps

No response

[TippyFlitsUK]

Hi @RobQuistNL

I believe that these files are garbage collected at miner restart. Can you please restart your instance and report back?

Many thanks!

[RobQuistNL]

no 😅

I don't want garbage collection (I can do that myself too) - I want Lotus to properly clean up after itself 😋

[RobQuistNL]

https://github.com/filecoin-project/go-fil-markets/blob/master/filestore/filestore.go I think its coming from here

[RobQuistNL]

Or actually, this one just isn't called properly;
https://github.com/filecoin-project/go-fil-markets/blob/v1.20.1/storagemarket/impl/provider.go#L339

[RobQuistNL]

@LexLuthr

[LexLuthr]

It seems cleanup function is called up manually after each error state. I haven't checked the complete path till addpiece() is called but a better way would be to defer cleanup() in the https://github.com/filecoin-project/go-fil-markets/blob/v1.20.1/storagemarket/impl/provider.go#L339 and use the original file path for the addpiece after verifying that car file is good to import. Not sure why we are importing from the temporary file path here.
Need more analysis to find the correct way to fix this.

[LexLuthr]

@dirkmc This is the issue we were discussing in the meeting. Can you take a look at this?

[LexLuthr]

@RobQuistNL Can you get me the deal state after import failure? I need to understand sectors state and piece's state. So, I can trace what is being traced in the market FSM to try to figure out why delete is not happening.

[TippyFlitsUK]

Hi 👋

The Legacy Lotus Markets sub-system reached EOL at the end of the 31st January 2023.

This ticket is being marked as won't fix and closed as the Lotus Team will no longer be making any further fixes or enhancements to the legacy markets subsystem.

Please feel free to re-open this ticket in the new Boost markets sub-system repository at https://github.com/filecoin-project/boost if you feel that it is still relevant.

Many thanks 🙏