cmd/gc: enhance progress display

[jiefenghuang]

What would you like to be added:
gc progress detail when doing ListSlices

Why is this needed:
Currently, in ListSlice, functions like cleanupLeakedInodes take a considerable amount of time to iterate without displaying progress, which may cause confusion for users.

[bobqian1007]

seems ListNode function already have showProgress func as the parameter to trace the progress? do you mean we also need to add the progress bar in the pre-cleanup stage? Like procedure cleanupLeakedInodes,cleanupLeakedChunks,cleanupOldSliceRefs,doCleanupSlices?

[jiefenghuang]

which ListNode? Currently, cleanupLeakedInodes is only added in the Redis engine, and there is no progress display for the current scanning process. I think it’s necessary to show the progress of time-consuming operations during the GC process, otherwise, users won’t know whether it’s working normally.

[bobqian1007]

which ListNode? Currently, cleanupLeakedInodes is only added in the Redis engine, and there is no progress display for the current scanning process. I think it’s necessary to show the progress of time-consuming operations during the GC process, otherwise, users won’t know whether it’s working normally.

I got it. Btw I am new to this project and would like to know what I should do if I want to take this task. Do I need to write a design document first?

[jiefenghuang]

For this issue, a simple description is sufficient, but we need to consider the implementation of multiple engines.

Going further, the GC currently scans two main categories: metadata and data. Within these, there are various scenarios, such as residues after compaction, records of asynchronously deleted files, residues caused by cloning, dangling inodes or edges, etc. It would be helpful to categorize the scanned data and ultimately produce a clearer statistical list. @SandyXSD @davies is this improvement needed?

[bobqian1007]

For this issue, a simple description is sufficient, but we need to consider the implementation of multiple engines.

Going further, the GC currently scans two main categories: metadata and data. Within these, there are various scenarios, such as residues after compaction, records of asynchronously deleted files, residues caused by cloning, dangling inodes or edges, etc. It would be helpful to categorize the scanned data and ultimately produce a clearer statistical list. @SandyXSD @davies is this improvement needed?

ok， I will deep dive into the code to sort out your mentioned scenarios. is there any ETA for this feature?

[jiefenghuang]

This can be included in version 1.4 next year

[bobqian1007]

This can be included in version 1.4 next year

Ok， let me finish the doc first

[issimo1]

Hello, I would like to attempt to resolve this issue. May I take it on?

[bobqian1007]

Hello, I would like to attempt to resolve this issue. May I take it on?

Hi, sir, I actually had previously told the maintainer that I'd take it on and write up the docs properly first — but things have been super busy at work lately, so I haven’t made as much progress as I wanted.

I’m still definitely interested in working on it, but totally understand if the maintainer thinks it’s better to reassign given the timeline. Happy to go with whatever works best for the project!

[hegdeadithyak]

So we need to add a progress bar in a new file, that serves for every engine including redis in longer run isn't it? or are we just focused on redis for now ? @jiefenghuang

[jiefenghuang]

Currently there are not many users reporting this issue, so a simple one- or two-line debug log (“We are currently processing some deletion operations.”) should suffice without causing misunderstanding. If you’re interested in contributing to the community (e.g. taking on other issues), you’re welcome to submit a PR.

“Going further, the GC currently scans two main categories: metadata and data. Within these, there are various scenarios, such as residues after compaction, records of asynchronously deleted files, residues caused by cloning, dangling inodes or edges, etc. It would be helpful to categorize the scanned data and ultimately produce a clearer statistical list. ”
For this part , our colleagues have already written an article summarizing the relevant explanations. This issue can be closed for now.