fix: deletion script will not OOM

[PeterChen13579]

fixes #1676 and #1678

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.36%. Comparing base (0282504) to head (4ebd219).
Report is 11 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1679      +/-   ##
===========================================
+ Coverage    69.92%   70.36%   +0.44%     
===========================================
  Files          260      266       +6     
  Lines         9976    10182     +206     
  Branches      5500     5618     +118     
===========================================
+ Hits          6976     7165     +189     
+ Misses        1586     1542      -44     
- Partials      1414     1475      +61     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[cindyyan317]

The logic looks good to me.
But the bookmark can be put into a class.
The workerCount seems incorrec. If I understand correctly, the number of goroutine running concurrently is workerCount*workerCount now.

[cindyyan317]

Maybe we can pack the continue.txt logic all in one class like:

type Marker struct {
	cmd  string
	file *os.File
}

func NewMarker(cmd string) *Marker {
	return &Marker{cmd: cmd}
}

func CloseMarker(m *Marker) {
	if m.file != nil {
		m.file.Close()
	}
	os.Remove("continue.txt")
}

func (m *Marker) EnterTable(table string) error {
	// Create the file
	file, err := os.OpenFile("continue.txt", os.O_WRONLY|os.O_TRUNC, 0644)
	m.file = file
	if err != nil {
		return fmt.Errorf("failed to create file: %w", err)
	}
	fmt.Fprintf(m.file, "%s\n", m.cmd)
	m.file.WriteString(fmt.Sprintf("%s\n", table))
	return nil
}

func (m *Marker) MarkProgress(x int64, y int64) {
	fmt.Fprintf(m.file, "%d, %d \n", x, y)
}

func (m *Marker) ExitTable() {
	m.file.Close()
	m.file = nil
}

[cindyyan317]

Thanks ~ LGTM