Skip to content

Optimize CheckpointSpec test runtime #4501

@Yicong-Huang

Description

@Yicong-Huang

Task Summary

CheckpointSpec (amber/src/test/scala/org/apache/texera/amber/engine/faulttolerance/CheckpointSpec.scala) takes ~70s in CI to run only two active assertions, making it the single slowest spec in the Scala test step (about 26% of test execution time on a recent run).

The two active tests just verify that a default ControllerProcessor and DataProcessor are serializable; the rest of the file is commented out. Despite that, the spec spins up two Pekko ActorSystems:

  • One in beforeAll (CheckpointSpec.scala:62), named "CheckpointSpec" — never referenced by the active tests
  • One created lazily by AmberRuntime.serde (AmberRuntime.scala:39-48), named "Amber" — used by chkpt.save(...) for SerializationExtension

There is no roundtrip across the two systems (no chkpt.load(...) is called), so the second ActorSystem boot looks redundant. Eliminating the duplicate boot, plus removing unused class-level setup (buildWorkflow(...) at line 48, which is only referenced by commented-out tests), should significantly cut spec runtime.

Priority

P2 – Medium

Task Type

  • Refactor / Cleanup
  • Testing / QA

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions