Skip to content

test(shared-tree): Add execution time performance benchmarks for SharedTree table APIs #24881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

chentong7
Copy link
Contributor

@chentong7 chentong7 commented Jun 20, 2025

Description

This PR introduces performance benchmark tests for SharedTree's table-related APIs, focusing on standard operations. The benchmarks cover:

Insertion of rows, columns, and cells

Includes insertion at the middle of the table for rows and columns. Similarly, tests removal for rows and columns.

Test Result

/home/tong/workspace/FluidFramework/node_modules/.bin/mocha ./tableTree.spec.js --no-timeouts --exit
Writing test results relative to package to nyc/junit-report.xml
mocharc-common.cjs:73
Warning: Cannot find any files matching pattern "lib/test"
collect-files.js:102

spec.js:54
  SharedTree execution time
spec.js:54
    Column Insertion
spec.js:54
      ✔ @Benchmark @Measurement @ExecutionTime Insert a column in the middle 5 times (192ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Undo insert the middle column 5 times (152ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Redo insert the middle column 5 times (131ms)
spec.js:83
    Row Insertion
spec.js:54
      ✔ @Benchmark @Measurement @ExecutionTime Insert a row in the middle 5 times (128ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Undo insert the middle row 5 times (151ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Redo insert the middle row 5 times (136ms)
spec.js:83
    Column and Row Insertion
spec.js:54
      ✔ @Benchmark @Measurement @ExecutionTime Insert a column and a row in the middle 5 times (132ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Undo insert the middle column and row 5 times (152ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Redo insert the middle column and row 5 times (147ms)
spec.js:83
    Column Removal
spec.js:54
      ✔ @Benchmark @Measurement @ExecutionTime Remove a column in the middle 5 times (145ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Undo remove the middle column 5 times (155ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Redo remove the middle column 5 times (150ms)
spec.js:83
    Row Removal
spec.js:54
      ✔ @Benchmark @Measurement @ExecutionTime Remove a row in the middle 5 times (134ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Undo remove the middle row 5 times (125ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Redo remove the middle row 5 times (128ms)
spec.js:83
    Column and Row Removal
spec.js:54
      ✔ @Benchmark @Measurement @ExecutionTime Remove a column and a row in the middle 5 times (135ms)
spec.js:83
    Insert a column and a row and remove right away
spec.js:54
      ✔ @Benchmark @Measurement @ExecutionTime Insert a column and a row and remove them right away 5 times (140ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Undo insert a column and a row and remove them right away 5 times (160ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Redo insert a column and a row and remove them right away 5 times (182ms)
spec.js:83
    Cell Value Setting
spec.js:54
      ✔ @Benchmark @Measurement @ExecutionTime Set a cell value 5 times (138ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Undo set a cell value 5 times (120ms)
spec.js:83
      ✔ @Benchmark @Measurement @ExecutionTime Redo set a cell value 5 times (135ms)
spec.js:83
  22 passing (3s)

@Copilot Copilot AI review requested due to automatic review settings June 20, 2025 01:47
@chentong7 chentong7 requested a review from a team as a code owner June 20, 2025 01:47
@github-actions github-actions bot added base: main PRs targeted against main branch area: dds Issues related to distributed data structures area: dds: tree labels Jun 20, 2025
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces performance benchmark tests for SharedTree's table-related APIs that measure execution times for various table operations such as insertion and removal.

  • Added benchmark tests for column, row, and combined insertion and removal operations.
  • Updated Mocha configuration to filter performance-relevant tests and included helper functions for table tree initialization.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
packages/dds/tree/src/test/time/tableTree.spec.ts Adds performance benchmark tests for table operations.
packages/dds/tree/src/test/time/.mocharc.cjs Configures Mocha for execution time tests.
packages/dds/tree/src/test/testHelper.ts Provides utility functions and type definitions for table trees.

@chentong7 chentong7 requested a review from a team June 20, 2025 20:46
@github-actions github-actions bot added the public api change Changes to a public API label Jun 20, 2025
@github-actions github-actions bot removed the public api change Changes to a public API label Jun 20, 2025
Copy link
Contributor

🔗 No broken links found! ✅

Your attention to detail is admirable.

linkcheck output


> fluid-framework-docs-site@0.0.0 ci:check-links /home/runner/work/FluidFramework/FluidFramework/docs
> start-server-and-test "npm run serve -- --no-open" 3000 check-links

1: starting server using command "npm run serve -- --no-open"
and when url "[ 'http://127.0.0.1:3000' ]" is responding with HTTP status code 200
running tests using command "npm run check-links"


> fluid-framework-docs-site@0.0.0 serve
> docusaurus serve --no-open

[SUCCESS] Serving "build" directory at: http://localhost:3000/

> fluid-framework-docs-site@0.0.0 check-links
> linkcheck http://localhost:3000 --skip-file skipped-urls.txt

Crawling...

Stats:
  225447 links
    1710 destination URLs
    1941 URLs ignored
       0 warnings
       0 errors


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tree already runs execution time tests using its root .mocharc.cjs file and some flags specified directly in the npm script. I like the idea of moving it to use a specific config file like this but then we should move to it for all execution time tests, not just the new ones, so we'll want to start by copying whatever the current root config file does, add the necessary settings for these tests, and update existing npm scripts that run them to point to the new file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. I haven't decided to update package.json or update the root .mocharc.js to includes local mocharc file. Will get back to this one later.

}
}

const unsubscribeFromCommitAppliedEvent = treeView.events.on(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Subscribing before filling the tree will result in us building up a large revertible stack from the column/row/cell insertions that were part of initialization. I don't think that's what we want. The undo/redo tests have specific operations they want inserted into the stack(s).

We also probably don't want to incur the overhead of these subscriptions during the measurements. I think my recommendation would be:

  1. Don't track undo/redo stacks outside of undo/redo tests.
  2. Only start populating the undo/redo stacks immediately before we do the pre-test operations that we plan to undo/redo
  3. Unsubscribe from event after we've built up the stack(s) we need for the test / before we begin measurement.

Copy link
Contributor

@Josmithr Josmithr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a couple more comments, but overall looking very good! Thanks!!

Copy link
Contributor

@jzaffiro jzaffiro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

matrix changes look fine!

(commit, getRevertible) => {
if (getRevertible !== undefined) {
const revertible = getRevertible(onDispose);
if (commit.kind === CommitKind.Undo) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic isn't quite right. One issue of note: if a non-undo commit is made, then we should clear the redo stack (otherwise we keep stale redo operations). See this code in our brainstorm example for a more standard way to set this up: https://github.com/microsoft/FluidExamples/blob/main/brainstorm/src/utils/undo.ts#L22

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: dds: tree area: dds Issues related to distributed data structures base: main PRs targeted against main branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants