-
Notifications
You must be signed in to change notification settings - Fork 549
Improve Assert APIs #24013
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Improve Assert APIs #24013
Conversation
## Description Automate previously manual checks for debugAssert removal as a mocha test. Split out from #24013
…to assertBetter
cursor.exitField(); // exit "color" | ||
hasNextEgg = cursor.nextNode(); // move to next egg | ||
} | ||
cursor.exitNode(); // exit the current egg |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test was incorrect, and should have omitted this line and added another exitField below. It used to pass due to the asserts not being enabled and the two bugs happening to cancel out.
It has been fixes, but also refactored to be less error prone/ more idiomatic with how it uses the cursor.
// Cursors have quite a few debugAsserts, so validate them with and without debug asserts enabled. | ||
// It is very unlikely for any of their debug asserts to cause issues in production mode: | ||
// this is mostly just a demonstration of how to run the tests in both modes, but also provides some extra validation. | ||
for (const emulateProduction of [true, false]) { | ||
describe(`emulateProductionBuild: ${emulateProduction}`, () => { | ||
if (emulateProduction) { | ||
before(() => { | ||
data = dataFactory(); | ||
emulateProductionBuild(); | ||
}); | ||
it("jsonableTreeFromCursor", () => { | ||
const cursor = cursorFactory(data); | ||
const jsonableClone = jsonableTreeFromCursor(cursor); | ||
// Check jsonable objects are actually json compatible | ||
const text = JSON.stringify(jsonableClone); | ||
const parsed = JSON.parse(text); | ||
assert.deepEqual(parsed, jsonableClone); | ||
after(() => { | ||
emulateProductionBuild(false); | ||
}); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The changes to this file are to just run the cursor test suite with and without debugAsserts, as documented by the comment above. The changes below are just an increase in indent.
// The debug asserts in cursors have some performance overhead (which is part of why they are debug asserts). | ||
// Running these benchmarks in both modes not only allows confirming that the mode selection works, | ||
// but also helps measure the performance impact of the debug asserts. | ||
// Note that real production mode would be even lower overhead as the debug asserts would be removed by the bundler instead of just early existing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Performance test data from this suite does show a significant win from the asserts being disabled. Sadly our performance tests no longer report suite geometric means in the summary table, so the data is a pain to interpret:
craig@large:~/Work/FluidFramework.git/4/packages/dds/tree$ pnpm run bench
@fluidframework/tree@2.42.0 bench /home/craig/Work/FluidFramework.git/4/packages/dds/tree
mocha --timeout 999999 --perfMode --parentProcess --fgrep @benchmark --fgrep @ExecutionTime --reporter @fluid-tools/benchmark/dist/MochaReporter.jsWriting test results relative to package to nyc/junit-report.xml
ITreeCursor emulateProductionBuild: true canada JsonCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_canada_JsonCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(JsonCursor) 5.04 46 4 26,793,579.35 ±590,579.46 ±2.20% ✔ jsonableTreeFromCursor(JsonCursor) 5.03 55 2 44,818,174.04 ±1,087,019.85 ±2.43% ✔ mapTreeFromCursor(JsonCursor) 5.05 51 2 48,527,929.62 ±2,263,518.38 ±4.66% ✔ sum(JsonCursor) 0.43 6 4 14,859,402.75 ±144,767.55 ±0.97% ✔ sum-map(JsonCursor) 0.54 7 4 16,355,574.00 ±162,031.79 ±0.99% ✔ averageValues(JsonCursor) 0.62 7 8 9,408,564.45 ±85,535.68 ±0.91%
ITreeCursor emulateProductionBuild: true canada TextCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_canada_TextCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(TextCursor) 5.00 46 4 26,578,288.12 ±568,362.83 ±2.14% ✔ jsonableTreeFromCursor(TextCursor) 5.02 57 2 43,223,837.32 ±1,164,851.15 ±2.69% ✔ mapTreeFromCursor(TextCursor) 5.08 54 2 46,150,133.72 ±2,264,744.98 ±4.91% ✔ sum(TextCursor) 0.42 6 4 14,314,275.29 ±119,940.87 ±0.84% ✔ sum-map(TextCursor) 0.85 12 4 15,952,620.60 ±148,272.78 ±0.93% ✔ averageValues(TextCursor) 0.85 10 8 9,217,011.11 ±88,133.79 ±0.96%
ITreeCursor emulateProductionBuild: true canada MapCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_canada_MapCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(MapCursor) 5.04 79 2 31,403,454.30 ±815,771.17 ±2.60% ✔ jsonableTreeFromCursor(MapCursor) 5.01 48 2 51,066,486.67 ±1,541,741.39 ±3.02% ✔ mapTreeFromCursor(MapCursor) 5.09 46 2 54,165,540.25 ±2,604,326.37 ±4.81% ✔ sum(MapCursor) 0.54 5 4 21,153,005.45 ±166,270.65 ±0.79% ✔ sum-map(MapCursor) 1.18 11 4 24,157,495.34 ±238,878.57 ±0.99% ✔ averageValues(MapCursor) 0.91 8 8 12,060,794.98 ±109,647.63 ±0.91%
ITreeCursor emulateProductionBuild: true canada object-forest Cursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_canada_object_forest_Cursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(object-forest Cursor) 5.04 80 2 30,964,374.34 ±830,640.74 ±2.68% ✔ jsonableTreeFromCursor(object-forest Cursor) 5.08 44 2 56,327,290.81 ±1,521,512.51 ±2.70% ✔ mapTreeFromCursor(object-forest Cursor) 5.00 45 2 54,222,194.52 ±2,716,271.77 ±5.01% ✔ sum(object-forest Cursor) 0.54 5 4 21,562,631.25 ±213,627.79 ±0.99% ✔ sum-map(object-forest Cursor) 1.59 30 2 25,166,800.97 ±246,762.17 ±0.98% ✔ averageValues(object-forest Cursor) 0.58 5 8 11,312,719.12 ±60,004.42 ±0.53%
ITreeCursor emulateProductionBuild: true canada BasicChunkCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_canada_BasicChunkCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(BasicChunkCursor) 5.02 78 4 15,802,300.71 ±401,389.25 ±2.54% ✔ jsonableTreeFromCursor(BasicChunkCursor) 5.05 70 2 35,404,982.04 ±669,086.06 ±1.89% ✔ mapTreeFromCursor(BasicChunkCursor) 5.07 57 2 43,638,022.72 ±1,854,813.30 ±4.25% ✔ sum(BasicChunkCursor) 4.17 47 8 10,769,675.73 ±107,121.91 ±0.99% ✔ sum-map(BasicChunkCursor) 2.24 22 8 11,983,981.76 ±119,518.88 ±1.00% ✔ averageValues(BasicChunkCursor) 1.24 14 16 5,015,204.20 ±49,899.53 ±0.99%
ITreeCursor emulateProductionBuild: true canada chunked-forest Cursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_canada_chunked_forest_Cursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(chunked-forest Cursor) 5.09 52 4 23,778,494.24 ±476,163.14 ±2.00% ✔ jsonableTreeFromCursor(chunked-forest Cursor) 5.01 107 1 46,081,306.14 ±918,342.00 ±1.99% ✔ mapTreeFromCursor(chunked-forest Cursor) 5.04 100 1 49,638,705.85 ±2,037,294.09 ±4.10% ✔ sum(chunked-forest Cursor) 2.35 35 4 15,998,251.40 ±156,383.23 ±0.98% ✔ sum-map(chunked-forest Cursor) 1.73 18 4 22,250,574.07 ±220,620.44 ±0.99% ✔ averageValues(chunked-forest Cursor) 2.06 36 4 13,638,350.19 ±134,405.24 ±0.99%
ITreeCursor emulateProductionBuild: true canada
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_canada_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ Clone JS Object 5.01 77 32 1,998,634.17 ±43,289.01 ±2.17%
ITreeCursor emulateProductionBuild: false canada JsonCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_canada_JsonCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(JsonCursor) 5.04 69 2 35,930,458.08 ±656,398.63 ±1.83% ✔ jsonableTreeFromCursor(JsonCursor) 5.04 84 1 59,013,869.46 ±1,476,046.78 ±2.50% ✔ mapTreeFromCursor(JsonCursor) 5.09 45 2 55,289,085.00 ±2,536,457.29 ±4.59% ✔ sum(JsonCursor) 0.79 8 4 21,505,631.66 ±199,020.54 ±0.93% ✔ sum-map(JsonCursor) 0.58 5 4 23,205,827.60 ±129,863.09 ±0.56% ✔ averageValues(JsonCursor) 0.42 6 4 14,182,361.00 ±141,369.69 ±1.00%
ITreeCursor emulateProductionBuild: false canada TextCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_canada_TextCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(TextCursor) 5.04 72 2 34,371,048.94 ±637,514.25 ±1.85% ✔ jsonableTreeFromCursor(TextCursor) 5.03 84 1 58,975,841.83 ±1,256,626.88 ±2.13% ✔ mapTreeFromCursor(TextCursor) 5.05 82 1 60,683,662.06 ±2,817,869.73 ±4.64% ✔ sum(TextCursor) 0.54 5 4 21,909,299.15 ±163,930.14 ±0.75% ✔ sum-map(TextCursor) 1.15 11 4 23,493,804.36 ±233,388.91 ±0.99% ✔ averageValues(TextCursor) 0.36 5 4 14,126,504.30 ±95,248.89 ±0.67%
ITreeCursor emulateProductionBuild: false canada MapCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_canada_MapCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(MapCursor) 5.06 61 2 40,563,379.15 ±849,527.55 ±2.09% ✔ jsonableTreeFromCursor(MapCursor) 5.06 72 1 69,084,999.11 ±1,900,975.80 ±2.75% ✔ mapTreeFromCursor(MapCursor) 5.01 76 1 64,925,967.99 ±3,006,276.04 ±4.63% ✔ sum(MapCursor) 0.59 9 2 28,519,653.33 ±266,677.63 ±0.94% ✔ sum-map(MapCursor) 0.52 7 2 30,578,482.36 ±284,713.00 ±0.93% ✔ averageValues(MapCursor) 0.65 7 4 19,246,363.86 ±180,842.72 ±0.94%
ITreeCursor emulateProductionBuild: false canada object-forest Cursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_canada_object_forest_Cursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(object-forest Cursor) 5.07 64 2 38,819,322.55 ±952,731.62 ±2.45% ✔ jsonableTreeFromCursor(object-forest Cursor) 5.04 67 1 73,835,055.04 ±1,716,902.81 ±2.33% ✔ mapTreeFromCursor(object-forest Cursor) 5.00 70 1 70,134,232.94 ±2,940,114.10 ±4.19% ✔ sum(object-forest Cursor) 0.48 7 2 28,373,456.36 ±247,410.49 ±0.87% ✔ sum-map(object-forest Cursor) 1.20 17 2 32,399,161.91 ±319,455.89 ±0.99% ✔ averageValues(object-forest Cursor) 2.24 32 4 16,691,936.59 ±158,297.42 ±0.95%
ITreeCursor emulateProductionBuild: false canada BasicChunkCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_canada_BasicChunkCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(BasicChunkCursor) 5.00 75 4 16,388,949.61 ±395,016.69 ±2.41% ✔ jsonableTreeFromCursor(BasicChunkCursor) 5.04 68 2 36,405,855.98 ±710,419.31 ±1.95% ✔ mapTreeFromCursor(BasicChunkCursor) 5.03 59 2 41,840,122.29 ±1,869,587.90 ±4.47% ✔ sum(BasicChunkCursor) 5.06 57 8 10,853,516.44 ±230,232.18 ±2.12% ✔ sum-map(BasicChunkCursor) 1.98 20 8 11,639,844.88 ±111,311.88 ±0.96% ✔ averageValues(BasicChunkCursor) 2.73 31 16 5,279,350.81 ±51,274.68 ±0.97%
ITreeCursor emulateProductionBuild: false canada chunked-forest Cursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_canada_chunked_forest_Cursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(chunked-forest Cursor) 5.02 52 4 23,485,737.76 ±553,882.69 ±2.36% ✔ jsonableTreeFromCursor(chunked-forest Cursor) 5.04 111 1 44,758,545.96 ±894,502.17 ±2.00% ✔ mapTreeFromCursor(chunked-forest Cursor) 5.05 94 1 52,768,391.04 ±2,118,160.93 ±4.01% ✔ sum(chunked-forest Cursor) 2.61 32 4 19,545,368.41 ±191,759.15 ±0.98% ✔ sum-map(chunked-forest Cursor) 0.79 9 4 18,630,128.61 ±181,404.04 ±0.97% ✔ averageValues(chunked-forest Cursor) 1.29 25 4 12,058,735.18 ±120,436.65 ±1.00%
ITreeCursor emulateProductionBuild: false canada
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_canada_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ Clone JS Object 5.04 80 32 1,937,284.51 ±21,497.76 ±1.11%
ITreeCursor emulateProductionBuild: true twitter JsonCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_twitter_JsonCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(JsonCursor) 1.02 16 2 29,445,976.91 ±290,517.48 ±0.99% ✔ jsonableTreeFromCursor(JsonCursor) 5.06 61 2 40,761,238.18 ±999,266.27 ±2.45% ✔ mapTreeFromCursor(JsonCursor) 5.04 74 2 33,589,019.78 ±1,530,685.03 ±4.56% ✔ sum(JsonCursor) 0.87 9 8 10,649,229.46 ±101,232.03 ±0.95% ✔ sum-map(JsonCursor) 0.88 8 8 12,017,314.47 ±108,963.98 ±0.91% ✔ averageValues(JsonCursor) 0.43 5 512 134,674.33 ±868.06 ±0.64%
ITreeCursor emulateProductionBuild: true twitter TextCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_twitter_TextCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(TextCursor) 0.90 13 2 31,176,768.27 ±301,317.67 ±0.97% ✔ jsonableTreeFromCursor(TextCursor) 5.05 58 2 42,735,750.03 ±947,978.69 ±2.22% ✔ mapTreeFromCursor(TextCursor) 5.03 68 2 36,380,038.10 ±1,671,609.46 ±4.59% ✔ sum(TextCursor) 1.06 10 8 11,778,089.59 ±111,625.70 ±0.95% ✔ sum-map(TextCursor) 2.31 39 4 14,293,895.20 ±139,892.45 ±0.98% ✔ averageValues(TextCursor) 0.51 5 512 161,808.23 ±1,542.92 ±0.95%
ITreeCursor emulateProductionBuild: true twitter MapCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_twitter_MapCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(MapCursor) 0.39 5 2 31,083,567.20 ±283,432.23 ±0.91% ✔ jsonableTreeFromCursor(MapCursor) 5.03 56 2 44,023,801.35 ±965,321.41 ±2.19% ✔ mapTreeFromCursor(MapCursor) 5.02 67 2 36,817,006.49 ±1,504,282.67 ±4.09% ✔ sum(MapCursor) 0.55 9 4 13,303,632.42 ±122,617.54 ±0.92% ✔ sum-map(MapCursor) 0.71 10 4 15,624,545.73 ±154,149.40 ±0.99% ✔ averageValues(MapCursor) 0.52 5 512 163,809.75 ±1,163.86 ±0.71%
ITreeCursor emulateProductionBuild: true twitter object-forest Cursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_twitter_object_forest_Cursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(object-forest Cursor) 1.08 15 2 33,099,338.93 ±329,963.22 ±1.00% ✔ jsonableTreeFromCursor(object-forest Cursor) 5.03 54 2 45,603,905.41 ±1,298,481.75 ±2.85% ✔ mapTreeFromCursor(object-forest Cursor) 5.02 65 2 37,914,381.71 ±1,407,112.02 ±3.71% ✔ sum(object-forest Cursor) 0.45 7 4 13,182,487.93 ±121,075.82 ±0.92% ✔ sum-map(object-forest Cursor) 0.89 13 4 15,313,447.56 ±149,170.35 ±0.97% ✔ averageValues(object-forest Cursor) 0.52 5 512 160,547.88 ±869.34 ±0.54%
ITreeCursor emulateProductionBuild: true twitter BasicChunkCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_twitter_BasicChunkCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(BasicChunkCursor) 0.78 9 4 18,850,302.03 ±172,874.56 ±0.92% ✔ jsonableTreeFromCursor(BasicChunkCursor) 5.03 91 2 27,256,681.65 ±467,410.61 ±1.71% ✔ mapTreeFromCursor(BasicChunkCursor) 5.09 47 4 26,467,567.84 ±1,162,383.33 ±4.39% ✔ sum(BasicChunkCursor) 2.63 34 16 4,649,911.44 ±46,352.70 ±1.00% ✔ sum-map(BasicChunkCursor) 4.64 90 8 6,337,570.32 ±63,272.47 ±1.00% ✔ averageValues(BasicChunkCursor) 0.51 5 1,024 78,905.26 ±259.57 ±0.33%
ITreeCursor emulateProductionBuild: true twitter chunked-forest Cursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_twitter_chunked_forest_Cursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(chunked-forest Cursor) 0.82 14 2 26,274,225.43 ±258,538.85 ±0.98% ✔ jsonableTreeFromCursor(chunked-forest Cursor) 5.05 74 2 33,548,162.70 ±650,023.62 ±1.94% ✔ mapTreeFromCursor(chunked-forest Cursor) 5.02 80 2 30,820,426.01 ±1,104,727.03 ±3.58% ✔ sum(chunked-forest Cursor) 3.60 49 8 8,928,115.78 ±88,713.53 ±0.99% ✔ sum-map(chunked-forest Cursor) 2.82 36 8 9,416,158.59 ±92,595.83 ±0.98% ✔ averageValues(chunked-forest Cursor) 5.01 61 512 156,968.05 ±1,635.91 ±1.04%
ITreeCursor emulateProductionBuild: true twitter
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_twitter_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ Clone JS Object 0.46 5 8 9,182,610.88 ±39,835.28 ±0.43%
ITreeCursor emulateProductionBuild: false twitter JsonCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_twitter_JsonCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(JsonCursor) 0.83 10 2 37,064,935.15 ±366,561.23 ±0.99% ✔ jsonableTreeFromCursor(JsonCursor) 5.02 50 2 49,195,856.70 ±895,480.33 ±1.82% ✔ mapTreeFromCursor(JsonCursor) 5.04 61 2 40,645,843.16 ±1,375,822.27 ±3.38% ✔ sum(JsonCursor) 0.57 8 4 15,397,207.91 ±131,433.15 ±0.85% ✔ sum-map(JsonCursor) 1.14 15 4 17,519,314.97 ±171,422.77 ±0.98% ✔ averageValues(JsonCursor) 0.35 5 256 214,217.99 ±994.53 ±0.46%
ITreeCursor emulateProductionBuild: false twitter TextCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_twitter_TextCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(TextCursor) 1.24 15 2 38,337,171.90 ±378,981.28 ±0.99% ✔ jsonableTreeFromCursor(TextCursor) 5.09 48 2 51,933,087.48 ±989,869.86 ±1.91% ✔ mapTreeFromCursor(TextCursor) 5.09 59 2 42,332,909.19 ±1,384,379.14 ±3.27% ✔ sum(TextCursor) 0.80 10 4 17,710,338.47 ±167,349.47 ±0.94% ✔ sum-map(TextCursor) 0.72 8 4 19,388,289.44 ±179,607.12 ±0.93% ✔ averageValues(TextCursor) 0.38 5 256 236,552.54 ±2,038.96 ±0.86%
ITreeCursor emulateProductionBuild: false twitter MapCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_twitter_MapCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(MapCursor) 1.72 21 2 38,607,702.14 ±384,710.76 ±1.00% ✔ jsonableTreeFromCursor(MapCursor) 1.05 9 2 51,792,169.11 ±461,965.00 ±0.89% ✔ mapTreeFromCursor(MapCursor) 5.02 53 2 46,391,897.77 ±1,713,784.82 ±3.69% ✔ sum(MapCursor) 2.24 30 4 17,870,167.17 ±177,220.24 ±0.99% ✔ sum-map(MapCursor) 3.30 37 4 21,543,250.35 ±214,623.80 ±1.00% ✔ averageValues(MapCursor) 1.73 26 256 246,465.28 ±2,398.02 ±0.97%
ITreeCursor emulateProductionBuild: false twitter object-forest Cursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_twitter_object_forest_Cursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(object-forest Cursor) 2.85 32 2 42,680,297.98 ±412,337.35 ±0.97% ✔ jsonableTreeFromCursor(object-forest Cursor) 5.06 89 1 56,011,615.40 ±1,081,117.97 ±1.93% ✔ mapTreeFromCursor(object-forest Cursor) 5.07 53 2 46,704,407.27 ±1,628,905.34 ±3.49% ✔ sum(object-forest Cursor) 0.48 5 4 18,974,451.90 ±167,797.52 ±0.88% ✔ sum-map(object-forest Cursor) 0.55 5 4 21,766,032.60 ±149,405.96 ±0.69% ✔ averageValues(object-forest Cursor) 0.38 5 256 228,224.17 ±1,384.31 ±0.61%
ITreeCursor emulateProductionBuild: false twitter BasicChunkCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_twitter_BasicChunkCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(BasicChunkCursor) 1.22 14 4 19,905,174.89 ±186,767.66 ±0.94% ✔ jsonableTreeFromCursor(BasicChunkCursor) 5.02 94 2 26,301,581.19 ±488,860.82 ±1.86% ✔ mapTreeFromCursor(BasicChunkCursor) 5.04 49 4 25,127,799.10 ±943,897.71 ±3.76% ✔ sum(BasicChunkCursor) 3.46 48 16 4,368,964.44 ±43,341.29 ±0.99% ✔ sum-map(BasicChunkCursor) 4.45 83 8 6,575,701.52 ±65,299.25 ±0.99% ✔ averageValues(BasicChunkCursor) 0.52 5 1,024 79,982.85 ±418.86 ±0.52%
ITreeCursor emulateProductionBuild: false twitter chunked-forest Cursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_twitter_chunked_forest_Cursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(chunked-forest Cursor) 2.00 38 2 25,327,933.89 ±250,860.50 ±0.99% ✔ jsonableTreeFromCursor(chunked-forest Cursor) 5.03 70 2 35,261,618.96 ±627,688.56 ±1.78% ✔ mapTreeFromCursor(chunked-forest Cursor) 5.05 81 2 30,567,091.07 ±1,013,585.42 ±3.32% ✔ sum(chunked-forest Cursor) 2.56 36 8 8,563,462.13 ±84,655.08 ±0.99% ✔ sum-map(chunked-forest Cursor) 3.03 39 8 9,361,741.91 ±92,469.22 ±0.99% ✔ averageValues(chunked-forest Cursor) 1.87 19 512 179,475.46 ±1,725.12 ±0.96%
ITreeCursor emulateProductionBuild: false twitter
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_twitter_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ Clone JS Object 0.45 5 8 9,086,757.22 ±80,158.65 ±0.88%
ITreeCursor emulateProductionBuild: true citm JsonCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_citm_JsonCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(JsonCursor) 1.72 25 1 65,474,591.16 ±642,959.01 ±0.98% ✔ jsonableTreeFromCursor(JsonCursor) 5.04 60 1 82,433,166.82 ±2,730,880.08 ±3.31% ✔ mapTreeFromCursor(JsonCursor) 5.03 70 1 70,724,396.43 ±3,894,594.95 ±5.51% ✔ sum(JsonCursor) 1.63 17 4 22,314,021.19 ±211,003.21 ±0.95% ✔ sum-map(JsonCursor) 0.74 13 2 25,437,612.92 ±252,580.87 ±0.99% ✔ averageValues(JsonCursor) 0.51 5 256 313,680.39 ±1,757.56 ±0.56%
ITreeCursor emulateProductionBuild: true citm TextCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_citm_TextCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(TextCursor) 0.54 6 1 72,820,779.17 ±614,402.67 ±0.84% ✔ jsonableTreeFromCursor(TextCursor) 0.74 7 1 89,707,147.71 ±788,682.87 ±0.88% ✔ mapTreeFromCursor(TextCursor) 5.05 59 1 83,900,639.08 ±4,285,611.39 ±5.11% ✔ sum(TextCursor) 0.41 6 2 27,087,013.83 ±263,896.33 ±0.97% ✔ sum-map(TextCursor) 0.52 7 2 30,164,770.07 ±257,502.12 ±0.85% ✔ averageValues(TextCursor) 1.83 34 128 403,121.16 ±3,989.74 ±0.99%
ITreeCursor emulateProductionBuild: true citm MapCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_citm_MapCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(MapCursor) 1.33 16 1 76,241,527.75 ±749,187.00 ±0.98% ✔ jsonableTreeFromCursor(MapCursor) 1.14 11 1 92,936,498.55 ±860,366.82 ±0.93% ✔ mapTreeFromCursor(MapCursor) 5.03 55 1 89,488,940.29 ±4,563,708.95 ±5.10% ✔ sum(MapCursor) 0.42 5 2 32,155,430.00 ±280,177.72 ±0.87% ✔ sum-map(MapCursor) 0.59 7 2 34,998,673.71 ±329,669.67 ±0.94% ✔ averageValues(MapCursor) 0.34 5 128 391,181.51 ±1,977.19 ±0.51%
ITreeCursor emulateProductionBuild: true citm object-forest Cursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_citm_object_forest_Cursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(object-forest Cursor) 0.81 9 1 77,402,555.56 ±699,486.66 ±0.90% ✔ jsonableTreeFromCursor(object-forest Cursor) 5.04 46 1 106,588,464.48 ±3,293,264.91 ±3.09% ✔ mapTreeFromCursor(object-forest Cursor) 5.04 43 1 114,614,397.70 ±8,620,410.32 ±7.52% ✔ sum(object-forest Cursor) 5.01 66 2 36,830,032.23 ±1,136,061.22 ±3.08% ✔ sum-map(object-forest Cursor) 5.07 61 2 40,563,902.50 ±904,698.43 ±2.23% ✔ averageValues(object-forest Cursor) 0.35 5 128 400,847.25 ±3,442.72 ±0.86%
ITreeCursor emulateProductionBuild: true citm BasicChunkCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_citm_BasicChunkCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(BasicChunkCursor) 1.15 11 2 46,568,561.95 ±455,872.35 ±0.98% ✔ jsonableTreeFromCursor(BasicChunkCursor) 5.05 80 1 62,011,824.82 ±1,764,611.18 ±2.85% ✔ mapTreeFromCursor(BasicChunkCursor) 5.01 68 1 72,156,047.91 ±3,147,255.87 ±4.36% ✔ sum(BasicChunkCursor) 5.03 88 4 14,016,038.58 ±155,404.44 ±1.11% ✔ sum-map(BasicChunkCursor) 5.01 66 4 18,535,950.20 ±251,706.00 ±1.36% ✔ averageValues(BasicChunkCursor) 0.69 8 256 279,830.69 ±2,743.41 ±0.98%
ITreeCursor emulateProductionBuild: true citm chunked-forest Cursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_citm_chunked_forest_Cursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(chunked-forest Cursor) 0.38 5 1 56,520,708.60 ±524,295.71 ±0.93% ✔ jsonableTreeFromCursor(chunked-forest Cursor) 5.02 67 1 73,330,626.49 ±2,112,395.84 ±2.88% ✔ mapTreeFromCursor(chunked-forest Cursor) 5.05 68 1 72,675,358.04 ±3,404,121.16 ±4.68% ✔ sum(chunked-forest Cursor) 2.08 24 4 20,385,799.76 ±199,934.81 ±0.98% ✔ sum-map(chunked-forest Cursor) 0.57 9 2 26,037,427.83 ±241,209.68 ±0.93% ✔ averageValues(chunked-forest Cursor) 0.70 6 256 362,623.80 ±3,014.24 ±0.83%
ITreeCursor emulateProductionBuild: true citm
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__true_citm_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ Clone JS Object 0.46 6 4 15,925,497.88 ±133,577.71 ±0.84%
ITreeCursor emulateProductionBuild: false citm JsonCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_citm_JsonCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(JsonCursor) 1.82 20 1 85,370,135.45 ±825,146.99 ±0.97% ✔ jsonableTreeFromCursor(JsonCursor) 5.09 49 1 101,719,338.63 ±3,289,721.36 ±3.23% ✔ mapTreeFromCursor(JsonCursor) 5.05 57 1 86,902,354.05 ±4,435,441.69 ±5.10% ✔ sum(JsonCursor) 0.48 6 2 32,741,761.17 ±307,590.95 ±0.94% ✔ sum-map(JsonCursor) 0.45 5 2 35,482,783.40 ±219,944.56 ±0.62% ✔ averageValues(JsonCursor) 0.42 5 128 510,623.13 ±2,845.25 ±0.56%
ITreeCursor emulateProductionBuild: false citm TextCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_citm_TextCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(TextCursor) 1.73 17 1 94,283,020.71 ±935,081.91 ±0.99% ✔ jsonableTreeFromCursor(TextCursor) 5.10 44 1 113,103,282.95 ±3,674,356.63 ±3.25% ✔ mapTreeFromCursor(TextCursor) 5.04 47 1 104,740,367.87 ±5,163,608.09 ±4.93% ✔ sum(TextCursor) 0.48 5 2 37,155,861.80 ±330,053.82 ±0.89% ✔ sum-map(TextCursor) 3.70 42 2 42,526,317.64 ±415,323.70 ±0.98% ✔ averageValues(TextCursor) 0.47 5 128 578,393.48 ±4,053.33 ±0.70%
ITreeCursor emulateProductionBuild: false citm MapCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_citm_MapCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(MapCursor) 1.14 10 1 100,019,914.50 ±938,026.62 ±0.94% ✔ jsonableTreeFromCursor(MapCursor) 5.06 41 1 119,946,198.95 ±4,138,685.94 ±3.45% ✔ mapTreeFromCursor(MapCursor) 5.16 50 1 100,695,613.98 ±5,264,197.97 ±5.23% ✔ sum(MapCursor) 4.35 49 2 43,197,915.47 ±426,954.76 ±0.99% ✔ sum-map(MapCursor) 1.01 19 1 48,926,802.00 ±472,836.21 ±0.97% ✔ averageValues(MapCursor) 0.48 5 128 570,445.25 ±3,497.45 ±0.61%
ITreeCursor emulateProductionBuild: false citm object-forest Cursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_citm_object_forest_Cursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(object-forest Cursor) 4.55 44 1 100,234,499.59 ±997,026.62 ±0.99% ✔ jsonableTreeFromCursor(object-forest Cursor) 5.01 37 1 131,257,791.97 ±4,081,257.30 ±3.11% ✔ mapTreeFromCursor(object-forest Cursor) 5.02 47 1 103,896,637.36 ±4,985,245.87 ±4.80% ✔ sum(object-forest Cursor) 0.65 6 2 43,986,429.58 ±414,596.83 ±0.94% ✔ sum-map(object-forest Cursor) 5.01 98 1 50,123,246.56 ±498,451.35 ±0.99% ✔ averageValues(object-forest Cursor) 0.51 5 128 564,101.08 ±4,257.94 ±0.75%
ITreeCursor emulateProductionBuild: false citm BasicChunkCursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_citm_BasicChunkCursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(BasicChunkCursor) 1.36 13 2 47,218,575.23 ±464,890.05 ±0.98% ✔ jsonableTreeFromCursor(BasicChunkCursor) 5.01 83 1 59,340,371.63 ±1,644,858.79 ±2.77% ✔ mapTreeFromCursor(BasicChunkCursor) 5.07 78 1 63,813,104.64 ±2,862,621.75 ±4.49% ✔ sum(BasicChunkCursor) 2.76 53 4 12,598,205.10 ±124,662.13 ±0.99% ✔ sum-map(BasicChunkCursor) 0.67 8 4 17,474,870.91 ±152,413.96 ±0.87% ✔ averageValues(BasicChunkCursor) 0.45 5 256 270,828.75 ±518.30 ±0.19%
ITreeCursor emulateProductionBuild: false citm chunked-forest Cursor
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_citm_chunked_forest_Cursor_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ cursorToJsonObject(chunked-forest Cursor) 0.43 6 1 56,099,395.83 ±542,618.47 ±0.97% ✔ jsonableTreeFromCursor(chunked-forest Cursor) 5.01 65 1 75,395,630.40 ±1,843,517.64 ±2.45% ✔ mapTreeFromCursor(chunked-forest Cursor) 5.04 70 1 70,377,162.40 ±2,929,241.40 ±4.16% ✔ sum(chunked-forest Cursor) 2.40 28 4 20,327,446.06 ±199,863.35 ±0.98% ✔ sum-map(chunked-forest Cursor) 0.71 11 2 27,471,537.41 ±272,728.36 ±0.99% ✔ averageValues(chunked-forest Cursor) 0.83 7 256 385,083.79 ±3,605.69 ±0.94%
ITreeCursor emulateProductionBuild: false citm
Results file: /home/craig/Work/FluidFramework.git/4/node_modules/.pnpm/@fluid-tools+benchmark@0.51.0/node_modules/@fluid-tools/benchmark/dist/.output/ITreeCursor_emulateProductionBuild__false_citm_perfresult.json
status name total time (s) Batch Count Iterations per Batch Period (ns/op) Margin of Error (ns) Relative Margin of Error
✔ Clone JS Object 0.60 8 4 16,098,627.94 ±151,443.25 ±0.94%
Overall summary
status suite name # of passed tests total time (s)
✔ ITreeCursor emulateProductionBuild: true canada JsonCursor 6 out of 6 16.7 ✔ ITreeCursor emulateProductionBuild: true canada TextCursor 6 out of 6 17.2 ✔ ITreeCursor emulateProductionBuild: true canada MapCursor 6 out of 6 17.8 ✔ ITreeCursor emulateProductionBuild: true canada object-forest Cursor 6 out of 6 17.8 ✔ ITreeCursor emulateProductionBuild: true canada BasicChunkCursor 6 out of 6 22.8 ✔ ITreeCursor emulateProductionBuild: true canada chunked-forest Cursor 6 out of 6 21.3 ✔ ITreeCursor emulateProductionBuild: true canada 1 out of 1 5.0 ✔ ITreeCursor emulateProductionBuild: false canada JsonCursor 6 out of 6 17.0 ✔ ITreeCursor emulateProductionBuild: false canada TextCursor 6 out of 6 17.2 ✔ ITreeCursor emulateProductionBuild: false canada MapCursor 6 out of 6 16.9 ✔ ITreeCursor emulateProductionBuild: false canada object-forest Cursor 6 out of 6 19.0 ✔ ITreeCursor emulateProductionBuild: false canada BasicChunkCursor 6 out of 6 24.9 ✔ ITreeCursor emulateProductionBuild: false canada chunked-forest Cursor 6 out of 6 19.8 ✔ ITreeCursor emulateProductionBuild: false canada 1 out of 1 5.0 ✔ ITreeCursor emulateProductionBuild: true twitter JsonCursor 6 out of 6 13.3 ✔ ITreeCursor emulateProductionBuild: true twitter TextCursor 6 out of 6 14.9 ✔ ITreeCursor emulateProductionBuild: true twitter MapCursor 6 out of 6 12.2 ✔ ITreeCursor emulateProductionBuild: true twitter object-forest Cursor 6 out of 6 13.0 ✔ ITreeCursor emulateProductionBuild: true twitter BasicChunkCursor 6 out of 6 18.7 ✔ ITreeCursor emulateProductionBuild: true twitter chunked-forest Cursor 6 out of 6 22.3 ✔ ITreeCursor emulateProductionBuild: true twitter 1 out of 1 0.5 ✔ ITreeCursor emulateProductionBuild: false twitter JsonCursor 6 out of 6 13.0 ✔ ITreeCursor emulateProductionBuild: false twitter TextCursor 6 out of 6 13.3 ✔ ITreeCursor emulateProductionBuild: false twitter MapCursor 6 out of 6 15.0 ✔ ITreeCursor emulateProductionBuild: false twitter object-forest Cursor 6 out of 6 14.4 ✔ ITreeCursor emulateProductionBuild: false twitter BasicChunkCursor 6 out of 6 19.7 ✔ ITreeCursor emulateProductionBuild: false twitter chunked-forest Cursor 6 out of 6 19.5 ✔ ITreeCursor emulateProductionBuild: false twitter 1 out of 1 0.5 ✔ ITreeCursor emulateProductionBuild: true citm JsonCursor 6 out of 6 14.7 ✔ ITreeCursor emulateProductionBuild: true citm TextCursor 6 out of 6 9.1 ✔ ITreeCursor emulateProductionBuild: true citm MapCursor 6 out of 6 8.8 ✔ ITreeCursor emulateProductionBuild: true citm object-forest Cursor 6 out of 6 21.3 ✔ ITreeCursor emulateProductionBuild: true citm BasicChunkCursor 6 out of 6 21.9 ✔ ITreeCursor emulateProductionBuild: true citm chunked-forest Cursor 6 out of 6 13.8 ✔ ITreeCursor emulateProductionBuild: true citm 1 out of 1 0.5 ✔ ITreeCursor emulateProductionBuild: false citm JsonCursor 6 out of 6 13.3 ✔ ITreeCursor emulateProductionBuild: false citm TextCursor 6 out of 6 16.5 ✔ ITreeCursor emulateProductionBuild: false citm MapCursor 6 out of 6 17.2 ✔ ITreeCursor emulateProductionBuild: false citm object-forest Cursor 6 out of 6 20.7 ✔ ITreeCursor emulateProductionBuild: false citm BasicChunkCursor 6 out of 6 15.3 ✔ ITreeCursor emulateProductionBuild: false citm chunked-forest Cursor 6 out of 6 14.4 ✔ ITreeCursor emulateProductionBuild: false citm 1 out of 1 0.6 total 222 out of 222 616.8
* @remarks | ||
* Use this instead of the node 'assert' package, which requires polyfills and has a big impact on bundle sizes. | ||
* | ||
* Assertions using this API will be included in all configurations: there is no option to disable or optimize them out. | ||
* Thus this API is suitable for detecting conditions that should terminate the application and produce a useful diagnostic message. | ||
* It can be used to ensure bad states are detected early and to avoid data corruption or harder to debug errors. | ||
* | ||
* In cases where the assert is very unlikely to have an impact on production code but is still useful as documentation and for debugging, consider using `debugAssert` instead | ||
* In cases where the assert is very unlikely to have an impact on production code but is still useful as documentation and for debugging, consider using {@link debugAssert} instead | ||
* to optimize bundle size. | ||
* | ||
* This API is not intended for use outside of the Fluid Framework client codebase: it will most likely be made internal in the future. | ||
* @privateRemarks | ||
* This should be deprecated (as a non internal API) then moved to purely internal. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we have an idea of if / how prevalently this is still used by our partners? I would love to finally make this internal...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No idea: I haven't checked. The old one duplicated assert in common/lib/common-utils/src/assert.ts is internal, so I'm not sure why it still exists either: I assumed most customers would still be using that, but I guess not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did a quick search in the Loop codebase - they have 76 modules that import this. @anthony-murphy do you happen to know if they have an alternative assert
library that we should be migrating them to? I would love to make this internal...
debugAssert(() => "This should be removed in production"); | ||
assert(true, "This should be kept 1"); | ||
debugAssert(() => "This should be removed in production 1"); | ||
assert(true, "This should be kept 1", () => "This should be removed in production 2"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know how our assert tagging works - would we need to make any changes to that infra to accommodate the API changes here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No. The tagging is based on function name and argument index: it leaves all other arguments alone. So the debug message will be left by tagging to be removed by the bundler in production configs.
If there is something likely impacted it would be code which looks at the message from asserts.
Looks like validateAssertionError probably needs and update to support this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated validateAssertionError
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes seem reasonable to me. I would very much like to be able to enable more input validation without impacting production perf.
I would love to get perspectives from others too, though, since this has wide implications for the repo.
@anthony-murphy @ChumpChief @noencke ?
Assuming we move forward with this, I think we should prioritize enabling a regular pipeline run that runs our test suite in production mode ASAP.
Co-authored-by: Joshua Smithrud <54606601+Josmithr@users.noreply.github.com>
…to assertBetter
import { shortCodeMap } from "../assertionShortCodesMap.js"; | ||
import { validateAssertionError } from "../validateAssertionError.js"; | ||
|
||
describe("validateAssertionError", () => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we intend to eventually run our tests under dual modes (debug asserts enabled and disabled), tests like these will be a problem. Seems like we probably don't want to capture the extended output for tests then?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we introduce such duel mode testing, we will need some policy for how we test development only functionality.
For the most part testing production functionality or mode agnostic functionality are the more important cases, and we have ways to do that (use emulateProductionBuild to make the code omit development specific stuff, or simply ignore the development specific things like debug messages passing a regex to validateAssertionError that matches the non development part of the message).
For testing development only stuff (like this code is doing, and also like what the tests for assert do for checking it works correctly), we will eventually want some pattern to either skip or adjust such tests if we do implement duel mode testing.
Currently we only do duel mode testing manually (for example using emulateProductionBuild), and such cases can already trigger additional logic based on which mode they are in by branching off of what ever controled the use of emulateProductionBuild.
If we do duel mode globally in the future, we will likely want some mechanism, for example a global (like isInProductrionMode for perf tests) or tags used to filter tests: I think it makes sense to punt worrying about that for now and address it as part of setting up dule mode testing if it ever happens.
Now a short aside about validateAssertionError in specific: this only really exists to validate developer facing messages, so I think having it include development mode stuff by default makes sense. Additionally, stripping out the development only part of the messages robustly isn't actually possible since we have no way to know if the assert isn't tagged where the separation is. We could heuristically search for \nDebug Message:
and strip it. Personally though I think it would make more sense for specific callers who wish to omit it to just provide a regex instead of a full string to match, which is what you can do with the current API.
That said, validateAssertionError is a bad API since it doesn't enforce the exception is from an assert, and it needlessly verbose to use. Tree's validateUsageError is way nicer and I'd love for us to migrate to something more like it (more ergonomic and more strict). If creating a newer nicer validateAssertionError that actually ensures the error is from an assert (maybe including an AssertionError class,and specific properties for the error parts) , having it take in a second optional string/regex to apply to the debug message which is only checked in production might be a nice API. I think fixing all that is out of scope here, but would be a nice followup.
🔗 No broken links found! ✅ Your attention to detail is admirable. linkcheck output
|
debugAssert(() => "This should be removed in production"); | ||
assert(true, "This should be kept 1"); | ||
debugAssert(() => "This should be removed in production 1"); | ||
assert(true, "This should be kept 1", () => "This should be removed in production 2"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will this impact bundle size? the primary reason we tag was to the reduce bundle size around assert messages
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also, how will this work in our test environments? we specifically alert on asserts, will that still work? Will it make investigating between prod and test environments harder?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will this impact bundle size?
No.
This is part of a test to validate this is removed from the bundle. Since the test is passing, it confirms this is removed from the bundle (when bundling in production mode anyway).
See examples/utils/bundle-size-tests/src/test/checkDebugAsserts.spec.ts for the test which inspects this bundle and validates its contents.
The purpose of this feature is you can include extra logic to create a more detailed message for development builds which is removed from production builds, and the tests confirm that it is working as intended.
also, how will this work in our test environments? we specifically alert on asserts, will that still work? Will it make investigating between prod and test environments harder?
This will have no impact on asserts that do not specifically opt into this feature, so it should no impact any ongoing issues or cause any major changes by just merging this change.
For asserts that do opt into this feature, the answer will depend on the configuration in our test environments. If they use production optimized bundles then they will behave the same as they did before, and this extra information with neither be available nor cause issues.
If they use development bundles (or don't use bundles at all, like our mocha unit tests), then any additional information developers put into the debug message will be available in the error message, which might be useful, but could also cause issues.
Can you point me to how our test environments configure their bundles?
Can you point me to how we do the alerting so I can inspect how it works and evaluate the potential impact? (maybe it needs a few new test cases to cover this)
I think the testing story really needs to be addressed before we expand usage here. Real bugs can happen when test code and prod code diverge, which this seems to increase the capability for. We should get a plan in place to address the testing gap with the current debug asserts before we move forward adding more capabilities for debug only code. Unfortunately, fixing the testing problem will likely be quite costly in terms of both building and maintaining the capability to test all our code under both debug and prod variants. |
Description
Enables debugAsserts by default in development configurations, but adds a more robust option for testing with them (and similar development only) code disabled.
Allows passing debug/development only formatted text into asserts via callbacks.
This change makes it possible to have descriptive assert messages with string interpolation for debug/development builds, but still get the benefits of minimized tagged asserts in production.
Breaking Changes
The format for assertion error messages in builds that do not enable production bundle optimizations can now include a second line with additional text.
This could presumably break some things, but it is opt in and has no impact on existing asserts.
Reviewer Guidance
The review process is outlined on this wiki page.