Move performance tests to phpbench/phpbench based benchmarks #5840

Ocramius · 2016-05-26T21:04:49Z

Sample output:

∙ ./vendor/bin/phpbench run tests/Doctrine/Performance --report=default                                                                                                                   23:03  ocramius@Marcos-MBP
PhpBench 0.11-dev (@git_sha@). Running benchmarks.
Using configuration file: /Users/ocramius/Documents/Projects/doctrine2/phpbench.json

\Doctrine\Performance\ChangeSet\UnitOfWorkComputeChangesBench

    benchChangeSetComputation     I0 P0     [μ Mo]/r: 10,708.000 10,708.000 (μs)  [μSD μRSD]/r: 0.000μs 0.00%

\Doctrine\Performance\Hydration\MixedQueryFetchJoinArrayHydrationPerformanceBench

    benchHydration                I0 P0     [μ Mo]/r: 384,761.000 384,761.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%

\Doctrine\Performance\Hydration\MixedQueryFetchJoinFullObjectHydrationPerformanceBench

    benchHydration                I0 P0     [μ Mo]/r: 425,848.000 425,848.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%

\Doctrine\Performance\Hydration\MixedQueryFetchJoinPartialObjectHydrationPerformanceBench

    benchHydration                I0 P0     [μ Mo]/r: 226,162.000 226,162.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%

\Doctrine\Performance\Hydration\SimpleHydrationBench

    benchHydration                I0 P0     [μ Mo]/r: 1,314,011.000 1,314,011.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%

\Doctrine\Performance\Hydration\SimpleInsertPerformanceBench

    benchHydration                I0 P0     [μ Mo]/r: 3,296,861.000 3,296,861.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%

\Doctrine\Performance\Hydration\SimpleQueryArrayHydrationPerformanceBench

    benchHydration                I0 P0     [μ Mo]/r: 211,959.000 211,959.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%

\Doctrine\Performance\Hydration\SimpleQueryFullObjectHydrationPerformanceBench

    benchHydration                I0 P0     [μ Mo]/r: 1,547,259.000 1,547,259.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%

\Doctrine\Performance\Hydration\SimpleQueryPartialObjectHydrationPerformanceBench

    benchHydration                I0 P0     [μ Mo]/r: 493,229.000 493,229.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%

\Doctrine\Performance\Hydration\SimpleQueryScalarHydrationPerformanceBench

    benchHydration                I0 P0     [μ Mo]/r: 168,561.000 168,561.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%

\Doctrine\Performance\Hydration\SingleTableInheritanceHydrationPerformanceBench

    benchHydrateFixContracts      I0 P0     [μ Mo]/r: 6,438.000 6,438.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%
    benchHydrateFlexContracts     I0 P0     [μ Mo]/r: 9,688.000 9,688.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%
    benchHydrateUltraContracts    I0 P0     [μ Mo]/r: 6,742.000 6,742.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%
    benchHydrateAllContracts      I0 P0     [μ Mo]/r: 12,120.000 12,120.000 (μs)  [μSD μRSD]/r: 0.000μs 0.00%

\Doctrine\Performance\Hydration\SingleTableInheritanceInsertPerformanceBench

    benchInsertFixContracts       I0 P0     [μ Mo]/r: 13,407.000 13,407.000 (μs)  [μSD μRSD]/r: 0.000μs 0.00%
    benchInsertFlexContracts      I0 P0     [μ Mo]/r: 14,660.000 14,660.000 (μs)  [μSD μRSD]/r: 0.000μs 0.00%
    benchInsertUltraContracts     I0 P0     [μ Mo]/r: 14,608.000 14,608.000 (μs)  [μSD μRSD]/r: 0.000μs 0.00%
    benchInsertAllContracts       I0 P0     [μ Mo]/r: 30,086.000 30,086.000 (μs)  [μSD μRSD]/r: 0.000μs 0.00%

\Doctrine\Performance\LazyLoading\ProxyInitializationTimeBench

    benchCmsUserInitialization    I0 P0     [μ Mo]/r: 659,678.000 659,678.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%
    benchCmsEmployeeInitializationI0 P0     [μ Mo]/r: 137,093.000 137,093.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%
    benchInitializationOfAlreadyInitializedCmsUsersI0 P0    [μ Mo]/r: 11,143.000 11,143.000 (μs)  [μSD μRSD]/r: 0.000μs 0.00%
    benchInitializationOfAlreadyInitializedCmsEmployeesI0 P0    [μ Mo]/r: 8,089.000 8,089.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%

\Doctrine\Performance\LazyLoading\ProxyInstantiationTimeBench

    benchCmsUserInstantiation     I0 P0     [μ Mo]/r: 881,038.000 881,038.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%
    benchCmsEmployeeInstantiation I0 P0     [μ Mo]/r: 372,925.000 372,925.000 (μs)    [μSD μRSD]/r: 0.000μs 0.00%

24 subjects, 24 iterations, 24 revs, 0 rejects
(best [mean mode] worst) = 6,438.000 [427,378.083 427,378.083] 6,438.000 (μs)
⅀T: 10,257,074.000μs μSD/r 0.000μs μRSD/r: 0.000%
suite: 133a00e92273cb425743468320756a5d22bd5346, date: 2016-05-26, stime: 21:04:02
+-----------------------------------------------------------+-----------------------------------------------------+--------+--------+------+------+-----+-------------+-----------------+---------+---------+
| benchmark                                                 | subject                                             | groups | params | revs | iter | rej | mem         | time            | z-value | diff    |
+-----------------------------------------------------------+-----------------------------------------------------+--------+--------+------+------+-----+-------------+-----------------+---------+---------+
| UnitOfWorkComputeChangesBench                             | benchChangeSetComputation                           |        | []     | 1    | 0    | 0   | 5,548,336b  | 10,708.000μs    | 0.00σ   | +39.88% |
| MixedQueryFetchJoinArrayHydrationPerformanceBench         | benchHydration                                      |        | []     | 1    | 0    | 0   | 31,754,640b | 384,761.000μs   | 0.00σ   | +98.33% |
| MixedQueryFetchJoinFullObjectHydrationPerformanceBench    | benchHydration                                      |        | []     | 1    | 0    | 0   | 18,816,960b | 425,848.000μs   | 0.00σ   | +98.49% |
| MixedQueryFetchJoinPartialObjectHydrationPerformanceBench | benchHydration                                      |        | []     | 1    | 0    | 0   | 12,578,840b | 226,162.000μs   | 0.00σ   | +97.15% |
| SimpleHydrationBench                                      | benchHydration                                      |        | []     | 1    | 0    | 0   | 67,909,120b | 1,314,011.000μs | 0.00σ   | +99.51% |
| SimpleInsertPerformanceBench                              | benchHydration                                      |        | []     | 1    | 0    | 0   | 23,329,640b | 3,296,861.000μs | 0.00σ   | +99.80% |
| SimpleQueryArrayHydrationPerformanceBench                 | benchHydration                                      |        | []     | 1    | 0    | 0   | 14,672,480b | 211,959.000μs   | 0.00σ   | +96.96% |
| SimpleQueryFullObjectHydrationPerformanceBench            | benchHydration                                      |        | []     | 1    | 0    | 0   | 65,055,072b | 1,547,259.000μs | 0.00σ   | +99.58% |
| SimpleQueryPartialObjectHydrationPerformanceBench         | benchHydration                                      |        | []     | 1    | 0    | 0   | 26,027,136b | 493,229.000μs   | 0.00σ   | +98.69% |
| SimpleQueryScalarHydrationPerformanceBench                | benchHydration                                      |        | []     | 1    | 0    | 0   | 14,932,704b | 168,561.000μs   | 0.00σ   | +96.18% |
| SingleTableInheritanceHydrationPerformanceBench           | benchHydrateFixContracts                            |        | []     | 1    | 0    | 0   | 7,221,224b  | 6,438.000μs     | 0.00σ   | 0.00%   |
| SingleTableInheritanceHydrationPerformanceBench           | benchHydrateFlexContracts                           |        | []     | 1    | 0    | 0   | 7,382,056b  | 9,688.000μs     | 0.00σ   | +33.55% |
| SingleTableInheritanceHydrationPerformanceBench           | benchHydrateUltraContracts                          |        | []     | 1    | 0    | 0   | 7,244,664b  | 6,742.000μs     | 0.00σ   | +4.51%  |
| SingleTableInheritanceHydrationPerformanceBench           | benchHydrateAllContracts                            |        | []     | 1    | 0    | 0   | 7,470,472b  | 12,120.000μs    | 0.00σ   | +46.88% |
| SingleTableInheritanceInsertPerformanceBench              | benchInsertFixContracts                             |        | []     | 1    | 0    | 0   | 6,698,104b  | 13,407.000μs    | 0.00σ   | +51.98% |
| SingleTableInheritanceInsertPerformanceBench              | benchInsertFlexContracts                            |        | []     | 1    | 0    | 0   | 6,710,896b  | 14,660.000μs    | 0.00σ   | +56.08% |
| SingleTableInheritanceInsertPerformanceBench              | benchInsertUltraContracts                           |        | []     | 1    | 0    | 0   | 6,756,424b  | 14,608.000μs    | 0.00σ   | +55.93% |
| SingleTableInheritanceInsertPerformanceBench              | benchInsertAllContracts                             |        | []     | 1    | 0    | 0   | 7,135,568b  | 30,086.000μs    | 0.00σ   | +78.60% |
| ProxyInitializationTimeBench                              | benchCmsUserInitialization                          |        | []     | 1    | 0    | 0   | 32,939,360b | 659,678.000μs   | 0.00σ   | +99.02% |
| ProxyInitializationTimeBench                              | benchCmsEmployeeInitialization                      |        | []     | 1    | 0    | 0   | 25,259,360b | 137,093.000μs   | 0.00σ   | +95.30% |
| ProxyInitializationTimeBench                              | benchInitializationOfAlreadyInitializedCmsUsers     |        | []     | 1    | 0    | 0   | 25,259,384b | 11,143.000μs    | 0.00σ   | +42.22% |
| ProxyInitializationTimeBench                              | benchInitializationOfAlreadyInitializedCmsEmployees |        | []     | 1    | 0    | 0   | 25,259,384b | 8,089.000μs     | 0.00σ   | +20.41% |
| ProxyInstantiationTimeBench                               | benchCmsUserInstantiation                           |        | []     | 1    | 0    | 0   | 5,470,976b  | 881,038.000μs   | 0.00σ   | +99.27% |
| ProxyInstantiationTimeBench                               | benchCmsEmployeeInstantiation                       |        | []     | 1    | 0    | 0   | 5,250,832b  | 372,925.000μs   | 0.00σ   | +98.27% |
+-----------------------------------------------------------+-----------------------------------------------------+--------+--------+------+------+-----+-------------+-----------------+---------+---------+

… executed queries, not actual performance

Ocramius · 2016-05-26T22:57:40Z

@guilhermeblanco remember to run with iterations and revolutions > 1, or else results may not represent the actual performance correctly ;-)

Ocramius added 30 commits May 26, 2016 18:00

Requiring PHPBench for performance testing

305582b

Not a performance test

264ae53

PHPBench base config

9d39afc

Benchmark autoloading definitions

8b676df

Factory used to create a new entity manager

9ac7406

Simple array hydration benchmark

e179085

Full hydration + query benchmark

8ce15bb

Array hydration benchmark

75825b5

Testing mixed array hydration (with scalars and fetch-joined results)

a2aa4d1

CS (alignment)

fa95b46

Renaming bench methods, for clarity

2663992

Partial hydration bench

cbc957d

Full object hydration benchmark

5f5244e

Removing unused $entityManager variable

dcf6d41

Benchmarking mixed fetch-join with partial hydration

f0652f2

Benchmarking full object hydration with fetch joins and mixed values

bbb7c51

Removing old HydrationPerformanceTest

53b57e3

Benchmarking simple insert operations (batched)

2478f29

Corrected class name - needs to end with Bench

fcd3562

Removing useless imports

e6d89d4

Removed old InsertPerformanceTest (moved to benchmark)

25fe06d

Removing instantiation overhead from benchmark

348b199

Benchmarking insert operations for a JTI

287ea0b

Corrected benchmark name: benchmark affects a STI

37be968

Benchmarking hydration of a STI

840e12f

Removing old inheritance persister test

8615848

Removing old PersisterPerformanceTest

cec381c

Unit of work changeset computation test

79fa636

Removing old UnitOfWorkPerformanceTest

88edf6f

The SecondLevelCacheTest is actually a test that verifies amount of…

96d0a5f

… executed queries, not actual performance

Ocramius added 4 commits May 26, 2016 22:38

Benchmarking instantiation of proxies

47a78ff

Benchmarking proxy initialization logic

5a0ce10

Mocks required by the proxy performance benchmarks

0a93f13

Removing old ProxyPerformanceTest

60d9c0c

Ocramius added the Improvement label May 26, 2016

Ocramius added this to the 3.0 milestone May 26, 2016

Ocramius assigned guilhermeblanco May 26, 2016

guilhermeblanco merged commit 25aee61 into develop May 26, 2016

guilhermeblanco deleted the cleanup/move-performance-tests-to-phpbench branch May 26, 2016 22:54

lcobucci mentioned this pull request Jul 23, 2017

Move performance tests to phpbench #6575

Merged

greg0ire added this to To consider 🤔 in Backports via automation Jun 27, 2021

greg0ire removed this from the 3.0.0 milestone Jun 27, 2021

beberlei moved this from To consider 🤔 to Done ✅ in Backports Jul 4, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move performance tests to phpbench/phpbench based benchmarks #5840

Move performance tests to phpbench/phpbench based benchmarks #5840

Ocramius commented May 26, 2016

Ocramius commented May 26, 2016

Move performance tests to phpbench/phpbench based benchmarks #5840

Move performance tests to phpbench/phpbench based benchmarks #5840

Conversation

Ocramius commented May 26, 2016

Ocramius commented May 26, 2016