Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache the relevance flag in rel contexts in an efficient way. #11152

Merged
merged 1 commit into from Nov 24, 2019

Conversation

ppedrot
Copy link
Member

@ppedrot ppedrot commented Nov 21, 2019

Fixes #11150: pattern is 10x slower in Coq 8.10.0

No test because performance tests suck in our infrastructure, but see @JasonGross's proposal.

Fixes #6502

@ppedrot
Copy link
Member Author

ppedrot commented Nov 21, 2019

Also, @SkySkimmer next time you use lists for an access-based data structure, you'll be publicly whipped against the Wall of Complexity™.

@herbelin
Copy link
Member

Maybe worth resurrecting the question of how to add regression tests for complexity?

@ppedrot
Copy link
Member Author

ppedrot commented Nov 21, 2019

Here is a less invasive version that relies on a somewhat hidden invariant of the reduction machine: every rel variable known in the context of the fconstr creation is static, and only those introduced after are handled in an untyped way.

Thus we simply reuse the environment to check irrelevance of FFlex (RelKey _) nodes and only use the relevances data for FRel nodes. This had the advantage of not requiring to change the environment signature nor duplicating data there.

I wouldn't mind somebody double-checking my DeBruijn mangling though, we've been bitten enough times never to be overly cautious...

@ppedrot ppedrot added this to the 8.10.2 milestone Nov 21, 2019
@ppedrot ppedrot added kind: fix This fixes a bug or incorrect documentation. kind: performance Improvements to performance and efficiency. labels Nov 21, 2019
@ppedrot
Copy link
Member Author

ppedrot commented Nov 21, 2019

(If people are fine with this variant I'll squash before merging.)

@SkySkimmer SkySkimmer self-assigned this Nov 21, 2019
@SkySkimmer SkySkimmer added the needs: squashing Some commits should be squashed together. label Nov 21, 2019
@ppedrot
Copy link
Member Author

ppedrot commented Nov 21, 2019

I am running a bench, just in case: https://ci.inria.fr/coq/view/benchmarking/job/benchmark-part-of-the-branch/806/.

@JasonGross
Copy link
Member

Maybe worth resurrecting the question of how to add regression tests for complexity?

@herbelin I've made #11156 to discuss this. (Feel free to make it a CEP instead and close the issue if that's better)

@JasonGross
Copy link
Member

that relies on a somewhat hidden invariant of the reduction machine:

@ppedrot can this invariant and the dependence be documented? I'm assuming this is what justifies using the empty range in some places, but that's a total guess on my part and I can imagine someone coming in and thinking it's suspicious and not realizing it's okay because of this invariant. (Or breaking this invariant and being confused.) I think dropping the text you already wrote in a comment in a relevant place would already be much better than nothing.

@ppedrot
Copy link
Member Author

ppedrot commented Nov 21, 2019

This invariant is not specific to the irrelevance marks. I think we can document it somehow, even though I have no idea about the actual invariants of the conversion machine. We do seem to be preserving that global rels are compiled to FFlex.

(To give a terrifying example, conversion does not preserve the fact that the two terms being compared have a common supertype. This is close from preventing me to sleep at night.)

Rels that exist inside the environment at the time of the closure creation
are fetched in the global environment, while we only use the local list of
relevance for FRels. All the infrastructure was implicitly relying on this
kind of behaviour before the introduction of SProp.

Fixes coq#11150: pattern is 10x slower in Coq 8.10.0
@ppedrot ppedrot removed the needs: squashing Some commits should be squashed together. label Nov 22, 2019
@SkySkimmer
Copy link
Contributor

Bench said

┌────────────────────────┬─────────────────────────┬─────────────────────────────────────────────┬─────────────────────────────────────────────┬───────────────────────────────┬────────────────────────────┐
│                        │      user time [s]      │                 CPU cycles                  │              CPU instructions               │     max resident mem [KB]     │         mem faults         │
│                        │                         │                                             │                                             │                               │                            │
│           package_name │     NEW     OLD PDIFF   │               NEW               OLD PDIFF   │               NEW               OLD PDIFF   │        NEW        OLD PDIFF   │     NEW     OLD    PDIFF   │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│             coq-sf-plf │   46.81   47.12 -0.66 % │      131418382122      131546835517 -0.10 % │      170428768378      170540903587 -0.07 % │     494076     487812 +1.28 % │     150      12 +1150.00 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│ coq-mathcomp-character │  176.28  177.14 -0.49 % │      491521847897      492849855299 -0.27 % │      647322220602      649058785808 -0.27 % │     854508     824560 +3.63 % │       6       4   +50.00 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│     coq-mathcomp-field │  225.33  226.28 -0.42 % │      628275158351      630640794118 -0.38 % │      895548356352      897362544852 -0.20 % │     755024     751188 +0.51 % │       0       0     +nan % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│  coq-mathcomp-solvable │  189.55  190.33 -0.41 % │      528285449118      531042793620 -0.52 % │      706506010919      708264484163 -0.25 % │     788268     795824 -0.95 % │      12       3  +300.00 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│        coq-fiat-crypto │ 4415.20 4425.28 -0.23 % │    12266691417702    12288520092060 -0.18 % │    19107036874514    19104623052317 +0.01 % │    2379380    2383440 -0.17 % │    1382    1307    +5.74 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│ coq-mathcomp-ssreflect │   42.10   42.18 -0.19 % │      116431109571      116181123214 +0.22 % │      140390309780      140459004503 -0.05 % │     522844     522716 +0.02 % │      89      11  +709.09 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│        coq-lambda-rust │ 1662.60 1665.71 -0.19 % │     4632385175642     4643264064089 -0.23 % │     6310133374297     6323231915743 -0.21 % │    1185440    1136100 +4.34 % │       0       0     +nan % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│   coq-mathcomp-algebra │  162.60  162.88 -0.17 % │      453242269352      453562760975 -0.07 % │      557607773947      557885698119 -0.05 % │     613672     612364 +0.21 % │       0       0     +nan % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│           coq-compcert │  779.32  780.62 -0.17 % │     2170792097838     2174596698817 -0.17 % │     2915486334904     2917618617569 -0.07 % │    1068084    1068056 +0.00 % │     355     370    -4.05 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│          coq-fiat-core │  126.88  127.06 -0.14 % │      360123856420      360169885516 -0.01 % │      442953959094      443061453644 -0.02 % │     507036     507216 -0.04 % │      38      16  +137.50 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│              coq-flocq │  258.14  258.37 -0.09 % │      716569884199      718138790873 -0.22 % │      913501830293      914086777881 -0.06 % │    1017528    1017524 +0.00 % │     235       5 +4600.00 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│       coq-fiat-parsers │  745.33  745.10 +0.03 % │     2087098632226     2084490729047 +0.13 % │     2991794546486     2994609620078 -0.09 % │    3200972    3213924 -0.40 % │      57     753   -92.43 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│             coq-geocoq │ 1602.60 1601.91 +0.04 % │     4475560348762     4471938991292 +0.08 % │     6542398737742     6544343157599 -0.03 % │    1358924    1359916 -0.07 % │     347     124  +179.84 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│            coq-coqutil │   61.77   61.72 +0.08 % │      172381355434      172245874027 +0.08 % │      210756710361      210762597747 -0.00 % │     544316     544796 -0.09 % │       2       3   -33.33 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│          coq-fourcolor │ 2170.60 2168.46 +0.10 % │     6052792704020     6047044768237 +0.10 % │    11369414650399    11370678878400 -0.01 % │     949328     949192 +0.01 % │       1       1    +0.00 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│         coq-verdi-raft │ 1336.34 1334.43 +0.14 % │     3729277544811     3722481095669 +0.18 % │     5035843594614     5043589443757 -0.15 % │    1822328    1822384 -0.00 % │       0       0     +nan % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│              coq-verdi │  122.54  122.35 +0.16 % │      339989087754      340131591537 -0.04 % │      424055280328      424544628474 -0.12 % │     556184     552384 +0.69 % │       7       8   -12.50 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│  coq-mathcomp-fingroup │   51.52   51.43 +0.17 % │      143716741821      143634047475 +0.06 % │      186706166627      187014846574 -0.17 % │     561240     560248 +0.18 % │       2       0     +nan % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│ coq-fiat-crypto-legacy │ 6730.36 6718.16 +0.18 % │    18726279169366    18705904786501 +0.11 % │    30150592750704    30161194493860 -0.04 % │    2389588    2368312 +0.90 % │    1022    1313   -22.16 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│         coq-coquelicot │   78.16   77.92 +0.31 % │      215002293402      214179414669 +0.38 % │      262259410468      261988545059 +0.10 % │     679704     679036 +0.10 % │     355      22 +1513.64 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│               coq-corn │ 1561.20 1556.05 +0.33 % │     4359478814424     4344787904688 +0.34 % │     6366837425881     6368483727572 -0.03 % │     855220     856584 -0.16 % │       1       4   -75.00 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│            coq-unimath │ 4179.88 4165.82 +0.34 % │    11648952469331    11608736504197 +0.35 % │    21360003734458    21346048416936 +0.07 % │    1068984    1072572 -0.33 % │     546     299   +82.61 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│ coq-mathcomp-odd-order │ 1439.48 1433.58 +0.41 % │     4012613846426     3994772179729 +0.45 % │     6806330505677     6831818547271 -0.37 % │    1290596    1290700 -0.01 % │      18      97   -81.44 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│           coq-bedrock2 │  522.55  519.94 +0.50 % │     1454365707252     1447760878406 +0.46 % │     2397916134082     2370765608139 +1.15 % │    3161560    3168080 -0.21 % │     520     604   -13.91 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│       coq-math-classes │  576.53  573.40 +0.55 % │     1608384919442     1597929918908 +0.65 % │     2146795871692     2146822970344 -0.00 % │     637492     637732 -0.04 % │     167      18  +827.78 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│            coq-bignums │   67.02   66.65 +0.56 % │      186611048860      185454042026 +0.62 % │      247577169664      247442890881 +0.05 % │     494748     494688 +0.01 % │     184     192    -4.17 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│              coq-color │ 3034.32 3010.93 +0.78 % │     8447469771283     8393083992571 +0.65 % │     9620689510328     9622653613699 -0.02 % │    1552152    1553924 -0.11 % │     300     450   -33.33 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│               coq-hott │  378.32  375.29 +0.81 % │     1027161225433     1021156062767 +0.59 % │     1576540400963     1576927869937 -0.02 % │     609548     607708 +0.30 % │     484     279   +73.48 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│           coq-coqprime │  187.74  186.23 +0.81 % │      521787768865      516840042053 +0.96 % │      919340049109      919678590132 -0.04 % │     807908     808064 -0.02 % │     436      81  +438.27 % │
├────────────────────────┼─────────────────────────┼─────────────────────────────────────────────┼─────────────────────────────────────────────┼───────────────────────────────┼────────────────────────────┤
│           coq-rewriter │  618.60  612.02 +1.08 % │     1720558993502     1703086574238 +1.03 % │     2479014760418     2479995521289 -0.04 % │    1012776    1015784 -0.30 % │     582     271  +114.76 % │
└────────────────────────┴─────────────────────────┴─────────────────────────────────────────────┴─────────────────────────────────────────────┴───────────────────────────────┴────────────────────────────┘

@gares
Copy link
Member

gares commented Nov 22, 2019

The bench seems to show the impact of the PR is noise. @JasonGross where is the code that got slower? Is there a opam package to test?

@JasonGross
Copy link
Member

No opam package. I could probably bring https://github.com/mit-plv/reification-by-parametricity up to date and add it to the CI and opam; it's a bunch of performance benchmarks. As far as I know there is no significant use of this code in the wild, at present. The two bug reports have small examples though.

@ppedrot
Copy link
Member Author

ppedrot commented Nov 24, 2019

(FTR, I consider this ready.)

SkySkimmer added a commit to SkySkimmer/coq that referenced this pull request Nov 24, 2019
@coqbot coqbot added this to Request inclusion in 8.10.2 in Coq 8.10 Nov 24, 2019
@coqbot coqbot added this to Request 8.11+beta1 inclusion in Coq 8.11 Nov 24, 2019
@SkySkimmer SkySkimmer merged commit b8b835c into coq:master Nov 24, 2019
@ppedrot ppedrot deleted the fix-11150 branch November 25, 2019 07:47
ppedrot added a commit to ppedrot/coq that referenced this pull request Nov 25, 2019
ppedrot added a commit to ppedrot/coq that referenced this pull request Nov 25, 2019
vbgl added a commit to vbgl/coq that referenced this pull request Nov 25, 2019
@coqbot coqbot moved this from Request 8.11+beta1 inclusion to Shipped in 8.11+beta1 in Coq 8.11 Nov 25, 2019
@coqbot coqbot moved this from Request inclusion in 8.10.2 to Shipped in 8.10.2 in Coq 8.10 Nov 26, 2019
@vbgl vbgl mentioned this pull request Nov 27, 2019
19 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: fix This fixes a bug or incorrect documentation. kind: performance Improvements to performance and efficiency.
Projects
No open projects
Coq 8.10
  
Shipped in 8.10.2
Coq 8.11
  
Shipped in 8.11+beta1
5 participants