Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing part of #5669: unification heuristics sensitive to alphabet #924

Conversation

herbelin
Copy link
Member

@herbelin herbelin commented Jul 26, 2017

A sensitivity to the order in which objects are stored in an OCaml Set structure.

See bug report #5669 and commit message for details.

May other similar bugs hang around?

@maximedenes
Copy link
Member

This reminds me a lot of universe minimization being sensitive to the file name, for similar reasons. So indeed, there may be other instances of the same problem.

@herbelin herbelin force-pushed the master+fix-5669-alphabet-unif-heuristic-sensitivity branch from 0771d63 to 29c0896 Compare July 26, 2017 23:02
@herbelin
Copy link
Member Author

You're right. We indeed had similar bugs already.

@mattam82
Copy link
Member

Wow, that one's deserve a praise indeed! The fix looks good to me.

@ejgallego
Copy link
Member

ejgallego commented Jul 26, 2017

May the fix for some of the cases be performance-sensitive?

@herbelin
Copy link
Member Author

May the fix for some of the cases be performance-sensitive?

It adds a search through the evar context for each possible solutions when more than one is available, and, in common problems, this number rarely exceeds two (one for the dependent case, one for the non dependent case). Even on the worst artificial examples, it would add at most a O(n^2) cost on the number of variables in the context when several solutions exist, which is a cost common in unification.

In theory, we could absorb this cost, by remembering the original position of each variable in the context, but by doing so we would allocate more, always. I'm not the complexity expert around, but I believe that working on compacting the identity part of instances would for instance be worther.

Also, worther, even if liable to randomly affect the complexity, shall be various expected improvements of the unification such as not dropping too early exploration paths as we do now.

…ce of names).

This surprising bug was caused by an Id.Set which was ordering
solutions to variable-projection problems in ascii order.

We fix it by re-considering the variables involved in the solutions
using the declaration order.

Note that in practice, this implies preferring a dependent solution
over a non-dependent one.
@herbelin herbelin force-pushed the master+fix-5669-alphabet-unif-heuristic-sensitivity branch from 29c0896 to c24bcae Compare July 27, 2017 09:15
@herbelin
Copy link
Member Author

Just realized that I did not correctly push an optimization (my previous answer was relative to this further simplification).

@maximedenes maximedenes added the needs: testing Some manual testing is required. label Jul 31, 2017
@maximedenes
Copy link
Member

I guess a pendulum run should tell us if there's any impact.

@Zimmi48 Zimmi48 added kind: fix This fixes a bug or incorrect documentation. needs: backport labels Aug 3, 2017
@Zimmi48 Zimmi48 added this to Request 8.7 inclusion in Coq 8.7 Aug 11, 2017
@herbelin
Copy link
Member Author

herbelin commented Sep 12, 2017

Hum, I feel that this will be without noticeable impact but a test is started here.

Edit: result gives minor variations with extrema +1.2 and -0.86.

@Zimmi48 Zimmi48 removed this from Request 8.7 inclusion in Coq 8.7 Sep 16, 2017
@Zimmi48 Zimmi48 added this to the 8.8 milestone Oct 5, 2017
@herbelin herbelin removed the needs: testing Some manual testing is required. label Oct 28, 2017
@herbelin
Copy link
Member Author

I took the initiative to remove the need:testing flag, since it seems that it lets think that the testing has not been done. However, whoever was interested in the result of the benchmark and did not acknowledge the result, it is still possible to intervene.

@Zimmi48
Copy link
Member

Zimmi48 commented Oct 30, 2017

Indeed, the complete results are as follow:

┌──────────────────────────┬─────────────────────────┬─────────────────────────────────────┬───────────────────────────────────────┬─────────────────────────┬─────────────────┐
│                          │      user time [s]      │             CPU cycles              │           CPU instructions            │  max resident mem [KB]  │   mem faults    │
│                          │                         │                                     │                                       │                         │                 │
│             package_name │     NEW     OLD PDIFF   │           NEW           OLD PDIFF   │            NEW            OLD PDIFF   │     NEW     OLD PDIFF   │ NEW OLD PDIFF   │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│   coq-mathcomp-ssreflect │   37.57   37.98 -1.08 % │  102504035481  102565947456 -0.06 % │   128072383280   128122611876 -0.04 % │  477716  477760 -0.01 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│             coq-compcert │  810.11  818.80 -1.06 % │ 2247013138259 2271950364204 -1.10 % │  3405112859929  3405195220699 -0.00 % │ 1388148 1387872 +0.02 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│   coq-mathcomp-odd_order │ 1390.62 1397.58 -0.50 % │ 3873085477632 3893674722088 -0.53 % │  6822900193600  6822901963568 -0.00 % │ 1086320 1086444 -0.01 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│    coq-mathcomp-solvable │  191.66  191.89 -0.12 % │  532677570556  533095769431 -0.08 % │   777185420787   776896443446 +0.04 % │  713984  712444 +0.22 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│              coq-unimath │ 1154.79 1155.57 -0.07 % │ 3199665966790 3204881621426 -0.16 % │  5502427931787  5503375429346 -0.02 % │ 1002156 1029476 -2.65 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│   coq-mathcomp-character │  271.84  271.97 -0.05 % │  755258759051  755816382292 -0.07 % │  1112325108891  1112352639019 -0.00 % │  980936  981084 -0.02 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│     coq-mathcomp-algebra │  168.74  168.70 +0.02 % │  468365481949  468192552422 +0.04 % │   667420417950   667525045418 -0.02 % │  588452  588540 -0.01 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│               coq-sf-plf │   40.70   40.65 +0.12 % │  111982059072  111878779943 +0.09 % │   145063261475   145154125623 -0.06 % │  511860  511984 -0.02 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│                  coq-vst │ 1363.83 1361.82 +0.15 % │ 3785987403927 3781448705743 +0.12 % │  5670208448305  5670456976242 -0.00 % │ 2045112 2048168 -0.15 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│                coq-flocq │   56.69   56.59 +0.18 % │  154886049501  154807601665 +0.05 % │   199479773762   199395028667 +0.04 % │  645304  644648 +0.10 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│               coq-sf-vfa │   17.95   17.91 +0.22 % │   48724291759   48616011598 +0.22 % │    65579610796    65666836386 -0.13 % │  536924  536936 -0.00 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│ coq-mathcomp-real_closed │  168.40  167.96 +0.26 % │  466940365575  466282948655 +0.14 % │   717230922896   717163855807 +0.01 % │  732632  732628 +0.00 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│           coq-coquelicot │   72.94   72.63 +0.43 % │  200064910225  199822578038 +0.12 % │   250661917977   250722780035 -0.02 % │  660796  660776 +0.00 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│          coq-fiat-crypto │ 3054.01 3039.92 +0.46 % │ 8458763680960 8420267328025 +0.46 % │ 14111081049985 14116222051979 -0.04 % │ 3184212 3197516 -0.42 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│       coq-mathcomp-field │  456.68  454.01 +0.59 % │ 1268887452036 1264213434665 +0.37 % │  2117422789320  2117260129421 +0.01 % │  731052  754356 -3.09 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│    coq-mathcomp-fingroup │   59.19   58.70 +0.83 % │  162645116746  162378349631 +0.16 % │   228869258815   228920390680 -0.02 % │  528780  526252 +0.48 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│               coq-geocoq │ 2177.08 2158.58 +0.86 % │ 6059920706959 6009886002325 +0.83 % │ 10091024530444 10096298115649 -0.05 % │ 1250468 1211416 +3.22 % │   0   0  +nan % │
├──────────────────────────┼─────────────────────────┼─────────────────────────────────────┼───────────────────────────────────────┼─────────────────────────┼─────────────────┤
│                coq-sf-lf │   14.29   14.12 +1.20 % │   37531947655   37315072489 +0.58 % │    51192890330    51255020556 -0.12 % │  422136  422336 -0.05 % │   0   0  +nan % │
└──────────────────────────┴─────────────────────────┴─────────────────────────────────────┴───────────────────────────────────────┴─────────────────────────┴─────────────────┘

and the needs: type of label is indeed supposed to indicate that the action has not been done so it's a good idea to remove it as soon it has been done, even if more discussion is needed on the results.

@coqbot coqbot merged commit c24bcae into coq:master Nov 3, 2017
coqbot pushed a commit that referenced this pull request Nov 3, 2017
herbelin added a commit to herbelin/github-coq that referenced this pull request Apr 15, 2018
This refines even further c24bcae (PR coq#924) and 6304c84:
- c24bcae fixed the order in the heuristic
- 6304c84 improved the order by preferring projections

There remained a dependency in the alphabetic order in selecting
unification candidates. The current commit fixes it.

We radically change the representation of the substitution to invert
by using a map indexed on the rank in the signature rather than on the
name of the variable.

More could be done to use numbers further, e.g. for representing
aliases.
herbelin added a commit to herbelin/github-coq that referenced this pull request Apr 15, 2018
This refines even further c24bcae (PR coq#924) and 6304c84:
- c24bcae fixed the order in the heuristic
- 6304c84 improved the order by preferring projections

There remained a dependency in the alphabetic order in selecting
unification candidates. The current commit fixes it.

We radically change the representation of the substitution to invert
by using a map indexed on the rank in the signature rather than on the
name of the variable.

More could be done to use numbers further, e.g. for representing
aliases.
herbelin added a commit to herbelin/github-coq that referenced this pull request Apr 15, 2018
This refines even further c24bcae (PR coq#924) and 6304c84:
- c24bcae fixed the order in the heuristic
- 6304c84 improved the order by preferring projections

There remained a dependency in the alphabetic order in selecting
unification candidates. The current commit fixes it.

We radically change the representation of the substitution to invert
by using a map indexed on the rank in the signature rather than on the
name of the variable.

More could be done to use numbers further, e.g. for representing
aliases.
herbelin added a commit to herbelin/github-coq that referenced this pull request Apr 16, 2018
This refines even further c24bcae (PR coq#924) and 6304c84:
- c24bcae fixed the order in the heuristic
- 6304c84 improved the order by preferring projections

There remained a dependency in the alphabetic order in selecting
unification candidates. The current commit fixes it.

We radically change the representation of the substitution to invert
by using a map indexed on the rank in the signature rather than on the
name of the variable.

More could be done to use numbers further, e.g. for representing
aliases.

Note that this has consequences on the test-suite (in
output/Notations.v) as some problems now infer a dependent return
clause.
herbelin added a commit to herbelin/github-coq that referenced this pull request Apr 17, 2018
This refines even further c24bcae (PR coq#924) and 6304c84:
- c24bcae fixed the order in the heuristic
- 6304c84 improved the order by preferring projections

There remained a dependency in the alphabetic order in selecting
unification candidates. The current commit fixes it.

We radically change the representation of the substitution to invert
by using a map indexed on the rank in the signature rather than on the
name of the variable.

More could be done to use numbers further, e.g. for representing
aliases.

Note that this has consequences on the test-suite (in
output/Notations.v) as some problems now infer a dependent return
clause.
herbelin added a commit to herbelin/github-coq that referenced this pull request Apr 18, 2018
This refines even further c24bcae (PR coq#924) and 6304c84:
- c24bcae fixed the order in the heuristic
- 6304c84 improved the order by preferring projections

There remained a dependency in the alphabetic order in selecting
unification candidates. The current commit fixes it.

We radically change the representation of the substitution to invert
by using a map indexed on the rank in the signature rather than on the
name of the variable.

More could be done to use numbers further, e.g. for representing
aliases.

Note that this has consequences on the test-suite (in
output/Notations.v) as some problems now infer a dependent return
clause.
herbelin added a commit to herbelin/github-coq that referenced this pull request Apr 18, 2018
This refines even further c24bcae (PR coq#924) and 6304c84:
- c24bcae fixed the order in the heuristic
- 6304c84 improved the order by preferring projections

There remained a dependency in the alphabetic order in selecting
unification candidates. The current commit fixes it.

We radically change the representation of the substitution to invert
by using a map indexed on the rank in the signature rather than on the
name of the variable.

More could be done to use numbers further, e.g. for representing
aliases.

Note that this has consequences on the test-suite (in
output/Notations.v) as some problems now infer a dependent return
clause.
herbelin added a commit to herbelin/github-coq that referenced this pull request Jul 28, 2018
This refines even further c24bcae (PR coq#924) and 6304c84:
- c24bcae fixed the order in the heuristic
- 6304c84 improved the order by preferring projections

There remained a dependency in the alphabetic order in selecting
unification candidates. The current commit fixes it.

We radically change the representation of the substitution to invert
by using a map indexed on the rank in the signature rather than on the
name of the variable.

More could be done to use numbers further, e.g. for representing
aliases.

Note that this has consequences on the test-suite (in
output/Notations.v) as some problems now infer a dependent return
clause.
herbelin added a commit to herbelin/github-coq that referenced this pull request Jul 28, 2018
This refines even further c24bcae (PR coq#924) and 6304c84:
- c24bcae fixed the order in the heuristic
- 6304c84 improved the order by preferring projections

There remained a dependency in the alphabetic order in selecting
unification candidates. The current commit fixes it.

We radically change the representation of the substitution to invert
by using a map indexed on the rank in the signature rather than on the
name of the variable.

More could be done to use numbers further, e.g. for representing
aliases.

Note that this has consequences on the test-suite (in
output/Notations.v) as some problems now infer a dependent return
clause.
herbelin added a commit to herbelin/github-coq that referenced this pull request Sep 11, 2018
This refines even further c24bcae (PR coq#924) and 6304c84:
- c24bcae fixed the order in the heuristic
- 6304c84 improved the order by preferring projections

There remained a dependency in the alphabetic order in selecting
unification candidates. The current commit fixes it.

We radically change the representation of the substitution to invert
by using a map indexed on the rank in the signature rather than on the
name of the variable.

More could be done to use numbers further, e.g. for representing
aliases.

Note that this has consequences on the test-suite (in
output/Notations.v) as some problems now infer a dependent return
clause.
herbelin added a commit to herbelin/github-coq that referenced this pull request Sep 12, 2018
This refines even further c24bcae (PR coq#924) and 6304c84:
- c24bcae fixed the order in the heuristic
- 6304c84 improved the order by preferring projections

There remained a dependency in the alphabetic order in selecting
unification candidates. The current commit fixes it.

We radically change the representation of the substitution to invert
by using a map indexed on the rank in the signature rather than on the
name of the variable.

More could be done to use numbers further, e.g. for representing
aliases.

Note that this has consequences on the test-suite (in
output/Notations.v) as some problems now infer a dependent return
clause.
herbelin added a commit to herbelin/github-coq that referenced this pull request Sep 14, 2018
This refines even further c24bcae (PR coq#924) and 6304c84:
- c24bcae fixed the order in the heuristic
- 6304c84 improved the order by preferring projections

There remained a dependency in the alphabetic order in selecting
unification candidates. The current commit fixes it.

We radically change the representation of the substitution to invert
by using a map indexed on the rank in the signature rather than on the
name of the variable.

More could be done to use numbers further, e.g. for representing
aliases.

Note that this has consequences on the test-suite (in
output/Notations.v) as some problems now infer a dependent return
clause.
silene pushed a commit that referenced this pull request Oct 16, 2018
This refines even further c24bcae (PR #924) and 6304c84:
- c24bcae fixed the order in the heuristic
- 6304c84 improved the order by preferring projections

There remained a dependency in the alphabetic order in selecting
unification candidates. The current commit fixes it.

We radically change the representation of the substitution to invert
by using a map indexed on the rank in the signature rather than on the
name of the variable.

More could be done to use numbers further, e.g. for representing
aliases.

Note that this has consequences on the test-suite (in
output/Notations.v) as some problems now infer a dependent return
clause.

(cherry picked from commit 99ffdbb)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: fix This fixes a bug or incorrect documentation.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants