Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX/ENHANCE: Small fixes and corrections, staged for 4.9.1 #2035

Closed
wants to merge 7 commits into from

Conversation

hulpke
Copy link
Contributor

@hulpke hulpke commented Dec 19, 2017

Includes #2025

Fix for #2055. #2104, #2131 (as far as LowIndex is concerned)

Multiple documenttion/testfile/formatting fixes.

@hulpke hulpke added the kind: enhancement Label for issues suggesting enhancements; and for pull requests implementing enhancements label Dec 19, 2017
@hulpke hulpke added this to the GAP 4.9.1 milestone Dec 19, 2017
hulpke added a commit that referenced this pull request Dec 19, 2017
FIX/ENHANCE: ImageKernelBlockHomomorphism memory and performance.

Merged now as all tests work fine and it can clearly go into master, there is a shadow commit #2035 to add the code into 4.9.1 once 4.9.0 is done.
@codecov
Copy link

codecov bot commented Dec 19, 2017

Codecov Report

Merging #2035 into stable-4.9 will increase coverage by 0.03%.
The diff coverage is 76.04%.

@@              Coverage Diff               @@
##           stable-4.9    #2035      +/-   ##
==============================================
+ Coverage       69.19%   69.22%   +0.03%     
==============================================
  Files             493      493              
  Lines          259991   260030      +39     
==============================================
+ Hits           179900   180008     +108     
+ Misses          80091    80022      -69
Impacted Files Coverage Δ
lib/grpffmat.gi 91.42% <100%> (+0.07%) ⬆️
lib/grplatt.gi 62.07% <100%> (+0.2%) ⬆️
lib/ghomperm.gi 90.63% <71.25%> (+0.19%) ⬆️
lib/stbcrand.gi 90.26% <0%> (-0.3%) ⬇️
hpcgap/lib/hpc/stdtasks.g 38.61% <0%> (-0.26%) ⬇️
src/hpc/threadapi.c 38.56% <0%> (-0.1%) ⬇️
src/hpc/traverse.c 84.1% <0%> (-0.07%) ⬇️
lib/grppclat.gi 80.1% <0%> (+0.13%) ⬆️
src/funcs.c 78.61% <0%> (+0.13%) ⬆️
src/scanner.c 87.5% <0%> (+0.19%) ⬆️
... and 5 more

Copy link
Member

@olexandr-konovalov olexandr-konovalov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#2025 with same changes has been merged into master too fast - it occurs that it now runs out of memory there if all packages are loaded. Please do not merge this until further investigation. This PR is assigned to GAP 4.9.1, so it should wait at least till GAP 4.9.0 will be out.

@hulpke
Copy link
Contributor Author

hulpke commented Dec 21, 2017

It seems either I have misunderstood the branching process or not explained clearly what this PR does vs. #2025:

The stable-4.9 branch has been forked off and will go to become the 4.9 releases, so master is going to be releases 4.10 and later. The change has been merged into master as it clearly is a long term improvement and any problems remaining seem to be rather with package interaction than the code itself.

I think this change is substantial enough in effect and harmless enough in actual change that it should go into 4.9, but with packing I understand if it should only go into 4.9.1 This PR is to merge the same changes there. I do not intend to merge it before 4.9.0 is done.

@olexandr-konovalov olexandr-konovalov changed the title Additions staged for 4.9.1 FIX/ENHANCE: ImageKernelBlockHomomorphism memory and performance (for 4.9.1) Jan 29, 2018
The old code added every generator found with
`AddGeneratorsExtendSchreierTree'. This caused the storage of lots of
generators and subsequent memory issues.

Also, if there are few large blocks, Butler's block homomorphism code is
inefficient and it needs to run through all points in one block. In this
case an ordinary orbit calculation is far more efficient.

Together these issues did cause severe problems in large degrees with few
blocks (observed by Thomas).

It it possible that it also rectifies some of the observed problems in
larger degree.

What one could still do is to use random subproducts instead of testing all
Schreier generators, but this is not the right time to do so.
@hulpke hulpke changed the title FIX/ENHANCE: ImageKernelBlockHomomorphism memory and performance (for 4.9.1) FIX/ENHANCE: Small fixes and corrections for 4.9.1 Feb 1, 2018
@hulpke hulpke changed the title FIX/ENHANCE: Small fixes and corrections for 4.9.1 FIX/ENHANCE: Small fixes and corrections, staged for 4.9.1 Feb 1, 2018
@hulpke hulpke dismissed olexandr-konovalov’s stale review February 1, 2018 21:04

I'm dismissing the review (in part to see what this does and) as it is not about the contents but only about not committing before 4.9.1 (which the title and milestone state). This also should make it clear that the changes still require a proper content review and do not have content changes pending.

as requested in gap-system#2131

Also added `LowLayerSubgroups` and minor rephrasing.

Also redid part of gap-system#683, as for some reason (probably my stupidity)
part of gap-system#683 had fallen out of master. Added it again.
@olexandr-konovalov olexandr-konovalov added the do not merge PRs which are not yet ready to be merged (e.g. submitted for discussion, or test results) label Feb 2, 2018
@olexandr-konovalov
Copy link
Member

PLEASE BE AWARE: these changes are already in master branch and are causing running out of memory in testextra/grpauto.tst when all packages are loaded. I have tried to reproduce this juts by running testextra/grpauto.tst with same memory settings and assertion level 2, but that was not enough to reproduce the problem :(

Copy link
Member

@fingolfin fingolfin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please be advised: For ongoing development of the 4.9.x branch, we (@ChrisJefferson @alex-konovalov @markuspf and myself) to change how we work: Instead of making changes on the stable branch, and then merging the stable-4.9 branch regulalry back into master, we will instead continue with PRs exclusively to the master branch. Then, any changes which should also go into stable-4.9 will be cherry-picked there (possibly via an intermediate PR).

The motivation for this is that this way, one can proceed making changes on master quickly, and does not have to wait for the changes to slowly trickle back from stable-4.9 into master. Moreover, fewer people have to worry about how to get changes "correctly" into stable-4.9: Just submit a PR against master, and indicate that you'd like the changes in there to also go into stable-4.9, and we'll take care of it.

Moreover, we expect stable-4.9 to be really treated as stable: I.e. no new developments should happen here, only strict bug fixes.

As it, this PR contains both new changes (e.g. to the documentation) which are not in master nor in stable-4.9. I have opened PR #2158 against the master branch with the new changes. Once that is merged, we can either merge this PR #2035 here, or perhaps replace it by an equivalent fresh PR full of cherry-picks (I could make it, no need to waste @hulpke's time).

Of course we can also do things differently, and e.g. @hulpke creates his own equivalent of my PR #2158, or yet something else; this all is not set in stone, and is not intended to cause extra work for @hulpke , so I hope we can work together to find a good solution.

@hulpke
Copy link
Contributor Author

hulpke commented Feb 5, 2018

@fingolfin
I'm very happy with this proposal. Do you want to suggest a way (so one can search for fixed text) how to indicate that a commit is also supposed to go into 4.9.x?

fingolfin
fingolfin previously approved these changes Mar 6, 2018
Copy link
Member

@fingolfin fingolfin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I missed that this PR had been updated and a comment being added :-(.

Looks OK now to me; only one tiny question on an operation vs. attribute -- but this could also be addressed in a future PR, if at all. I.e. feel free to just merge now.

@fingolfin fingolfin dismissed their stale review March 6, 2018 21:17

Err, wrong PR

@fingolfin
Copy link
Member

Sorry for the confusion, I meant to review #2057 and then got confused by too many open browser tabs sigh. Anyway, this PR is alone one where I totally missed an update. I'll now look into this PR, too, to figure out where we add, and will comment again later.

@hulpke
Copy link
Contributor Author

hulpke commented Mar 6, 2018

@fingolfin
Thank you. My understanding was that these should go into master (as they actually have already) and then would be cherry-picked into 4.9. Let me know if I can help in making this easier.

@fingolfin
Copy link
Member

I now made PR #2253 which contains the same changes as in this PR, but freshly cherry-picked from master, so that they are really identical (a few of the manual changes differed between this PR and master).

As to marking commits for backporting: Yeah, I'd include a fixed string somewhere in the commit message. E.g. "Should be backported to stable-4.9" ? I'll open an issue for this so others can join the discusson.

@fingolfin fingolfin closed this Mar 7, 2018
@fingolfin fingolfin added release notes: added PRs introducing changes that have since been mentioned in the release notes and removed release notes: added PRs introducing changes that have since been mentioned in the release notes labels Mar 22, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do not merge PRs which are not yet ready to be merged (e.g. submitted for discussion, or test results) kind: enhancement Label for issues suggesting enhancements; and for pull requests implementing enhancements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants