Add Brettell's matroids and reorganize database

[gmou3]

This PR reorganizes the catalog of available matroids into separate files (and inside a new "database" folder). It also expands the catalog with the addition of Brettell's "interesting matroids" (as found here: https://homepages.ecs.vuw.ac.nz/~bretteni/code.html). It accurately reorders and fills some gaps in the matroids found in Oxley's Matroid Theory appendix.

It also adds some functions to easily access these collection of matroids. This can be useful to easily experiment and check conjectures.

This PR partially addresses Issue #36014.

Some changes require an update of the documentation.

📝 Checklist

• The title is concise, informative, and self-explanatory.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation accordingly.

⌛ Dependencies

[mkoeppe]

@xuluze

[gmou3]

I have updated the documentation and got rid of some build errors. (There still appears an error when testing my fork; an html file not available at the "Copy docs" stage of the doc build. Not sure how to fix this.)

Can we maybe get a rerun of the tests here? @mkoeppe

[tscrim]

Overall, this seems fine with me. Although your way of listing the matroids is very strange from a programming perspective. I would just have a list, but organize the list into the appropriate blocks.

Also, how do you want to deal with the redundancy between the catalog and the named_matroids? It seems like you are making things more complicated as stuff could fall out of sync easily.

[gmou3]

Overall, this seems fine with me. Although your way of listing the matroids is very strange from a programming perspective. I would just have a list, but organize the list into the appropriate blocks.

I could move all code under the folder database into one database.py file. This would be more minimal in terms of code files and documentation pages. However it would be a bit less structured for the programmer. Overall I think that it is indeed preferable in one file. I am planning to do more PRs (adding AllMatroids function, relabeling matroids, etc.) and I will probably do this (but I can do it here if you want).

Also, how do you want to deal with the redundancy between the catalog and the named_matroids? It seems like you are making things more complicated as stuff could fall out of sync easily.

Glad you noticed this as I was planning to bring it up. I would remove the named_matroids module and only keep the catalog module. I find it better as a name (and shorter). The reason I kept both is as to not break any external script that would work with previous versions of sage.

All other minor suggested changes have been implemented, and, seizing the chance, I added a few more parametrized Oxley matroids, the code for which I wrote.

[tscrim]

Overall, this seems fine with me. Although your way of listing the matroids is very strange from a programming perspective. I would just have a list, but organize the list into the appropriate blocks.

I could move all code under the folder database into one database.py file. This would be more minimal in terms of code files and documentation pages. However it would be a bit less structured for the programmer.

Sorry, I realize my comment was vague. I was talking about the, say, OxleyMatroids function. Constructing a dict and then converting that to a list is overly complicated.

Overall I think that it is indeed preferable in one file. I am planning to do more PRs (adding AllMatroids function, relabeling matroids, etc.) and I will probably do this (but I can do it here if you want).

Indeed, your functions are redoing the work that you have done by splitting them into separate files. It might make sense to instead simply have the functions iterate over the modules following, e.g., https://stackoverflow.com/questions/21885814/how-to-iterate-through-a-modules-functions. That being said, like you, I also lean towards having them all in one file. However, I leave the decision up to you as you have more of a vision for this than I do.

Also, how do you want to deal with the redundancy between the catalog and the named_matroids? It seems like you are making things more complicated as stuff could fall out of sync easily.

Glad you noticed this as I was planning to bring it up. I would remove the named_matroids module and only keep the catalog module. I find it better as a name (and shorter). The reason I kept both is as to not break any external script that would work with previous versions of sage.

There are two ways to approach this: The first would be simply to have an alias (so they are actually the same list). The second would be to issue a deprecation warning to named_matroids, which would then be removed after a year. I just wouldn't do things manually. My preference would be having an alias, but it is a very slight preference.

[gmou3]

Sorry, I realize my comment was vague. I was talking about the, say, OxleyMatroids function. Constructing a dict and then converting that to a list is overly complicated.

I quite like the dict because they are kept organized by number of elements. It is a good way to keep them in manageable chunks. Especially now with the matroids being moved into one file (database_matroids.py), it becomes clumsier to have them automatically imported (the file also contains parametrized matroids that I don't want to import).

All other recommendations I implemented and the result is becoming rather satisfactory I think. :)

[tscrim]

Sorry, I realize my comment was vague. I was talking about the, say, OxleyMatroids function. Constructing a dict and then converting that to a list is overly complicated.

I quite like the dict because they are kept organized by number of elements. It is a good way to keep them in manageable chunks. Especially now with the matroids being moved into one file (database_matroids.py), it becomes clumsier to have them automatically imported (the file also contains parametrized matroids that I don't want to import).

I don't understand this. Where the matroids are located has nothing to do with how the function is implemented. I fully agree with you that it is useful to have them organized by rank as output. By having the implementation be a dict that gets converted to a list, you make the function logic more complicated than necessary when you are simply constructing a list. Code comments can do the job just as well IMO:

    lst = [
        U24, # rank 4
        U25, U35, # rank 5
        K4, Whirl3, Q6, P6, U36, R6, # rank 6
        Fano, FanoDual, NonFano, NonFanoDual, O7, P7, # rank 7
        # rank 8
            AG32, AG32prime,
            R8, F8, Q8, L8, S8,
            Vamos, T8, J, P8, P8pp,
            Wheel4, Whirl4,
        K33dual, K33, AG23, TernaryDowling3, Pappus, NonPappus, # rank 9
        K5, K5dual, R10, # rank 10
        R12, ExtendedTernaryGolayCode, T12, # rank 12
        PG23 # rank 13
    ]

Although if you added some additional optional arguments (such as as_dict or rank), then it would make sense to organize it first into a dict of lists.

[tscrim]

Thank you. This is looking good now. One small change that you can decide. Since you already create the list, I would simply return it (plus it is not going to grow based on user input). The other option would be to use the Python3 yield from lst.

[gmou3]

The list contains the references to the functions, which I only call when yielding. So it makes sense I think.

[tscrim]

Ah, right, you do something with the elements. Then it's no problem. I will do one more look-over in the next day or two, but any changes will likely be small.

[tscrim]

As a technical point, we shouldn't delete files but instead leave a redirect from new_file import * that also raises a deprecation.

The other thing is we (generally and are moving towards uniformity) end the docstrings as

def foo():
    """
    Some doc

     EXAMPLES:: 

        sage: foo()
        5
    """
    return 5

with no blanklines; this includes after REFERENCES:. Compare with the previous catalog.

[tscrim]

Thank you for all the changes. I think this is now ready to be included into Sage.

[gmou3]

Great, thank you @tscrim.

I did some small refinements and added a few extra tests given the merging of #36962.

[tscrim]

Great, thank you @tscrim.

I did some small refinements and added a few extra tests given the merging of #36962.

Please try to avoid doing that after a positive review as I now have to review your new changes. It is generally easier and better to just do a new PR when making further improvements (as opposed to fixes).

You also made a more substantive change with the name of the Fano dual matriod. While this is minor as the matroid was not in the catalog before, it would be appreciated if this had been mentioned.

I will try to check in the next day or two, but you could help me by running sage -tp --warn-long [files_changed] and seeing if any tests take longer than half a second. If they do, please mark them as # long time. (I am mainly looking at the change on line 751 for F8 in database_matoids.py.)

Since changes are being made, looking more closely at the files database_matroids.py and database_functions.py, the "titles" of the modules is somewhat strange in comparison to the file names. Should database_matroids.py be titled "Database of matroids"? On the other hand, I feel like database_functions.py is better named as database_collections.py too.

[gmou3]

Please try to avoid doing that after a positive review as I now have to review your new changes. It is generally easier and better to just do a new PR when making further improvements (as opposed to fixes).

Excuse me for that; noted for next time.

Since changes are being made, looking more closely at the files database_matroids.py and database_functions.py, the "titles" of the modules is somewhat strange in comparison to the file names. Should database_matroids.py be titled "Database of matroids"? On the other hand, I feel like database_functions.py is better named as database_collections.py too.

These names are supposed to be a "tree" description, meaning that "database" is the word of highest significance, and the next word describes the contents. I think the database_collections.py suggestion is good and I will implement it.

[tscrim]

Please try to avoid doing that after a positive review as I now have to review your new changes. It is generally easier and better to just do a new PR when making further improvements (as opposed to fixes).

Excuse me for that; noted for next time.

No problem. For the record, I was making just a neutral comment about it.

Since changes are being made, looking more closely at the files database_matroids.py and database_functions.py, the "titles" of the modules is somewhat strange in comparison to the file names. Should database_matroids.py be titled "Database of matroids"? On the other hand, I feel like database_functions.py is better named as database_collections.py too.

These names are supposed to be a "tree" description, meaning that "database" is the word of highest significance, and the next word describes the contents.

That was clear to me, but I don't quite see the relevance to my comment.

Anyways, I think Matroids in the database is better, but there is no "the" database. I still say that "Database of matroids" is the most clear header for the file (and hence the doc link).

[github-actions]

Documentation preview for this PR (built with commit b10acf0; changes) is ready! 🎉

[tscrim]

I believe this is now ready for inclusion. Thank you for all of the changes.

Feel free to request a review from me for anything matroid related. I am not an expert, but I know my way around the area to some degree (including the code). I am also happy to review code outside my expertise too.

[gmou3]

Certainly. I will send more PRs after this is merged. Thanks.