Moving quadrature coefficients generation in a separate package

[tlunet]

Hi everyone,

I intend to move in a very near future all the functionalities in pySDC focused on node generation, $Q$ matrix, and $Q_\Delta$ coefficients in a separate repo of in Parallel-in-Time, create an associated pip package and make pySDC depend on it.

Main reason is : more libraries that are not pySDC start to use the same tools to generate those coefficients (SWEET, Alex-Brown integration into Gusto, SDC implementation for Dedalus, etc ...), and each time there is a code duplication for that (I'm especially aware of that because I'm the one doing it 😅).

Also, quadrature nodes, Q and QDelta coefficients are not necessarily something only related to pySDC => putting it in a separate repo would help its sharing amongst other SDC developers, and centralize all contributions (new coefficients generation, etc ...) into one same package.

This would require moving some code belonging to pySDC into this new package at some point, hence @pancetta @danielru let me know if you are okay with this. If I don't get any strong objection for it, I'll start to work on it in the next few days.

Cheers 😇

[tlunet]

Potential names for this package

• qmatrix
• qmat ✨
• kewmat
• kewmatrix
• kewpy 🚫
• qsdc
• quadrat
• quadrature 🚫
• spectralquadrature
• sqmatrix
• skewpy

[brownbaerchen]

Hm. I didn't know your such a fan of botanical gardens :D How about leaning in to the quadrature aspect? Like quadrat? quadrature is already taken for a similar package. But you could work "spectral" into the name somehow?

[lisawim]

I really like your intention to write a own package for that! :) Hm, what about simply spectralquadrature (although I quite enjoy kewpy)? But this is rather a long name and probably not wanted. Or sqmatrix? skewpy? :D

[tlunet]

I'm also in favor of short names, not very sure about quadrature as it is a very generic name, and there is already the quadpy package that can be confused with.

Didn't understood the joke about botanical gardens tho 😓

[brownbaerchen]

I thought kew... was a reference to the famous Kew Garden. I guess I didn't get your joke :D I think kewpy is too easily confused with cupy.

[pancetta]

I'd be fine with taking out Q and Q_delta matrix generation of pySDC. I'm very much not in favor of anything kew, because besides the pun I don't see the connection. spectralquadrature is nice, but long and should this package actually DO quadrature? Like, this would need to ship with all bells and whistles, showing how to do quadrature with it.

In any case, if you make this a standalone package, all the corresponding tests need to get out of pySDC (no testing of external dependencies). You'd also need your own tutorials, your own website, PyPI, etc.. This will be quite an effort.

[tlunet]

I know about the additional effort, in particular that it needs to be uploaded to conda-forge (and not only pip) since pySDC is using exclusively conda for dependencies.

But :

• most tests are already done in pySDC, so lot of code will simply be copy-pasted (hence my request @pancetta)
• tutorials : I kinda already have that, since in many repos (gfm, sweet, dedalus_sdc), I made some time tutorials and documentations already ... that would simply be centralized here. Some adaptation need to be done of course, but nothing that cannot be solved with some quick jupyter notebooks 😉
• own website : I'm getting more and more experienced for that, thanks to some personal projects, and github deployment makes that even more easier ... needs some coordination with the domain name owner (@danielru I beleive), but should not be too complicated.
• and never neglect the power of chatGPT when it's about doing the hard work first, so you can correct and refine after it.

So without being over-optimistic, I believe that I would need between 7 to 10 days of full work to get everything done (distributed over one month maybe, since I do have other tasks atm). I may have 10% chance of being wrong, but am also aware of my pride and stubbornness that are non-negligible driving forces ...

That said, what about qmat then ? It is short, summarize what this package would do, would not be confused with quadrature or something else ... there is also its big brother qmatrix if you think qmat is too short

[brownbaerchen]

I like qmat. You can checkout git submodules for the transition process. This would let you move the whole thing to a separate repo now and you can just include that repo in pySDC. I do this in the pipeline for using my own fork of mpi4py-fft, which is not on PyPy or conda-forge. Not sure if it is really useful, though.

[tlunet]

qmat it is then !

I know already about git submodules, but I beleive it's mostly a half way solution before publishing a package, and since the goal is to make qmat as generic as possible, I would prefer avoid submodules for now and go the whole distance up to conda-forge packaging, if I can.

[pancetta]

If you really want to do that (and @danielru does not find you), then go. But please no copy&paste, rather cut&paste. If the quadrature stuff goes out of pySDC, then there should be no tests and documentation in there. So, these are two construction sites at once.

[pancetta]

Also, this sounds like a great topic for a student's project (CS or CSE). There is no(t really) math in there, but a lot of software engineering. And this is nothing to publish as a paper or to put in a CV.

[tlunet]

If you really want to do that (and @danielru does not find you), then go. But please no copy&paste, rather cut&paste. If the quadrature stuff goes out of pySDC, then there should be no tests and documentation in there. So, these are two construction sites at once.

Of course, that's what I intend to do ! (and hide from @danielru ...)

[tlunet]

Also, this sounds like a great topic for a student's project (CS or CSE). There is no(t really) math in there, but a lot of software engineering. And this is nothing to publish as a paper or to put in a CV.

Maybe not a paper for this package only, but it's more a long term vision here => the more generic it is, the more people can use that, the more it can be cited (will be already for Sweet and maybe the dedalus_sdc coupling ...).

And I disagree about the CV ... being the developer of such package is worth putting into a CV, especially since I want to find (or maybe get funding for) CSE/RSE positions

[tlunet]

@pancetta until when pySDC will keep support for python 3.8 ? Currently using some type hint features in qmat that are not supported before 3.9 ... just to know if I should avoid it 😅

[pancetta]

According to the Status of Python versions, version 3.8 is approaching end of life in October 2024. That would be the latest date for us to drop 3.8.