Make project build process and infrastructure modern

[tommorris]

mf2py is not really a modern Python project anymore. This probably needs fixing, because the further behind current best practice a project gets, the less motivation people have to fix issues.

This is basically a tracking issue for all the component parts of modernisation.

• Get CI working again: Specify parser to avoid warning in CI #162 and Update CI setup to install libxml2 #163
• Decide whether to get rid of Python 2 support: Deprecate Python 2 support? #87
• Release a final python 2.7 package and document deprecation
• Switch from nose2 to pytest
• Use pyproject.toml per PEP 518 and use Poetry for dependency management and building
• Setup Dependabot to automatically create PRs for dependency updates
• [ ] Have GitHub CI manage PyPI releases
• [ ] Pre-commit
• Code style lint in CI (Black?)

Feedback on this plan is more than welcome (pings for all and sundry: @kylewm, @kartikprabhu, @aaronpk, @kevinmarks, @sknebel, @snarfed, @barnabywalters, @pawelmhm, @tantek)

[snarfed]

Really glad to see renewed interest in this project. Thanks @tommorris!

My biggest question would be, is there an official mf2py maintainer? Are you volunteering to take up the torch again @tommorris? 😁 It's been without a maintainer for many years now, which isn't ideal. It's obviously survived ok, but it'd be nice to have clear ownership.

As for these tasks, they sound good. Some feel maybe lower priority to me - poetry, automating releases, linter - and I'm curious what you'd put in a precommit hook, the same linter? - but I'm fully on board in general with someone like you doing much-needed cleanup and modernization here. Thank you!!!

[sknebel]

first 4: agreed

Poetry: I personally haven't gotten there yet and honestly not sure how useful it is. Personally skeptical about a lot of the "we'll fix packaging this time, honest!" stuff, but willing to be convinced otherwise.

Rest: 🤷 - fine to do, but "modernizing" for sake of being modern is also not the highest priority compared to actually improving features etc, which also hasn't really happened. Not using an opinionated code formatter is really not the main barrier to mf2py improvements right now ;)

Personally, having two people again that are willing to review and release stuff would be great though - my personal motivation to do anything went down quite a bit once it seemingly was down to only me.

[snarfed]

Personally, having two people again that are willing to review and release stuff would be great though - my personal motivation to do anything went down quite a bit once it seemingly was down to only me.

Definitely understood! I'm not quite ready to volunteer to be a/the official maintainer, but I'm happy to pitch in and help with triaging issues, reviewing PRs, and admin work like cutting releases.

(I checked just now out of curiosity, and this is surprisingly not that big of a project, just ~5kloc plus ~2kloc of tests. Not that that's a hugely useful metric, but it's a data point.)

[tommorris]

I can't promise long term support, but getting it back on track and maybe doing some work on it this year would be nice.

[pawelmhm]

Thank you for flagging these problems @tommorris I am a user of this package, so I'll try to help.

re:

Decide whether to get rid of Python 2 support

I think there is not much discuss, there is no point supporting Python 2 long after end of life.

Have GitHub CI manage PyPI releases

I can help setting of GH, it is just a matter of using file along these lines here: https://github.com/scrapy/scrapyd/blob/master/.github/workflows/publish.yml

To list of problems mentioned i would also add

• update README https://github.com/microformats/mf2py#mf2py because it looks old, I can help here too

[capjamesg]

I'm excited to see this discussion! I have extensively used mf2py throughout various projects. It's been an incredibly useful resource to use. All of the first four points make sense. All of the points in the original message make sense. I'm happy to assist in any way that I can.

Might it also be worth looking at type hints for code clarity? It's not a big deal but seems to be a good decision for maintaining code clarity (not to mention the autocomplete benefits in IDEs).

[capjamesg]

@sknebel Re:

actually improving features etc, which also hasn't really happened

What features would you like to see?

[angelogladding]

I wasn't sure how much I'd be able to contribute so I just gave it a whirl over at https://github.com/canopy/mf2py

• switched to pyproject.toml via poetry
• switched from nose to pytest
• ran codebase through isort and black
• raised lowest version supported to 3.7
  • manually removed from __future__ import ...
  • manually removed if sys.version < "3": ...
• updated CI to use recent actions, recent Python versions, poetry and pytest

I've tried not to change the semantics of the code. As a result there are at least a few redundant str(...) calls that may be carefully refactored out later: https://github.com/canopy/mf2py/commit/0080814d4eccc5cb2b50108c3aa9e8f9237517f4

Tests are passing across all versions.

I'd be willing to redo these steps in the form of pull requests or whatever process makes most sense.

Poetry has come a long way and has stabilized a great deal with its most recent release.

isort is nice. Black will change your life.

I can look into linting and releasing via CI as @pawelmhm has exemplified.

I'm willing to help review/maintain.

For what it's worth, I do agree that these modernizations will lower the bar for future feature/documentation work.

EDIT: moved prototyped repo to canopy/mf2py to create a clean fork

[angelogladding]

I think we can all agree we need to:

• Release a final Python 2.7 package and document deprecation
• Switch from nose2 to pytest
• Deprecate Python 2 support

Can we move forward with main...angelogladding:mf2py:final-py2 ? I haven't turned it into a pull request for lack of a release date for the changelog.

The remaining two tasks have pull requests pending. CI only triggers on the main branch so I wasn't able to test in my fork without modifying the workflow (it's working on my local machine via act).

Can I work with someone on a pull request for the final 2.7 release so I can be approved for running CI workflows?

@tommorris, @kylewm, @kartikprabhu, @aaronpk, @kevinmarks, @sknebel, @snarfed, @barnabywalters, @pawelmhm, @tantek

Thanks

[capjamesg]

I am curious to get perspectives on @angelogladding's comment above. It would be exciting to have a maintainer who could help lead these changes. I think it is reasonable for us to consider deprecating Python 2.7 support in future package versions, leaving a frozen-in-time final version for Python 2 users available on pypi and appropriate deprecation notices in the project repository.

Setting up pytests and bringing the repository inline with more modern Python repository standards -- black for linting, isort for sorting import statements, etc. -- would be amazing, but we would need an owner who would be willing to either make or oversee these changes. To the extent I can help with such changes, I am happy to help!

[capjamesg]

To be clear, I would be happy to co-maintain this project, assuming there was interest among contributors.

[snarfed]

@angelogladding @capjamesg that sounds great! I fully support both of you as additional/new maintainers here. I've added you both with those permissions. Thank you!!!

[snarfed]

(btw, if you're taking requests, I vote for #181! 😎)

[capjamesg]

I am indeed taking requests! I am going to work with @angelogladding on the modernisation work required, then work through open Issues.

[capjamesg]

We have completed all of the tasks aside from the automated PyPi deploys and the precommit. We are going to write a CONTRIBUTING.md file that outlines contribution guidelines for this project, in place of the precommit. @angelogladding and I discussed the automated PyPi deploys, but implementing this would result in having to set version numbers in two places: a tag, and in the pyproject.toml. This is as much work as a traditional, manual deploy, and is open to a mistake that may cause an issue.