fix(snakemake): update to v7.32.4 to make Python 3.11 clients compatible

[giuseppe-steduto]

Update Snakemake version to v7.32.4 (latest one before v8 refactoring),
to support newer features and resolve problems for clients using Python 3.11.

• reana_commons/config.py: update default Snakemake environment image.
• reana_commons/snakemake.py: modify first_rule directive to
  default_target directive to adhere to the changes made in
  snakemake/snakemake!638ec1a.
• setup.py: adjust Snakemake version in the dependencies.

Closes: reanahub/reana-client#655.
Closes: reanahub/reana-workflow-engine-snakemake#31.

---

Note that, even though the external Snakemake API did not change, quite a few things
are different internally from v6.8.0 to v7.32.4, so proper testing is needed.
Also note that bigger efforts in improving the way in which Snakemake and REANA
interact are probably better suited when upgrading to Snakemake v8 and use a plugin-based
approach.

[codecov]

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (d3035dc) 36.36% compared to head (b8caf55) 36.38%.

❗ Current head b8caf55 differs from pull request most recent head 20ae9ce. Consider uploading reports for the commit 20ae9ce to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #435      +/-   ##
==========================================
+ Coverage   36.36%   36.38%   +0.01%     
==========================================
  Files          26       26              
  Lines        1573     1575       +2     
==========================================
+ Hits          572      573       +1     
- Misses       1001     1002       +1     

Files	Coverage Δ
reana_commons/config.py	0.00% <0.00%> (ø)
reana_commons/snakemake.py	91.78% <75.00%> (-1.18%)	⬇️

[giuseppe-steduto]

This is needed to also support Python 3.6, that uses a Snakemake version (6.15) in which the new default_target directive is present, but internally persistence had not been replaced by _persistence yet in the snakemake.Workflow class. I think this approach of directly checking the attribute (rather than checking, say, the Python version) is correct, but I'm more than happy to get advice on how to do this better, if you think there's a more appropriate way!

[mdonadoni]

I think this is good and I prefer it over checking Snakemake/Python version

[tiborsimko]

This change should rather be a feature, not a fix. While it's technically true that we are fixing things for py311 clients, we are also upgrading Snakemake to a new major version, which might affect all clients for all other Python versions. So we should rather bring attention to that. I would suggest putting something like feat(snakemake): upgrade to Snakemake 7.32.4 for the release announcement headline.

[mdonadoni]

Working well! Also reana-client works with Python 3.11+.

I went through the code and everything seems good, except for one thing that we should modify in r-w-e-snakemake. In particular, the method _wait_for_jobs is now an async coroutine (see commit snakemake/snakemake@50b8f16). I think the code still works because the coroutine is called like this (source):

def _wait_thread(self):
    try:
        asyncio.run(self._wait_for_jobs())
    except Exception as e:
        print(e)
        self.workflow.scheduler.executor_error_callback(e)

So when _wait_for_jobs is called it should return immediately the coroutine, but in our case it is just a function, so it is executed as normal. Given the while True, the function never returns until self.wait is set to false; at that point, the non-async _wait_for_jobs returns None, and asyncio.run() fails after all the jobs are already executed. Indeed, at the end of the logs I see:

a coroutine was expected, got None

PS: there is also the small issue with the missing user/home directory that we have talked about

[mdonadoni]

I think this is good and I prefer it over checking Snakemake/Python version

[mdonadoni]

This should also be removed from tests in reana-client