Collect test cases for known bugs

[hoodmane]

We should have a test_bugs.py file to collect known bugs. When people file bug reports, if there is a consistent test that demonstrates the bug we should make a PR adding an xfail test and cross referencing the test against the bug report.

[rth]

Usually we put non regression tests at the moment of fixing the issue, in the most appropriate test file, with the minimal example from the issue. It could indeed be useful to put tests as xfail in advance in case they got fixed by themselves after updating emscripten or some other parts of the build system.

The downside is if each issue which is a bug has 2 PRs (one for test and one for the fix) it creates more activity on the issue tracker, and makes it a bit harder to follow for contributors.

[hoodmane]

That makes sense for bugs that we are fixing quickly.

The current organization scheme seems a bit limited. Issue #461 was opened on Jun 11, 2019, @mdboom opened #463 to fix it the next day, #463 was never merged, #768 was opened on October 14th 2020 about the same bug. There are a handful of bug reports in old issues that never seem to have been resolved.

[hoodmane]

Maybe the correct solution is to be more consistent about adding the "bug" label to bug reports.
This list is definitely missing things:
https://github.com/iodide-project/pyodide/issues?q=is%3Aissue+is%3Aopen+label%3Abug
Do you know if any of those reports you made in 2018 have been fixed?

[hoodmane]

I think we also need a "typeconversions" label, since a lot of bugs are related to that.