Use Conda Forge graph for package name mappings and less downloads

[flying-sheep]

Fixes #12

However, not everything seems to be in there, e.g. black is missing. Maybe the file only contains mappings that are not trivial?

[saulshanabrook]

Oh nice, this is awesome! @jaimergp do you know how to best use this file?

[flying-sheep]

I investigated a bit, but I can’t find a simple list of packages available. I think:

• cf-graph-countyfair/master/graph.json contains all conda packages (and their for us useless dependency info)
• cf-graph-countyfair/master/mappings/pypi/name_mapping.yaml contains all combinations of PyPI name, conda name, and import where the three are not equivalent
• cf-scripts/master/conda_forge_tick/pypi_name_mapping_static.yaml the same as the one before, but manually maintained. IDK if the info in this one is included in the previous one.

[jaimergp]

• cf-graph-countyfair/master/mappings/pypi/name_mapping.yaml contains all combinations of PyPI name, conda name, and import where the three are not equivalent

Yep this is the one you need. It should only contain known packages (to conda-forge) that don't share a name with PyPI. @viniciusdc might have some more info, since he's more involved with the bot infrastructure there.

[viniciusdc]

• cf-graph-countyfair/master/mappings/pypi/name_mapping.yaml contains all combinations of PyPI name, conda name, and import where the three are not equivalent

Yep this is the one you need. It should only contain known packages (to conda-forge) that don't share a name with PyPI. @viniciusdc might have some more info, since he's more involved with the bot infrastructure there.

Indeed that's the best source for your problem. There is a notebook where you can inspect some of the metadata within the conda-forge json in that repo as well, if needed. About the usage of the static maps, this one pypi_mapping.py generates the magic

[flying-sheep]

Great, thank you!

This is looking good. Would be nice if there was a smaller JSON blob to download for the list of packages on conda forge, but with some caching, the graph.json will work too.

[flying-sheep]

Using it now looks like this (once an hour, otherwise no progress bars)

$ python -m beni pyproject.toml
Creating cache for name_mapping.yaml at /home/phil/.cache/beni: 100%|█████████████████████████████| 636k/636k [00:00<00:00, 5.46MB/s]
Creating cache for graph.json at /home/phil/.cache/beni: 100%|██████████████████████████████████| 8.28M/8.28M [00:01<00:00, 5.70MB/s]
channels:
- conda-forge
dependencies:
- ...

[saulshanabrook]

Awesome! That looks great 🎈