charm build fails when compiling c modules in Python dependencies

[pengale]

If you attempt to build a charm that needs to compile c modules for one of the Python dependencies, it will fail, as charm tools can't see the system python headers.

To reproduce, try building charm-octavia (https://github.com/openstack/charm-octavia), and you should see the error.

Note that this may not be fixable, and it's not a great idea to build c modules in a charm, anyway, as you introduce dependencies on the architecture of your build machine. The snap should fail more gracefully, however, and we should document workarounds.

[pengale]

You can workaround the issue by running the following:

snap run --shell charm
$SNAP/usr/bin/python3.6 $SNAP/bin/charm-build

[johnsca]

We're using pip download specifically to avoid installation / building, so I'm really unclear why pip is trying to do any compilation at all.

[johnsca]

I think a big part of the issue with pip download is that because Python packages historically have used code (setup.py) to define their metadata, dependencies, and installation all together, it's perhaps impossible to realistically separate dependency resolution from installation. The move toward the pyproject.toml approach seems like it may eventually improve this situation, but inertia and legacy mean that this is not going to resolve the issue for a long time. So we're left with the situation that snaps, even classically confined, seem fundamentally opposed to the idea of downloading and building arbitrary code / packages within the snap's namespace.

We should look to how snapcraft handles this process and use the same approach. Specifically, I like the idea of using multipass or lxd to do the building "remotely" with hooks to allow for setting up the build environment while keeping it contained and separated from the building snap.