External libraries cannot have a requirements.txt that needs to install a cpp_extension

[zou3519]

🐛 Describe the bug

If an external library (e.g. pytorch/tutorials) has a requirements.txt with the following:

torch
git+https://github.com/pytorch/functorch.git@v0.2.0

then pip install -r requirements.txt errors out with "module torch not found". Here's why:

1. pip reads requirements.txt
2. pip tries to determine each package's dependencies before installing anything
3. pip tries to read functorch's dependencies by running its setup.py
4. functorch's setup.py imports torch to get the cpp_extension
5. torch does not exist, so python throws an error

This was discovered in pytorch/tutorials#1968

Pitch

People might need to pin libraries like this (I'm surprised this is the first time we've run into it). So it would be cool if we could offer a way for pip to read the torch dependency in a c++ extension library (e.g. functorch) without actually importing torch.

I am not sure if this is actually possible, since we need to pass setup.py a BuildExtension object somehow

Versions

main

cc @malfet @zou3519