The plugin only detects changes to the requirements file itself. Changes to anything listed within it will not trigger a re-run of pip. This includes directories, sdists, wheels, and other requirements files, recursively.
Also need to check directories which are installed directly, i.e. not via a requirements file. This may work, but I don't think it's in the integration tests.