Ideally Umbra can do something so that Penumbra knows that there's new files to discover. This would then let you, for example, edit and save a file to a modpack and then Penumbra is aware of the new file and will load it the next time the file is requested in game.
Alternatively, we can just listen for filesystem events for the mod directory and load from there, which would make it tool independent.