Add source_dir option for ProcessingStep objects #2857
Description
Describe the feature you'd like
A clear and concise description of the functionality you want.
I'd like to be able to submit a source directory to a ProcessingStep so that I can utilize other dependent code in the source directory, including but not limited to requirements.txt files
How would this feature be used? Please describe.
A clear and concise description of the use case for this feature. Please provide an example, if possible.
We want to manage dependencies for our SageMaker Pipeline jobs using requirements.txt files that are managed and kept separately and without having to hackily call python -m pip install commands in the middle of our scripts.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
We've tried many ways of having depedencies be available in our ProcessingStep objects, the closest we can get to is manually manipulating the directory in the processing container by passing ProcessingInput objects that take our dependencies and store them somewhere in the code - this works for most use cases but when it comes to requirements.txt, we have to call some sort of "os.system("python -m pip install -r") or a subprocess or some other way of manually executing the pip installs which we believe should be handling automatically (just like it is when you use a FrameworkProcessor and can pass it a source_dir during the .run() call, unfortunately we cannot do this when we want to use a SageMaker Pipeline to execute this workflows.
Additional context
Add any other context or screenshots about the feature request here.