refactor sdlf-ddk-lightweight

[RLashofRegas]

Description of changes:

• refactor sdlf-ddk-lightweight example to better support custom pipelines (individual changes summarized below)
• renamed various files to make purpose more clear, e.g. sdlf_pipeline_stack -> sdlf_base_stack, sdlf_dataset_stack -> standard_dataset_stack
• re-organized files to make clear association with each pipeline.
  • moved code defining "standard pipeline" out of sdlf_pipeline_stack into a subfolder pipelines/standard_pipeline/standard_pipeline.py also moved sdlf_dataset_stack.py into this folder and renamed as mentioned above.
  • moved sdlf_heavy_transform.py and sdlf_light_transform.py to pipelines/common_stages/...
  • moved foundations stack to foundations folder
• refactored light and heavy transform stages for better code re-use
• added more type hints and fixed linting errors on the files that I refactored (i.e. not everywhere)
• Made standard pipeline no longer dependent on dataset (each pipeline should support more than 1 dataset). Included removing glue crawler from pipeline and replacing with code in each glue job to update catalog on write
• Added CustomPipeline and CustomDataset as an example of how customers can add custom pipelines.
• added SDLFPipeline protocol (interface)
• updated README and architecture diagrams to reflect the changes
• moved routing function within pipeline definition (because it is specific to the standard pipeline) and updated S3 object created EventBridge rules to only pick up events for the specific datasets registered to that pipeline.
• explicitly specified stack names so that they are easier to understand from the CloudFormation UI
• changed parameters.json to have a config element for pipeline specific configuration to support custom pipelines.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[malachi-constant]

Awesome work, thanks for the cleanup and additions here! Haven't tested a deployment yet but will try that now.

[kukushking]

This is awesome! Thank you very much for the refactoring. Added a few minor comments & clarifications

[LeonLuttenberger]

I left a couple of nitpicky comments but overall this looks great! Awesome work

[RLashofRegas]

I have tested the most recent version end-to-end and everything works.