Welcome to the 2023 IGARSS Summer School repository! This repository is a comprehensive guide divided into three key chapters, aimed to equip you with the knowledge and skills to fine-tune foundation models on High-Performance Computing (HPC), push the model to the cloud, establish a SageMaker endpoint, and finally test the end-to-end inference pipeline.
Chapter 1: Fine-tuning on HPC and Uploading Model to Cloud The first chapter dives into the process of fine-tuning a foundation model on an HPC. You will gain hands-on experience in adapting a pre-trained model to a specific use case - either burn scar or flood detection. The chapter also guides you on how to upload the fine-tuned model to the cloud for seamless scalability and accessibility.
Chapter 2: Establishing SageMaker Endpoint The second chapter walks you through the process of setting up an Amazon SageMaker endpoint with the uploaded model. With SageMaker, you can run your model on the cloud, which enhances the ease of deployment and utilization of the model.
Chapter 3: Testing the Inference Pipeline End-to-End In the third and final chapter, you will learn how to test the end-to-end inference pipeline. This process includes sending requests to your SageMaker endpoint and receiving the model's inferences, which allows you to verify the performance and correctness of your deployed model.
This course material primarily focuses on "Foundation Models" and "Transformers". You will learn how to fine-tune a foundation model, specifically trained on HLS (Harmonized Landsat Sentinel), to fit a particular use case - either burn scar or flood detection. This fine-tuning process enables the model to perform specific tasks with greater precision.
Additionally, you will discover how to leverage the cloud's power by pushing the fine-tuned model to the cloud and running it on the cloud using Amazon SageMaker. The course also teaches you how to establish a SageMaker endpoint, which can be used in subsequent applications to interact with the model, providing a user-friendly method for model inference.
This course is perfect for students, researchers, and professionals interested in learning more about foundation models, particularly in the areas of image recognition and disaster detection. Basic understanding of Python and machine learning concepts would be beneficial.
Join us in this exciting journey of exploring foundation models and cloud deployment. Get hands-on experience and boost your skillset!