With a strong foundation in data engineering, I've now transitioned into the field of AI, focusing on building intelligent applications powered by AI agents and Retrieval-Augmented Generation (RAG). I’m excited to share my journey so far.
-
☁️ Data Extraction & Management: I started by leveraging AWS services like Glue, Lambda, and S3 for data extraction and storage. These tools have been essential for managing and securing data from various sources in the cloud.
-
🐍 Data Processing: PySpark was my go-to for efficient data processing and transformation. I’ve handled large datasets and performed complex transformations to make data AI-ready.
-
🏗️ Data Modeling & Optimization: My journey involved working on data modeling and performance optimization. I focused on data integrity, leveraging tools like Delta Lake and Snowflake to streamline data management.
-
🤖 AI-Powered Solutions: Now, I apply my skills to develop AI applications, particularly using tools like LangChain for Retrieval-Augmented Generation (RAG) and Pinecone for intelligent information retrieval.
-
🔄 End-to-End Pipelines: From data extraction to AI model interaction, I build full pipelines. My experience includes automating AI-based workflows with AWS services like Fargate, Lambda, and Step Functions, while ensuring seamless integration between components.
-
🎩 Workflow Orchestration: Tools like Apache Airflow have been instrumental in orchestrating workflows in both data engineering and AI projects. I’ve used it to manage complex pipelines efficiently.
-
🚀 AI Agents & Applications: In the realm of AI, I focus on building intelligent agents that can use multiple tools to solve tasks, making AI solutions more effective and responsive to user needs.
-
📈 Visualization & Insights: For data-driven insights, I use tools like QuickSight, creating intuitive visualizations to derive meaningful interpretations from data and AI model outputs.
-
🧩 Problem Solving & Growth: Throughout this journey, problem-solving remains key. Whether in data engineering or AI, I continuously learn and grow by overcoming challenges.
- LangChain
- Pinecone
- Bedrock
- Python
- RAG
- AI Agents
- AWS Fargate
- AWS Lambda
- AWS S3
- AWS Glue
- AWS Redshift Serverless
- AWS RDS
- AWS LakeFormation
- Amazon Athena
- AWS CloudFormation
- AWS IAM (Identity and Access Management)
- Amazon CloudWatch
- AWS Step Functions
- Amazon DynamoDB
- Amazon API Gateway
- MWAA
- Snowflake
- Apache Airflow
- CI/CD
And learning more in the realm of AI every day...
