A comprehensive 3-week learning resource for AI application development using C# and Python, covering machine learning, data engineering, neural networks, and modern AI technologies.
- Prerequisites
- Learning Objectives
- 3-Week Learning Plan
- Project Structure
- Recommended Tools & Setup
- Dataset Recommendations
- Hands-On Projects
- Resources & References
- Contributing
- Programming: Basic understanding of C# (.NET Core/5+) and Python (3.8+)
- Mathematics: Linear algebra, statistics, and calculus fundamentals
- Development: Git version control, command line usage
- Concepts: Object-oriented programming, data structures
- Previous work with databases (SQL/NoSQL)
- Basic understanding of web APIs and HTTP protocols
- Familiarity with JSON and data serialization
- Experience with package managers (NuGet, pip)
- Minimum: 8GB RAM, 50GB free disk space
- Recommended: 16GB+ RAM, SSD storage, GPU (NVIDIA with CUDA support)
By the end of this 3-week program, you will be able to:
- Develop AI-powered applications using C# and Python
- Implement neural network models for various use cases
- Design and build real-time streaming data pipelines
- Create effective prompt engineering strategies
- Build data visualization and plotting solutions
- Design scalable data engineering architectures
- Implement data warehousing solutions
- Deploy AI models in production environments
Focus: Core concepts, environment setup, and fundamental implementations
Learning Goals:
- Set up development environment for C# and Python AI development
- Understand AI/ML terminology and concepts
- Explore the AI development ecosystem
Activities:
- Install Visual Studio/VS Code, .NET SDK, Python, and essential packages
- Complete "Hello World" projects in both C# and Python
- Study machine learning types: supervised, unsupervised, reinforcement learning
Hands-on Tasks:
// C# - Create a simple linear regression model
using MathNet.Numerics.LinearAlgebra;
using MathNet.Numerics.Statistics;
public class SimpleLinearRegression
{
public (double slope, double intercept) FitLine(double[] x, double[] y)
{
// Implementation here
}
}# Python - Data exploration with pandas
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
# Load and explore a dataset
df = pd.read_csv('sample_data.csv')
df.describe()Learning Goals:
- Master ML.NET framework and ecosystem
- Understand C# numerical computing libraries
- Implement basic machine learning algorithms
Key Libraries:
- ML.NET: Microsoft's machine learning framework
- Math.NET Numerics: Mathematical library for .NET
- Accord.NET: Scientific computing framework
- TensorFlowSharp: TensorFlow bindings for C#
Practice Project: Build a classification model using ML.NET
using Microsoft.ML;
using Microsoft.ML.Data;
public class IrisData
{
[LoadColumn(0)] public float SepalLength;
[LoadColumn(1)] public float SepalWidth;
[LoadColumn(2)] public float PetalLength;
[LoadColumn(3)] public float PetalWidth;
[LoadColumn(4)] public string Label;
}
// Train iris classification model
var mlContext = new MLContext();
var dataView = mlContext.Data.LoadFromTextFile<IrisData>("iris.csv", hasHeader: true);Learning Goals:
- Master essential Python libraries for AI
- Understand data manipulation and analysis
- Implement basic neural networks
Essential Libraries:
- NumPy: Numerical computing
- Pandas: Data manipulation and analysis
- Scikit-learn: Machine learning library
- TensorFlow/PyTorch: Deep learning frameworks
- Matplotlib/Seaborn: Data visualization
Practice Project: Build a neural network from scratch
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
class SimpleNeuralNetwork:
def __init__(self, input_size, hidden_size, output_size):
self.W1 = np.random.randn(input_size, hidden_size)
self.W2 = np.random.randn(hidden_size, output_size)
def forward(self, X):
self.z1 = np.dot(X, self.W1)
self.a1 = self.sigmoid(self.z1)
self.z2 = np.dot(self.a1, self.W2)
self.a2 = self.sigmoid(self.z2)
return self.a2Learning Goals:
- Create compelling data visualizations
- Understand different chart types and use cases
- Implement interactive plotting solutions
C# Plotting Tools:
- OxyPlot: Cross-platform plotting library
- ScottPlot: Simple plotting library
- LiveCharts: Animated charts
Python Plotting Tools:
- Matplotlib: Comprehensive plotting library
- Seaborn: Statistical data visualization
- Plotly: Interactive web-based plots
- Bokeh: Interactive visualization library
Practice Projects:
- Create a real-time data dashboard
- Build interactive plots for model performance
- Design data exploration visualizations
Focus: Practical applications, streaming, and prompt engineering
Learning Goals:
- Master prompt design techniques
- Understand large language model interactions
- Implement effective prompt strategies
Key Concepts:
- Zero-shot prompting: Getting results without examples
- Few-shot prompting: Learning from examples
- Chain-of-thought: Step-by-step reasoning
- Prompt templates: Reusable prompt structures
Prompt Engineering Techniques:
# Example: Structured prompt for data analysis
prompt_template = """
You are a data scientist analyzing the following dataset:
{dataset_description}
Please provide:
1. Key insights from the data
2. Potential patterns or anomalies
3. Recommendations for further analysis
Dataset sample:
{sample_data}
Analysis:
"""
# C# implementation with semantic kernel
using Microsoft.SemanticKernel;
var kernel = Kernel.CreateBuilder()
.AddOpenAIChatCompletion("gpt-3.5-turbo", apiKey)
.Build();
var prompt = """
Analyze this data and provide insights:
{{$data}}
""";Learning Goals:
- Implement real-time data streaming
- Build event-driven architectures
- Handle high-throughput data processing
Streaming Technologies:
- Apache Kafka: Distributed streaming platform
- Azure Event Hubs: Cloud-based event streaming
- Redis Streams: Lightweight streaming
- SignalR: Real-time web functionality
Implementation Examples:
// C# - Real-time data processing with SignalR
public class DataStreamHub : Hub
{
public async Task JoinDataStream(string streamName)
{
await Groups.AddToGroupAsync(Context.ConnectionId, streamName);
}
public async Task ProcessData(DataPoint data)
{
var result = await _mlModel.PredictAsync(data);
await Clients.Group("predictions").SendAsync("NewPrediction", result);
}
}# Python - Kafka streaming with ML inference
from kafka import KafkaConsumer, KafkaProducer
import json
import numpy as np
consumer = KafkaConsumer('data-stream',
bootstrap_servers=['localhost:9092'],
value_deserializer=lambda x: json.loads(x.decode('utf-8')))
for message in consumer:
data = np.array(message.value['features'])
prediction = model.predict(data.reshape(1, -1))
# Send prediction to output streamLearning Goals:
- Design custom neural network architectures
- Understand deep learning principles
- Implement specialized layers and functions
Architecture Patterns:
- Feedforward Networks: Basic neural networks
- Convolutional Networks: Image processing
- Recurrent Networks: Sequential data
- Transformer Networks: Attention mechanisms
Custom Implementation:
# PyTorch custom neural network
import torch
import torch.nn as nn
import torch.nn.functional as F
class CustomNNP(nn.Module):
def __init__(self, input_dim, hidden_dims, output_dim):
super(CustomNNP, self).__init__()
self.layers = nn.ModuleList()
prev_dim = input_dim
for hidden_dim in hidden_dims:
self.layers.append(nn.Linear(prev_dim, hidden_dim))
self.layers.append(nn.BatchNorm1d(hidden_dim))
self.layers.append(nn.Dropout(0.2))
prev_dim = hidden_dim
self.output = nn.Linear(prev_dim, output_dim)
def forward(self, x):
for layer in self.layers:
if isinstance(layer, nn.Linear):
x = F.relu(layer(x))
else:
x = layer(x)
return self.output(x)Learning Goals:
- Implement in-memory data structures
- Understand caching strategies
- Build high-performance data access layers
Technologies:
- Redis: In-memory data structure store
- MemoryDB: Amazon's Redis-compatible service
- Hazelcast: Distributed in-memory computing
- System.Memory: .NET memory management
Implementation:
// C# - In-memory caching with IMemoryCache
using Microsoft.Extensions.Caching.Memory;
public class InMemoryMLCache
{
private readonly IMemoryCache _cache;
public async Task<PredictionResult> GetOrPredict(string key, InputData data)
{
if (_cache.TryGetValue(key, out PredictionResult cachedResult))
return cachedResult;
var result = await _mlModel.PredictAsync(data);
_cache.Set(key, result, TimeSpan.FromMinutes(10));
return result;
}
}Focus: Data engineering, warehousing, and production deployment
Learning Goals:
- Design robust data pipelines
- Implement ETL/ELT processes
- Handle data quality and validation
Data Engineering Stack:
- Apache Airflow: Workflow orchestration
- dbt: Data transformation
- Apache Spark: Big data processing
- Azure Data Factory: Cloud ETL service
Pipeline Implementation:
# Python - Data pipeline with Pandas and validation
import pandas as pd
from great_expectations import DataContext
class DataPipeline:
def __init__(self):
self.context = DataContext()
def extract(self, source):
return pd.read_sql(source.query, source.connection)
def transform(self, df):
# Data cleaning and transformation
df_clean = df.dropna()
df_clean['processed_date'] = pd.Timestamp.now()
return df_clean
def validate(self, df):
# Data quality checks
expectation_suite = self.context.get_expectation_suite("data_quality")
return self.context.run_validation_operator(
"action_list_operator",
assets_to_validate=[df],
run_id="pipeline_run"
)
def load(self, df, destination):
df.to_sql('processed_data', destination, if_exists='append')Learning Goals:
- Design dimensional data models
- Implement star and snowflake schemas
- Build OLAP cubes and analytics
Data Warehouse Technologies:
- SQL Server Analysis Services: Microsoft OLAP solution
- Azure Synapse Analytics: Cloud data warehouse
- Snowflake: Cloud-native data platform
- ClickHouse: Columnar database
Schema Design:
-- Star schema example for sales analytics
CREATE TABLE FactSales (
SaleKey BIGINT IDENTITY(1,1) PRIMARY KEY,
DateKey INT FOREIGN KEY REFERENCES DimDate(DateKey),
ProductKey INT FOREIGN KEY REFERENCES DimProduct(ProductKey),
CustomerKey INT FOREIGN KEY REFERENCES DimCustomer(CustomerKey),
SalesAmount DECIMAL(10,2),
Quantity INT,
UnitPrice DECIMAL(10,2)
);
CREATE TABLE DimProduct (
ProductKey INT IDENTITY(1,1) PRIMARY KEY,
ProductID NVARCHAR(50),
ProductName NVARCHAR(255),
Category NVARCHAR(100),
SubCategory NVARCHAR(100)
);Learning Goals:
- Deploy ML models to production
- Implement CI/CD for ML projects
- Monitor model performance
MLOps Tools:
- MLflow: ML lifecycle management
- Kubeflow: Kubernetes-native ML workflows
- Azure ML: Cloud ML platform
- Docker: Containerization
Deployment Example:
# Dockerfile for ML model deployment
FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /src
COPY ["MLModelAPI/MLModelAPI.csproj", "MLModelAPI/"]
RUN dotnet restore "MLModelAPI/MLModelAPI.csproj"
COPY . .
WORKDIR "/src/MLModelAPI"
RUN dotnet build "MLModelAPI.csproj" -c Release -o /app/build
RUN dotnet publish "MLModelAPI.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=build /app/publish .
ENTRYPOINT ["dotnet", "MLModelAPI.dll"]Learning Goals:
- Integrate all learned concepts
- Build a complete AI application
- Prepare for production deployment
Final Project: End-to-end AI application with:
- Real-time data ingestion
- ML model inference
- Interactive dashboard
- Data warehousing backend
csharp-ai-study/
├── src/
│ ├── CSharp.AI.Core/ # Core AI libraries and utilities
│ │ ├── Models/ # ML models and algorithms
│ │ ├── Data/ # Data access and processing
│ │ └── Utils/ # Helper utilities
│ ├── CSharp.AI.Web/ # Web API and interfaces
│ │ ├── Controllers/ # API controllers
│ │ ├── Services/ # Business logic
│ │ └── Hubs/ # SignalR hubs for real-time
│ ├── Python.AI.Scripts/ # Python analysis scripts
│ │ ├── data_processing/ # ETL and data cleaning
│ │ ├── model_training/ # ML model training
│ │ └── visualization/ # Plotting and dashboards
│ └── Streaming.Services/ # Real-time processing
│ ├── Kafka/ # Kafka consumers/producers
│ └── EventProcessing/ # Event handling logic
├── data/
│ ├── raw/ # Raw datasets
│ ├── processed/ # Cleaned and transformed data
│ └── models/ # Trained model artifacts
├── notebooks/ # Jupyter notebooks for exploration
├── tests/ # Unit and integration tests
├── docs/ # Documentation and guides
└── docker/ # Container configurations
# Install .NET SDK
wget https://dot.net/v1/dotnet-install.sh
chmod +x dotnet-install.sh
./dotnet-install.sh --channel 8.0
# Essential NuGet packages
dotnet add package Microsoft.ML
dotnet add package Microsoft.ML.Vision
dotnet add package Microsoft.ML.TensorFlow
dotnet add package Math.NET.Numerics
dotnet add package Accord.MachineLearning# Create virtual environment
python -m venv ai-study-env
source ai-study-env/bin/activate # Linux/Mac
# ai-study-env\Scripts\activate # Windows
# Install essential packages
pip install numpy pandas scikit-learn tensorflow torch
pip install matplotlib seaborn plotly bokeh
pip install jupyter notebook jupyterlab
pip install great-expectations mlflow
pip install kafka-python redis# Docker containers for development
docker run -d --name redis-ai -p 6379:6379 redis:alpine
docker run -d --name postgres-ai -p 5432:5432 -e POSTGRES_PASSWORD=password postgres:13
docker run -d --name kafka-ai -p 9092:9092 confluentinc/cp-kafka:latest- C# Dev Kit
- Python
- Jupyter
- Docker
- GitLens
- Thunder Client (API testing)
- ML.NET Model Builder
- Python Tools for Visual Studio
- Azure Tools
- Iris Dataset: Classic flower classification (150 samples, 4 features)
- Wine Quality: Predict wine quality based on chemical properties
- Titanic: Passenger survival prediction
- MNIST: Handwritten digit recognition (28x28 images)
- Boston Housing: Predict house prices (506 samples, 13 features)
- California Housing: Larger housing dataset with geographic data
- Bike Sharing: Predict bike rental demand based on weather/time
- Stock Prices: Historical stock market data
- Weather Data: Temperature, humidity, pressure over time
- Sales Forecasting: Retail sales data with seasonality
- CIFAR-10: 32x32 color images in 10 classes (60,000 images)
- Fashion-MNIST: Clothing items classification
- Cats vs Dogs: Binary image classification
- IMDB Movie Reviews: Sentiment analysis (50,000 reviews)
- 20 Newsgroups: Text classification across topics
- Twitter Sentiment: Social media sentiment analysis
- MovieLens: Movie ratings and recommendations
- Amazon Product Reviews: E-commerce recommendation data
- Last.fm: Music listening and recommendation data
- ImageNet: 1.2M images across 1,000 categories
- COCO: Object detection and segmentation
- Open Images: Google's large-scale image dataset
- NYC Taxi Data: Large-scale transportation data
- Wikipedia Clickstream: Web traffic patterns
- Twitter Stream: Real-time social media data
- Medical Imaging: X-rays, MRI scans (with proper permissions)
- Financial Data: High-frequency trading data
- IoT Sensor Data: Industrial sensor readings
Goal: Build a C# application that analyzes CSV data and provides insights
Features:
- File upload and parsing
- Statistical analysis
- Basic visualizations
- Export reports
Technologies: C#, Math.NET Numerics, OxyPlot
Goal: Create an end-to-end machine learning pipeline
Features:
- Data preprocessing
- Model training and evaluation
- Hyperparameter tuning
- Model persistence
Technologies: Python, scikit-learn, pandas, joblib
Goal: Build a real-time sentiment analysis dashboard
Features:
- Live data streaming (Twitter API simulation)
- Sentiment analysis with prompt engineering
- Real-time chart updates
- Alert system for sentiment changes
Technologies: C# (SignalR), Python (sentiment analysis), JavaScript (frontend)
Goal: Interactive neural network design and training tool
Features:
- Visual network architecture builder
- Real-time training visualization
- Parameter adjustment controls
- Performance metrics display
Technologies: Python (PyTorch/TensorFlow), Streamlit/Dash
Goal: Build a complete data warehousing solution
Features:
- ETL pipeline automation
- Dimensional modeling
- OLAP cube creation
- Business intelligence dashboard
Technologies: SQL Server/PostgreSQL, dbt, Power BI/Tableau
Goal: Deploy a scalable AI microservice
Features:
- RESTful API for ML predictions
- Model versioning and A/B testing
- Performance monitoring
- Auto-scaling capabilities
Technologies: .NET Core, Docker, Kubernetes, MLflow
Goal: Integrate all learned concepts into a comprehensive platform
System Architecture:
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Data Sources │ │ Stream Proc. │ │ AI Models │
│ • APIs │────│ • Kafka │────│ • C# ML.NET │
│ • Databases │ │ • Redis │ │ • Python ML │
│ • Files │ │ • Event Hubs │ │ • Deep Learning │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │ │
└───────────────────────┼───────────────────────┘
│
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Data Storage │ │ Web API │ │ Visualization │
│ • Data Lake │ │ • .NET Core │ │ • React/Blazor│
│ • Data Warehouse│ │ • FastAPI │ │ • D3.js │
│ • Vector DB │ │ • GraphQL │ │ • Plotly │
└─────────────────┘ └─────────────────┘ └─────────────────┘
Features:
- Multi-source data ingestion
- Real-time and batch processing
- Multiple ML model deployment
- Interactive dashboards
- Natural language querying
- Automated insights generation
-
C# AI Development:
- "Hands-On Machine Learning with ML.NET" by Jarred Capellman
- "C# Machine Learning Projects" by Yoon Hyup Hwang
-
Python AI/ML:
- "Hands-On Machine Learning" by Aurélien Géron
- "Python Machine Learning" by Sebastian Raschka
-
Data Engineering:
- "Designing Data-Intensive Applications" by Martin Kleppmann
- "The Data Warehouse Toolkit" by Ralph Kimball
- Microsoft Learn: AI and Machine Learning paths
- Coursera: Machine Learning by Andrew Ng
- edX: MIT Introduction to Machine Learning
- Udacity: Machine Learning Engineer Nanodegree
- Pluralsight: C# and Python AI tracks
- Medium: Towards Data Science publication
- Reddit: r/MachineLearning, r/datascience, r/csharp
- Stack Overflow: AI/ML tagged questions
- GitHub: Awesome Machine Learning repositories
- Microsoft Build: Annual developer conference
- PyCon: Python community conference
- NeurIPS: Premier AI research conference
- Strata Data Conference: Data science and engineering
-
Cloud Platforms:
- Azure Machine Learning
- AWS SageMaker
- Google Cloud AI Platform
-
Development Tools:
- Jupyter Notebooks
- Apache Zeppelin
- Databricks
-
Collaboration:
- GitHub/GitLab
- MLflow
- Weights & Biases
We welcome contributions from learners and experts alike! Here's how you can contribute:
- Learning Resources: Add new tutorials, examples, or explanations
- Code Samples: Contribute working code examples for different concepts
- Dataset Recommendations: Suggest new datasets for practice
- Project Ideas: Propose new hands-on projects
- Bug Fixes: Fix issues in existing code samples
- Performance Improvements: Optimize existing implementations
- New Features: Add new functionality to example projects
- Documentation: Improve existing documentation
- Fork the repository
- Create a feature branch:
git checkout -b feature/your-feature-name - Make your changes following our coding standards
- Test your changes thoroughly
- Submit a pull request with a clear description
C# Code:
// Use PascalCase for public members
public class DataProcessor
{
// Use camelCase for private fields
private readonly ILogger _logger;
// Add XML documentation for public methods
/// <summary>
/// Processes the input data and returns predictions
/// </summary>
/// <param name="inputData">The data to process</param>
/// <returns>Prediction results</returns>
public async Task<PredictionResult> ProcessAsync(InputData inputData)
{
// Implementation
}
}Python Code:
# Follow PEP 8 style guidelines
class DataProcessor:
"""Process data for machine learning tasks."""
def __init__(self, model_path: str):
"""Initialize the processor with model path."""
self.model_path = model_path
self.model = None
def process_data(self, data: pd.DataFrame) -> np.ndarray:
"""Process input data and return predictions.
Args:
data: Input DataFrame with features
Returns:
Array of predictions
"""
# Implementation
pass- Use clear, concise language
- Include code examples for complex concepts
- Add links to relevant resources
- Ensure examples are tested and working
- Be respectful and inclusive
- Help others learn and grow
- Share knowledge generously
- Provide constructive feedback
- Issues: Report bugs or request features via GitHub Issues
- Discussions: Join community discussions in GitHub Discussions
- Discord: Join our learning community (link in repository)
- Office Hours: Weekly virtual help sessions (see calendar)
Contributors will be recognized in:
- README contributors section
- Annual contributor awards
- Speaking opportunities at meetups
- Recommendation letters for outstanding contributors
License: This project is licensed under the GNU General Public License v3.0 - see the LICENSE file for details.
Disclaimer: This educational resource is for learning purposes. Always follow best practices and security guidelines when implementing AI solutions in production environments.