ET.CodeFixer-Agent

Purpose

ET.CodeFixer-Agent is an autonomous DevOps platform that integrates SonarQube, Jira, and Git workflows (code under ET.CodeFixer.Agent/). It uses AI and machine learning to automatically detect SonarQube findings such as bugs, code smells, and vulnerabilities, create corresponding Jira tickets, and generate intelligent code fixes through autonomous agents.

Getting Started

Clone the repository
Configure services in appsettings.json
Set up Cosmos DB and Redis (optional)
Run the application - it will auto-execute based on your configuration
Monitor logs for pipeline execution and any issues
Use API endpoints for manual control when needed

Key Features

Autonomous Automation Pipeline: Automatically processes SonarQube bugs, code smells, and vulnerabilities → Jira tickets → Git fixes
AI-Powered Code Fixes: Uses LLM integration for intelligent code generation and fixes
Machine Learning Intelligence: Learns from past fixes to improve future recommendations
Multi-Agent Architecture: Specialized agents for different aspects of the DevOps workflow
Redis Working Memory: Fast caching and agent communication
Cosmos DB Persistence: Long-term storage of decisions, experiences, and metrics

Current Project Codebase Structure

ET.CodeFixer.Agent/
├── ET.CodeFixer.Agent.Api/              # Web API layer
│   ├── Controllers/                           # API endpoints
│   ├── Program.cs                            # Application entry point
│   └── appsettings.json                      # Configuration settings
├── ET.CodeFixer.Agent.Application/      # Application layer
│   └── Services/
│       ├── Agents/                           # Autonomous agent implementations
│       ├── BackgroundServices/               # Background processing services
│       ├── Integration/                      # External service integrations
│       ├── MachineLearning/                  # ML.NET implementations
│       ├── Pipeline/                         # Pipeline orchestration
│       └── Utilities/                        # Utility services
├── ET.CodeFixer.Agent.Core/            # Domain layer
│   ├── Entities/                             # Domain entities and DTOs
│   ├── Enums/                                # Enumerations
│   ├── Interfaces/                           # Service contracts
│   └── Configuration/                        # Configuration models
├── ET.CodeFixer.Agent.Infrastructure/  # Infrastructure layer
│   ├── Clients/                              # External API clients
│   ├── Data/                                 # Database contexts
│   ├── Factories/                            # Factory implementations
│   ├── Repositories/                         # Data access implementations
│   └── Services/                             # Infrastructure services
└── tests/                                    # Test projects
    ├── Integration/                          # Integration tests
    └── Unit/                                 # Unit tests

Technology Used

Core Technologies

.NET 9.0: Latest .NET framework with enhanced performance
ASP.NET Core Web API: RESTful API framework
Entity Framework Core: ORM for database operations
Clean Architecture: Separation of concerns with layered architecture

External Integrations

Azure DevOps API: Git operations and pull request management
SonarCloud/SonarQube API: Code quality analysis and issue detection
Jira REST API: Issue tracking and ticket management (Atlassian Cloud)
OpenAI API: GPT-5 integration for intelligent code generation
Microsoft Semantic Kernel: AI orchestration framework

Data Storage & Caching

Azure Cosmos DB: NoSQL database for scalable data storage (EastAsia region)
Redis: In-memory caching and agent working memory (optional)
ML.NET: Machine learning model training and predictions

Package Management & Build

NuGet: Package management with Azure DevOps Artifacts integration
Build Validation: Automated build verification for code changes

Resilience & Observability

Serilog: Structured logging with Console and File sinks
Background Services: .NET hosted services for autonomous operation
Health Checks: Built-in health monitoring endpoints

Development Tools

xUnit: Unit testing framework
Docker: Containerized deployment with multi-stage builds
Visual Studio: IDE support with solution file

Which Component Does What

API Layer (`ET.CodeFixer.Agent.Api`)

MLController: Endpoints for machine learning operations and model management
PipelineController: Pipeline orchestration and manual trigger endpoints

Application Services

Agents (`Application/Services/Agents/`)

BaseAutonomousAgent: Foundation for all autonomous agents with decision-making capabilities
SonarToJiraAgent: Processes SonarQube issues and creates corresponding Jira tickets
JiraToGitAgent: Takes Jira tickets and generates code fixes with pull requests

Background Services (`Application/Services/BackgroundServices/`)

AutonomousPipelineService: Main orchestrator for autonomous pipeline execution
MLTrainingBackgroundService: Handles periodic ML model training and retraining
HealthMonitorService: Monitors system health and agent performance
AgentLifecycleService: Manages agent lifecycle, memory cleanup, and resource optimization

Integration Services (`Application/Services/Integration/`)

SonarToJiraService: Bridges SonarQube analysis results to Jira ticket creation
JiraToGitFixService: Converts Jira tickets into actionable code fixes
PipelineOrchestrationService: Coordinates the entire pipeline flow

Machine Learning (`Application/Services/MachineLearning/`)

MLModelTrainingService: Trains predictive models using historical data
FixSuccessPredictor: Predicts success probability of proposed fixes
MLTrainingDataGenerator: Generates and prepares training datasets
PredictionEngine: Provides real-time predictions for decision-making

Infrastructure Layer

Clients (`Infrastructure/Clients/`)

AzureDevOpsClient: Handles Git operations, repository management, and PR creation
SonarQubeClient: Interfaces with SonarQube API for issue retrieval
JiraClient: Manages Jira ticket operations (create, update, query)
LLMClient: Integrates with AI services for code generation

Data & Services (`Infrastructure/Data/`, `Infrastructure/Services/`)

DevOpsCosmosDbContext: Entity Framework context for Cosmos DB operations
AgentMemoryService: Hybrid Redis/Cosmos DB memory management for agents
CosmosDbInitializerService: Database initialization and container setup

Core Domain (`ET.CodeFixer.Agent.Core`)

Entities: Domain models (AgentExperience, DecisionLog, PerformanceMetrics)
Interfaces: Service contracts and abstractions
Configuration: Strongly-typed configuration models

How to Run This App

Prerequisites

.NET 9.0 SDK
Azure Cosmos DB instance (configured for EastAsia region)
Redis server (optional - graceful fallback available)
Access to SonarCloud/SonarQube, Jira (Atlassian), and Azure DevOps instances
Valid API tokens for all external services
Docker (optional - for containerized deployment)

1. Configuration Setup

Update appsettings.json with your service credentials:

{
  "SonarQube": {
    "Url": "https://sonarcloud.io",
    "Token": "your-sonarcloud-token",
    "ProjectKey": "your-project-key",
    "AllowedSeveritiesStrings": ["Blocker", "High"]
  },
  "Jira": {
    "BaseUrl": "https://your-company.atlassian.net",
    "ApiToken": "your-jira-api-token",
    "Email": "your-email@company.com",
    "ProjectKey": "SP",
    "StatusDeveloping": "Developing",
    "StatusCodeReview": "Code Review"
  },
  "Git": {
    "Provider": "AzureDevOps",
    "RepositoryUrl": "https://dev.azure.com/org/project/_git/repo",
    "Token": "your-azure-devops-pat",
    "Organization": "your-org",
    "Project": "your-project",
    "AzureRepository": "repository-name"
  },
  "NuGet": {
    "Sources": [
      {
        "Name": "ETCodeFixer",
        "Url": "https://pkgs.dev.azure.com/org/_packaging/feed/nuget/v3/index.json",
        "Username": "DevOpsAutomation",
        "Token": "your-nuget-token",
        "IsDefault": true
      }
    ],
    "EnableBuildValidation": true,
    "BuildTimeoutMinutes": 10
  },
  "CosmosDbSettings": {
    "EndPoints": [
      {
        "RegionName": "EastAsia",
        "Url": "https://your-cosmosdb.documents.azure.com:443/",
        "AuthenticationKey": "your-cosmosdb-key"
      }
    ]
  }
}

2. Build and Run

# Build the solution
dotnet build

# Run the API
dotnet run --project ET.CodeFixer.Agent.Api

# Or use Visual Studio / VS Code to run with debugging

3. Docker Deployment (Alternative)

# Build Docker image
docker build -t et-codefixer-automation .

# Run with Docker
docker run -p 8080:8080 -p 8081:8081 \
  -e ASPNETCORE_ENVIRONMENT=Production \
  --name devops-automation \
  et-codefixer-automation

# Check health
curl http://localhost:8080/health

4. Function Execution Modes

A. Autonomous Mode (Automatic Execution) 🚀

The system runs automatically with configurable intervals:

Auto-Execution Configuration (appsettings.json):

"AutonomousService": {
  "EnableBackgroundProcessing": true,
  "ProcessingIntervalMinutes": 30,
  "MaxConcurrentTickets": 5,
  "SonarToJiraPipelineIntervalMinutes": 45,
  "JiraToGitPipelineIntervalMinutes": 30,
  "FullPipelineIntervalMinutes": 60,
  "MLTrainingIntervalDays": 7,
  "HealthCheckIntervalMinutes": 5,
  "EnableAutonomousOperation": true,
  "EnableMLTraining": true,
  "EnableHealthMonitoring": true,
  "EnableAgentLifecycle": true
}

Background Services:

AutonomousPipelineService: Processes pipeline every 30 minutes
MLTrainingBackgroundService: Retrains models weekly
HealthMonitorService: Health checks every 5 minutes
AgentLifecycleService: Manages agent memory and lifecycle

B. Manual Mode (API Endpoints)

Trigger specific functions via REST endpoints:

# Trigger full pipeline
POST /api/pipeline/trigger-full

# Process SonarQube to Jira only
POST /api/pipeline/sonar-to-jira

# Process Jira to Git fixes only
POST /api/pipeline/jira-to-git

# Train ML models
POST /api/ml/train-models

# Get system health
GET /api/pipeline/health

5. Testing

# Run unit tests
dotnet test tests/ET.CodeFixer.Agent.Tests.Unit/

# Run integration tests (requires configured services)
dotnet test tests/ET.CodeFixer.Agent.Tests.Integration/

How to Configure

1. Service Connections

Configure external service connections in appsettings.json:

SonarQube Configuration

"SonarQube": {
  "Url": "https://sonarcloud.io",
  "Token": "your-sonarqube-user-token",
  "ProjectKey": "your-sonarqube-project-key",
  "AllowedSeverities": ["Blocker", "High", "Medium", "Low"]
}

Jira Configuration

"Jira": {
  "BaseUrl": "https://your-company.atlassian.net",
  "ApiToken": "your-jira-api-token",
  "Email": "your-email@company.com",
  "ProjectKey": "DEV",
  "StatusDeveloping": "In Progress",
  "StatusCodeReview": "Code Review"
}

Azure DevOps Configuration

"Git": {
  "Provider": "AzureDevOps",
  "RepositoryUrl": "https://dev.azure.com/org/project/_git/repository",
  "Token": "your-personal-access-token",
  "Organization": "your-organization",
  "Project": "your-project",
  "AzureRepository": "repository-name"
}

2. Database Configuration

Cosmos DB Setup

"CosmosDbSettings": {
  "EndPoints": [
    {
      "RegionName": "EastAsia",
      "Url": "https://your-cosmosdb.documents.azure.com:443/",
      "AuthenticationKey": "your-primary-key"
    }
  ],
  "Collections": [
    {
      "Name": "AgentExperiences",
      "DatabaseId": "ETCodeFixerDb",
      "CollectionId": "AgentExperiences",
      "CollectionPartitionKeyName": "partitionKey",
      "CollectionDefaultTimeToLive": -1
    },
    {
      "Name": "DecisionLogs",
      "DatabaseId": "ETCodeFixerDb",
      "CollectionId": "DecisionLogs",
      "CollectionPartitionKeyName": "partitionKey",
      "CollectionDefaultTimeToLive": -1
    },
    {
      "Name": "PerformanceMetrics",
      "DatabaseId": "ETCodeFixerDb",
      "CollectionId": "PerformanceMetrics",
      "CollectionPartitionKeyName": "partitionKey",
      "CollectionDefaultTimeToLive": -1
    },
    {
      "Name": "TrainingData",
      "DatabaseId": "ETCodeFixerDb",
      "CollectionId": "TrainingData",
      "CollectionPartitionKeyName": "partitionKey",
      "CollectionDefaultTimeToLive": -1
    },
    {
      "Name": "MLModelMetrics",
      "DatabaseId": "ETCodeFixerDb",
      "CollectionId": "MLModelMetrics",
      "CollectionPartitionKeyName": "partitionKey",
      "CollectionDefaultTimeToLive": -1
    },
    {
      "Name": "JiraAuditLogs",
      "DatabaseId": "ETCodeFixerDb",
      "CollectionId": "JiraAuditLogs",
      "CollectionPartitionKeyName": "ProjectKey",
      "CollectionDefaultTimeToLive": -1
    }
  ]
}

Redis Configuration (Optional)

"Redis": {
  "ConnectionString": "localhost:6379"
}

3. AI/LLM Configuration

"LLM": {
  "Provider": "OpenAI",
  "Temperature": 0.1,
  "MaxTokens": 100000,
  "OpenAI": {
    "ApiKey": "your-openai-api-key",
    "Model": "gpt-5"
  }
},
"Processing": {
  "MaxRetriesPerTicket": 30
}

4. Auto-Execution Timing Configuration

Fine-tune autonomous operation intervals:

"AutonomousService": {
  "EnableBackgroundProcessing": true,          // Enable/disable auto-execution
  "ProcessingIntervalMinutes": 30,            // Main processing cycle
  "MaxConcurrentTickets": 3,                  // Concurrent ticket processing limit
  "EnableMLPredictions": true,                // Enable ML predictions
  "FullPipelineIntervalMinutes": 60,          // Complete pipeline cycle
  "SonarToJiraPipelineIntervalMinutes": 45,   // SonarQube to Jira frequency
  "JiraToGitPipelineIntervalMinutes": 30,     // Jira to Git frequency
  "MLTrainingIntervalDays": 7,                // ML model retraining frequency
  "HealthCheckIntervalMinutes": 5,            // Health monitoring frequency
  "ModelWarmupDelayMinutes": 2,               // ML model warmup delay
  "EnableAutonomousOperation": true,          // Enable autonomous operation
  "EnableMLTraining": true,                   // Enable ML training
  "EnableHealthMonitoring": true,             // Enable health monitoring
  "EnableAgentLifecycle": true,               // Enable agent lifecycle management
  "MaxFailureRetriesBeforePause": 3,          // Failure threshold before pause
  "PauseAfterFailuresMinutes": 120            // Pause duration after failures
}

5. Machine Learning Configuration

"MLTraining": {
  "MinimumTrainingSamples": 50,               // Minimum samples required for training
  "MinimumSamplesForRetraining": 25,          // Minimum samples for retraining
  "ValidationSplitRatio": 0.2,                // Validation data percentage
  "MinimumAccuracy": 0.75,                    // Minimum model accuracy threshold
  "MinimumF1Score": 0.70,                     // Minimum F1 score threshold
  "RetrainingIntervalDays": 7,                // Model retraining frequency
  "ModelStoragePath": "MLModels"              // Model storage directory
},
"MLModelStorage": {
  "StoragePath": "MLModels",                   // Model storage path
  "MaxModelVersionsToKeep": 10,               // Maximum model versions to retain
  "EnableVersionCleanup": true,               // Enable automatic version cleanup
  "ModelRetentionPeriodDays": 90              // Model retention period
}

6. Logging Configuration

"Serilog": {
  "MinimumLevel": {
    "Default": "Information",
    "Override": {
      "Microsoft": "Warning",
      "System": "Warning"
    }
  },
  "WriteTo": [
    { "Name": "Console" },
    {
      "Name": "File",
      "Args": {
        "path": "logs/et-codefixer-.txt",
        "rollingInterval": "Day",
        "retainedFileCountLimit": 7
      }
    }
  ]
}

Monitoring and Logging

Request Logging

The system automatically logs all OpenAI requests and responses for debugging and analysis:

Location: logs/openai-requests/
Structure: Organized by Jira ticket (e.g., SP-3208/, SP-3205-correction-1/)
Files: JSON request/response pairs with timestamps
Usage: Helps track AI interactions and troubleshoot generation issues

System Logs

Daily rotation: logs/et-codefixer-YYYYMMDD.txt
Structured logging: JSON format with Serilog
Log levels: Debug, Information, Warning, Error
Enrichment: Machine name, thread ID, and log context

Health Monitoring

Endpoint: GET /health
Docker: Built-in health checks every 30 seconds
Background: Automated health monitoring every 5 minutes
Metrics: Agent performance, system health, pipeline status

Architecture Features

Multi-Agent System

The platform uses specialized autonomous agents:

SonarToJiraAgent: Converts SonarQube issues to Jira tickets
JiraToGitAgent: Generates code fixes from Jira tickets
BaseAutonomousAgent: Shared decision-making and memory capabilities

Machine Learning Integration

Fix Success Prediction: ML.NET models predict fix success probability
Training Data Collection: Automated collection of historical fix outcomes
Model Retraining: Weekly automated retraining with new data
Performance Tracking: Model accuracy and F1 score monitoring

Memory and Context Management

Agent Memory: Hybrid Redis/Cosmos DB for short and long-term memory
Decision Logging: All agent decisions logged for analysis
Context Awareness: Agents maintain context across pipeline executions

Recent Activity

The system shows active development with recent processing of multiple Jira tickets:

SP-3208, SP-3207, SP-3206, SP-3205 with corrections and iterations
OpenAI request logging shows active AI-powered code generation
Docker containerization with health checks implemented
Multi-stage build process for production deployment

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
ET.CodeFixer.Agent.Api		ET.CodeFixer.Agent.Api
ET.CodeFixer.Agent.Application		ET.CodeFixer.Agent.Application
ET.CodeFixer.Agent.Core		ET.CodeFixer.Agent.Core
ET.CodeFixer.Agent.Infrastructure		ET.CodeFixer.Agent.Infrastructure
tests		tests
.dockerignore		.dockerignore
.env.template		.env.template
.gitignore		.gitignore
Directory.Build.props		Directory.Build.props
Dockerfile		Dockerfile
ET.CodeFixer.Agent.sln		ET.CodeFixer.Agent.sln
NuGet.config		NuGet.config
README.md		README.md
docker-compose.yml		docker-compose.yml

Folders and files

Latest commit

History

Repository files navigation