[D] documentaion of Operators

[michalharakal]

KSP Gradle Module Structure for SKaiNET

Based on the analysis of the project structure, here's a mermaid diagram visualizing the KSP-related Gradle modules and their relationships:

graph TD
    %% Core Modules
    Core["skainet-lang-core<br/>📦 Multiplatform Core<br/>• Tensor operations<br/>• Common interfaces"]
    Models["skainet-lang-models<br/>📦 Model Definitions<br/>• Neural network models<br/>• MNIST, CNN implementations"]
    
    %% KSP Infrastructure
    Annotations["skainet-lang-ksp-annotations<br/>📝 Annotation Definitions<br/>• @NotImplemented<br/>• @InProgress<br/>• Backend status markers"]
    
    Processor["skainet-lang-ksp-processor<br/>🔧 KSP Processor<br/>• SymbolProcessor implementation<br/>• Code generation logic<br/>• kotlinpoet integration"]
    
    ExportOps["skainet-lang-export-ops<br/>🚀 Ops Renderer/Exporter<br/>• Uses KSP for doc generation<br/>• Operator metadata extraction<br/>• JSON output generation"]
    
    %% Generated Artifacts
    JSON["operators.json<br/>📄 Generated Metadata<br/>• Function signatures<br/>• Backend status<br/>• Version info"]
    
    Generated["Generated Code<br/>⚡ KSP Output<br/>• Documentation fragments<br/>• AsciiDoc files<br/>• Status matrices"]
    
    %% Dependencies and Flow
    Core --> Annotations
    Models --> Core
    
    Processor --> Annotations
    
    ExportOps --> Core
    ExportOps --> Annotations
    ExportOps --> |"ksp dependency"| Processor
    
    %% KSP Processing Flow
    Processor --> |"processes"| JSON
    Processor --> |"generates"| Generated
    
    %% Build Configuration
    subgraph "Build Process"
        KSP["KSP Plugin<br/>🔨 Kotlin Symbol Processing"]
        Gradle["Gradle Tasks<br/>📋 Build Orchestration"]
        
        KSP --> Processor
        Gradle --> KSP
        Gradle --> ExportOps
    end
    
    %% Notes
    classDef coreModule fill:#e1f5fe
    classDef kspModule fill:#f3e5f5
    classDef generatedModule fill:#e8f5e8
    classDef errorModule fill:#ffebee
    
    class Core,Models coreModule
    class Annotations,Processor,ExportOps kspModule
    class JSON,Generated generatedModule
    class miKrograd errorModule

Loading

Module Breakdown

Core Modules

• skainet-lang-core: Multiplatform core module with tensor operations and common interfaces
• skainet-lang-models: Contains neural network model implementations (MNIST, CNN, etc.)

KSP Infrastructure

• skainet-lang-ksp-annotations: Defines annotations for backend status tracking

  • @NotImplemented(backends: Array<String>)
  • @InProgress(backends: Array<String>, owner: String, issue: String)
  • Multiplatform with JVM target

• skainet-lang-ksp-processor: KSP processor implementation

  • Uses KotlinPoet for code generation
  • Processes annotations to extract metadata
  • Issue: References legacy :miKrograd-annotations instead of :skainet-lang-ksp-annotations

• skainet-lang-export-ops: Ops renderer/documentation generator

  • Consumes KSP-generated metadata
  • Exports operator documentation
  • Generates operators.json with function signatures and backend status
  • Issue: Also references legacy :miKrograd projects

Generated Artifacts

• operators.json: Machine-readable metadata about operators
• Generated Documentation: AsciiDoc fragments and status matrices

Key Issues Identified

1. Naming Inconsistency: Build files reference :miKrograd-* modules but actual directories are named skainet-lang-*

2. Incomplete Module Registration: settings.gradle.kts only includes 2 modules but 5 modules exist in the filesystem

3. Broken Dependencies: KSP processor cannot find its annotation dependencies due to incorrect module references

KSP Workflow

sequenceDiagram
    participant Gradle
    participant KSP
    participant Processor
    participant Annotations
    participant ExportOps
    participant Output

    Gradle->>KSP: Execute kspJvm task
    KSP->>Processor: Load OperatorDocProcessor
    Processor->>Annotations: Read @NotImplemented/@InProgress
    Processor->>Processor: Scan operator interfaces
    Processor->>Output: Generate operators.json
    ExportOps->>Output: Read operators.json
    ExportOps->>Output: Generate AsciiDoc fragments

Loading

Required Fixes

1. Update settings.gradle.kts to include all modules:

include("skainet-lang:skainet-lang-ksp-annotations")
include("skainet-lang:skainet-lang-ksp-processor") 
include("skainet-lang:skainet-lang-export-ops")

2. Fix module references in build files:

   • Change :miKrograd-annotations → :skainet-lang:skainet-lang-ksp-annotations
   • Change :miKrograd-processor → :skainet-lang:skainet-lang-ksp-processor

3. Establish proper dependency chain for the KSP documentation pipeline