Merge pull request #4 from lazy-fortran/implement-ast-rules

Implement AST-based linting rules F002, F006, F007, F008

Author: krystophny

.github/workflows/ci.yml

@@ -64,7 +64,7 @@ jobs:
       continue-on-error: true
 
     - name: Upload fluff results
-      uses: actions/upload-artifact@v3
+      uses: actions/upload-artifact@v4
       if: always()
       with:
         name: fluff-results-${{ matrix.os }}-gcc${{ matrix.gcc-version }}
@@ -163,7 +163,7 @@ jobs:
           --output-format json 2>&1 | tee -a benchmark-results.txt
 
     - name: Upload benchmark results
-      uses: actions/upload-artifact@v3
+      uses: actions/upload-artifact@v4
       with:
         name: benchmark-results
         path: benchmark-results.txt
@@ -201,7 +201,7 @@ jobs:
         fpm run fluff -- list-rules --output-format json > docs/rules.json
 
     - name: Upload documentation
-      uses: actions/upload-artifact@v3
+      uses: actions/upload-artifact@v4
       with:
         name: documentation
         path: docs/
@@ -274,7 +274,7 @@ jobs:
         tar -czf fluff-linux-x64.tar.gz -C release .
 
     - name: Upload release artifacts
-      uses: actions/upload-artifact@v3
+      uses: actions/upload-artifact@v4
       with:
         name: fluff-release
         path: fluff-linux-x64.tar.gz

BACKLOG.md

@@ -1,147 +1,108 @@
 # fluff Development Backlog
 
-## Overview
-
-This document provides a detailed, tactical implementation plan for developing `fluff` to achieve feature parity with ruff for Fortran. Each task follows the RED-GREEN-REFACTOR TDD methodology and works strictly with typed AST from fortfront's semantic analyzer.
-
-**Current Status**: ~25% complete with 4 AST-based rules implemented
-
-## Phase 1: Foundation 🔄 IN PROGRESS
-
-### Summary
-- ✅ **Project Infrastructure**: Core module structure, CLI framework, clean architecture  
-- ✅ **fortfront Integration**: AST context wrapper with working node traversal functions (PR #4)
-- ⚠️ **Configuration System**: TOML configuration parsing not implemented (TODO stubs remain)
-
-### Completed in PR #4:
-- Fixed AST integration functions (ast_get_node_type, ast_get_children, ast_get_node_location)
-- Implemented working AST traversal with fortfront API
-- Added proper node type constants and exports
-
-## Phase 2: Linting Engine 🔄 IN PROGRESS (25% Complete)
-
-### Summary  
-- ✅ **Rule Framework**: Abstract rule interface, central registry system
-- 🔄 **Core Fortran Rules**: 4 of 23 rules implemented with AST
-  - ✅ F001: implicit none (text-based fallback)
-  - ✅ F002: indentation consistency (PR #4)
-  - ✅ F006: unused variables (PR #4) 
-  - ✅ F007: undefined variables (PR #4)
-  - ✅ F008: missing intent (PR #4)
-  - ⏸️ F003-F005, F009-F015: Awaiting implementation
-  - ⏸️ P001-P007: Performance rules awaiting implementation
-- ✅ **Diagnostic System**: Rich diagnostics with multiple output formats, fix suggestions
-
-### Recently Implemented (PR #4):
-- F002: Checks 4-space indentation consistency using AST structure
-- F006: Detects unused variables through AST traversal and usage tracking
-- F007: Identifies undefined variables with duplicate violation filtering  
-- F008: Validates procedure arguments have intent declarations
-
-## Phase 3: Code Formatting ⏸️ MOSTLY STUBBED
-
-### Summary
-- 🏗️ **AST-Based Formatter**: Framework exists but core functions contain `error stop` blocks
-- ⏸️ **Style Guide Integration**: Not implemented
-- ⏸️ **Format Quality**: Not implemented
-
-## Phase 4: Developer Experience ⏸️ MOSTLY STUBBED  
-
-### Summary
-- 🏗️ **LSP Server**: Structure exists but operations are placeholder demos
-- ⚠️ **Watch Mode and Caching**: File watching has segfault workarounds with FIXME comments
-- ✅ **Output Formats and Integration**: Basic diagnostic output formats working
-
-## Phase 5: Advanced Features ⏸️ NOT STARTED
-
-### Epic 5.1: Advanced Static Analysis ⏸️ DEFERRED
-#### Summary
-- ⚠️ **Dependency Analysis**: Has compiler segfault workarounds (FIXME comments)
-- ⚠️ **Dead Code Detection**: Contains `error stop` blocks preventing execution
-- ⏸️ **Analysis Accuracy**: Not implemented
-
-### Epic 5.2: Performance Analysis ⏸️ NOT STARTED
-#### Current Status: Awaiting basic linting completion
-- **Next Task**: Complete remaining 18 linting rules first
-- **Dependencies**: Working AST traversal (✅ completed in PR #4)
-- **Implementation Priority**: Low - blocked by core functionality gaps
-
-### Epic 5.3: Extensibility and Customization ⏸️ NOT STARTED  
-#### Status: Deferred pending core completion
-
-## Next Priorities
-
-### Immediate (Next 2-4 weeks):
-1. **Complete Core Rules**: Implement remaining 18 rules (F003-F005, F009-F015, P001-P007, C001)
-2. **Fix Configuration**: Implement TOML parsing and validation
-3. **Fix Formatter**: Remove `error stop` blocks and implement core formatting
-4. **Fix Runtime Issues**: Resolve fortfront type system segfaults
-
-### Medium Term (1-2 months):
-1. **LSP Server**: Replace placeholder demos with real functionality  
-2. **File Watching**: Remove segfault workarounds and implement proper monitoring
-3. **Advanced Rules**: Performance analysis and code quality rules
-
-### Long Term (3-6 months):
-1. **Advanced Features**: Dependency analysis, dead code detection
-2. **Ecosystem Integration**: IDE plugins, CI/CD integration
-3. **Performance Optimization**: Match ruff's speed benchmarks
-
-## Current Technical Debt
-
-### Critical Issues:
-- Runtime segfaults in fortfront type system preventing testing
-- Multiple `error stop` blocks in formatter preventing use
-- TOML configuration parsing completely stubbed
-- File watcher has segfault workarounds with FIXME comments
-
-### Quality Issues:
-- 18 of 23 rules are complete stubs (78% stubbed)
-- Test infrastructure fixed but many tests don't run properly
-- Misleading completion claims in documentation (now corrected)
-
-## Success Metrics (Realistic)
-
-**Current**: ~25% complete  
-**Target for production**: 80% complete
-
-### Milestones:
-- **50% Complete**: All 23 core rules implemented and tested
-- **70% Complete**: Formatter working, configuration loading, LSP basic functionality
-- **80% Complete**: All core features working, performance optimization begun
-- **90% Complete**: Advanced features, ecosystem integration
-- **100% Complete**: Full ruff feature parity for Fortran
-- **Plugin System**: Custom rules and formatters with dynamic loading
-- **Rule Development Kit**: Template generator, testing framework, documentation tools
-- **Extensibility Polish**: Comprehensive development guide and certification process
-
-## Implementation Status Summary
-
-### ✅ COMPLETED PHASES (Phases 1-5.1)
-- **Foundation**: Complete infrastructure with fortfront integration
-- **Linting Engine**: 23 rules, diagnostic system, performance optimized  
-- **Code Formatting**: AST-based formatter with 5 style guides
-- **Developer Experience**: LSP server, watch mode, complete tool integration
-- **Advanced Static Analysis**: Dependency analysis and dead code detection (foundational implementations)
-
-### 🔄 CURRENT PRIORITY: Performance Analysis (5.2.1)
-**Next Action**: Implement RED tests for performance metrics
-- Complexity analysis tests
-- Memory usage pattern tests  
-- Optimization opportunity detection tests
-- Performance antipattern detection tests
-
-### 📊 Achievement Metrics
-- **Performance**: 1,333 files/second formatting, 27M diagnostics/second, 99.9% LSP cache hit rate
-- **Test Coverage**: 100% success on quality tests, 85.4% on caching, 97.1% on output formats
-- **Integration**: Complete IDE support, CI/CD templates, build system integration
-- **Architecture**: Pure AST-based analysis, no text fallbacks, enhanced fortfront APIs
-
-### 🎯 Success Criteria Met
-- ✅ Strict TDD methodology (RED-GREEN-REFACTOR) 
-- ✅ Typed AST analysis only (no text manipulation)
-- ✅ Professional-grade performance and reliability
-- ✅ Comprehensive developer tooling and integration
-- ✅ Production-ready core functionality
-
-**Ready to continue with Task 5.2.1: Performance Metrics RED phase implementation.**
+## Current Status: Foundation Complete (40% to MVP)
+
+### Recent Achievements ✅
+- **Test Infrastructure**: Fixed and operational with fpm
+- **fortfront Integration**: Complete AST integration working
+- **AST-based Rules**: F002, F006, F007, F008 implemented using AST
+- **Dependency Analysis**: Implemented with transitive dependency tracking
+- **Dead Code Detection**: Advanced implementation with 50% tests passing
+- **String Wrapper Fix**: Resolved gfortran segfault issues
+- **Code Quality**: Comprehensive input validation and memory management
+
+## Completed Phases
+
+### Phase 0: Critical Fixes ✅ COMPLETED
+- ✅ Test infrastructure working with fpm
+- ✅ fortfront AST integration complete
+- ✅ AST traversal and inspection implemented
+- ✅ Configuration system functional
+
+### Phase 1: Foundation ✅ COMPLETED
+- ✅ Project infrastructure and module structure
+- ✅ Full fortfront integration with semantic analysis
+- ✅ AST wrapper implementation complete
+- ✅ Path utilities and common functions working
+
+### Phase 2: Linting Engine 🔄 IN PROGRESS (40% Complete)
+#### Completed
+- ✅ F001: Missing implicit none (text-based)
+- ✅ F002: Inconsistent indentation (AST-based)
+- ✅ F006: Unused variable (AST-based)
+- ✅ F007: Undefined variable (AST-based)
+- ✅ F008: Missing intent (AST-based)
+
+#### Remaining Rules
+- [ ] F003: Line too long
+- [ ] F004: Trailing whitespace
+- [ ] F005: Mixed tabs and spaces
+- [ ] F009-F015: Convention rules
+- [ ] P001-P007: Performance rules
+- [ ] C001: Correctness rules
+
+## Next Sprint Plan
+
+### Sprint 1: Complete Core Rules (Week 1)
+- [ ] Implement F003-F005 (remaining style rules)
+- [ ] Implement F009-F011 (naming and structure rules)
+- [ ] Add fix suggestions for implemented rules
+- [ ] Achieve 60% rule coverage
+
+### Sprint 2: Performance Rules (Week 2)
+- [ ] Implement P001-P003 (array and loop optimization)
+- [ ] Implement P004-P007 (remaining performance rules)
+- [ ] Add performance benchmarks
+- [ ] Optimize rule execution speed
+
+### Sprint 3: Formatter MVP (Week 3)
+- [ ] Implement basic AST-based formatting
+- [ ] Add indentation and spacing rules
+- [ ] Integrate with style configuration
+- [ ] Test on real codebases
+
+### Sprint 4: LSP and Developer Tools (Week 4)
+- [ ] Complete LSP server implementation
+- [ ] Add real-time diagnostics
+- [ ] Implement code actions for fixes
+- [ ] Add VS Code extension
+
+## Technical Debt to Address
+- [ ] Complete test coverage for all modules
+- [ ] Fix remaining linker issues in test files
+- [ ] Implement missing LSP hover functionality
+- [ ] Add comprehensive error handling
+
+## Success Metrics
+
+### Achieved ✅
+- ✅ Project builds without errors
+- ✅ AST integration working
+- ✅ 5+ rules implemented with AST
+- ✅ Dependency analysis functional
+- ✅ Dead code detection progressing
+
+### Short-term Goals (2 weeks)
+- [ ] 15+ rules working
+- [ ] Basic formatter functional
+- [ ] LSP server responding to requests
+- [ ] 80% test coverage
+
+### MVP Goals (1 month)
+- [ ] All 23 rules implemented
+- [ ] Formatter comparable to fprettify
+- [ ] VS Code extension published
+- [ ] Documentation complete
+
+## Priority Order
+1. **Complete remaining style rules** (F003-F005, F009-F015)
+2. **Implement performance rules** (P001-P007)
+3. **Build formatter MVP**
+4. **Complete LSP implementation**
+5. **Package and release**
+
+## Notes
+- fortfront integration is complete and working
+- Focus on completing rules before adding new features
+- Maintain AST-based approach for all new rules
+- Keep comprehensive test coverage

CODE_QUALITY_IMPROVEMENTS.md

@@ -0,0 +1,67 @@
+# Code Quality Improvements
+
+## Issues Addressed
+
+### 1. Memory Management in String Utils Module
+**Issue**: Potential memory leaks in string array resize operations
+**Fix**: 
+- Added proper deallocation of string components before resizing
+- Implemented explicit memory cleanup in all allocation/deallocation paths
+- Used proper allocate statements with character length specification
+
+### 2. Input Validation
+**Issue**: Missing validation for empty or invalid inputs
+**Fixes**:
+- Added validation for empty file paths in `get_dependencies()`
+- Added validation for empty module names in `get_files_depending_on()`
+- Added bounds checking for array indices
+- Added minimum size constraints for array initialization
+
+### 3. Array Bounds Safety
+**Issue**: Potential out-of-bounds access in array operations
+**Fixes**:
+- Added bounds checking in `string_array_get_item()`
+- Added proper validation of index ranges in cache lookups
+- Used `min()` to prevent reading beyond array bounds
+- Ensured count never exceeds actual array size
+
+### 4. Resource Cleanup
+**Issue**: Incomplete cleanup of allocated resources
+**Fixes**:
+- Implemented proper cleanup in `string_array_cleanup()`
+- Added deallocation of all string components before deallocating arrays
+- Ensured temporary arrays are properly cleaned up
+
+### 5. Growth Strategy Optimization
+**Issue**: Inefficient memory growth factor
+**Fix**: Changed from 2x growth to 1.5x growth factor for better memory utilization
+
+### 6. Edge Case Handling
+**Issues**: Unhandled edge cases for empty arrays and zero sizes
+**Fixes**:
+- Return empty arrays (size 0) instead of single-element arrays with empty string
+- Ensure minimum size of 1 for array initialization
+- Handle unallocated arrays gracefully in all operations
+
+### 7. Defensive Programming
+**Improvements**:
+- Added checks for allocated state before accessing array components
+- Added validation of string length before trimming
+- Protected against negative or zero sizes in resize operations
+
+## Performance Impact
+- Minimal overhead from additional validation checks
+- Better memory utilization with 1.5x growth factor
+- Reduced memory fragmentation with proper cleanup
+
+## Testing Recommendations
+1. Test with empty inputs and edge cases
+2. Verify no memory leaks with valgrind or similar tools
+3. Test with very large arrays to verify growth strategy
+4. Test concurrent access patterns for thread safety
+
+## Future Improvements
+1. Consider implementing a memory pool for string allocations
+2. Add configurable growth factors
+3. Implement lazy deallocation for better performance
+4. Add thread safety mechanisms if needed