Skip to content

fix: security hardening and code quality improvements#1

Merged
HugoMSFT merged 1 commit into
mainfrom
fix/security-and-code-quality
Apr 2, 2026
Merged

fix: security hardening and code quality improvements#1
HugoMSFT merged 1 commit into
mainfrom
fix/security-and-code-quality

Conversation

@HugoMSFT

@HugoMSFT HugoMSFT commented Apr 2, 2026

Copy link
Copy Markdown
Owner

Summary

Comprehensive security hardening and code quality improvements across the Python backend.

Security Fixes

  • Path traversal in CSV download endpoint — validate filename chars and realpath containment
  • Code injection — validate table_name and write_mode before embedding in PySpark code
  • ast.literal_eval on untrusted data — replaced with json.loads for CSV keyword parsing
  • JWT handling — validate 3-part token structure before decoding payload
  • Secret key warning — log a warning when FLASK_SECRET_KEY is not set
  • Input validation — keyword count/length limits and Flask MAX_CONTENT_LENGTH (16 MB)

Reliability Fixes

  • DB connection leaks — added finally: conn.close() to 8 methods across fabric_sql_writer and state_manager
  • Session resource leaks — added close() to all 6 collector classes and call it after collect()
  • Thread safetythreading.Lock around global mutable state (collection_status, last_collected_feedback)

Code Quality

  • Replaced stderr debug prints with proper logger calls in fabric_writer
  • Removed unused xml.etree.ElementTree import
  • Moved inline import json to module top-level in fabric_sql_writer
  • Added exc_info=True to exception handlers missing stack traces
  • Removed overly-aggressive regex that stripped legitimate text in utils.py

Files Changed

  • src/app.py — thread lock, path traversal fix, input validation, collector close calls
  • src/collectors.pyclose() methods, unused import removal, error logging
  • src/fabric_writer.py — input validation, debug print cleanup
  • src/fabric_sql_writer.py — connection leak fixes, import cleanup
  • src/state_manager.py — JWT validation, connection leak fixes
  • src/utils.py — regex fix

- Path traversal: validate filename and realpath containment in CSV download
- Thread safety: add threading.Lock around global mutable state
- Connection leaks: add finally blocks to all DB methods in fabric_sql_writer and state_manager
- Session leaks: add close() to all collector classes; call after collect()
- Code injection: validate table_name and write_mode in PySpark payload generation
- JWT handling: validate token structure before decoding payload
- Input validation: add keyword count/length limits and Flask MAX_CONTENT_LENGTH
- Replace ast.literal_eval with json.loads for CSV keyword parsing
- Replace stderr debug prints with proper logger calls in fabric_writer
- Remove unused xml.etree.ElementTree import and redundant inline json imports
- Add exc_info=True to exception handlers missing stack traces
- Remove overly-aggressive regex that stripped legitimate text in utils.py
- Warn when FLASK_SECRET_KEY is not set instead of silently using default
@HugoMSFT
HugoMSFT merged commit 5116a5f into main Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant