Skip to content

DDoS-Detector v61 - WGANGPArchitectureAndCommentNormalization

Latest
Latest

Choose a tag to compare

View all tags
@BrenoFariasdaSilva BrenoFariasdaSilva released this 06 Aug 20:30
· 960 commits to main since this release
v61-refactor-WGANGPArchitectureAndCommentNormalization
7d44da9

This release substantially refactors the WGAN-GP execution architecture into smaller, focused orchestration functions while consolidating configuration namespaces, strengthening result-file validation, and standardizing internal comments and class placement.

It resolves edge cases involving empty augmented outputs, unsafe PNG generation, inconsistent Telegram reporting, duplicated configuration classes, unused parameters, and oversized top-level control flows. These changes improve maintainability, execution traceability, and reliability across single-file, batch, training-only, generation-only, and combined WGAN-GP workflows.

Key Features

WGAN-GP Main Flow Decomposition

  • Extracted the ConfigNamespace class from the main WGAN-GP execution function.
  • Simplified the top-level main function by delegating setup and execution responsibilities to focused helpers.
  • Added initialize_cli_and_config to centralize command-line parsing and configuration initialization.
  • Added extract_runtime_parameters to resolve runtime values from the loaded configuration.
  • Added register_exit_handlers to centralize process-exit and interruption handling.
  • Added print_execution_summary to report the final WGAN-GP execution state.
  • Added run_wgangp as the primary orchestration entry point.
  • Reduced the size and complexity of the previous monolithic control flow.
  • Improved separation of concerns between initialization, configuration, execution, reporting, and cleanup.

Batch and Dataset Processing Architecture

  • Added run_batch_mode to coordinate multi-dataset WGAN-GP execution.
  • Added process_dataset_path to process configured dataset paths consistently.
  • Added process_single_dataset_file to isolate execution for each individual dataset file.
  • Added mark_file_as_processed to track completed dataset files.
  • Added apply_dataset_ordering to apply the configured generation order.
  • Added build_file_progress_prefix to standardize per-file progress messages.
  • Improved dataset-level execution traceability during long-running batch operations.
  • Strengthened handling of ordered and repeated multi-file augmentation workflows.

Single-File and Execution-Mode Dispatch

  • Added handle_single_file_mode to coordinate execution for a single dataset file.
  • Added dispatch_mode_for_file to select the configured execution mode for each file.
  • Added dispatch_single_file_mode to route single-file execution safely.
  • Added run_both_mode_for_csv to coordinate combined training and generation.
  • Added execute_training_with_timing to isolate training execution and timing.
  • Added resolve_checkpoint_after_training to determine the checkpoint produced by training.
  • Added execute_generation_with_verification to generate samples and verify the resulting output.
  • Improved separation between:
    • training-only execution,
    • generation-only execution,
    • combined training and generation,
    • per-file timing,
    • checkpoint resolution,
    • generated-output verification.
  • Made execution-mode behavior easier to inspect, test, and maintain.

Per-File Output and Result Schema Handling

  • Added setup_per_file_output to prepare output state for each dataset file.
  • Added setup_single_file_output_path to resolve output locations for single-file execution.
  • Added create_results_csv_if_absent to initialize result files only when required.
  • Added validate_results_csv_columns to verify the expected result schema before appending data.
  • Improved safeguards around existing result CSV files.
  • Reduced the risk of incompatible or malformed result persistence across repeated executions.
  • Strengthened consistency between per-file outputs, generated datasets, checkpoints, and result records.

Empty Output and Export Fixes

  • Fixed an issue that could produce an empty _data_augmented.csv file.
  • Added verification around generated sample output before considering generation successful.
  • Guarded PNG export operations to prevent invalid or unavailable figures from being saved.
  • Corrected Telegram reporting associated with generated outputs.
  • Improved consistency between actual generated files and remote execution notifications.
  • Reduced the risk of reporting successful generation when the expected dataset output was missing or empty.

Configuration Namespace Consolidation

  • Merged duplicated ConfigNamespace implementations into a single standardized class.
  • Corrected the placement of ConfigNamespace within the WGAN-GP module.
  • Reorganized class definitions into their appropriate architectural locations.
  • Removed redundant configuration-class behavior.
  • Improved consistency when exposing nested YAML configuration values through attribute access.
  • Reduced maintenance overhead caused by duplicated namespace logic.

Typing, Dead Code, and Internal Cleanup

  • Cast feature_dim to Any before integer conversion to satisfy Pylance type validation.
  • Annotated dataset-descriptor fallback keyword arguments as dict[str, Any].
  • Removed unused parameters from WGAN-GP helper functions.
  • Removed a dead local variable from the WGAN-GP execution flow.
  • Applied minor internal cleanup after the architectural refactoring.
  • Improved static-analysis compatibility and reduced misleading or unused implementation paths.

Comment and Structural Normalization

  • Normalized comments throughout wgangp.py.
  • Standardized comment wording and placement after the major function decomposition.
  • Improved alignment between comments and the refactored execution structure.
  • Reduced outdated or inconsistent documentation inside the module.
  • Made the reorganized WGAN-GP architecture easier to navigate and review.

Function Size Reporter Enhancements

  • Added resolve_target_path to centralize target-file resolution.
  • Added process_multiple_files for batch source-file analysis.
  • Added discover_python_files to locate Python files for multi-file processing.
  • Added process_single_file to isolate analysis of one target file.
  • Updated the Function Size Reporter main flow to support both single-file and multi-file execution.
  • Refined imports, target-path handling, and minor internal behavior.
  • Improved the reporter’s usability across individual modules and larger repository-wide inspections.

Impact

This release makes the WGAN-GP module significantly easier to maintain and safer to operate across its supported execution modes.

By replacing oversized control flows with focused orchestration helpers, the framework gains clearer boundaries for configuration loading, dataset iteration, mode dispatch, training, checkpoint resolution, generation, output verification, result persistence, and execution reporting.

The output-validation fixes, configuration-class consolidation, dead-code removal, and comment normalization also reduce ambiguity and failure risk during large augmentation runs, while preserving the framework’s existing WGAN-GP methodology and experimental behavior.

Full Changelog: v60-feat-FunctionSizeReporterAndWGANGPScaffolding...v61-refactor-WGANGPArchitectureAndCommentNormalization

Assets 2
Loading