Generalize architecture: remove hardcoding (e.g., "cwru") and make config-driven

[abhiprd2000]

Background

The repository currently contains hardcoded values (for example: references to "cwru") that prevent the codebase from being reused with other datasets, organizations, or environments. This makes onboarding, testing, and deployment brittle and error-prone.

Goal

Generalize the project architecture and remove all hardcoding so the codebase is configurable and reusable across datasets/environments.

Scope / Acceptance criteria

• Remove any hardcoded references to "cwru" and replace with configurable values (config file, environment variables, or CLI args).
• Identify and remove other hardcoded values (paths, dataset names, organization identifiers, endpoints, file names) across the repo.
• Add or update a single configuration mechanism (e.g., YAML/JSON config + environment variable overrides + CLI flags).
• Ensure dataset loaders, paths, and model checkpoints support configurable base directories and dataset identifiers.
• Maintain backward compatibility where reasonable or provide a migration note.
• Remove the second - person tone in the comments and make it minimal and informative to address a third person.

Acceptance Tests

• Run existing examples or a small smoke test with default config (should behave as before).
• Verify that pointing config to another dataset path/name makes the code run against that dataset without code changes.

Estimate

• Small-to-medium refactor (depends on how many hardcodings are present). Start with inventory and config design as a PR.

[abhiprd2000]

Hii @TasfinMahmud ,

Can you take a look at this. This was the most severe enhancement we discussed.

[TasfinMahmud]

@abhiprd2000, I'll take this on! I’m going to dig through the codebase tonight to map out all the hardcodings. I’ll drop an inventory and a proposed config design here soon so we can align before I start touching the loaders.

[TasfinMahmud]

@abhiprd2000,

I’ve finished scanning the codebase. Here is the proposed YAML config structure to replace the hardcoding, followed by the inventory of files that need to be refactored.

Proposed Config Structure (cnsd_config.yaml)

To keep dependencies light, I propose using a standard yaml config loaded via a Python dataclass. Here is the base structure:

dataset:
  name: "cwru" # Configurable for any dataset
  base_dir: "./data/"
  sampling_rate_hz: 12000
  num_classes: 10

physics:
  bearing_type: "6205-2RS"
  motor_load_rpm:
    0: 1797
    1: 1772
    2: 1750
    3: 1730

taxonomy:
  classes:
    0: ["Normal", null]
    1: ["Ball", 0.007]
    # ...

🔍 Click to view the full Inventory of Hardcoded References

I mapped out all the dataset-specific hardcodings that need to be swapped out for the config loader:

Core Framework

• cnsd/causal/estimators.py: Hardcoded domain='CWRU' default argument.
• cnsd/datasets/contract.py: Hardcoded dataset routing (if name == 'cwru') and sampling rate defaults.
• cnsd/diagnosis/system.py: Hardcoded load_dataset('cwru').
• cnsd/physics/bearing.py: Hardcoded physics mapping (CWRU_LOAD_RPM, CWRU_FS, BEARING_6205).
• cnsd/physics/configs.py: Hardcoded CWRU_PHYSICS instantiation.
• cnsd/symbolic/engine.py: Hardcoded CWRU_CLASSES taxonomy and CWRU_FS.

Examples & Tests

• examples/cross_rig.py: Hardcoded CWRU_TO_SUPER mapping.
• examples/quickstart.py: Hardcoded load_dataset('cwru').
• test/test_layers.py: Hardcoded tests checking specifically for CWRU published frequencies.

If this YAML structure and inventory looks complete to you, give me the green light and I will start building out the config.py parser and updating the loaders!

[abhiprd2000]

@TasfinMahmud

The inventory looks comprehensive and matches the areas I expected to be dataset-specific. I have one architectural suggestion before implementation: let's avoid embedding bearing-specific concepts directly into the root config schema. Since the goal is a general industrial framework, I'd like the config to remain domain-agnostic where possible so future gearbox, motor, and process-monitoring datasets can plug in without schema changes. Adding configurable target columns and feature definitions would also help. If you're aligned with that direction, you can start with the config loader and dataset abstraction layer.

For example see this section:

physics:
  bearing_type: "6205-2RS"
  motor_load_rpm:

This can still be assumed as a bearing hardcode. I would add something like

domain:
  type: "bearing"

Or

physics:
  parameters:

Where later a user can customize according to need, like replacing bearing with gearbox.

Another thing I'd request:

Add

target:
  column: "fault"

And

dataset:
  features:

For example

target:
  column: "fault"

features:
  - vibration
  - temperature
  - current

Which can make the api more simple and cleaner.

Current refactor objective is to Move all dataset-specific assumptions (CWRU frequencies, taxonomies, RPM mappings, physics constants, labels, sampling rates) into the Configuration Layer and Dataset Abstraction Layer. All downstream layers should operate on a unified internal representation and remain dataset-agnostic and universal.

[abhiprd2000]

I am attaching the architecture flowcharts for the layers of cnsd and the live deployment we are seeking. We are currently optimizing the five layers to be universal and generalized (not bound to a dataset or a domain). I am converting this issue into a discussion. NOTE: I missed adding counterfactual layer in the diagrams, please consider them too, these flowcharts are just mere representation.

[TasfinMahmud]

@abhiprd2000, the flowcharts are a huge help. The shift to a live streaming architecture (< 85ms inference constraint) puts this config refactor into perfect perspective.

Decoupling the new ingestion buffer from the inference layers is exactly why we need this schema to be strictly domain-agnostic. By abstracting the sensor specifics (like the 12kHz rate and bearing physics) out of the core logic and into a generic parameters config block, we guarantee the 5 layers can execute universally without caring where the data came from.

I'll incorporate your structural feedback (adding features, target, and the generic physics: parameters block) into the schema today. I'm going to start drafting the config.py parser and spin up a Draft PR so we can lock down the dataset abstraction layer!