Open
Description
Describe the bug
With the update of anomalib to V1, the way to create callbacks changed. Now you can pass some callbacks in the config.yaml file, with the desired init_args. Example below:
trainer:
...
callbacks:
- class_path: anomalib.callbacks.checkpoint.ModelCheckpoint
init_args:
dirpath: weights/lightning
filename: best_model-{epoch}-{image_F1Score:.2f}
monitor: image_F1Score
save_last: true
mode: max
auto_insert_metric_name: true
So your models will be logged into "weights/lightning". However, if you see engine.py line 421:
# Add ModelCheckpoint if it is not in the callbacks list.
has_checkpoint_callback = any(isinstance(c, ModelCheckpoint) for c in self._cache.args["callbacks"])
if has_checkpoint_callback is False:
_callbacks.append(
ModelCheckpoint(
dirpath=self._cache.args["default_root_dir"] / "weights" / "lightning",
filename="model",
auto_insert_metric_name=False,
),
)
the default ModelCheckpoint callback uses self._cache.args["default_root_dir"] which in engine.py _setup_workspace() method (line 315) was updated. So the default callbacks will log into this self._cache.args["default_root_dir"] but the custom callbacks will not. And I think the correct would be that all stuff related with the same run to be logged within the same dafault_root_dir.
Dataset
N/A
Model
N/A
Steps to reproduce the behavior
- Train a model with anomalib last code, providing a custom callback in your
config.yaml, setting thedirpathinit_arg to whatever. - You will see some things of your run are logged in one folder locally while the callback is logged in another one (the specified en
dirpath)
OS information
OS information:
- OS: Windows 10 Pro
- Python version: 3.10.14
- Anomalib version: 1.1.0
- PyTorch version: 2.2.+cu121
- CUDA/cuDNN version: 12.5/8.9.7.29
- GPU models and configuration: 1x GeForce RTX 3090
Expected behavior
That all custom callbacks to be logged in the default_root_dir path (plus some aditional path provided by the user).
Screenshots
No response
Pip/GitHub
pip
What version/branch did you use?
main
Configuration YAML
data:
class_path: anomalib.data.Folder
init_args:
name: my_dataset
normal_dir: train/good
root: data/processed/my_dataset
abnormal_dir:
- val/defect1
- val/defect2
normal_test_dir: val/good
mask_dir: null
normal_split_ratio: 0.2
extensions:
- .png
train_batch_size: 1
eval_batch_size: 1
num_workers: 8
task: classification
image_size:
- 1024
- 608
transform: null
train_transform: null
eval_transform: null
test_split_mode: from_dir
test_split_ratio: 0.2
val_split_mode: same_as_test
val_split_ratio: 0.5
seed: 6120
model:
class_path: anomalib.models.EfficientAd
init_args:
imagenet_dir: data/external/imagenette
teacher_out_channels: 384
model_size: S
lr: 0.0001
weight_decay: 1.0e-05
padding: false
pad_maps: true
normalization:
normalization_method: min_max
metrics:
image:
- F1Score
- AUROC
pixel:
- F1Score
- AUROC
threshold:
class_path: anomalib.metrics.F1AdaptiveThreshold
init_args:
default_value: 0.5
logging:
log_graph: false
seed_everything: 6120
task: classification
default_root_dir: results
ckpt_path: null
trainer:
accelerator: auto
strategy: auto
devices: 1
num_nodes: 1
precision: 32
logger:
- class_path: anomalib.loggers.AnomalibWandbLogger
init_args:
project: my_project_name
- class_path: anomalib.loggers.AnomalibMLFlowLogger
init_args:
experiment_name: my_project_name
callbacks:
- class_path: anomalib.callbacks.checkpoint.ModelCheckpoint
init_args:
dirpath: weights/lightning
filename: best_model-{epoch}-{image_F1Score:.2f}
monitor: image_F1Score
save_last: true
mode: max
auto_insert_metric_name: true
fast_dev_run: false
max_epochs: 200
min_epochs: null
max_steps: 70000
min_steps: null
max_time: null
limit_train_batches: 1.0
limit_val_batches: 1.0
limit_test_batches: 1.0
limit_predict_batches: 1.0
overfit_batches: 0.0
val_check_interval: 1.0
check_val_every_n_epoch: 1
num_sanity_val_steps: 0
log_every_n_steps: 50
enable_checkpointing: true
enable_progress_bar: true
enable_model_summary: true
accumulate_grad_batches: 1
gradient_clip_val: 0
gradient_clip_algorithm: norm
deterministic: false
benchmark: false
inference_mode: true
use_distributed_sampler: true
profiler: null
detect_anomaly: false
barebones: false
plugins: null
sync_batchnorm: false
reload_dataloaders_every_n_epochs: 0
default_root_dir: nullLogs
NoneCode of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Labels
No labels