HUST-SOFT tactile modulus estimation

中文使用说明

This repository contains the training and evaluation code for Young's modulus estimation from three-frame tactile image sequences. The network combines a ResNet-18 visual encoder, force-guided cross-attention, deformable spatial convolution, temporal aggregation, and a regression decoder.

Quick start

Run all commands from the repository root.

# 1. Create and activate the environment
conda create -n hust-soft python=3.10 -y
conda activate hust-soft

# 2. Install dependencies
pip install -r requirements.txt

# 3. Verify the installation with generated demo data
python scripts/create_demo_dataset.py
python train.py --config configs/demo.yaml --dry-run
python train.py --config configs/demo.yaml

# 4. After placing the real HUST-SOFT data under data/, validate it
python train.py --config configs/default.yaml --dry-run

# 5. Start the full experiment
python train.py --config configs/default.yaml

The demo run should create outputs/demo/, including a checkpoint, a metrics CSV file, a regression plot, and a confusion matrix. The synthetic demo data are only for checking the software installation.

Repository structure

.
├── configs/
│   ├── default.yaml              # HUST-SOFT experiment configuration
│   └── demo.yaml                 # one-epoch software smoke test
├── data/
│   └── README.md                 # expected dataset layout
├── scripts/
│   └── create_demo_dataset.py    # generates synthetic test data
├── tests/
│   ├── test_data_pipeline.py
│   └── test_model.py
├── BioSyS.py
├── train.py
├── train_BioSyS.py  # legacy filename wrapper
├── README_CN.md                     # Chinese usage guide
├── requirements.txt
└── requirements-dev.txt

Installation

Python 3.10 or newer is recommended. Create an isolated environment and install the dependencies:

conda create -n hust-soft python=3.10 -y
conda activate hust-soft
pip install -r requirements.txt

torch and torchvision must be a compatible pair because the model uses torchvision.ops.DeformConv2d. If a specific CUDA version is required, install the matching PyTorch build first and then run pip install -r requirements.txt.

Weights & Biases is optional:

pip install wandb

For a CUDA-specific PyTorch build, follow the command generated by the official PyTorch installation selector, then install the remaining packages from requirements.txt.

Installation smoke test

The following commands generate a small synthetic dataset, validate the full data/model interface, and run one training epoch:

python BioSyS.py
python scripts/create_demo_dataset.py
python train.py --config configs/demo.yaml --dry-run
python train.py --config configs/demo.yaml

The synthetic data are only for checking the software installation. They must not be used for scientific evaluation.

HUST-SOFT data

The dataset is available from HUST-SOFT on Hugging Face. Place the CSV file and image folders under data/, or change data.root in configs/default.yaml.

The CSV schema is:

Column	Meaning
group_id	Identifier shared by all frames in one temporal sample
image_path	Image path relative to data.root
force_n	Contact force in newtons
youngs_modulus_mpa	Young's modulus label in MPa
shore_a	Shore A hardness used for grouped visualization

For each group_id, rows are sorted by force_n, and the highest three force levels are selected by default. The train/validation split is performed at the sequence level, so frames from one sequence cannot appear in both sets.

Before a full training run, always validate the real data:

python train.py --config configs/default.yaml --dry-run

The command checks the CSV columns, every image path, sequence construction, train/validation splitting, tensor shapes, and one forward pass without updating the model.

Training

Review configs/default.yaml, especially the dataset path, batch size, and output run name, and then run:

python train.py --config configs/default.yaml

Common command-line overrides:

# Validate paths and one forward pass without optimizing
python train.py --config configs/default.yaml --dry-run

# Change the device and batch size
python train.py --device cuda:0 --batch-size 16

# Run without downloading ImageNet weights
python train.py --no-pretrained

# Enable optional W&B logging
python train.py --wandb

# Continue an interrupted run
python train.py --resume outputs/hust_soft_default/last_checkpoint.pt

The default configuration preserves the original experiment choices: three tactile frames, four-way spatial augmentation, force normalization by 35 N, modulus normalization to 0–18 MPa, Adam optimization, and seed 27.

Outputs

Each run is written to outputs/<run_name>/:

File	Description
resolved_config.yaml	exact configuration used by the run
metrics.csv	epoch-level loss, MAE, RMSE, and MAPE
last_checkpoint.pt	latest model and optimizer state
best_checkpoint.pt	checkpoint with the lowest validation MAPE
best_model_state_dict.pt	model weights only
regression_plot.png	clean train/validation regression plot
confusion_matrix.png	validation-set modulus-class confusion matrix
classification_metrics.json	accuracy and confusion-matrix values

The final plots and confusion matrix are generated from the best validation checkpoint, not from an arbitrary last epoch.

Tests

Install the development requirements and run:

pip install -r requirements-dev.txt
pytest -q

The packaged repository was verified with Python 3.12.13, PyTorch 2.13.0 (CPU), and torchvision 0.28.0. The model forward tests passed, all three automated tests passed, and the one-epoch demo completed through checkpoint, metric, regression-plot, and confusion-matrix generation. This software check does not replace reproduction on the full HUST-SOFT dataset.

Troubleshooting

• If pretrained ResNet weights cannot be downloaded, use --no-pretrained. For an explicit fallback to random initialization, use --allow-pretrained-fallback.
• An error involving torchvision::deform_conv2d usually means the installed torch and torchvision builds are incompatible.
• If images are not found, check that every CSV image_path is relative to data.root. Windows and POSIX separators are both supported.
• Reduce training.batch_size if GPU memory is insufficient.

Citation

Citation information will be updated after the article is formally published.