Datasets

[ZhuYun97]

When I run the command python -m experiments.scripts.train -d bioavailability -m mat --model.pretrained_name mat_masking_20M --train.gpus 1 --train.num_epochs 100, the error will raise.
But I can run successfully on freesolv dataset. When I use bioavailability and PPBR, the errors will appear.

---

[21:05:19] UFFTYPER: Unrecognized atom type: Au6 (7)
[21:05:19] UFFTYPER: Unrecognized atom type: Au6 (7)
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1,2,3,4,5]

  | Name    | Type              | Params
----------------------------------------------
0 | model   | MatModel          | 42.1 M
1 | loss_fn | BCEWithLogitsLoss | 0     
----------------------------------------------
42.1 M    Trainable params
0         Non-trainable params
42.1 M    Total params
168.231   Total estimated model params size (MB)
Validation sanity check:  50%|██████████████████████████████████████████████████████████████████████████████████████████████████                                                                                                  | 1/2 [00:00<00:00,  3.84it/s]
WARNING:root:AUROC requires both negative and positive samples. Returning None

Traceback (most recent call last):
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/zhuyun/huggingmolecules/experiments/scripts/train.py", line 13, in <module>
    train_model()
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/zhuyun/huggingmolecules/experiments/src/training/training_train_model.py", line 65, in train_model
    trainer.fit(pl_module,
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 458, in fit
    self._run(model)
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 756, in _run
    self.dispatch()
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 797, in dispatch
    self.accelerator.start_training(self)
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 96, in start_training
    self.training_type_plugin.start_training(trainer)
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 144, in start_training
    self._results = trainer.run_stage()
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 807, in run_stage
    return self.run_train()
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 842, in run_train
    self.run_sanity_check(self.lightning_module)
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1107, in run_sanity_check
    self.run_evaluation()
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 988, in run_evaluation
    self.evaluation_loop.evaluation_epoch_end(outputs)
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/evaluation_loop.py", line 200, in evaluation_epoch_end
    self.trainer.logger_connector.evaluation_epoch_end()
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py", line 251, in evaluation_epoch_end
    self.cached_results.has_batch_loop_finished = True
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/logger_connector/epoch_result_store.py", line 404, in has_batch_loop_finished
    self.update_logger_connector()
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/logger_connector/epoch_result_store.py", line 345, in update_logger_connector
    epoch_log_metrics = self.get_epoch_log_metrics()
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/logger_connector/epoch_result_store.py", line 417, in get_epoch_log_metrics
    return self.run_epoch_by_func_name("get_epoch_log_metrics")
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/logger_connector/epoch_result_store.py", line 410, in run_epoch_by_func_name
    results = [func() for func in results]
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/logger_connector/epoch_result_store.py", line 410, in <listcomp>
    results = [func() for func in results]
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/logger_connector/epoch_result_store.py", line 130, in get_epoch_log_metrics
    return self.get_epoch_from_func_name("get_epoch_log_metrics")
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/logger_connector/epoch_result_store.py", line 123, in get_epoch_from_func_name
    self.run_epoch_func(results, opt_metrics, func_name, *args, **kwargs)
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/logger_connector/epoch_result_store.py", line 111, in run_epoch_func
    metrics_to_log = func(*args, add_dataloader_idx=self.has_several_dataloaders, **kwargs)
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/pytorch_lightning/core/step_result.py", line 289, in get_epoch_log_metrics
    result[dl_key] = self[k].compute().detach()
AttributeError: 'NoneType' object has no attribute 'detach'
  In call to configurable 'train' (<function train_model at 0x7f15a2a6eca0>)

[ZhuYun97]

When I use bbbp dataset, other errors will appear. FileNotFoundError: [Errno 2] No such file or directory: './data/bbbp/bbbp.csv'
But after downloading bbbp.csv, other error will come up.

Traceback (most recent call last):
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/zhuyun/huggingmolecules/experiments/scripts/train.py", line 13, in <module>
    train_model()
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/zhuyun/huggingmolecules/experiments/src/training/training_train_model.py", line 60, in train_model
    train_loader, val_loader, test_loader = get_data_loaders(featurizer,
  File "/home/zhuyun/huggingmolecules/experiments/src/training/training_utils.py", line 175, in get_data_loaders
    split = get_data_split()
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/zhuyun/anaconda3/envs/huggingmolecules/lib/python3.8/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/zhuyun/huggingmolecules/experiments/src/training/training_utils.py", line 218, in get_data_split
    split = _get_data_split_from_csv(dataset_name, assay_name, dataset_path,
  File "/home/zhuyun/huggingmolecules/experiments/src/training/training_utils.py", line 268, in _get_data_split_from_csv
    raise NotImplementedError()
NotImplementedError: 
  In call to configurable 'data' (<function get_data_split at 0x7f532b1dc430>)
  In call to configurable 'train' (<function train_model at 0x7f532b1dfca0>)

[panpiort8]

Hello, sorry for delay.

1. Could you please provide Your version of Pytorch Lightning and PyTDC?
2. The error says that scaffold splitting has not been implemented yet, so we need to use pre-defined splits. I've added them in the above MR.