DFTYaml Loader #3295
DFTYaml Loader #3295
Conversation
deepchem/data/data_loader.py
Outdated
|
|
||
| Parameters | ||
| ---------- | ||
| featurizer: Featurizer |
There was a problem hiding this comment.
Formatting looks off here. Can you also add more details about the featurizers that we support?
deepchem/data/data_loader.py
Outdated
| featurizer: Featurizer | ||
| """ | ||
|
|
||
| def create_dftdataset(self, |
There was a problem hiding this comment.
We shouldn't create a new method name here. Instead, we should follow the existing API (create_dataset and not create_dftdataset)
deepchem/data/data_loader.py
Outdated
| X = np.array([self._featurize_shard(shard) for shard in entries]) | ||
| y = np.array([0]) | ||
| w = w | ||
| return NumpyDataset(X, y, w=w) |
There was a problem hiding this comment.
We shouldn't need to create only a NumpyDataset. If we implement _get_shards we should directly be able to construct a DiskDataset with the inherited superclass method of create_dataset
deepchem/data/data_loader.py
Outdated
|
|
||
| Returns | ||
| ------- | ||
| x: DFTEntry object |
There was a problem hiding this comment.
This looks possibly off? A shard has multiple datapoints. That should be multiple DFTEntry objects right
There was a problem hiding this comment.
Almost good to go, but you need to add yaml to the requirements file for tests https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
deepchem/data/data_loader.py
Outdated
| true_val = shard['true_val'] | ||
| systems = shard['systems'] | ||
| except KeyError: | ||
| print("Unknown key") |
There was a problem hiding this comment.
Instead of a print, you want to raise an error:
raise ValueError("Unknown key in yaml file. Please check format for correctness.")
There was a problem hiding this comment.
On ubuntu-latest, I'm seeing this CI failure due to dqc. @advikavs could you take a look and fix?
=========================== short test summary info ============================
ERROR deepchem/feat/dft_data.py - ModuleNotFoundError: No module named 'dqc'
ERROR deepchem/models/dft/nnxc.py - ModuleNotFoundError: No module named 'dqc'
!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!
======================== 12 warnings, 2 errors in 8.87s ========================
I have fixed the issue in this PR: #3315 |
Description
Load and featurize .yaml files