Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some questions about "ut preprocess" #25

Closed
ljyljy opened this issue Jun 3, 2021 · 0 comments
Closed

Some questions about "ut preprocess" #25

ljyljy opened this issue Jun 3, 2021 · 0 comments

Comments

@ljyljy
Copy link

ljyljy commented Jun 3, 2021

I have skipped the 'ut preprocess' command and used the 'ut train' command to train directly before, and there is no error.

However, when trying the 'ut preprocess' command, I encountered some problems.
I don't quite understand the use of 'ut preprocess'. Is it related to settings such as YAML? > <
How to set up correctly to successfully perform 'ut preprocess'?
THX a lot!

my folder:
demo->my_utime_project2->...
->datasets->processed->dcsm

command:
ut preprocess --out_path ../datasets/processed/dcsm/processed_data.h5 --dataset_splits train_data val_data --overwrite

error screenshot:
(1)Use RandomChannelSelector (default in YAML)

Logged by: 'run' in 'preprocess.py'
Args dump: {'out_path': '../datasets/processed/dcsm/processed_data.h5', 'dataset_splits': ['train_data', 'val_data'], 'overwrite': True, 'num_threads': 1}

Logged by: 'init' in 'hparams.py'
YAML path: /export/ljy/demo/my_utime_project2/hyperparameters/hparams.yaml

Logged by: 'log' in 'abc_sleep_study_dataset.py'
[Dataset: dcsm/TRAIN] SleepStudyDataset(identifier: dcsm/TRAIN, N pairs: 189, N loaded: 0)
[Dataset: dcsm/VAL] SleepStudyDataset(identifier: dcsm/VAL, N pairs: 26, N loaded: 0)
[Dataset: dcsm/TRAIN] Setting load-time random channel selector: RandomChannelSelector(group_0: ['F3-M2', 'F4-M1', 'C3-M2', 'C4-M1', 'O1-M2', 'O2-M1'], group_1: ['E1-M2', 'E2-M2'])
[Dataset: dcsm/VAL] Setting load-time random channel selector: RandomChannelSelector(group_0: ['F3-M2', 'F4-M1', 'C3-M2', 'C4-M1', 'O1-M2', 'O2-M1'], group_1: ['E1-M2', 'E2-M2'])
[Dataset: dcsm/TRAIN] Setting sample rate of 128 Hz
[Dataset: dcsm/VAL] Setting sample rate of 128 Hz
[Dataset: dcsm/TRAIN] Setting 'strip_to_match' strip function with parameters {}...
[Dataset: dcsm/VAL] Setting 'strip_to_match' strip function with parameters {}...
[Dataset: dcsm/TRAIN] Setting 'clip_noisy_values' quality control function with parameters {'min_max_times_global_iqr': 20}...
[Dataset: dcsm/VAL] Setting 'clip_noisy_values' quality control function with parameters {'min_max_times_global_iqr': 20}...
[Dataset: dcsm/TRAIN] Setting 'RobustScaler' scaler...
[Dataset: dcsm/VAL] Setting 'RobustScaler' scaler...
Traceback (most recent call last):
File "/export/anaconda3/envs/ljy_u-sleep2/bin/ut", line 33, in
sys.exit(load_entry_point('utime', 'console_scripts', 'ut')())
File "/export/ljy/U-Time/utime/bin/ut.py", line 71, in entry_func
mod.entry_func(script_args + help_agrs)
File "/export/ljy/U-Time/utime/bin/preprocess.py", line 207, in entry_func
run(args=args)
File "/export/ljy/U-Time/utime/bin/preprocess.py", line 170, in run
value=hparams_out_path, overwrite=True)
File "/export/anaconda3/envs/ljy_u-sleep2/lib/python3.7/site-packages/mpunet/hyperparameters/hparams.py", line 234, in set_value
add_if_missing=add_if_missing)
File "/export/anaconda3/envs/ljy_u-sleep2/lib/python3.7/site-packages/mpunet/hyperparameters/hparams.py", line 217, in _set_value_in_subdir
subdir, name, value, str_value, overwrite, add_if_missing
File "/export/anaconda3/envs/ljy_u-sleep2/lib/python3.7/site-packages/mpunet/hyperparameters/hparams.py", line 190, in _set_value_in_existing_dir
self._update_string_line_by_name(name, str_value, subdir=subdir)
File "/export/anaconda3/envs/ljy_u-sleep2/lib/python3.7/site-packages/mpunet/hyperparameters/hparams.py", line 157, in _update_string_line_by_name
raise AttributeError("No field has the name '{}'".format(name))
AttributeError: No field has the name 'dcsm'

(2)Use select_channels: ['C4-M1', 'E1-M2'] in YAML:

ut preprocess --out_path ../datasets/processed/dcsm/processed_data.h5 --dataset_splits train_data val_data --overwrite

Logged by: 'run' in 'preprocess.py'
Args dump: {'out_path': '../datasets/processed/dcsm/processed_data.h5', 'dataset_splits': ['train_data', 'val_data'], 'overwrite': True, 'num_threads': 1}

Logged by: 'init' in 'hparams.py'
YAML path: /export/ljy/demo/my_utime_project2/hyperparameters/hparams.yaml

Logged by: 'log' in 'abc_sleep_study_dataset.py'
[Dataset: dcsm/TRAIN] SleepStudyDataset(identifier: dcsm/TRAIN, N pairs: 189, N loaded: 0)
[Dataset: dcsm/VAL] SleepStudyDataset(identifier: dcsm/VAL, N pairs: 26, N loaded: 0)
[Dataset: dcsm/TRAIN] Setting select channels: ['C4-M1', 'E1-M2']
[Dataset: dcsm/VAL] Setting select channels: ['C4-M1', 'E1-M2']
[Dataset: dcsm/TRAIN] Setting sample rate of 128 Hz
[Dataset: dcsm/VAL] Setting sample rate of 128 Hz
[Dataset: dcsm/TRAIN] Setting 'strip_to_match' strip function with parameters {}...
[Dataset: dcsm/VAL] Setting 'strip_to_match' strip function with parameters {}...
[Dataset: dcsm/TRAIN] Setting 'clip_noisy_values' quality control function with parameters {'min_max_times_global_iqr': 20}...
[Dataset: dcsm/VAL] Setting 'clip_noisy_values' quality control function with parameters {'min_max_times_global_iqr': 20}...
[Dataset: dcsm/TRAIN] Setting 'RobustScaler' scaler...
[Dataset: dcsm/VAL] Setting 'RobustScaler' scaler...
Traceback (most recent call last):
File "/export/anaconda3/envs/ljy_u-sleep2/bin/ut", line 33, in
sys.exit(load_entry_point('utime', 'console_scripts', 'ut')())
File "/export/ljy/U-Time/utime/bin/ut.py", line 71, in entry_func
mod.entry_func(script_args + help_agrs)
File "/export/ljy/U-Time/utime/bin/preprocess.py", line 207, in entry_func
run(args=args)
File "/export/ljy/U-Time/utime/bin/preprocess.py", line 170, in run
value=hparams_out_path, overwrite=True)
File "/export/anaconda3/envs/ljy_u-sleep2/lib/python3.7/site-packages/mpunet/hyperparameters/hparams.py", line 234, in set_value
add_if_missing=add_if_missing)
File "/export/anaconda3/envs/ljy_u-sleep2/lib/python3.7/site-packages/mpunet/hyperparameters/hparams.py", line 217, in _set_value_in_subdir
subdir, name, value, str_value, overwrite, add_if_missing
File "/export/anaconda3/envs/ljy_u-sleep2/lib/python3.7/site-packages/mpunet/hyperparameters/hparams.py", line 190, in _set_value_in_existing_dir
self._update_string_line_by_name(name, str_value, subdir=subdir)
File "/export/anaconda3/envs/ljy_u-sleep2/lib/python3.7/site-packages/mpunet/hyperparameters/hparams.py", line 157, in _update_string_line_by_name
raise AttributeError("No field has the name '{}'".format(name))
AttributeError: No field has the name 'dcsm'

@ljyljy ljyljy closed this as completed Jun 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant