CAE Fitting Error

[nonsenseWHO]

When trying to fit the CAE on my own data, I've run into a roadblock running a simple grid search to train the autoencoder.

Filled out stuff and ran jupyter notebook for 00_data seemingly without issue
Now when I’m running this:

python behavenet/fitting/ae_grid_search.py --data_config C:/Users/taliaferro/.behavenet/ck_lab_coh1_params.json --model_config C:/Users/taliaferro/.behavenet/ae_model.json --training_config C:/Users/taliaferro/.behavenet/ae_training.json --compute_config C:/Users/taliaferro/.behavenet/ae_compute.json

as suggested in https://behavenet.readthedocs.io/en/latest/source/user_guide.autoencoders.html

I get two errors:

AttributeError: Can't pickle local object 'HyperOptArgumentParser.optimize_parallel_gpu..init'

And later

EOFError: Ran out of input

I’ve attached a .txt file that has the full output from the Anaconda command prompt
JT_CAE_Error1.txt

Of note:

• Data are in the correct format

• I've cd'd to the suggested directory

• CUDA toolkit and drivers are installed

Selected computer specs:

• Dell New Optiplex 7071 Desktop

• Processor: Intel i9-9900 CPU @ 3.10 GHz

• 48 GB RAM

• Windows 10 Enterprise

• Nvidia GeForce RTX 2080 8GB GPU

• CUDA Version 10.2

(I know Ella and Matt have done all their stuff on Macs, so I'm wondering if this could be a PC thing?)

Any help or thoughts would be greatly appreciated--thanks!

[nonsenseWHO]

Of note, got the same errors when trying to train autoencoder with example dataset

[ebatty]

I haven't see this error before - I will try this out on a PC this weekend and get back to you

[johnmbarrett]

I also ran into this error today, it appears to be related to an open issue in test-tube. Per this comment on the issue, I moved the nested init functions defined on lines 316 and 355 of test_tube/argparse_hopt.py outside the HyperOptArgumentParser class definition (e.g. to line 66) and it got rid of the error. Whether it actually fixes the problem remains to be seen, but at least my autoencoder grid search appears to be running...

[themattinthehatt]

@ebatty did you fix this by any chance in your recent round of updates?