New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Understanding the prediction_dir format for leaderboard submission #89
Comments
Hi Jean-Michel, Those log files should be in the |
Hi, thank for your response. No the folder isn't empty. After training it has the following files:
But the files (EDIT) I think I get it, once we have finished training, we should run the same command with Note that in the previous command, I did not used I think that precising how to get the predictions in the readme could help. The problem I faced is that using Windows as operating system does not allow to have colons ":" in filenames, so I used a hack: https://stackoverflow.com/questions/10386344/how-to-get-a-file-in-windows-with-a-colon-in-the-filename to change the colons in many files and functions to save the model and result. Then it becames difficult to see where I use the hacky colon or the normal colon. Maybe it would be more democratic to use underscore "_" in place of colon ":". |
Hi Jean-Michel, Hmm, you shouldn't need to run it with
it should look like
Perhaps this is related to the Windows colon issue? I hadn't known about that. Sorry for the trouble. (If it helps, using Windows Subsystem for Linux would allow you to bypass these issues.) |
I wonder if the log folder used during training is the prediction_dir described in Get Started: Evaluating trained models.
I tried to reproduce the ERM result on a subset of camelyon with the following command:
python examples/run_expt.py --dataset camelyon17 --algorithm ERM--root_dir data --frac 0.1 --log_dir log_erm_01
.Training goes well.
But my file
camelyon17_split:id_val_seed:0_epoch
is empty.Then I ran the following command:
python examples/evaluate.py log_erm_01 erm_01_output --root-dir data --dataset camelyon17
And I got this:
So my question is whether the log file is the prediction_dir described in Get Started ?
The text was updated successfully, but these errors were encountered: