[FEAT] Report missing sample file

[shlanger]

🚀 Feature Request

Report missing sample file. If I make a typo, the file name in the samples section may not match the file name in my samples generation command. Merlin responds by silently not doing anything. I think it would be better to say "Sample file my_samples_1024.npy not found" or something of that sort.

[ben-bay]

@shlanger can you give us more details on how to duplicate this? Feel free to edit the text body of this issue.

EDIT: I think I understand, will look at fixing this

[shlanger]

Ben: Try putting this in your yaml spec: merlin: resources: task_server: celery workers: simworkers: args: -Ofair --prefetch-multiplier 1 -E -l info --concurrency $(SIM_CONCURRENCY) steps: [prep, run-hydra, post-process-hydra-summary, fit-hydra-model] samples: column_labels: - LAS_MULT - LASLAM file: $(MERLIN_INFO)/samples_8.npy generate: cmd: |- python $(SPECROOT)/../../make_samples.py -dims 2 -n 8 -outfile=$(MERLIN_INFO)/samples_88.npy -scale "[(0.5,9.0,'log'),(1.0,12.0,'log')]" The typo is “samples_88.npy”. That kind of error is easy to make. When merlin runs, it looks for samples in $(MERLIN_INFO)/samples_8.npy, but doesn’t find the file. Merlin doesn’t say anything, but there aren’t any tasks on the queue and the workers just sit there. An error message saying “samples_8.npy not found” would lead me to the error instantly. Scanning through the whole yaml spec and my batch script wondering what I did wrong turns out to be a slow way to find errors like this. Please let me know if this isn’t clear. Steve From: Benjamin Bay <notifications@github.com> Reply-To: LLNL/merlin <reply@reply.github.com> Date: Wednesday, June 3, 2020 at 1:45 PM To: LLNL/merlin <merlin@noreply.github.com> Cc: "Langer, Steve" <langer1@llnl.gov>, Mention <mention@noreply.github.com> Subject: Re: [LLNL/merlin] [FEAT] Report missing sample file. If I make a typo, the file name in the samples section may not match the file name in my samples generation command. Merlin responds by silently not doing anything. I think it would be better to say ... @shlanger<https://github.com/shlanger> can you give us more details on how to duplicate this bug? Feel free to edit the text body of this issue. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#227 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AOGRLJBMNNHVYEUSIFF3TT3RU2R4LANCNFSM4NREFM7Q>.

[ben-bay]

@shlanger When I run this (using feature_demo and changing the samples filename) in merlin 1.6.1, I get the error:

[2020-06-08 13:08:35: ERROR] [Errno 2] No such file or directory: 'feature_demo_20200608-130832/merlin_info/samples.npy'

...which seems like the right error to me.