FileNotFoundError when running Madgraph trough pinefarm #47

comane · 2023-11-13T12:13:07Z

Hi @cschwan, I am trying to reproduce one of the pinecards using pinefarm. For ATLAS_TTB_13TEV_TOT (as well as for others) I get this error that seems related to pineappl

INFO:  
INFO: Checking test output: 
INFO: P0_gg_ttx 
INFO:  Result for test_ME: 
Command "launch auto " interrupted with error:
FileNotFoundError : [Errno 2] No such file or directory: '/store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/results/200-ATLAS_TTB_13TEV_TOT--20231113103915/ATLAS_TTB_13TEV_TOT/SubProcesses/P0_gg_ttx/test_ME.log'
Please report this bug on https://bugs.launchpad.net/mg5amcnlo
More information is found in '/store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/results/200-ATLAS_TTB_13TEV_TOT--20231113103915/ATLAS_TTB_13TEV_TOT/run_01_tag_1_debug.log'.
Please attach this file to your report.
INFO:  
quit
INFO:  
quit
quit
Error calling StartServiceByName for org.freedesktop.Notifications: Timeout was reached
Traceback (most recent call last):
  File "/store/DAMTP/mnc33/miniconda3/envs/pinefarm/bin/pinefarm", line 8, in <module>
    sys.exit(command())
  File "/store/DAMTP/mnc33/miniconda3/envs/pinefarm/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/store/DAMTP/mnc33/miniconda3/envs/pinefarm/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/store/DAMTP/mnc33/miniconda3/envs/pinefarm/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/store/DAMTP/mnc33/miniconda3/envs/pinefarm/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/store/DAMTP/mnc33/miniconda3/envs/pinefarm/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/store/DAMTP/mnc33/miniconda3/envs/pinefarm/lib/python3.9/site-packages/pinefarm/cli/run.py", line 31, in subcommand
    main(dataset, theory_card, pdf)
  File "/store/DAMTP/mnc33/miniconda3/envs/pinefarm/lib/python3.9/site-packages/pinefarm/cli/run.py", line 67, in main
    run_dataset(runner)
  File "/store/DAMTP/mnc33/miniconda3/envs/pinefarm/lib/python3.9/site-packages/pinefarm/cli/run.py", line 122, in run_dataset
    runner.generate_pineappl()
  File "/store/DAMTP/mnc33/miniconda3/envs/pinefarm/lib/python3.9/site-packages/pinefarm/external/mg5/__init__.py", line 184, in generate_pineappl
    grid = pineappl.grid.Grid.read(mg5_grids[0])
IndexError: list index out of range
Thanks for using LHAPDF 6.4.0. Please make sure to cite the paper:
  Eur.Phys.J. C75 (2015) 3, 132  (http://arxiv.org/abs/1412.7420)

The text was updated successfully, but these errors were encountered:

felixhekhorn · 2023-11-13T12:16:49Z

Can you check this directory

pinefarm/src/pinefarm/external/mg5/__init__.py

Line 181 in c007afc

for p in self.mg5_dir.glob("Events/run_01*/amcblast_obs_*.pineappl")

?
did mg5 actually run?

comane · 2023-11-13T13:43:19Z

Hi @felixhekhorn , I am not sure I do have that directory as I have installed pinefarm with pip.

However, if I go in the .prefix/mg5amc folder Madgraph is there and runs.

felixhekhorn · 2023-11-13T13:57:47Z

However, if I go in the .prefix/mg5amc folder Madgraph is there and runs.

that is good, that at least mg5 got correctly installed - but the question is rather did it run for the grid you requested?

Hi @felixhekhorn , I am not sure I do have that directory as I have installed pinefarm with pip.

i.e. whenever you request a new calculation pinefarm should create a new folder (the name of the dataset + some timestamp) and that should be the mg5_dir in the snippet above ...

actually, I just had a closer look to your snippet (which I should have done in the first place 🙈 ) and the directory we're talking about is /store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/results/200-ATLAS_TTB_13TEV_TOT--20231113103915/

However, I was trying to understand why there are no grids (which is the python error you encounter), but the true problem is the mg5 error just as you said in the first place "FileNotFoundError" ... sorry for the noise ... on mg5 I've no idea, maybe @cschwan knows more ... is the debug file mentioned in the error revealing more information? else this might be better discussed in the mg5 bugtracker?

comane · 2023-11-13T14:10:28Z

The mg5 log doesn't say much more:

launch auto
Traceback (most recent call last):
File "/store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/.prefix/mg5amc/madgraph/interface/extended_cmd.py", line 1544, in onecmd
return self.onecmd_orig(line, **opt)
File "/store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/.prefix/mg5amc/madgraph/interface/extended_cmd.py", line 1493, in onecmd_orig
return func(arg, **opt)
File "/store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/.prefix/mg5amc/madgraph/interface/amcatnlo_run_interface.py", line 1783, in do_launch
self.compile(mode, options)
File "/store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/.prefix/mg5amc/madgraph/interface/amcatnlo_run_interface.py", line 5407, in compile
self.check_tests(test, this_dir)
File "/store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/.prefix/mg5amc/madgraph/interface/amcatnlo_run_interface.py", line 5418, in check_tests
return self.parse_test_mx_log(pjoin(dir, '%s.log' % test))
File "/store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/.prefix/mg5amc/madgraph/interface/amcatnlo_run_interface.py", line 5425, in parse_test_mx_log
content = open(log).read()
FileNotFoundError: [Errno 2] No such file or directory: '/store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/results/200-ATLAS_TTB_13TEV_TOT--20231113103915/ATLAS_TTB_13TEV_TOT/SubProcesses/P0_gg_ttx/test_ME.log'
Related File: /store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/results/200-ATLAS_TTB_13TEV_TOT--20231113103915/ATLAS_TTB_13TEV_TOT/SubProcesses/P0_gg_ttx/test_ME.log
Value of current Options:
pythia8_path : None
hwpp_path : None
thepeg_path : None
hepmc_path : None
madanalysis_path : None
madanalysis5_path : None
pythia-pgs_path : None
rivet_path : None
yoda_path : None
contur_path : None
td_path : None
delphes_path : None
exrootanalysis_path : None
syscalc_path : None
timeout : 60
web_browser : None
eps_viewer : None
text_editor : None
fortran_compiler : None
f2py_compiler : None
f2py_compiler_py2 : None
f2py_compiler_py3 : None
cpp_compiler : None
cluster_type : condor
cluster_queue : None
cluster_status_update : (600, 30)
fastjet : None
golem : None
samurai : None
ninja : /store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/.prefix/mg5amc/HEPTools/lib
collier : /store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/.prefix/mg5amc/HEPTools/lib
lhapdf : lhapdf-config
pineappl : pineappl
lhapdf_py2 : None
lhapdf_py3 : None
cluster_temp_path : None
mg5amc_py8_interface_path : None
cluster_local_path : None
OLP : MadLoop
cluster_nb_retry : 1
cluster_retry_wait : 300
cluster_size : 100
output_dependencies : external
crash_on_error : False
auto_convert_model : True
acknowledged_v3.1_syntax : False
group_subprocesses : Auto
ignore_six_quark_processes : False
low_mem_multicore_nlo_generation : False
complex_mass_scheme : False
include_lepton_initiated_processes : False
gauge : unitary
stdout_level : 20
loop_optimized_output : True
loop_color_flows : False
max_npoint_for_channel : 0
default_unset_couplings : 99
max_t_for_channel : 99
zerowidth_tchannel : True
nlo_mixed_expansion : True
auto_update : 7
automatic_html_opening : False
run_mode : 2
nb_core : 112
notification_center : True
mg5_path : /store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/.prefix/mg5amc

felixhekhorn · 2023-11-13T14:10:42Z

Okay, digging a bit more around ... I think we're looking at the processed version of this file https://github.com/NNPDF/pinecards/blob/master/ATLAS_TTB_13TEV_TOT/launch.txt right? this should configure mg5 with the correct setting, I think

can you check that the generated version /store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/results/200-ATLAS_TTB_13TEV_TOT--20231113103915/launch.txt (?) makes sense? if that is fine, pinefarm has done it's job and it is a mg5 bug ...

(Just for the sake of it:

do the parent directories exist?
do you have the rights to write there?
)

comane · 2023-11-13T14:15:20Z

So I think it has to be a Madgraph bug given that the launch card seems to have been generated correctly (see below):

$ sdiff launch.txt ../../runcards/ATLAS_TTB_13TEV_TOT/launch.txt 
launch ATLAS_TTB_13TEV_TOT				      |	launch @OUTPUT@
fixed_order = ON						fixed_order = ON
set maxjetflavor 5						set maxjetflavor 5
set gf 1.1663787e-5					      |	set gf @GF@
set mh 125.0						      |	set mh @MH@
set mt 172.5						      |	set mt @MT@
set mw 80.352						      |	set mw @MW@
set mz 91.1535						      |	set mz @MZ@
set wh 4.07468e-3					      |	set wh @WH@
set wt 0.0							set wt 0.0
set ww 2.084						      |	set ww @WW@
set wz 2.4943						      |	set wz @WZ@
set ebeam1 6500							set ebeam1 6500
set ebeam2 6500							set ebeam2 6500
set pdlabel lhapdf						set pdlabel lhapdf
set lhaid 324900					      |	set lhaid @LHAPDF_ID@
set dynamical_scale_choice 10					set dynamical_scale_choice 10
set reweight_scale True						set reweight_scale True
set req_acc_FO 0.001						set req_acc_FO 0.001
set pineappl True						set pineappl True
done								done
quit								quit

felixhekhorn · 2023-11-13T14:24:24Z

So I think it has to be a Madgraph bug given that the launch card seems to have been generated correctly (see below):

good (i.e. someone else needs to worry 😇 ) - just to exclude pinefarm 100%: can you just run mg5 independently (i.e. not through the CI)? something like /store/DAMTP/mnc33/Projects_store/PhD/nnpdf40_pheno/pinefarm_runs/.prefix/mg5amc launch.txt (it should hopefully crash with the very same error)

comane · 2023-11-13T14:48:08Z

Indeed, by running directly trough Madgraph I get the same error 👍

Maybe a weird thing might be the following: although fixed_order = ON in launch.txt it seems (see below) that it is set to OFF?

113142105/ATLAS_TTB_13TEV_TOT/Cards/amcatnlo_configuration.txt
launch auto
WARNING: NLO+PS mode is not allowed for processes including electroweak corrections
The following switches determine which programs are run:
/================== Description ==================|=========== values ===========|================ other options ================
| 1. Type of perturbative computation | order = NLO | LO |
| 2. No MC@[N]LO matching / event generation | fixed_order = OFF | No NLO+PS available for EW correction |
================================================================================================================================/

comane · 2023-11-13T14:49:01Z

Btw @felixhekhorn , thanks for the answers! feel free to close the issue if you think that it has nothing to do with pinefarm and its an mg5 issue only!

felixhekhorn · 2023-11-13T14:57:22Z

Maybe a weird thing might be the following: although fixed_order = ON in launch.txt it seems (see below) that it is set to OFF?

as said, I've no idea of mg5 - but, guessing in the wild, I think mg5 somehow makes a test run, before doing the actual calculation (see Checking test output: or Result for test_ME:) and that is the one that fails ... maybe there the settings are weird ...

Btw @felixhekhorn , thanks for the answers! feel free to close the issue if you think that it has nothing to do with pinefarm and its an mg5 issue only!

let's do this

felixhekhorn added the bug Something isn't working label Nov 13, 2023

felixhekhorn closed this as completed Nov 13, 2023

comane mentioned this issue Nov 15, 2023

No valid pineappl installation found when running pinefarm #50

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FileNotFoundError when running Madgraph trough pinefarm #47

FileNotFoundError when running Madgraph trough pinefarm #47

comane commented Nov 13, 2023

felixhekhorn commented Nov 13, 2023

comane commented Nov 13, 2023 •

edited

Loading

felixhekhorn commented Nov 13, 2023

comane commented Nov 13, 2023

felixhekhorn commented Nov 13, 2023

comane commented Nov 13, 2023

felixhekhorn commented Nov 13, 2023 •

edited

Loading

comane commented Nov 13, 2023

comane commented Nov 13, 2023

felixhekhorn commented Nov 13, 2023

FileNotFoundError when running Madgraph trough pinefarm #47

FileNotFoundError when running Madgraph trough pinefarm #47

Comments

comane commented Nov 13, 2023

felixhekhorn commented Nov 13, 2023

comane commented Nov 13, 2023 • edited Loading

felixhekhorn commented Nov 13, 2023

comane commented Nov 13, 2023

felixhekhorn commented Nov 13, 2023

comane commented Nov 13, 2023

felixhekhorn commented Nov 13, 2023 • edited Loading

comane commented Nov 13, 2023

comane commented Nov 13, 2023

felixhekhorn commented Nov 13, 2023

comane commented Nov 13, 2023 •

edited

Loading

felixhekhorn commented Nov 13, 2023 •

edited

Loading