-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
piXedfit not working in HPC #15
Comments
Hi @Nikhil0504, Sorry to know that you got this issue. first of all, can I see your model_rest_frame_spectra.py script? and the way you run this Python script seems incorrect -- no need to mpirun again. Please try this instead And regarding the slow run on Macbook Air M1, how many cores does your laptop have and how many of them are used (nproc) for running your piXedfit script?. And did you put mpirun on terminal command when executing the script (which shouldn't be)? On my MBP using 5 cores, generating 100,000 models usually takes around 10-15min. |
Hey @aabdurrouf, sorry for the late reply! It's end of the semester for me and is filled with lots of finals. Here is the I am not sure why it's super slow and buggy for me. # %%
import numpy as np
from astropy.cosmology import FlatLambdaCDM
from piXedfit.piXedfit_model import save_models_rest_spec
# %%
imf_type = 1
sfh_form = 1
dust_law = 1
duste_switch = 1
add_neb_emission = 1
add_agn = 0
nmodels=100000
nproc=16
min_z = 2.47 # minimuz redshift which determines the maximum age of the models
cosmo = FlatLambdaCDM(H0=70.0, Om0=0.3)
age_univ = cosmo.age(min_z)
max_log_age = np.log10(age_univ.value)
# # we fix the ionization parameter to log(U)=-2.0
# params_range = {'dust1':[0.0,4.0],
# 'dust2':[0.0,4.0],
# 'log_age':[-1.0,max_log_age],
# 'log_tau':[-1.0,1.5],
# 'gas_logu':[-2.0,-2.0]}
name_out = 'model_specs.hdf5'
save_models_rest_spec(imf_type=imf_type, sfh_form=sfh_form, dust_law=dust_law,
duste_switch=duste_switch, add_neb_emission=add_neb_emission, add_agn=add_agn,
nmodels=nmodels, nproc=nproc, name_out=name_out)
# %% |
Update: But, upon installing none of the codes were working because of a conflict in the packages defined in the installation (mainly astropy). So, I had to fix the versions manually. I can do a new push fixing the errors of the packages with a new There was also a bug where the code breaks when |
Hello,
I am trying to run the code for a JWST project on HPC. But, when I try to make the model rest frame spectra my code breaks due to some MPI error that is being cause by piXedfit.
I did test the MPI installation in my cluster and everything seems to be fine there with no conflicts and only breaks when I run piXedfit.
The error seems to be some networking error but upon checking with my HPC support team they didn't find any problems with the way I set up my cluster and seems to be leaning in the way that it's the problem of the way MPI implementation might be written here.
Please let me know how I can fix it because this code takes a very long time to run on my laptop (Macbook Air M1). I would like to also know if there are any other new features implemented for JWST images and spectroscopy.
Thanks,
Nikhil
FILE DUMPS:
Here is HPC job shell file:
Here is the output from the run:
slurm-1653886.out.txt
The text was updated successfully, but these errors were encountered: