Skip to content

Canfar new#283

Merged
martinkilbinger merged 83 commits intoCosmoStat:masterfrom
martinkilbinger:canfar_new
Jun 9, 2020
Merged

Canfar new#283
martinkilbinger merged 83 commits intoCosmoStat:masterfrom
martinkilbinger:canfar_new

Conversation

@martinkilbinger
Copy link
Copy Markdown
Contributor

This branch is running on canfar, and provides consistent results with Axel's previous runs on candide. This might be the one!

…x, [pixel scale, PSF init guess, more output fields], make_catalogue [entire file copied].
Comment thread shapepipe/modules/find_exposures_package/find_exposures_script.py
Comment thread shapepipe/modules/get_images_runner.py
Comment thread shapepipe/modules/get_images_runner2.py
Comment thread shapepipe/modules/merge_sep_cats_runner.py
Comment thread shapepipe/modules/ngmix_runner.py
Comment thread shapepipe/modules/uncompress_fits_image_runner.py
Comment thread shapepipe/pipeline/run_log.py
@martinkilbinger martinkilbinger requested a review from sfarrens June 8, 2020 13:29
@martinkilbinger
Copy link
Copy Markdown
Contributor Author

@aguinot should probably also review this merge...

Copy link
Copy Markdown
Member

@sfarrens sfarrens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall it looks fine to me. There are just a couple of points that I think need clarification.

Comment thread environment.yml
Comment thread install_shapepipe
Comment thread scripts/python/canfar_run_analyse.py Outdated
Comment thread scripts/sh/canfar_sp.bash Outdated
Comment thread scripts/sh/canfar_submit_selection.sh Outdated
Comment thread shapepipe/modules/get_images_runner.py Outdated
Comment thread shapepipe/modules/get_images_runner.py Outdated
Comment thread shapepipe/modules/get_images_runner2.py
Comment on lines +332 to +382
# def save_galsim_shapes(final_cat_file, galsim_cat_path):
# """ Save ngmix data
#
# Save the ngmix catalog into the final one.
#
# Parameters
# ----------
# final_cat_file : io.FITSCatalog
# Final catalog.
# ngmix_cat_path : str
# Path to ngmix catalog to save.
#
# """
#
# final_cat_file.open()
# obj_id = np.copy(final_cat_file.get_data()['NUMBER'])
#
# galsim_cat_file = io.FITSCatalog(galsim_cat_path)
# galsim_cat_file.open()
# galsim_id = galsim_cat_file.get_data()['id']
# # max_epoch = np.max(ngmix_n_epoch)
#
# output_dict = {'GALSIM_GAL_ELL': np.ones((len(obj_id), 2)) * -10.,
# 'GALSIM_GAL_ELL_ERR': np.ones(len(obj_id)) * -1.,
# 'GALSIM_GAL_SIGMA': np.zeros(len(obj_id)),
# 'GALSIM_GAL_FLAG': np.ones(len(obj_id), dtype='int16'),
# 'GALSIM_GAL_ELL_U': np.ones((len(obj_id), 2)) * -10.,
# 'GALSIM_GAL_RES': np.ones(len(obj_id)) * -1.,
# 'GALSIM_PSF_ELL': np.ones((len(obj_id), 2)) * -10.,
# 'GALSIM_PSF_SIGMA': np.zeros(len(obj_id)),
# 'GALSIM_PSF_FLAG': np.ones(len(obj_id), dtype='int16')}
# for i, id_tmp in enumerate(obj_id):
# ind = np.where(id_tmp == galsim_id)[0]
# if len(ind) > 0:
# output_dict['GALSIM_GAL_ELL'][i][0] = galsim_cat_file.get_data()['gal_g1'][ind[0]]
# output_dict['GALSIM_GAL_ELL'][i][1] = galsim_cat_file.get_data()['gal_g2'][ind[0]]
# output_dict['GALSIM_GAL_ELL_ERR'][i] = galsim_cat_file.get_data()['gal_g1_err'][ind[0]]
# output_dict['GALSIM_GAL_SIGMA'][i] = galsim_cat_file.get_data()['gal_sigma'][ind[0]]
# output_dict['GALSIM_GAL_FLAG'][i] = galsim_cat_file.get_data()['gal_flag'][ind[0]]
# output_dict['GALSIM_GAL_RES'][i] = galsim_cat_file.get_data()['gal_resolution'][ind[0]]
# output_dict['GALSIM_GAL_ELL_U'][i][0] = galsim_cat_file.get_data()['gal_uncorr_g1'][ind[0]]
# output_dict['GALSIM_GAL_ELL_U'][i][1] = galsim_cat_file.get_data()['gal_uncorr_g2'][ind[0]]
# output_dict['GALSIM_PSF_ELL'][i][0] = galsim_cat_file.get_data()['psf_g1'][ind[0]]
# output_dict['GALSIM_PSF_ELL'][i][1] = galsim_cat_file.get_data()['psf_g2'][ind[0]]
# output_dict['GALSIM_PSF_SIGMA'][i] = galsim_cat_file.get_data()['psf_sigma'][ind[0]]
#
# for key in output_dict.keys():
# final_cat_file.add_col(key, output_dict[key])
#
# final_cat_file.close()
# galsim_cat_file.close()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this function being saved for later?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this is used when galsim instead of ngmix is used to measure shapes on the tiles. I think @aguinot should check whether this function is needed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function was there when I was running galsim without the metacal. I think you can remove it.

Comment thread shapepipe/modules/ngmix_runner.py Outdated
@sfarrens
Copy link
Copy Markdown
Member

sfarrens commented Jun 9, 2020

Hi @martinkilbinger I think we can merge this after the following things have been resolved:

  • PEP8 errors fixed
  • Resolve comment regarding empty return statement in find_exposures_runner.py.
  • Resolve commented out function in make_catalog_runner.py after feedback from @aguinot

@martinkilbinger
Copy link
Copy Markdown
Contributor Author

PEP errors and empty return are resolved. Is the commented function ok as it is? I don't know what Axel recommended. If yes, we can merge.

@martinkilbinger
Copy link
Copy Markdown
Contributor Author

@martinkilbinger martinkilbinger merged commit 85c2745 into CosmoStat:master Jun 9, 2020
@martinkilbinger martinkilbinger deleted the canfar_new branch June 9, 2020 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants