Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stage to compute CL shear individual profiles and stacked profile, using CLMM functionalities #325

Draft
wants to merge 45 commits into
base: master
Choose a base branch
from
Draft

Stage to compute CL shear individual profiles and stacked profile, using CLMM functionalities #325

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
8cde3d1
Start new ensemble profile stage
combet Jul 19, 2023
7f1e876
remove weights computation from select.py
combet Jul 19, 2023
50024fd
putting data folder back in
combet Jul 19, 2023
d708b6b
reproduced Celine's crossing pipeline diagram
cavestruz Jul 19, 2023
69a6e93
added cluster_shear_profiles as one of the inputs to CLClusterEnsembl…
cavestruz Jul 19, 2023
8f5443d
added nersc version to notebook
cavestruz Jul 19, 2023
b5727aa
Merge branch 'master' into issue/307/CL_individual_profile_stage
cavestruz Jul 19, 2023
0c76603
Merge branch 'master' into issue/307/CL_individual_profile_stage
cavestruz Jul 20, 2023
90d1b51
merged the NERSC instructions in the notebook from master and now hav…
cavestruz Jul 20, 2023
576ca44
reorgnise new stuff in to new files
combet Jul 20, 2023
8acd286
fixing merge conflict
combet Jul 20, 2023
81d972f
fix merge conflict
combet Jul 20, 2023
d0230f4
one more fix
combet Jul 20, 2023
8d644fa
rename select.py to sources_select_compute.py to perform all individu…
combet Jul 20, 2023
acbcccc
compute tan and cross component of individual galaxies
combet Jul 20, 2023
8b5ebb0
Fix galaxy source selection to be per cluster
combet Jul 20, 2023
743d930
Fix sigma_c keyword
combet Jul 20, 2023
9a075a0
modification to make_ensemble_profile so we can access source galaxy …
cavestruz Jul 20, 2023
8ac846e
no major change
cavestruz Jul 20, 2023
6efd50a
Modified step 3 to have appropriate input/output files available
cavestruz Jul 20, 2023
0210cb4
needed to add a colon
cavestruz Jul 21, 2023
f62a426
starting code for getting source galaxy redshifts per cluster
cavestruz Jul 21, 2023
aaf9846
added some rough lines of code to read in source galaxy data and pull…
cavestruz Jul 21, 2023
80ed6b9
aiming to replicate the source reading from sources_select_compute.py…
cavestruz Jul 21, 2023
4f18eac
Actually didn't need to access source photoz from the cluster backgro…
combet Jul 21, 2023
cea36a8
Started adding HDF5 save for the profile. Probbaly need to output ind…
combet Jul 21, 2023
259f9ce
added all necessary pieces for the output file in the run portion of …
cavestruz Jul 21, 2023
bdf0b72
minor bugs fixed, now need to understand necessary datastructure for …
cavestruz Jul 21, 2023
e318d76
Save HDF5 file of the individual profiles computed by CLMM
combet Jul 21, 2023
c24dfad
Fix indexing of the catalog group. All profiles are correctly populated
combet Jul 22, 2023
d5007cd
made modiications to the collect method in this new stage for profile…
cavestruz Jul 22, 2023
b492ffb
Minor thing
combet Aug 29, 2023
970ff4c
Merge branch 'master' into issue/307/CL_individual_profile_stage
combet Aug 29, 2023
2417c76
Change column name source_bin --> bin so following the change merged …
combet Sep 7, 2023
e9d1787
Comment out line currently giving error when importing txpipe
combet Sep 7, 2023
efe8f53
small test
combet Oct 17, 2023
db604f3
Merge branch 'master' into issue/307/CL_individual_profile_stage
combet Oct 17, 2023
446c980
First attempt to build CLMM cluster ensemble object and compute stack…
combet Oct 18, 2023
f7f5f5f
Reorder
combet Oct 18, 2023
ab67fa9
Switching to pre-defined radial bins
combet Oct 18, 2023
20517c1
Extend to 10 Mpc to avoid empty bins
combet Oct 18, 2023
b9b5d30
Comment out MPI-specific code and add explanatory comments for the ne…
combet Oct 18, 2023
403a5d0
Update select.py
marina-ricci Feb 12, 2024
77a4665
Update make_ensemble_profile.py
marina-ricci Feb 12, 2024
9fc9f83
Update make_ensemble_profile.py
marina-ricci Feb 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion examples/cosmodc2/config-20deg2-clmm.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ Inform_BPZ_lite:
zp_errors: [0.01, 0.01, 0.01]
hdf5_groupname: photometry



BPZ_lite:
aliases:
Expand Down
44 changes: 44 additions & 0 deletions examples/cosmodc2/pipeline-20deg2-CLdev.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
launcher:
name: mini
interval: 3.0

site:
name: cc-parallel
mpi_command: "mpirun -n"


modules: >
txpipe
rail.estimation.algos.bpz_lite

python_paths: []

stages:
- name: TXSourceSelectorMetadetect
nprocess: 30
- name: Inform_BPZ_lite
nprocess: 1
- name: BPZ_lite
nprocess: 30
- name: CLClusterShearCatalogs
nprocess: 30
- name: CLClusterEnsembleProfiles
nprocess: 30


output_dir: data/cosmodc2/outputs-20deg2
config: examples/cosmodc2/config-20deg2-clmm.yml

inputs:
# See README for paths to download these files
shear_catalog: ./data/cosmodc2/20deg2/shear_catalog.hdf5
photometry_catalog: ./data/cosmodc2/20deg2/photometry_catalog.hdf5
fiducial_cosmology: data/fiducial_cosmology.yml
calibration_table: ./data/cosmodc2/20deg2/sample_cosmodc2_w10year_errors.dat
spectroscopic_catalog: ./data/cosmodc2/20deg2/spectroscopic_catalog.hdf5
cluster_catalog: ./data/cosmodc2/20deg2/cluster_catalog.hdf5

resume: True
log_dir: data/cosmodc2/logs
pipeline_log: data/cosmodc2/log.txt

45 changes: 45 additions & 0 deletions examples/cosmodc2/pipeline-20deg2-clmm-nersc-test2.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
launcher:
name: mini
interval: 3.0

site:
name: cori-batch
image: ghcr.io/lsstdesc/txpipe-dev


modules: >
txpipe
rail.estimation.algos.bpz_lite

python_paths: []

stages:
- name: TXSourceSelectorMetadetect
nprocess: 30
- name: Inform_BPZ_lite
nprocess: 1
- name: BPZ_lite
nprocess: 30
- name: CLClusterShearCatalogs
nprocess: 30
- name: CLClusterEnsembleProfiles
nprocess: 30



output_dir: data/cosmodc2/outputs-20deg2
config: examples/cosmodc2/config-20deg2-clmm.yml

inputs:
# See README for paths to download these files
shear_catalog: ./data/cosmodc2/20deg2/shear_catalog.hdf5
photometry_catalog: ./data/cosmodc2/20deg2/photometry_catalog.hdf5
fiducial_cosmology: data/fiducial_cosmology.yml
calibration_table: ./data/cosmodc2/20deg2/sample_cosmodc2_w10year_errors.dat
spectroscopic_catalog: ./data/cosmodc2/20deg2/spectroscopic_catalog.hdf5
cluster_catalog: ./data/cosmodc2/20deg2/cluster_catalog.hdf5

resume: True
log_dir: data/cosmodc2/logs
pipeline_log: data/cosmodc2/log.txt

4 changes: 2 additions & 2 deletions notebooks/Reading TXPipe Outputs.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -562,7 +562,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand All @@ -576,7 +576,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.4"
"version": "3.8.5"
}
},
"nbformat": 4,
Expand Down
Loading