New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allows DataStoreMaker to be used with IRFs not following CALDB structure #3846
Merged
Merged
Changes from all commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
14524c3
read irf filename from events header without CALDB
QRemy d7d8834
add example in the notebook
QRemy f89498e
use expandvars before glob
QRemy a9ccfab
irf_paths in input
QRemy 8de837b
adapt notebook
QRemy 6be690c
docstring
QRemy fce9393
fix keyword
QRemy cea0cfc
events_paths list from loop
QRemy 0d45bfb
support only one path for irf
QRemy 241a737
remove unused import
QRemy 6d64268
use iterdir instead of glob
QRemy 197eb44
adapt tests
QRemy 3a4bd63
fix tests
QRemy 58f58a8
$CALDB replace for monkeypatch env
QRemy a07117c
move datastore for dc1 with monkeypatch into fixture
QRemy ec12488
remove unused import
QRemy 44f2eda
typo in notebook
QRemy 9b37bec
define caldb in docstring example
QRemy 81f6a80
notebook typo
QRemy File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -144,7 +144,7 @@ | |
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"filename = \"$GAMMAPY_DATA/cta-caldb/Prod5-South-20deg-AverageAz-14MSTs37SSTs.180000s-v0.1.fits.gz\"\n", | ||
"irf_filename = \"$GAMMAPY_DATA/cta-caldb/Prod5-South-20deg-AverageAz-14MSTs37SSTs.180000s-v0.1.fits.gz\"\n", | ||
"\n", | ||
"pointing = SkyCoord(0.0, 0.0, frame=\"galactic\", unit=\"deg\")\n", | ||
"livetime = 1 * u.hr" | ||
|
@@ -165,7 +165,7 @@ | |
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"irfs = load_cta_irfs(filename)\n", | ||
"irfs = load_cta_irfs(irf_filename)\n", | ||
"location = observatory_locations['cta_south']\n", | ||
"\n", | ||
"observation = Observation.create(\n", | ||
|
@@ -375,12 +375,11 @@ | |
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"events.table.meta[\"OBJECT\"] = dataset.models[0].name" | ||
"events.table.meta[\"OBJECT\"] = dataset.models[0].name\n" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "874b911e", | ||
"metadata": {}, | ||
"source": [ | ||
"Let's write the event list and its GTI extension to a FITS file. We make use of `fits` library in `astropy`:" | ||
|
@@ -747,8 +746,11 @@ | |
"outputs": [], | ||
"source": [ | ||
"%%time\n", | ||
"n_obs = len(tstarts)\n", | ||
"irf_paths = [Path(irf_filename)] * n_obs\n", | ||
"events_paths = []\n", | ||
"for idx, tstart in enumerate(tstarts):\n", | ||
"\n", | ||
" irfs = load_cta_irfs(irf_paths[idx])\n", | ||
" observation = Observation.create(\n", | ||
" obs_id=idx,\n", | ||
" pointing=pointing,\n", | ||
|
@@ -760,20 +762,20 @@ | |
"\n", | ||
" dataset = maker.run(empty, observation)\n", | ||
" dataset.models = models\n", | ||
"\n", | ||
" sampler = MapDatasetEventSampler(random_state=idx)\n", | ||
" events = sampler.run(dataset, observation)\n", | ||
" events.table.write(\n", | ||
" f\"./event_sampling/events_{idx:04d}.fits\", overwrite=True\n", | ||
" )" | ||
" \n", | ||
" path = Path(f\"./event_sampling/events_{idx:04d}.fits\")\n", | ||
" events_paths.append(path)\n", | ||
" events.table.write(path, overwrite=True)" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "626a8673", | ||
"metadata": {}, | ||
"source": [ | ||
"You can now load the event list with `Datastore.from_events_files()` and make your own analysis following the instructions in the [`analysis_2`](analysis_2.ipynb) tutorial." | ||
"You can now load the event list and the corresponding IRFs with `Datastore.from_events_files()` : " | ||
] | ||
}, | ||
{ | ||
|
@@ -784,8 +786,8 @@ | |
"outputs": [], | ||
"source": [ | ||
"path = Path(\"./event_sampling/\")\n", | ||
"paths = list(path.rglob(\"events*.fits\"))\n", | ||
"data_store = DataStore.from_events_files(paths)\n", | ||
"events_paths = list(path.rglob(\"events*.fits\"))\n", | ||
"data_store = DataStore.from_events_files(events_paths, irf_paths)\n", | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is probably where you could define the list of irf_path no? |
||
"data_store.obs_table" | ||
] | ||
}, | ||
|
@@ -794,7 +796,17 @@ | |
"id": "bbf88446", | ||
"metadata": {}, | ||
"source": [ | ||
"For completeness, `data_store` is a `~gammapy.data.Datastore` object. You can find more information about it [here](https://docs.gammapy.org/dev/tutorials/data/cta.html#Datastore)." | ||
"Then you can create the obervations from the Datastore and make your own analysis following the instructions in the [`analysis_2`](analysis_2.ipynb) tutorial." | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"observations = data_store.get_observations()\n", | ||
"observations[0].peek()" | ||
] | ||
}, | ||
{ | ||
|
@@ -812,6 +824,8 @@ | |
"id": "38a38c6a", | ||
"metadata": {}, | ||
"source": [ | ||
"For completeness, `data_store` is a `~gammapy.data.Datastore` object. You can find more information about it [here](https://docs.gammapy.org/dev/tutorials/data/cta.html#Datastore).\n", | ||
"\n", | ||
"## Exercises\n", | ||
"- Try to sample events for an extended source (e.g. a radial gaussian morphology);\n", | ||
"- Change the spatial model and the spectrum of the simulated Sky model;\n", | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it useful here? It is only when reading that you need the path to the irf now, no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general observations can have different irf files so the list of irfs_path have to be defined before the loop. I wanted to stress that in the notebook, even if it is not necessary for this use case (but we could use 2 different zenith angles for example).