-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Small demo notebook for team meeting (#26)
* Putting together small demo notebook for team meeting. * Add a zgrid definition. * Fix file path.
- Loading branch information
Showing
5 changed files
with
209 additions
and
5 deletions.
There are no files selected for viewing
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,202 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "This example notebook uses synthetic data produced by PZFlow in combination with several predefined SED templates and filter definition files." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from rail.estimation.algos.lephare import LephareInformer, LephareEstimator\n", | ||
| "import numpy as np\n", | ||
| "import lephare as lp\n", | ||
| "from rail.core.stage import RailStage\n", | ||
| "import matplotlib.pyplot as plt\n", | ||
| "\n", | ||
| "DS = RailStage.data_store\n", | ||
| "DS.__class__.allow_overwrite = True" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Here we load previously created synthetic data" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import tables_io\n", | ||
| "\n", | ||
| "trainFile = './data/output_table_conv_train.hdf5'\n", | ||
| "testFile = './data/output_table_conv_test.hdf5'\n", | ||
| "\n", | ||
| "traindata_io = tables_io.read(trainFile)\n", | ||
| "testdata_io = tables_io.read(testFile)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Retrieve all the required filter and template files" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "lephare_config_file = \"../src/rail/estimation/algos/lsst.para\"\n", | ||
| "lephare_config = lp.read_config(lephare_config_file)\n", | ||
| "\n", | ||
| "lp.data_retrieval.get_auxiliary_data(keymap=lephare_config)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "We use the inform stage to create the library of SEDs with various redshifts, extinction parameters, and reddening values. This typically takes ~3-4 minutes." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "inform_lephare = LephareInformer.make_stage(\n", | ||
| " name=\"inform_Lephare\",\n", | ||
| " nondetect_val=np.nan,\n", | ||
| " model=\"lephare.pkl\",\n", | ||
| " hdf5_groupname=\"\",\n", | ||
| ")\n", | ||
| "\n", | ||
| "inform_lephare.inform(traindata_io)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Now we take the sythetic test data, and find the best fits from the library. This results in a PDF, zmode, and zmean for each input test data. Takes ~2 minutes to run on 1500 inputs." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "estimate_lephare = LephareEstimator.make_stage(\n", | ||
| " name=\"test_Lephare\",\n", | ||
| " nondetect_val=np.nan,\n", | ||
| " model=inform_lephare.get_handle(\"model\"),\n", | ||
| " hdf5_groupname=\"\",\n", | ||
| " aliases=dict(input=\"test_data\", output=\"lephare_estim\"),\n", | ||
| ")\n", | ||
| "\n", | ||
| "lephare_estimated = estimate_lephare.estimate(testdata_io)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "An example lephare PDF and comparison to the true value" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "indx = 0\n", | ||
| "zgrid = np.linspace(0,3,300)\n", | ||
| "plt.plot(zgrid, np.squeeze(lephare_estimated.data[indx].pdf(zgrid)), label='Estimated PDF')\n", | ||
| "plt.axvline(x=testdata_io['redshift'][indx], color='r', label='True redshift')\n", | ||
| "plt.legend()\n", | ||
| "plt.xlabel('z')\n", | ||
| "plt.show()" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "More example fits" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "indxs = [8, 16, 32, 64, 128, 256, 512, 1024]\n", | ||
| "zgrid = np.linspace(0,3,300)\n", | ||
| "fig, axs = plt.subplots(2,4, figsize=(20,6))\n", | ||
| "for i, indx in enumerate(indxs):\n", | ||
| " ax = axs[i//4, i%4]\n", | ||
| " ax.plot(zgrid, np.squeeze(lephare_estimated.data[indx].pdf(zgrid)), label='Estimated PDF')\n", | ||
| " ax.axvline(x=testdata_io['redshift'][indx], color='r', label='True redshift')\n", | ||
| " ax.set_xlabel('z')" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Histogram of the absolute difference between lephare estimate and true redshift" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "estimate_diff_from_truth = np.abs(lephare_estimated.data.ancil['zmode'] - testdata_io['redshift'])\n", | ||
| "\n", | ||
| "plt.figure()\n", | ||
| "plt.hist(estimate_diff_from_truth, 100)\n", | ||
| "plt.xlabel('abs(z_estimated - z_true)')\n", | ||
| "plt.show()" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "new_rail", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.12.3" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters