Merge pull request #16 from lsst/tickets/DM-26603

tickets/DM-26603: Update to latest scarlet

.gitignore

@@ -4,12 +4,14 @@ build/
 var/
 dist/
 *.egg-info
+*.eggs
 *.so
 .ipynb_checkpoints/
 tmp/
 include/
 _eupspkg/
-
+_version.txt
+scarlet/_version.py
 docs/_build
 docs/api
 docs/_templates

.travis.yml

@@ -21,7 +21,7 @@ install:
   - conda info -a
 
   # Replace dep1 dep2 ... with your dependencies
-  - conda create -q -n test-environment python=$TRAVIS_PYTHON_VERSION numpy scipy astropy pybind11 numpydoc jupyter pytest
+  - conda create -q -n test-environment python=$TRAVIS_PYTHON_VERSION numpy scipy astropy pybind11 pytest pandoc
   - conda activate test-environment
   # Use pip to install our rtd required packages
   - pip install -r docs/rtd-pip-requirements
@@ -33,16 +33,3 @@ script:
 # Calculate coverage
 #after_success:
 #  - coveralls --config_file .coveragerc
-
-# deploy to github pages
-before_deploy:
-  - cd docs && make html && touch _build/html/.nojekyll && cd ..
-
-deploy:
-  provider: pages:git
-  cleanup: false
-  token: $github_token
-  local_dir: docs/_build/html/
-  edge: true
-  on:
-    branch: master

MANIFEST.in

@@ -0,0 +1 @@
+include _version.txt

docs/0-quickstart.ipynb

@@ -18,11 +18,11 @@
     "# Import Packages and setup\n",
     "import numpy as np\n",
     "import scarlet\n",
-    "import scarlet.display\n",
     "\n",
     "%matplotlib inline\n",
     "import matplotlib\n",
     "import matplotlib.pyplot as plt\n",
+    "\n",
     "# use a good colormap and don't interpolate the pixels\n",
     "matplotlib.rc('image', cmap='inferno', interpolation='none', origin='lower')"
    ]
@@ -57,6 +57,7 @@
    "metadata": {},
    "source": [
     "### Display Image Cube\n",
+    "\n",
     "This is an example of how to display an RGB image from an image cube of multiband data. In this case the image uses a $sinh^{-1}$ function to normalize the flux in each filter consistently to create an RGB image."
    ]
   },
@@ -71,12 +72,10 @@
     "stretch = 0.2\n",
     "Q = 10\n",
     "norm = AsinhMapping(minimum=0, stretch=stretch, Q=Q)\n",
+    "\n",
     "img_rgb = scarlet.display.img_to_rgb(images, norm=norm)\n",
     "plt.imshow(img_rgb)\n",
-    "\n",
-    "# Mark all of the sources from the detection cataog\n",
-    "for k, src in enumerate(catalog):\n",
-    "    plt.text(src[\"x\"], src[\"y\"], str(k), color=\"red\")"
+    "plt.show()"
    ]
   },
   {
@@ -92,7 +91,9 @@
   },
   {
    "cell_type": "raw",
-   "metadata": {},
+   "metadata": {
+    "raw_mimetype": "text/restructuredtext"
+   },
    "source": [
     ".. note::\n",
     "\n",
@@ -152,16 +153,57 @@
     "2. construct observation\n",
     "3. match it to the model frame\n",
     "\n",
-    "Steps 2 and 3 are combined above using a fluent pattern.\n",
+    "Steps 2 and 3 are combined above using a fluent pattern."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We can now make use of the convenience function `scarlet.display.show_observation` to plot observations as RGB images, with individual sources labeled at their position on the sky. It can also show the PSF in the same scaling."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "sky_coords = [[src['y'], src['x']] for src in catalog] # y/x!\n",
+    "scarlet.display.show_observation(observation, norm=norm, sky_coords=sky_coords, show_psf=True)\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Since we use a trivial `wcs` in this `Observation`, all coordinates are already in image pixels, otherwise RA/Dec pairs are expected as sky coordinates. Also:"
+   ]
+  },
+  {
+   "cell_type": "raw",
+   "metadata": {
+    "raw_mimetype": "text/restructuredtext"
+   },
+   "source": [
+    ".. warning::\n",
     "\n",
+    "    Coordinates in *scarlet* are given in the C/numpy notation (y,x) as opposed to the more conventional mathematical (x,y) ordering."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
     "## Initialize sources\n",
     "\n",
-    "You now need to define sources that are going to be fit. The full model, which we will call `Blend`, is a collection of those sources. We provide several pre-built source types:\n",
+    "You now need to define sources that are going to be fit. The full model, which we will call `Blend`, is a collection of those sources. We provide several pre-built source types, e.g.:\n",
     "\n",
     "* `RandomSource` fit per-band amplitude and non-parametric morphology starting from uniform random draws for both.\n",
     "* `PointSource` fits centers and per-band amplitude using the observed PSF model.\n",
     "* `ExtendedSource` fits per-band amplitude and a non-parametric morphology (which can be constrained to be symmetric and/or monotonic with respect to the center).\n",
-    "* `MultiComponentSource` splits an `ExtendedSource` into multiple components that are initially radially separated.\n",
+    "* `ExtendedSource(K=2)` splits an `ExtendedSource` into `K` components that are initially radially separated and stacked on top of each other like a pyramid.\n",
     "\n",
     "In our example, we assume *prior* knowledge that object 0 is a star, and object 1 should be modeled with two components. Everything else is assumed a single-component galaxy. **We generally recommend `ExtendedSource` as default** if additional information about the source is not available."
    ]
@@ -177,21 +219,12 @@
     "    if k == 0:\n",
     "        new_source = scarlet.PointSource(model_frame, (src['y'], src['x']), observation)\n",
     "    elif k == 1:\n",
-    "        new_source = scarlet.MultiComponentSource(model_frame, (src['y'], src['x']), observation)\n",
+    "        new_source = scarlet.ExtendedSource(model_frame, (src['y'], src['x']), observation, K=2)\n",
     "    else:\n",
     "        new_source = scarlet.ExtendedSource(model_frame, (src['y'], src['x']), observation)\n",
     "    sources.append(new_source)"
    ]
   },
-  {
-   "cell_type": "raw",
-   "metadata": {},
-   "source": [
-    ".. warning::\n",
-    "\n",
-    "    Coordinates in *scarlet* are given in the C/numpy notation (y,x) as opposed to the more conventional mathematical (x,y) ordering."
-   ]
-  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -207,11 +240,10 @@
    "outputs": [],
    "source": [
     "blend = scarlet.Blend(sources, observation)\n",
-    "%time blend.fit(200)\n",
-    "print(\"scarlet ran for {0} iterations to logL = {1}\".format(len(blend.loss), -blend.loss[-1]))\n",
-    "plt.plot(-np.array(blend.loss))\n",
-    "plt.xlabel('Iteration')\n",
-    "plt.ylabel('log-Likelihood')"
+    "%time it, logL = blend.fit(100, e_rel=1e-4)\n",
+    "print(f\"scarlet ran for {it} iterations to logL = {logL}\")\n",
+    "scarlet.display.show_likelihood(blend)\n",
+    "plt.show()"
    ]
   },
   {
@@ -237,9 +269,11 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Load the model and calculate the residual\n",
+    "# Compute model\n",
     "model = blend.get_model()\n",
+    "# Render it in the observed frame\n",
     "model_ = observation.render(model)\n",
+    "# Compute residual\n",
     "residual = images-model_\n",
     "\n",
     "# Create RGB images\n",
@@ -256,11 +290,12 @@
     "ax[2].imshow(residual_rgb)\n",
     "ax[2].set_title(\"Residual\")\n",
     "\n",
-    "for k,component in enumerate(blend):\n",
-    "    y,x = component.center\n",
-    "    ax[0].text(x, y, k, color=\"w\")\n",
-    "    ax[1].text(x, y, k, color=\"w\")\n",
-    "    ax[2].text(x, y, k, color=\"w\")\n",
+    "for k,src in enumerate(blend):\n",
+    "    if hasattr(src, \"center\"):\n",
+    "        y,x = src.center\n",
+    "        ax[0].text(x, y, k, color=\"w\")\n",
+    "        ax[1].text(x, y, k, color=\"w\")\n",
+    "        ax[2].text(x, y, k, color=\"w\")\n",
     "plt.show()"
    ]
   },
@@ -282,20 +317,24 @@
     "scarlet.display.show_sources(sources, \n",
     "                             norm=norm, \n",
     "                             observation=observation,\n",
+    "                             show_model=True,\n",
     "                             show_rendered=True, \n",
-    "                             show_observed=True)\n",
+    "                             show_observed=True,\n",
+    "                             add_markers=False,\n",
+    "                             add_boxes=True\n",
+    "                            )\n",
     "plt.show()"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We can see that each source \"lives\" in a smaller box and is then placed into the larger scene. The model of object 0 assumes the simple Gaussian shape of the model PSF, which is the internal representation of a point source. Source 1 uses the freedom of the 2-compoent model to represent a slightly redder core; the difference in SEDs is clearly noticeable.\n",
+    "We can see that each source \"lives\" in a smaller box and is then placed into the larger scene. The model of object 0 assumes the simple Gaussian shape of the model PSF, which is the internal representation of a point source. Source 1 uses the freedom of the 2-compoent model to represent a slightly redder core; the difference in spectrum is clearly noticeable.\n",
     "\n",
     "### Measure Fluxes\n",
     "\n",
-    "The color information in these plots stems from the per-band amplitude, which could be obtained as `source.sed`. However, it is more useful to compute per-band fluxes, which integrate over the morphology. The source SED plots above have done exactly that. The convention of these fluxes is given by the units and ordering of the original data cube. In the case of multi-component sources, the fluxes of all components are combined."
+    "The color information in these plots stems from the per-band amplitude, which could be obtained as `source.spectrum`. However, it is more useful to compute per-band fluxes, which integrate over the morphology. The source spectra plots above have done exactly that. The convention of these fluxes is given by the units and ordering of the original data cube. In the case of multi-component sources, the fluxes of all components are combined."
    ]
   },
   {
@@ -339,7 +378,9 @@
   },
   {
    "cell_type": "raw",
-   "metadata": {},
+   "metadata": {
+    "raw_mimetype": "text/restructuredtext"
+   },
    "source": [
     ".. note::\n",
     "\n",
@@ -363,7 +404,7 @@
     "sources_ = pickle.load(fp)\n",
     "fp.close()\n",
     "\n",
-    "scarlet.display.show_scene(sources_, norm=norm)\n",
+    "scarlet.display.show_scene(sources_, norm=norm, add_boxes=True)\n",
     "plt.show()"
    ]
   },
@@ -383,7 +424,7 @@
    "outputs": [],
    "source": [
     "# add two sources at their approximate locations\n",
-    "model_frame_ = sources_[0].model_frame\n",
+    "model_frame_ = sources_[0].frame\n",
     "yx = (14., 44.)\n",
     "new_source = scarlet.ExtendedSource(model_frame_, yx, observation, shifting=True)\n",
     "sources_.append(new_source)\n",
@@ -394,14 +435,9 @@
     "# generate a new Blend instance\n",
     "blend_ = scarlet.Blend(sources_, observation)\n",
     "# tighten relative change in likelihood for convergence\n",
-    "blend_.fit(200, e_rel=1e-4)\n",
-    "\n",
-    "# show convergence of logL\n",
-    "print(\"scarlet ran for {0} iterations to logL = {1}\".format(len(blend_.loss), -blend_.loss[-1]))\n",
-    "plt.plot(-np.array(blend_.loss), label='7+2 sources')\n",
-    "plt.xlabel('Iteration')\n",
-    "plt.ylabel('log-Likelihood')\n",
-    "plt.legend()\n",
+    "%time it_, logL_ = blend_.fit(100, e_rel=1e-4)\n",
+    "print(f\"scarlet ran for {it_} iterations to logL = {logL_}\")\n",
+    "scarlet.display.show_likelihood(blend_)\n",
     "plt.show()"
    ]
   },
@@ -431,19 +467,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We can see the two new sources 7 and 8, and that most of the features are very similar to before. As expected, the residuals have visibly improved and are now dominated by a Yin-Yang-shaped orange-blue pattern in the center of source 1, which we already fit with 2 components. Maybe we should try with 3 ..."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "nbsphinx": "hidden"
-   },
-   "outputs": [],
-   "source": [
-    "# minimal regression testing (hidden in sphinx)\n",
-    "np.testing.assert_almost_equal(-blend_.loss[-1], 31262.29884949654, decimal=2)"
+    "We can see the two new sources 7 and 8, and that most of the features are very similar to before. As expected, the residuals have visibly improved and are now dominated by a red-blue pattern in the center of source 1, which we already fit with 2 components. Maybe we should try with 3 ..."
    ]
   }
  ],
@@ -464,7 +488,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.3"
+   "version": "3.7.4"
   }
  },
  "nbformat": 4,