From 44d1fa767cf8ededf354a76665cfcc24c0758e75 Mon Sep 17 00:00:00 2001 From: luthienliu Date: Sat, 6 Aug 2022 17:01:48 -0400 Subject: [PATCH 1/3] Change to 4_Coordinates_Crossmatch.ipynb --- .../4_Coordinates_Crossmatch.ipynb | 557 ++++++++++++++++++ .../astropy-coordinates/requirements.txt | 4 - 2 files changed, 557 insertions(+), 4 deletions(-) create mode 100644 tutorials/astropy-coordinates/4_Coordinates_Crossmatch.ipynb delete mode 100644 tutorials/astropy-coordinates/requirements.txt diff --git a/tutorials/astropy-coordinates/4_Coordinates_Crossmatch.ipynb b/tutorials/astropy-coordinates/4_Coordinates_Crossmatch.ipynb new file mode 100644 index 00000000..c2ddd557 --- /dev/null +++ b/tutorials/astropy-coordinates/4_Coordinates_Crossmatch.ipynb @@ -0,0 +1,557 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "pvmxjgnDopS6" + }, + "source": [ + "# Astronomical Coordinates 4: Cross-matching Catalogs Using astropy.coordinates and astroquery\n", + "\n", + "## Authors\n", + "Adrian Price-Whelan, Lúthien Liu, Zihao Chen, Saima Siddiqui\n", + "\n", + "## Learning Goals\n", + "* Demonstrate how to retrieve a catalog from Vizier using astroquery\n", + "* Show how to perform positional cross-matches between catalogs of sky coordinates\n", + "\n", + "## Keywords\n", + "coordinates, OOP, astroquery, gaia\n", + "\n", + "\n", + "## Summary\n", + "\n", + "In the previous tutorials in this series, we introduced many of the key concepts underlying how to represent and transform astronomical coordinates using `astropy.coordinates`, including how to work with both position and velocity data within the coordinate objects.\n", + "\n", + "In this tutorial, we will explore how the `astropy.coordinates` package can be used to cross-match two catalogs that contain overlapping sources that may have been observed at different times. You may find it helpful to keep [the Astropy documentation for the coordinates package](http://docs.astropy.org/en/stable/coordinates/index.html) open alongside this tutorial for reference or additional reading. In the text below, you may also see some links that look like ([docs](http://docs.astropy.org/en/stable/coordinates/index.html)). These links will take you to parts of the documentation that are directly relevant to the cells from which they link. \n", + "\n", + "*Note: This is the 4th tutorial in a series of tutorials about astropy.coordinates. If you are new to astropy.coordinates, you may want to start from the beginning or an earlier tutorial.*\n", + "- [Previous tutorial: Astronomical Coordinates 3: Working with Velocity Data](3-Coordinates-Velocities)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "sKhydue_opS-" + }, + "source": [ + "## Imports\n", + "\n", + "We start by importing some general packages that we will need below:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "741ulTSBopS_", + "outputId": "1fea5dc8-ffd2-4aae-b9b6-d5c3ea534a2f" + }, + "outputs": [], + "source": [ + "import warnings\n", + "\n", + "import matplotlib.pyplot as plt\n", + "%matplotlib inline\n", + "import numpy as np\n", + "\n", + "from astropy import units as u\n", + "from astropy.coordinates import SkyCoord, Distance\n", + "from astropy.io import fits\n", + "from astropy.table import QTable\n", + "from astropy.time import Time\n", + "from astropy.utils.data import download_file\n", + "\n", + "from astroquery.vizier import Vizier" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ZKJjNwxiopTB" + }, + "source": [ + "## Cross-matching and comparing catalogs\n", + "\n", + "In this tutorial, we are going to return to a set of data that we downloaded from the *Gaia* archive back in [Tutorial 1](1-Coordinates-Intro) of this series.\n", + "\n", + "Let's recap what we did in that tutorial: We defined a `SkyCoord` object to represent the center of an open cluster (NGC 188), we queried the *Gaia* DR2 catalog to select stars that are close (on the sky) to the center of the cluster, and we used the parallax values from *Gaia* to select stars that are near NGC 188 in 3D position. Here, we will briefly reproduce those selections so that we can start here with a catalog of sources that are likely members of NGC 188 (see [Tutorial 1](1-Coordinates-Intro) for more information):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "pEWYbW8UopTC", + "outputId": "0fed4911-59bc-4980-d4f2-9319ffe76dec" + }, + "outputs": [], + "source": [ + "ngc188_table = QTable.read('gaia_results.fits')\n", + "ngc188_table = ngc188_table[ngc188_table['parallax'] > 0.25*u.mas]\n", + "\n", + "ngc188_center_3d = SkyCoord(12.11*u.deg, 85.26*u.deg,\n", + " distance=1.96*u.kpc,\n", + " pm_ra_cosdec=-2.3087*u.mas/u.yr,\n", + " pm_dec=-0.9565*u.mas/u.yr)\n", + "\n", + "# Deal with masked quantity data in a backwards-compatible way:\n", + "parallax = ngc188_table['parallax']\n", + "if hasattr(parallax, 'mask'):\n", + " parallax = parallax.filled(np.nan)\n", + " \n", + "velocity_data = {\n", + " 'pm_ra_cosdec': ngc188_table['pmra'],\n", + " 'pm_dec': ngc188_table['pmdec'],\n", + " 'radial_velocity': ngc188_table['radial_velocity']\n", + "}\n", + "for k, v in velocity_data.items():\n", + " if hasattr(v, 'mask'):\n", + " velocity_data[k] = v.filled(0.)\n", + " velocity_data[k][np.isnan(velocity_data[k])] = 0.\n", + "\n", + "ngc188_coords_3d = SkyCoord(\n", + " ra=ngc188_table['ra'], \n", + " dec=ngc188_table['dec'],\n", + " distance=Distance(parallax=parallax),\n", + " obstime=Time('J2015.5'),\n", + " **velocity_data\n", + ")\n", + "\n", + "sep3d = ngc188_coords_3d.separation_3d(ngc188_center_3d)\n", + "pm_diff = np.sqrt(\n", + " (ngc188_table['pmra'] - ngc188_center_3d.pm_ra_cosdec)**2 + \n", + " (ngc188_table['pmdec'] - ngc188_center_3d.pm_dec)**2)\n", + "\n", + "ngc188_members_mask = (sep3d < 50*u.pc) & (pm_diff < 1.5*u.mas/u.yr)\n", + "ngc188_members = ngc188_table[ngc188_members_mask]\n", + "ngc188_members_coords = ngc188_coords_3d[ngc188_members_mask]\n", + "len(ngc188_members)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "HH-OAdGiopTD" + }, + "source": [ + "From the selections above, the table `ngc188_members` and the `SkyCoord` instance `ngc188_members_coords` contain 216 sources that, based on their 3D positions and proper motions, are consistent with being members of the open cluster NGC 188." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "j3vx-xpzopTE" + }, + "source": [ + "Let's assume that we now want to cross-match our catalog of candidate members of NGC 188 — here, based on *Gaia* data — to some other catalog. In this tutorial, we will demonstrate how to manually cross-match these *Gaia* sources with the 2MASS photometric catalog to retrieve infrared magnitudes for these stars, and then we will plot a color–magnitude diagram. To do this, we first need to query the 2MASS catalog to retrieve all sources in a region around the center of NGC 188, as we did for *Gaia*. Here, we will also take into account the fact that the *Gaia* data release 2 reference epoch is J2015.5, whereas the 2MASS coordinates are likely reported at their time of observation (in the late 1990's). \n", + "\n", + "*Note that some data archives, like the Gaia science archive, support running cross-matches at the database level and even support epoch propagation. If you need to perform a large cross-match, it will be much more efficient to use these services!*\n", + "\n", + "We will again use `astroquery` to execute this query. This will again require an internet connection, but we have included the results of this query in a file along with this notebook in case you are not connected to the internet. To query 2MASS, we will use the `astroquery.vizier` module ([docs](https://astroquery.readthedocs.io/en/latest/vizier/vizier.html)) to run a cone search centered on the sky position of NGC 188 with a search radius of 0.5º:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "nKlO1UBmopTF" + }, + "outputs": [], + "source": [ + "# NOTE: skip this cell if you do not have an internet connection\n", + "\n", + "# II/246 is the catalog name for the main 2MASS photometric catalog\n", + "v = Vizier(catalog=\"II/246\", columns=['*', 'Date']) \n", + "v.ROW_LIMIT = -1\n", + "\n", + "result = v.query_region(ngc188_center_3d, radius=0.5*u.deg)\n", + "tmass_table = result[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "et4ax7AqopTF" + }, + "source": [ + "Alternatively, we can read the 2MASS table provided along with this tutorial:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "_KNyVfsjopTG" + }, + "outputs": [], + "source": [ + "# the .read() below produces some warnings that we can safely ignore\n", + "with warnings.catch_warnings(): \n", + " warnings.simplefilter('ignore', UserWarning)\n", + " \n", + " tmass_table = QTable.read('2MASS_results.ecsv')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "gH3GmuxEopTH" + }, + "source": [ + "As with the *Gaia* results table, we can now create a single `SkyCoord` object to represent all of the sources returned from our query to the 2MASS catalog. Let's look at the column names in this table by displaying the first few rows:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 165 + }, + "id": "rfe_mIhGopTI", + "outputId": "50270998-1eb3-459d-89c4-22da8078b643" + }, + "outputs": [], + "source": [ + "tmass_table[:3]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "m3bAC2kCopTI" + }, + "source": [ + "From looking at the column names, the two relevant sky coordinate columns are `RAJ2000` for `ra` and `DEJ2000` for `dec`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "LWFl1JM9opTJ", + "outputId": "d201a13c-7659-4262-9b75-b6bc26f0a19c" + }, + "outputs": [], + "source": [ + "tmass_coords = SkyCoord(tmass_table['RAJ2000'], \n", + " tmass_table['DEJ2000'])\n", + "len(tmass_coords)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "cWGVnrWRopTJ" + }, + "source": [ + "Note also that the table contains a \"Date\" column that specifies the epoch of the coordinates. Are all of these epochs the same?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 58 + }, + "id": "djfwkEGbopTK", + "outputId": "9ab50944-fc07-49e4-9fb2-d7e3f359de7c" + }, + "outputs": [], + "source": [ + "np.unique(tmass_table['Date'])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "2HVHIOzPopTK" + }, + "source": [ + "It looks like all of the sources in our 2MASS table have the same epoch, so let's create an `astropy.time.Time` object to represent this date:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "1KeO3N9AopTK" + }, + "outputs": [], + "source": [ + "tmass_epoch = Time(np.unique(tmass_table['Date']))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "BTV_a1GlopTL" + }, + "source": [ + "We now want to cross-match our *Gaia*-selected candidate members of NGC 188, `ngc_members_coords`, with this table of photometry from 2MASS. However, as noted previously, the *Gaia* coordinates are given at a different epoch J2015.5, which is nearly ~16 years after the 2MASS epoch of the data we downloaded (1999-10-19 or roughly J1999.88). We will therefore first use the `SkyCoord.apply_space_motion()` method ([docs](http://docs.astropy.org/en/latest/api/astropy.coordinates.SkyCoord.html#astropy.coordinates.SkyCoord.apply_space_motion)) to transform the *Gaia* positions back to the 2MASS epoch before we do the cross-match:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Krr18s9gopTM" + }, + "outputs": [], + "source": [ + "# you can ignore the warning raised here\n", + "ngc188_members_coords_1999 = ngc188_members_coords.apply_space_motion(\n", + " tmass_epoch)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "6RVxtRLdopTM" + }, + "source": [ + "The object `ngc188_members_coords_1999` now contains the coordinate information for our *Gaia*-selected members of NGC 188, as we think they would appear if observed on 1999-10-19.\n", + "\n", + "We can now use the ``SkyCoord.match_to_catalog_sky`` method to match these two catalogs ([docs](http://docs.astropy.org/en/latest/coordinates/matchsep.html#astropy-coordinates-matching)), using the `ngc188_members_coords_1999` as our NGC 188 members coordinates. \n", + "\n", + "Note that order matters with this method: Here we will match *Gaia* to 2MASS. `SkyCoord.match_to_catalog_sky` returns three objects: (1) the indices into `tmass_coords` that get the closest matches in `ngc188_members_coords_1999`, (2) the angular separation between each `ngc188_members_coords_1999` coordinate and the closest source in `tmass_coords`, and (3) the 3D distance between each `ngc188_members_coords_1999` coordinate and the closest source in `tmass_coords`. Here, the 3D distances will not be useful because the 2MASS coordinates do not have associated distance information, so we will ignore these quantities:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "UZHtgJ00opTM" + }, + "outputs": [], + "source": [ + "idx_gaia, sep2d_gaia, _ = ngc188_members_coords_1999.match_to_catalog_sky(\n", + " tmass_coords)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "t1qUzhHWopTN" + }, + "source": [ + "Let's now look at the distribution of separations (in arcseconds) for all of the cross-matched sources:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 297 + }, + "id": "Acm50SzTopTN", + "outputId": "490adeed-9d80-4720-9576-c9293aafd15b" + }, + "outputs": [], + "source": [ + "plt.hist(sep2d_gaia.arcsec, histtype='step', \n", + " bins=np.logspace(-2, 2., 64))\n", + "plt.xlabel('separation [arcsec]')\n", + "plt.xscale('log')\n", + "plt.yscale('log')\n", + "plt.tight_layout()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "SG0nUr7HopTN" + }, + "source": [ + "From this, it looks like all of sources in our *Gaia* NGC 188 member list cross-match to another sources within a few arcseconds, so these all seem like they are correctly matches to a 2MASS source!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "5bk543X9opTN", + "outputId": "5af58652-f62b-41fc-9dbf-172dc0a76149" + }, + "outputs": [], + "source": [ + "(sep2d_gaia < 2*u.arcsec).sum(), len(ngc188_members)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "kXdqAdfnopTO" + }, + "source": [ + "With our cross-match done, we can now make `Gaia`+2MASS color–magnitude diagrams of our candidate NGC 188 members using the information returned by the cross-match:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "OHI0B-TYopTO" + }, + "outputs": [], + "source": [ + "Jmag = tmass_table['Jmag'][idx_gaia] # note that we use the index array returned above\n", + "Gmag = ngc188_members['phot_g_mean_mag']\n", + "Bmag = ngc188_members['phot_bp_mean_mag']" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 369 + }, + "id": "fuePUx8uopTO", + "outputId": "46515b44-4c91-4615-b830-3211c9252843" + }, + "outputs": [], + "source": [ + "fig, axes = plt.subplots(1, 2, figsize=(10, 5))\n", + "\n", + "ax = axes[0]\n", + "ax.scatter(Gmag - Jmag, Gmag, \n", + " marker='o', color='k', \n", + " linewidth=0, alpha=0.5)\n", + "ax.set_xlabel('$G - J$')\n", + "ax.set_ylabel('$G$')\n", + "ax.set_xlim(0, 3)\n", + "ax.set_ylim(19, 10) # backwards because magnitudes!\n", + "\n", + "ax = axes[1]\n", + "ax.scatter(Bmag - Gmag, Jmag, \n", + " marker='o', color='k', \n", + " linewidth=0, alpha=0.5)\n", + "ax.set_xlabel('$G_{BP} - G$')\n", + "ax.set_ylabel('$J$')\n", + "ax.set_xlim(0.2, 1)\n", + "ax.set_ylim(17, 8) # backwards because magnitudes!\n", + "\n", + "fig.tight_layout()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "01UZ3GFVopTP" + }, + "source": [ + "Those both look like color-magnitude diagrams of a main sequence + red giant branch of an intermediate-age stellar cluster, so it looks like our selection and cross-matching has worked!\n", + "\n", + "For more on what matching options are available, check out the [separation and matching section of the Astropy documentation](https://astropy.readthedocs.io/en/stable/coordinates/matchsep.html). Or for more on what you can do with `SkyCoord`, see [its API documentation](http://astropy.readthedocs.org/en/stable/api/astropy.coordinates.SkyCoord.html)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "95uKx2_E-Lfd" + }, + "source": [ + "## Exercises" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "J4TWmYPf-O-z" + }, + "source": [ + "Instead of transforming the Gaia positions back to the 2MASS epoch for cross-matching, try to move the 2MASS positions to the Gaia's epoch (J2015.5), and name it as `tmass_coords_2015`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "EfNzCmP1opTP" + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "pZyezlr--Uyo" + }, + "source": [ + "Now apply the `SkyCoord.match_to_catalog_sky` method to do a new cross-matching between these 2 catalogs, using the `tmass_coords_2015` as our NGC 188 members coordinates this time. Plot a histogram to show the distribution of separation. Is there any difference between this trial and the result above?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Th2k_vW_-XiP" + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "a0pCmK01-349" + }, + "source": [ + "Instead of picking all stars within 50 pc of the cluster center, try to enlarge to a larger range of 80 pc for cross-match and make a comparison between the result above." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "gCKRdYix-704" + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "colab": { + "name": "4-Coordinates-Crossmatch.ipynb", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.12" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} diff --git a/tutorials/astropy-coordinates/requirements.txt b/tutorials/astropy-coordinates/requirements.txt deleted file mode 100644 index f3cb4ae9..00000000 --- a/tutorials/astropy-coordinates/requirements.txt +++ /dev/null @@ -1,4 +0,0 @@ -astropy -astroquery -matplotlib -numpy From d5d947dfeee5dcec61ec21de99e7fa41a1b7d711 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat, 6 Aug 2022 21:03:36 +0000 Subject: [PATCH 2/3] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- .../astropy-coordinates/4_Coordinates_Crossmatch.ipynb | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/tutorials/astropy-coordinates/4_Coordinates_Crossmatch.ipynb b/tutorials/astropy-coordinates/4_Coordinates_Crossmatch.ipynb index c2ddd557..df617c72 100644 --- a/tutorials/astropy-coordinates/4_Coordinates_Crossmatch.ipynb +++ b/tutorials/astropy-coordinates/4_Coordinates_Crossmatch.ipynb @@ -534,11 +534,6 @@ "name": "4-Coordinates-Crossmatch.ipynb", "provenance": [] }, - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, "language_info": { "codemirror_mode": { "name": "ipython", @@ -548,8 +543,7 @@ "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.12" + "pygments_lexer": "ipython3" } }, "nbformat": 4, From 2370957ff636da65f9cef46156736bbe6c8ea73e Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 18 Aug 2022 02:11:07 +0000 Subject: [PATCH 3/3] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- .../4_Coordinates_Crossmatch.ipynb | 194 ++++-------------- 1 file changed, 39 insertions(+), 155 deletions(-) diff --git a/tutorials/astropy-coordinates/4_Coordinates_Crossmatch.ipynb b/tutorials/astropy-coordinates/4_Coordinates_Crossmatch.ipynb index df617c72..13d222c4 100644 --- a/tutorials/astropy-coordinates/4_Coordinates_Crossmatch.ipynb +++ b/tutorials/astropy-coordinates/4_Coordinates_Crossmatch.ipynb @@ -2,9 +2,7 @@ "cells": [ { "cell_type": "markdown", - "metadata": { - "id": "pvmxjgnDopS6" - }, + "metadata": {}, "source": [ "# Astronomical Coordinates 4: Cross-matching Catalogs Using astropy.coordinates and astroquery\n", "\n", @@ -31,9 +29,7 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "sKhydue_opS-" - }, + "metadata": {}, "source": [ "## Imports\n", "\n", @@ -43,13 +39,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "741ulTSBopS_", - "outputId": "1fea5dc8-ffd2-4aae-b9b6-d5c3ea534a2f" - }, + "metadata": {}, "outputs": [], "source": [ "import warnings\n", @@ -70,9 +60,7 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "ZKJjNwxiopTB" - }, + "metadata": {}, "source": [ "## Cross-matching and comparing catalogs\n", "\n", @@ -84,13 +72,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "pEWYbW8UopTC", - "outputId": "0fed4911-59bc-4980-d4f2-9319ffe76dec" - }, + "metadata": {}, "outputs": [], "source": [ "ngc188_table = QTable.read('gaia_results.fits')\n", @@ -137,18 +119,14 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "HH-OAdGiopTD" - }, + "metadata": {}, "source": [ "From the selections above, the table `ngc188_members` and the `SkyCoord` instance `ngc188_members_coords` contain 216 sources that, based on their 3D positions and proper motions, are consistent with being members of the open cluster NGC 188." ] }, { "cell_type": "markdown", - "metadata": { - "id": "j3vx-xpzopTE" - }, + "metadata": {}, "source": [ "Let's assume that we now want to cross-match our catalog of candidate members of NGC 188 — here, based on *Gaia* data — to some other catalog. In this tutorial, we will demonstrate how to manually cross-match these *Gaia* sources with the 2MASS photometric catalog to retrieve infrared magnitudes for these stars, and then we will plot a color–magnitude diagram. To do this, we first need to query the 2MASS catalog to retrieve all sources in a region around the center of NGC 188, as we did for *Gaia*. Here, we will also take into account the fact that the *Gaia* data release 2 reference epoch is J2015.5, whereas the 2MASS coordinates are likely reported at their time of observation (in the late 1990's). \n", "\n", @@ -160,9 +138,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "nKlO1UBmopTF" - }, + "metadata": {}, "outputs": [], "source": [ "# NOTE: skip this cell if you do not have an internet connection\n", @@ -177,9 +153,7 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "et4ax7AqopTF" - }, + "metadata": {}, "source": [ "Alternatively, we can read the 2MASS table provided along with this tutorial:" ] @@ -187,9 +161,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "_KNyVfsjopTG" - }, + "metadata": {}, "outputs": [], "source": [ "# the .read() below produces some warnings that we can safely ignore\n", @@ -201,9 +173,7 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "gH3GmuxEopTH" - }, + "metadata": {}, "source": [ "As with the *Gaia* results table, we can now create a single `SkyCoord` object to represent all of the sources returned from our query to the 2MASS catalog. Let's look at the column names in this table by displaying the first few rows:" ] @@ -211,14 +181,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 165 - }, - "id": "rfe_mIhGopTI", - "outputId": "50270998-1eb3-459d-89c4-22da8078b643" - }, + "metadata": {}, "outputs": [], "source": [ "tmass_table[:3]" @@ -226,9 +189,7 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "m3bAC2kCopTI" - }, + "metadata": {}, "source": [ "From looking at the column names, the two relevant sky coordinate columns are `RAJ2000` for `ra` and `DEJ2000` for `dec`:" ] @@ -236,13 +197,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "LWFl1JM9opTJ", - "outputId": "d201a13c-7659-4262-9b75-b6bc26f0a19c" - }, + "metadata": {}, "outputs": [], "source": [ "tmass_coords = SkyCoord(tmass_table['RAJ2000'], \n", @@ -252,9 +207,7 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "cWGVnrWRopTJ" - }, + "metadata": {}, "source": [ "Note also that the table contains a \"Date\" column that specifies the epoch of the coordinates. Are all of these epochs the same?" ] @@ -262,14 +215,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 58 - }, - "id": "djfwkEGbopTK", - "outputId": "9ab50944-fc07-49e4-9fb2-d7e3f359de7c" - }, + "metadata": {}, "outputs": [], "source": [ "np.unique(tmass_table['Date'])" @@ -277,9 +223,7 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "2HVHIOzPopTK" - }, + "metadata": {}, "source": [ "It looks like all of the sources in our 2MASS table have the same epoch, so let's create an `astropy.time.Time` object to represent this date:" ] @@ -287,9 +231,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "1KeO3N9AopTK" - }, + "metadata": {}, "outputs": [], "source": [ "tmass_epoch = Time(np.unique(tmass_table['Date']))" @@ -297,9 +239,7 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "BTV_a1GlopTL" - }, + "metadata": {}, "source": [ "We now want to cross-match our *Gaia*-selected candidate members of NGC 188, `ngc_members_coords`, with this table of photometry from 2MASS. However, as noted previously, the *Gaia* coordinates are given at a different epoch J2015.5, which is nearly ~16 years after the 2MASS epoch of the data we downloaded (1999-10-19 or roughly J1999.88). We will therefore first use the `SkyCoord.apply_space_motion()` method ([docs](http://docs.astropy.org/en/latest/api/astropy.coordinates.SkyCoord.html#astropy.coordinates.SkyCoord.apply_space_motion)) to transform the *Gaia* positions back to the 2MASS epoch before we do the cross-match:" ] @@ -307,9 +247,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "Krr18s9gopTM" - }, + "metadata": {}, "outputs": [], "source": [ "# you can ignore the warning raised here\n", @@ -319,9 +257,7 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "6RVxtRLdopTM" - }, + "metadata": {}, "source": [ "The object `ngc188_members_coords_1999` now contains the coordinate information for our *Gaia*-selected members of NGC 188, as we think they would appear if observed on 1999-10-19.\n", "\n", @@ -333,9 +269,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "UZHtgJ00opTM" - }, + "metadata": {}, "outputs": [], "source": [ "idx_gaia, sep2d_gaia, _ = ngc188_members_coords_1999.match_to_catalog_sky(\n", @@ -344,9 +278,7 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "t1qUzhHWopTN" - }, + "metadata": {}, "source": [ "Let's now look at the distribution of separations (in arcseconds) for all of the cross-matched sources:" ] @@ -354,14 +286,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 297 - }, - "id": "Acm50SzTopTN", - "outputId": "490adeed-9d80-4720-9576-c9293aafd15b" - }, + "metadata": {}, "outputs": [], "source": [ "plt.hist(sep2d_gaia.arcsec, histtype='step', \n", @@ -374,9 +299,7 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "SG0nUr7HopTN" - }, + "metadata": {}, "source": [ "From this, it looks like all of sources in our *Gaia* NGC 188 member list cross-match to another sources within a few arcseconds, so these all seem like they are correctly matches to a 2MASS source!" ] @@ -384,13 +307,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "5bk543X9opTN", - "outputId": "5af58652-f62b-41fc-9dbf-172dc0a76149" - }, + "metadata": {}, "outputs": [], "source": [ "(sep2d_gaia < 2*u.arcsec).sum(), len(ngc188_members)" @@ -398,9 +315,7 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "kXdqAdfnopTO" - }, + "metadata": {}, "source": [ "With our cross-match done, we can now make `Gaia`+2MASS color–magnitude diagrams of our candidate NGC 188 members using the information returned by the cross-match:" ] @@ -408,9 +323,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "OHI0B-TYopTO" - }, + "metadata": {}, "outputs": [], "source": [ "Jmag = tmass_table['Jmag'][idx_gaia] # note that we use the index array returned above\n", @@ -421,14 +334,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 369 - }, - "id": "fuePUx8uopTO", - "outputId": "46515b44-4c91-4615-b830-3211c9252843" - }, + "metadata": {}, "outputs": [], "source": [ "fig, axes = plt.subplots(1, 2, figsize=(10, 5))\n", @@ -456,9 +362,7 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "01UZ3GFVopTP" - }, + "metadata": {}, "source": [ "Those both look like color-magnitude diagrams of a main sequence + red giant branch of an intermediate-age stellar cluster, so it looks like our selection and cross-matching has worked!\n", "\n", @@ -467,18 +371,14 @@ }, { "cell_type": "markdown", - "metadata": { - "id": "95uKx2_E-Lfd" - }, + "metadata": {}, "source": [ "## Exercises" ] }, { "cell_type": "markdown", - "metadata": { - "id": "J4TWmYPf-O-z" - }, + "metadata": {}, "source": [ "Instead of transforming the Gaia positions back to the 2MASS epoch for cross-matching, try to move the 2MASS positions to the Gaia's epoch (J2015.5), and name it as `tmass_coords_2015`." ] @@ -486,17 +386,13 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "EfNzCmP1opTP" - }, + "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", - "metadata": { - "id": "pZyezlr--Uyo" - }, + "metadata": {}, "source": [ "Now apply the `SkyCoord.match_to_catalog_sky` method to do a new cross-matching between these 2 catalogs, using the `tmass_coords_2015` as our NGC 188 members coordinates this time. Plot a histogram to show the distribution of separation. Is there any difference between this trial and the result above?" ] @@ -504,17 +400,13 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "Th2k_vW_-XiP" - }, + "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", - "metadata": { - "id": "a0pCmK01-349" - }, + "metadata": {}, "source": [ "Instead of picking all stars within 50 pc of the cluster center, try to enlarge to a larger range of 80 pc for cross-match and make a comparison between the result above." ] @@ -522,28 +414,20 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "gCKRdYix-704" - }, + "metadata": {}, "outputs": [], "source": [] } ], "metadata": { - "colab": { - "name": "4-Coordinates-Crossmatch.ipynb", - "provenance": [] - }, "language_info": { "codemirror_mode": { - "name": "ipython", - "version": 3 + "name": "ipython" }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3" + "nbconvert_exporter": "python" } }, "nbformat": 4,