Skip to content

Commit

Permalink
Merge aa0d7d6 into 21902eb
Browse files Browse the repository at this point in the history
  • Loading branch information
@jason-neal
jason-neal committed Mar 28, 2019
2 parents 21902eb + aa0d7d6 commit d9171c4
Show file tree
Hide file tree
Showing 70 changed files with 3,120 additions and 2,782 deletions.
21 changes: 21 additions & 0 deletions .readthedocs.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Read the Docs Configuration File

# Build PDF aswell as HTML and JSON
formats:
- pdf

# Requirements file
requirements_file:
requirements_dev.txt

build:
image: latest

python:
version: 3.6
setup_py_install: true
# From extras_require in setup.py
extra_requirements:
- dev
- tests
- docs
3 changes: 2 additions & 1 deletion .shippable.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ build:

cache_dir_list:
- $HOME/.cache/pip
- $HOME/data/test_data
- $HOME/tests/data
- $HOME/data

ci:
- pip install --upgrade pip
Expand Down
7 changes: 5 additions & 2 deletions .travis.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ python:
cache:
directories:
- $HOME/.cache/pip
- $HOME/data/test_data
- $HOME/tests/data
- $HOME/data

install:
- pip install --upgrade pip
Expand All @@ -29,12 +30,14 @@ before_script:
# Prepare atmosphere models
- make atmos V=1
# - python scripts/split_atmmodel.py -b K H J Y Z TEST
# - python scripts/bary_shift_atmmodel.py -b K H J Y Z TEST
# - python scripts/barycenter_broaden_atmmodel.py -b K H J Y Z TEST

script:
- pytest --cov=eniric --cov-report term-missing --durations 5
- coverage xml -i

- rstcheck -r docs

after_success:
- coveralls
- codeclimate-test-reporter
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ To get a list of commit messages since last version to help write change log try
- Example Notebooks.
- Add `Contributing.md` and `Contributors.md`.
- Zip required testing data, Delete unneeded data.
- Refactor `eniric.Qcalculator` module to `eniric.precision`.

Other:
- `Blacken` source code.
Expand Down
15 changes: 6 additions & 9 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@

#### Do you have more general questions about eniric?

* Please check the [documentation](https://github.com/jason-neal/eniric/wiki) to make sure your question is not answered there.
* Please check the [documentation](https://eniric.readthedocs.io/en/latest) to make sure your question is not answered there.
* If not, feel free to open a new issue

#### **Did you find a bug?**

* Please open a GitHub [issue](https://github.com/jason-neal/eniric/issues). to tell us about it.
* Please open a GitHub [issue](https://github.com/jason-neal/eniric/issues). to tell us about it.
Would be even more awesome if you could provide a possible solution!
  maybe check if the bug was not already reported (and solved) by searching the closed issues.

Expand All @@ -25,17 +25,14 @@


#### Coding conventions
The style guide for eniric is governed by [black](https://github.com/ambv/black) wth default settings.
You can run this yourself but it is also included in the pre-commit hooks.
The style guide for eniric is governed by [black](https://github.com/ambv/black) wth default settings.
You can run this yourself but it is also included in the pre-commit hooks.

Before commiting changes you can activate the [pre-commit hooks](https://github.com/pre-commit/pre-commit) using
Before commiting changes you can activate the [pre-commit hooks](https://github.com/pre-commit/pre-commit) using

pip install pre-commit
pre-commit install

This will then preform the pre-commit checks before commiting.

This will then preform the pre-commit checks before commiting.

Thank you!

(Adapted from [kima](https://github.com/j-faria/kima/blob/master/CONTRIBUTING.md))
2 changes: 1 addition & 1 deletion Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ clean-data:

atmos:
split_atmmodel.py -b ALL
bary_shift_atmmodel.py -b ALL
barycenter_broaden_atmmodel.py -b ALL

clean-build:
rm --force --recursive build/
Expand Down
218 changes: 44 additions & 174 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# [ENIRIC](https://github.com/jason-neal/eniric) - Extended Near InfraRed Information Content

[![Documentation Status](https://readthedocs.org/projects/eniric/badge/?version=latest)](https://eniric.readthedocs.io/en/latest/?badge=latest)
[![Python 3.6+](https://img.shields.io/badge/python-3.6+-blue.svg)](https://www.python.org/downloads/release/python-360/)
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/24d3d525a79d4ae493de8c527540edef)](https://www.codacy.com/app/jason-neal/eniric?utm_source=github.com&utm_medium=referral&utm_content=jason-neal/eniric&utm_campaign=badger)
[![Codacy Badge](https://api.codacy.com/project/badge/Coverage/24d3d525a79d4ae493de8c527540edef)](https://www.codacy.com/app/jason-neal/eniric?utm_source=github.com&utm_medium=referral&utm_content=jason-neal/eniric&utm_campaign=Badge_Coverage)
Expand All @@ -13,203 +14,70 @@
`Eniric` is an overhaul and extension to the code used in [Figueria et al. 2016](http://dx.doi.org/10.1051/0004-6361/201526900) to analysis the precision of M-dwarf stars.
Extending the performance and usability, it is able to be used on any synthetic spectra from the [PHOENIX-ACES](http://phoenix.astro.physik.uni-goettingen.de) and [BT-Settl](https://phoenix.ens-lyon.fr/Grids/BT-Settl/CIFIST2011_2015/FITS/) (CIFIST2001-2015) libraries.

Checkout the [wiki here](https://github.com/jason-neal/eniric/wiki)!
Checkout the documentation on [Read the Docs](https://eniric.readthedocs.io/en/latest/)!

## Features:
`Eniric` contains a number of features to transform and prepare the spectra (observed and synthetic).
- [Spectral broadening](https://github.com/jason-neal/eniric/wiki/Broadening)
- Rotational
- Instrumental
- [Atmospheric transmission masking](https://github.com/jason-neal/eniric/wiki/Atmospheric-Transmission)
- Relative RV precision
- The RV precision can be calculated relative to a specified SNR per pixel in the center of a spectroscopic band.
The default as used in the Figueira et al. 2016 is a SNR of 100 at the center of the J-band.
- Spectral re-sampling
- n pixels per FWHM
- Band selection
- Analysis in individual spectroscopic bands.
- Incremental quality & precision
- Synthetic libraries available
- Available through [Starfish]'s() grid_tools.
- [PHOENIX-ACES](http://phoenix.astro.physik.uni-goettingen.de)
- [BT-Settl](https://phoenix.ens-lyon.fr/Grids/BT-Settl/CIFIST2011_2015/FITS/)


# [Installation](https://github.com/jason-neal/eniric/wiki/Installation)
Currently to install `Eniric` you need to clone the repo:
```
git clone https://github.com/jason-neal/eniric
cd eniric
pip install -r requirements.txt
python setup.py develop
```

A pip installable version is in the works...

You also need to manually install [Starfish](https://github.com/iancze/Starfish)

```
git clone git@github.com:iancze/Starfish.git
cd Starfish
python setup.py build_ext --inplace
python setup.py develop
cd ..
```
see [here](https://github.com/iancze/Starfish) for more information about installing Starfish.
For **windows** installation of `Starfish` also see [starfish#87](see https://github.com/iancze/Starfish/issues/87).

##### Requirements for `Eniric` :
These should be automatically installed (if not present) when installing `eniric`.
- astropy
- joblib>=0.12.3
- matplotlib
- numpy
- pandas
- pyyaml
- scipy
- tqdm

If want to use known working pinned versions they can be found in `requirements_dev.txt`.
To install these run `pip install -r requirements_dev.txt`.

##### Extra requirements for Starfish:
- corner
- cython
- h5py
- scikit-learn

If you are not going to use `eniric` to analyze PHOENIX-ACES or BT-Settl synthetic spectral models then you may
get away with not installing it (some tests will just xfail).


## Preparation
#### Data download
To download the data for eniric, an atmopsheric transmission spectra and some test Spectra run the
following from the main repo directory

Linux:
`download_eniric_data.sh`

Windows:
`... `

This should place the data in `data/atmmodel` and `data/testdata` where it can be found for testing.

### Configuration
`Eniric` uses a `config.yaml` file which is required in directory where you are running `Eniric`. (i.e. the current directory)
to specify some paths, such as the location the the synthetic spectral library.

```
paths:
phoenix-raw: path/to/phoenix/aces/spectra
btsettl-raw: ["path", "to", "btsettl" ,"spectra"]
...
```
The paths can either be a string or a list of strings to pass to `os.path.join` (os independant).

You can use the `config.yaml` to specify custom wavelength ranges to use
```
bands:
all: [..., myband] # add myband to all list
custom_bands:
myband: [1.5, 1.6] # micron
```

You can then pass `myband` to the `band` arguments in `Eniric` scripts/functions.
- [Spectral broadening](https://eniric.readthedocs.io/en/latest/broadening.html)

This based off `Starfish` and although many keywords are needed to be present
for `Starfish` to run they are not used for `Eniric`'s usage of `Starfish` and are fine left blank.
Allows for Rotational and Instrumental broadening of synthetic spectra given a rotation speed ``vsini`` and resolution ``R``.

- [Atmospheric transmission masking](https://eniric.readthedocs.io/en/latest/telluric_corection.html)

#### Atmospheric data:
To perform telluric masking and account for the transmission of Earth's atmosphere a telluric spectra is required.
`Eniric` includes the telluric spectra uses in Figueira et al. 2016, averaged over 2014.
To automatically prepare the telluric masks, splitting into bands and applying the barycentric expansion run the following scripts:
- `split_atmmodel.py`
- `bary_shift_atmmodel.py`
Analyzing the RV precision attainable under the different masking conditions presented in `Figueira et al. 2016`_.
* No treatment of atmospheric transmission
* Masking all regions affected by atmospheric absorption of a given % over the course of the year.
* Assuming perfect telluric correction in which the variance of the measured flux is impacted.

These will split the large telluirc spectra into the bands specified in the `config.yaml` so that the
opening and slicing of the large telluric spectrum is not performed continually.

To change the telluric line cutoff depth you to 4% can pass (default = 2%) you can pass it like this

`split_atmmodel.py --cutoff-depth 4`

You can specify your own telluric mask instead.
By keeping it in the same format and setting atmmodel parameters in `config.yaml` you can make use of the
these scripts and the`Atmosphere` class which can perform the mask cutoff and doppler shifting.

Or you can manually apply your own masking function as the mask parameter to the `rv_precision` function.

### Convolutions
The most computational component in `Eniric` is the convolutions. To help with this we use parallel prcessing and caching.

- *Caching*:
The convolution results are cached using Joblib to avoid repeating the convoutions. This can be disabled by
setting the `location: None` in the `config.yaml`.

- *Parallel Processing*:
The default number of processors used is one less then the total number of cores (N-1).
You can change this by specifying the `num_procs`.
Setting `num_procs = 1` disables parallel processing.

## Usage
You can now calculate the theoretical RV precision for any PHOENIX-ACES model.
You will need to configure the path to the phoenix models in ´config.yaml´

e.g.

phoenix_precision.py -t 3900 -l 4.5, -m 0.5 -r 100000 -v 1.0 -b J K

Will calculate the RV precision in the `J` and `K`-band of the PHOENIX-ACES spectra with parameters \[Teff=3900K, logg=4.5, \[Fe/H\]=0.5\] observed at a resolution of 100,000 and rotating with 1.0 km/s.
For more details on the command line arguments to use see the [wiki](https://github.com/jason-neal/eniric/wiki) or type

phoenix_precision.py -h


# The Readme below this point needs amended....

## Outline

The code works in two main stages, "spectral preparation" and "precision calculation".

#### Spectrum preparation
- Relative RV precision

`eniric/nIRanalysis.py`
The RV precision can be calculated relative to a specified SNR per pixel in the center of a spectroscopic band.
The default as used in the Figueira et al. 2016 is a SNR of 100 at the center of the J-band.

This stage takes in the raw PHOENIX-ACES spectral models and transforms them, saving the results of this computation as .dat files.
- Spectral Resampling

It includes:
- Conversion from flux to photon counts.
- Resolution convolution
- Re-sampling
Allows for resampling of synthetic spectra to ``N`` pixels per FWHM.

Some scripts are given in `scripts` to run this preparation over all desired parameters automatically. You will have to modify the paths to things.
- SNR normalization.

Normalize spectral flux to a defined SNR level.

#### Precision Calculations
- Band selection

This takes in the processed spectra and performs the precision calculations for all 3 conditions outlined in the original paper.
- Cond1. Total information
- Cond2. +/-30km/s telluric line > 2% masking
- Cond3. Perfect telluric correction with variance correction
Analysis splitable into individual photometric bands ``Z``\ , ``Y``\ , ``J``\ , ``H``\ , ``K``.
User definable.

It also *scales the flux level* to a desired SNR level in a desired band, see below, as this affects the RV precision calculated. By default this is a SNR of 100 in the J band.
- Theoretical RV precision

Compute spectral RV precision and spectral quality.

## Band SNR Scaling.
By default, in accordance with the initial paper, each spectra band is normalized to 100 SNR in the center of the J band.
- Incremental quality & precision

This now does this automatically by measuring the SNR in 1 pixel resolution (3 points) in the center of the band. And scales accordingly. This adds a spectral model dependent factor on the RV precision.
To get around you can manually specify the SNR level to normalize to and which specific band to normalize to. (it can be itself for instance).
Determine the RV precision and spectral quality on narrow wavelength slices across the entire spectrum, similar to that present in Figure 1 of `Artigau et al. 2018 <http://adsabs.harvard.edu/abs/2018AJ....155..198A>`_.

* Analyse relative precision of synthetic libraries

The RV precision of are present relative to a specified SNR per pixel in the center of a photometric band.
The default as used in `Figueira et al. 2016`_ is a SNR of 100 at the center of the J-band.
- Available through [Starfish]'s() grid_tools.
- [PHOENIX-ACES](http://phoenix.astro.physik.uni-goettingen.de)
- [BT-Settl](https://phoenix.ens-lyon.fr/Grids/BT-Settl/CIFIST2011_2015/FITS/)

## Instructions

Copy config.yaml and adjust the paths relative to what you created and to the raw phoenix spectra.
## Contents

- [Installation](https://eniric.readthedocs.io/en/latest/installation.html)
- [Configuration](https://eniric.readthedocs.io/en/latest/configuration.html)
- [Basic Usage](https://eniric.readthedocs.io/en/latest/basic_usage.html)
- [Broadening](https://eniric.readthedocs.io/en/latest/broadening.html)
- [Atmospheric Transmission](https://eniric.readthedocs.io/en/latest/telluric_corection.html)
- [Normalization](https://eniric.readthedocs.io/en/latest/normalization.html)
- [Resampling](https://eniric.readthedocs.io/en/latest/resampling.html)
- [Theoretical Precision of Synthetic Spectra](https://eniric.readthedocs.io/en/latest/theoretical_precision.html)
- [Scripts](https://eniric.readthedocs.io/en/latest/scripts.html)
- [Example Notebooks](https://eniric.readthedocs.io/en/latest/examples.html)
- [Utilities](https://eniric.readthedocs.io/en/latest/utilities.html)


## Background
Expand All @@ -223,4 +91,6 @@ It had a number of efficiency issues with convolution which were improved upon

To reproduce the updated results for [Figueira et al. 2016](http://dx.doi.org/10.1051/0004-6361/201526900) run

phoenix_precision.py -t 3900 3500 2800 2600 -l 4.5, -m 0.5 -r 60000 80000 100000 -v 1.0 5.0 10.0 -b Z Y J H K
phoenix_precision.py -t 3900 3500 2800 2600 -l 4.5 -m 0.5 -r 60000 80000 100000 -v 1.0 5.0 10.0 -b Z Y J H K

after installation and configuration.
8 changes: 4 additions & 4 deletions appveyor.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@ environment:
# For Python versions available on Appveyor, see
# http://www.appveyor.com/docs/installed-software#python
- PYTHON: "C:\\Python36"
- PYTHON: "C:\\Python37"
- PYTHON: "C:\\Python36-x64"
- PYTHON: "C:\\Python37-x64"

cache:
- "%LOCALAPPDATA%\\pip\\Cache -> appveyor.yml" # Use cache unless appveyor.yml changes.
- "data\\test_data -> appveyor.yml" # Use cache unless appveyor.yml changes.
- "tests\\data -> appveyor.yml"
- "data\\ -> appveyor.yml"
- "tests\\data -> appveyor.yml"

install:
- SET PATH=%PYTHON%;%PYTHON%\Scripts\;%path%
Expand All @@ -32,7 +32,7 @@ install:

# Prepare atmosphere models
- cmd: "%PYTHON%\\python.exe scripts\\split_atmmodel.py -b ALL"
- cmd: "%PYTHON%\\python.exe scripts\\bary_shift_atmmodel.py -b ALL"
- cmd: "%PYTHON%\\python.exe scripts\\barycenter_broaden_atmmodel.py -b ALL"

build: off

Expand Down

0 comments on commit d9171c4

Please sign in to comment.