Skip to content

Latest commit

 

History

History
331 lines (323 loc) · 27.6 KB

developers_board.md

File metadata and controls

331 lines (323 loc) · 27.6 KB
Raw

Next big tasks - think about the order:

  1. full Keplerian motion of the lens (because 2-parameter approximation is unphysical and is already implemented)
  2. triple sources
  3. triple lenses
  4. astrometric microlensing
  5. better support of FFP calculations (see "next smaller tasks" below)
  6. terrestial parallax

Next smaller tasks:

  1. ModelParameters - note that t_0_1 etc. are for "binary source" models
  2. make directory with reference plots from examples
  3. remove unused branches
  4. UC40 - finish it and implement Event.get_lc and plot_lc
  5. UC38 - finish it
  6. faster FFP calculations
  7. "import MulensModel as mm" everywhere: use_cases/->30,31,34 examples/run_time_tests/check_system.py examples/checks.py TUTORIALS
  8. satellite positions read from a file similar to data challenge files
  9. check open issues on github
  10. "add a list of public datasets:" - search below
  11. In longest files re-order functions to have documentation in expected order: modelparameters.py, model.py, event.py, mulensdata.py, fitdata.py (in that order)
  12. FitData.get_residuals - test for binary source

Specific tasks to be performed

boldfaced tasks are most important because requested by the users

italics mark important tasks

Changes for planned v3 are here: documents/MM_v3.md

  • Install
    • PYPI website - some links: 1 2
    • test Windows installation
    • in setup.py in setup() add keywords: long_description, classifiers
    • virtualenv; pip install -r requirements.txt; its best to install the dependencies first
    • more metadata in setup.py
    • more on setup.py: link
    • compile with "pedantic" flags for compilers
  • Documentation
    • magnification_methods.pdf - add full references there
    • Sagan workshop hands-on activity in MM
    • examples as ipython notebooks
    • Add __repr__ functions to Lens and Source
    • Include full documentation via setup.py data_files mechanism.
    • note that all plotting functions require plt.show() or plt.save()
    • try removing Attributes from docstrings - just make short @property functions
    • add a note that pi_E is "geocentric" (and "heliocentric" has the same length of vector but is rotated)
    • example 8 corrections - PSBL, not PSPL; clarify removing the anomaly
    • make sure that website shows correct version of MM
    • note that we are not checking for negative source or blending flux
    • add a list of public datasets: VVV paper was published?, second VVV paper was published?, add link to K2/MCPM?; LINK the file documents/public_data_list.md somewhere
  • Effects:
    • Binary source - see documents/binary_source_notes.md:
      • extract flux ratio for binary source models when fluxes are fitted via regression
      • finish use cases
      • Fit.fit_fluxes docstring to be updated
      • which_parameters() - note that it doesnt work for binary source parameters, but the parameters work properly; just BSPL and rho_2 etc. are optional
      • parallax models
      • different t_E for each source (correct Model.set_times)
      • test binary source with exactly one rho_X defined
      • add t_eff_1, t_eff_2
    • Magnification function provided by the user - already started in user_method branch; also this could be used to model variable source events - note that
    • triple lens
      • test ob07349 to see if center of mass can reference point for t0 and u0
      • caustics calculations
      • plan use cases and unit tests - note in documents/TRIPLE_LENS.md
      • "reset" triple_lens branch? (i.e., copy code, remove branch, start new branch e.g. tripleLens, use the code copied at begin)
      • documents/TRIPLE_LENS.md - make notes in order there
      • check triple lens use cases in master branch
      • use cases, point source, hexadecapole...
    • triple source calculations
    • Finite Source
      • FSPL with low magnification - do Witt & Mao 94 or Witt 95 give the right formulas?
      • FSPL ray shooting (ala getmag_rs_single.f)
      • Yoo+04 full formalism
      • get gamma/u LD coeffs from Claret papers etc.
      • Lee+09 - gradient calculations for uniform source, also faster calculations - profile
      • FSPL for large sources using Agol 2003
    • Quadratic limb darkening
    • Multi-lens ray shooting:
      • mapmaking version which adds new rays as needed (but remember that it runs for fixed (s,q) only!)
      • Yossi idea to find all the images
    • Orbital motion like in VBBL 2.0
    • calculate jerk parallax degeneracy: Park+04 Gould 04
    • topocentric/Earth parallax
    • Chang-Refsdal binary calculations
    • elliptical source magnification Heyrovsky & Loeb 1997
    • magnification calculated for a set of points, not just a trajectory - this way we could, e.g., plot magnification maps
  • Parameterization
    • Cassan 2008 binary lens parameters:
      • option to change scaling (from [0,1] to C08 params) to work well near topology change
    • Albrow et al. 1999 (also Cassan 2008 Sec. 5)
    • t_eff as a parameter for large u_0 - see Andy paper and maybe also other from Jen 2012 paper, i.e., f_lim=f_s/u_0 and q*t_E
    • caustic size w Dong+09 refers to Chung+05; see also Skowron+11 Eq.2
    • check if new parameters are defined here: Liebig, DAgo, Bozza, and Dominik 2015
    • Heyrovsky 2003 parametrization of limb-darkening
    • t_0_planet, u_0_planet, t_E_planet instead of s, q, alpha
    • Dominik 2009 for PSPL
    • Jung+17 - rotating triple lens - somehow special version of xallarap
  • Function Improvements/Expansion:
    • BinaryLens class:
      • VBBL2.0 - are we using accuracy limit as default? If so then we should switch to relative accuracy
      • should BinaryLens() accept source_x/y as lists or arrays?
      • function for center of mass shift (currently: shift_x in trajectory.py, x_shift in binarylens.py, xcm_offset in caustics.py)
      • central and planetary caustic properties: Chung et al. 2005 and Han 2006
      • consider using Utils.complex_fsum() in BinaryLens functions: _jacobian_determinant_ok_WM95()
      • faster hexadecapole using Cassan 2017 (code)
      • there should be an option for the user to ignore "wrong number of solutions" error and replace it with a warning or dismiss it fully
      • error message for not 3 or 5 images - This can be improved by checking the denominators in the equation (before polynomial is fully formed - see Zoey Mathematica notes)
      • error message for not 3 or 5 images - in "Consider ..." note that one must use Model.set_magnification_methods() for epochs far from peak
      • VBBL additional parameter so that it works for huge source and tiny caustic
    • Caustics class:
      • Caustics._calculate - optimize using vectors instead of a loop
      • Caustics calculations using Erdl & Schneider 1993 approach; there is also Asada 2002 and I dont know if it makes sense to code that one as well
      • root solver can be used the same as in binarylens.py - not needed for binary lens and Erdl & Schneider 1993
      • smaller points
      • correct conditions in get_caustics()
    • Event class:
      • Allow fluxes to be fixed in chi^2 calculation (e.g. given a particular fs, fb, which you might want to do if you want fs as a chain parameter); also think how it will work for binary sources
      • give access to all fluxes without changing data_ref
      • plot magnitude difference between 2 models for residuals plot
      • it seems it doesnt have plot_trajectory()
      • add plot_source() etc. already implemented in Model
      • Event should sync information on which of the 3 types of parallax are used, so that if it is specified for event, then there will be exception if one dataset is missing earth_coords etc. In general there should be some way to make sure which parallax types are used in which calculation of magnification.
      • Class Event should have not only set_datasets() methods but also add_datasets(), i.e. a similar method that appends datasets to self._datasets.
      • reduce calls to Fit.fit_fluxes()
      • chi2_gradient() should cope NaN values in a way similar to get_chi2()
      • check all functions that should pass fit_blending parameter - Event.plot_model, what else??? Already done: Event.get_ref_fluxes()
      • chi2 with maximum value provided - if the chi2 for point-source gives chi2 larger than specified limit, then finite source calculations are not undertaken (this should significantly speed-up MultiNest)
      • get flux and its error in reference system
      • change order to improve the website
      • gradient - fluxes as well? if so, then start using the second test in test_event_chi2_gradient()
      • for consistency, it would be good to combine get_chi2_for_dataset() and get_chi2_per_point()
      • other likelihoods, e.g., Dominik+18, ARTEMiS, SIGNALMEN, RoboTAP
      • function that calculates cumulative chi2 so that it can be plotted easier
      • Binary source - see optimization comment at begin of Event.get_chi2_for_dataset()
      • plot cumulative chi2 difference between 2 models - magnification and magnitude spaces
    • Fit class:
      • should use marginalized distributions of fluxes (if those are from linear fits); JCY - it needs UC
      • n_sources somehow inconsistent in different places
    • Horizons class:
      • JPL Horizons
        • correct JPL Horizons => CSV file format; also example usage
        • check if Horizons e-mail is for correct satellite
      • BJD
    • Lens class:
      • __repr__ function needs work
      • a_proj, couples with source distance in mulensmodel to determine s.
      • 2-body example 3 is missing s. Why? Does that work?
      • problem with tracking number of masses, esp when successively defining masses (see test_Lens.py)
      • implement triple+ systems
    • MagnificationCurve class:
      • what to do if some magnifications are nan, inf, or None?; maybe give warning and return None in these places
      • re-write magnification() to use lazy loading (here or in model.py)
    • Model class:
      • in functions magnification(), plot_magnification(), and plot_trajectory() use satellite_skycoord from __init__ if available
      • plot_lc() - add satellite option like in plot_magnification(), other options as well - use keywords passed to self.magnification()
      • reorder functions so that it looks good on website
      • bandpass option could simultaneously account for limb-darkening and source_flux_ratio for a given band (to be applied in a few options that have bandpass option)
      • Model.set_parameters() should remember previously set values (of course unless they are overwritten)
      • Class Model should not allow accessing attributes that should not be there, eg., q for single lens case.
      • Function that prints RA, Dec, t_0_par, t_0_kep, types of parallaxes turned on, satellite info, limb coeffs and textual description of type of model
      • plot_trajectory() - mark epochs using colorscale? Maybe it should be passed by kwargs (if so, then add example)
      • Should get_satellite_coords() use caching?
      • we should have versions of all plot functions to use magnifications instead of magnitudes; also add access via Event
      • add option to use random zorder when plotting multiple datasets (e.g. gaussian with sigma depending on number of plotted datapoints)
      • does Model.plot_lc() give docstring for flux_ratio_constraint option? If not, then present it in plot_magnification() docstring
      • use plt.axhline() to plot 0 line in residuals plots at the end, using t_min,t_max
      • get_residuals needs unit test
      • plot_trajectory - mark data epochs (as pyplot kwargs use MulensData.plot_properties)
      • plot_data & plot_residuals - if color is not set by the user and show_bad is True, then X-s are plotted using different color
      • allow rotating plots, so that source travel "exactly" from left to right
      • when plotting residuals allow keywords to be passed for plotting 0-line
      • get_data_magnification() - should there be data.good masking?
      • get_ref_fluxes - use caching (self._fit) inside it
      • plot_lc() should have satellite_skycoord keyword as plot_magnification() has
      • plot_source and binary sources - allow different kwargs for each of the sources?
      • check_if_caustic_crossing() function can be added - for non-parallax, no-orbital motion it requires calculation of cusps positions for each caustic and checking if they are on the same side of the trajectory
      • try/except in pyplot commands and catch AttributeError - these may be miss-spelling etc.
    • ModelParameters class:
      • values in dimensionless astropy.quantity should be changed to float, other types should be rejected (unless it is a time unit etc.)
      • LaTeX strings with parameters names (useful e.g. for corner plots or tables with results)
      • a check if each PARAMETER.setter function changes parameter that was previously set
      • Transform t_E and other parameters between geocentric and heliocentric frames.
      • option to return alpha, dalpha_dt, and ds_dt as floats instead of astropy.quantities
      • why .rho returns None if it is not defined? In other similar cases we have KeyError. Should that be changed? (if so, then maybe only after changing version to 2.0.0)
      • to make _check_valid_combination_1_source_...() shorter, make a boolean dict that says if given parameter is defined or not
      • change order to improve the website
      • check if t_eff and t_star can be used as input simultaneously
      • check if input values are floats (or something else accepted)
      • warning if t_0_par is changed too many times
      • add is_Cassan08 or is_standard_binary?
      • __repr__ - if Cassan 2008 then should we print standard parameters as well?
    • MulensData class:
      • Errorbar scaling, in particular the two parameter.
      • quick look alignment for MulensData objects - just use interpolation
      • make __init__() shorter
      • add version of n_epochs that uses only good epochs
      • read settings from file header: flux vs. mag, filter, satellite info
      • change order to improve the website
      • docstring phot_fmt vs. input_fmt
      • data_and_err_in_input_fmt() and Fit.get_input_format() possible can be deprecated or removed because we shifted to chi2_fmt instead of input_fmt
      • when plotting data, make sure that max/min limits on Y axis include errorbars, if the errorbars are shown
      • export/save given data file in scale of other dataset and model
      • this line may be wrong for some values of char: kwargs['fmt'] = kwargs['fmt'].replace(char, "")
      • for plt.scatter() the color can be set as 'facecolor', 'facecolors', or 'edgecolors' and this should be dealt with in _set_plot_properties()
      • for plotting X for bad data use large size and/or thinner line
      • separate colors (or even kwargs) for X-es as an option (to get contrasting colors see https://itsphbytes.wordpress.com/2016/08/29/complementary-colors-python-code/)
    • PointLens class:
      • in _integrand_Lee09_v2() use loop and math.fsum in "values = ...", check performance as u_ and theta_ have shape of (90, 1000)
      • make WM method faster: 1) interpolation done once for many values; 2) interpolate different function; 3) allow changing number of annuli; 4) divide A by 2 different functions (z<1 or z>1) and interpolate these - like in VBBL (arguments of interpolation are: 1) (rho, z), 2) (log rho, log z) etc.)
      • add Witt and Atrio-Barandela 2019?
      • get_pspl_magnification() - change it to operate on u^2, not u, so that np.sqrt() calls are reduced
      • 1+2/u^4 approximation for very large u
      • try to remove sympy and use scipy instead (not possible in Sep 2020, because there is no elliptical integral of third kind in scipy)
    • SatelliteSkyCoord class:
      • attach magnification_methods to SatelliteSkyCoord so that they overwrite Model and MagnificationCurve settings when given SatelliteSkyCoord is used
    • Trajectory class:
      • warning when too many annual parallax calculations are conducted
      • _get_delta_satellite() should be using self.times
      • annual parallax caching - if moved to MulensData, then would be faster because hashing of coords and time vector takes time
      • colorscale time or magnification (see Fig 2 in Ranc+19)
      • plot in units of theta_star (or even days) instead of theta_E
    • UniformCausticSampling class:
      • add function that calculated t_in & t_out in analogy to get_x_in_x_out() - the problem is that for given (u_0, alpha) - there are 2, 4, or 6 caustic crossing points
      • prior probability (in get_uniform_sampling()) of hitting given caustic based on (d, q, alpha) - see Kains+12
      • prior only in given box (in get_uniform_sampling())
      • prior includes t_E prior provided by user (also add option to use Mroz+19 etc) in get_uniform_sampling()
      • trajectories that cross 2 different caustics
      • profile calculations
    • Utils class:
      • in np.any() ifs give more information in warning e.g., "out of 1234 values provided, the fails are: 12, 345, 678 (0-based)"
      • add u(a) function: u = np.sqrt(2A/np.sqrt(A^2-1.) - 2.)
      • utils.py:57 - code produces 2 warnings, should produce just one; use masking
      • documentation - use ITALICS
      • uses masks there if warnings and apply None etc. - check examples after change!
      • add a function that calculates mag/flux in a different flux space - we currently do it in MulensData.plot() and Model.get_residuals()
    • MulensObjects submodule:
    • Plotting:
      • plt.plot() .scatter() and .errorbar() should share default colors, so that when you plot one dataset and one model and dont set colors, then they are of different colors
      • for plotting functions option to pass pyplot.Axis and pyplot.Figure instances and call e.g. Axis.scatter() instead of pyplot.scatter(); for a simple example see here
      • subplots with shared X-axis (plt.subplots(2, 1, sharex=True, gridspec_kw={'height_ratios': [4, 1]}, figsize=???, dpi=100)) - start in Example 5
      • add option to plot satellite coordinates as in Henderson+16 where K2 and Spitzer orbits were compared; i.e., make Trajectory._get_delta_satellite() public and add appropriate example
      • add plotting with fit_blending=False for functions that use magnitude space
      • add plt.xlim() and ylim in plotting functions (using t_start) etc.; then also update (simplify) tutorials, examples etc.
      • caustics for trajectory plot with single lens models
      • plot upper limits instead of photometry with negative flux
    • Examples and use cases:
      • use case 34 - make the output meaningful
      • example 4 - use "with open() as ...:"
      • Hamiltonian MCMC link 1 and link 2 and link 3
      • example with parallel EMCEE
      • plot many models from posterior
      • scipy.curve_fit() and print parameter uncertainties
      • corner plots; they require corner, pyhdust, or pyGTC
      • F_s for MOA data for MB08310 differs from Janczak paper - is it caused by FSPL vs. FSBL models?
      • add example that shows 'log_' in the name of the parameter; central caustic anomaly planet would be best,
      • example 08: PSBL, not PSPL
      • gaussian process - see Li+19 (maybe it requires input in Fit class, hence would not be just an example)
      • Dan FM approach for periodic variables - two variables instead of one (identified by some postfix) - make it for alpha or x_caustic_in/out
      • add illustration on how to remove airmass trends
      • add example of fitting PSPL model using Albrow (2004) method link
      • plotting - sharex where possible
      • note in PSPL tutorial about plotting data in MulensData
      • add example that after calling Event.get_chi2() use Event.fit to get e.g. magnifications so that the magnification is not calculated twice
      • satellite data fitted and plotted - what is missing now?
      • some cfg files use "../data/..." - change it to MM.DATA_PATH somehow
      • in emcee we should check if all the starting points are in prior
      • check if MM correctly interacts with scipy.optimize.leastsq and maybe add an example
      • add Coordinates.velocity_of_Earth() example
      • Example 13 - make x_caustic_in/_out periodic variables
      • add example with well-known code like https://mc-stan.org/
      • MulensData.bad needs example note that one has to substitute full vector, not single values
    • Miscellaneous:
      • COVERAGE : "coverage run --source MulensModel -m py.test" and then "coverage report" or "coverage report -m" or "coverage html" (and then open htmlcov/index.html); https://coverage.readthedocs.io/en/v4.5.x/
      • u_0 sign for satellite or just parallax model - some way of following u(t) evolution
      • warnings - give type for each one of them
      • Dave idea on caustic-crossing epochs as parameters t_cc1 and t_cc2 - see david.p.bennettATnasa.gov e-mail on Feb 11, 2019
      • when checking units use Unit.physical_type - search for physical_type in mulensobjects/lens.py as an example; to find places to be changed search for "isinstance" (to find these places run grep isinstance *py mulensobjects/*py | grep Quantity
      • use lazy loading in MagnificationCurve.magnification and/or Model.magnification
      • guessing parameters of PSPL model (Kim+17 as an example)
      • add calculation of Caustic Region of Influence (CROIN) - Penny 2014
      • anything from use cases that does not work yet -- see TODO.md file
      • comments at begin of each use case and example
      • use case 25 doesnt work
      • OB161195 is used only in UC25 - change it and remove directory
      • interaction with fitting routines - see list of them
      • caching of results in trajectory.py should stop at some point - if the user changes t_0_par or coords, then there is no point in remembering huge indexes (whole self.times)
      • profile the code (python -m cProfile script.py - you only should check 2nd and 4th column)
      • Leap seconds library - barycorrpy
      • documents/TODO.md file - move content and remove
      • add transformation Jacobians, see: Batista+11 and Skowron+11
      • use cython, numba, pypy or similar (numba seems best) to speed-up calculations in Lee+09 and/or Cassan08
      • remove "import *" from __init__.py and mulensobjects/__init__.py
  • Other Tests:
    • test_event_chi2_gradient() - add parallax without flux gradient
    • add unit tests for Horizons and MulensData.satellite_skycoord
    • Coordinates - write tests, possibly remove test_Coords.py
    • t_eff is not tested
    • plt.scatter -> plt.plot; after that we can start unit tests for plt calls
    • check for memory leaks by running long calculations and monitoring RAM usage
  • Style/Architecture:
    • Are we consistent with PEP8? check here or pycodestyle command
    • PEP8 for tests/ (test_Event is already done: test_Event test_ModelParameters test_Model_Parallax
    • for examples/
    • better import of the module so that all main classes are accessible (use __all__ = [...] in all files?)
    • Utils - Make subpackage/submodules that group related functions (e.g. flux2mag conversions)?

reStructuredText:

1st tutorial, 2nd tutorial, example

Xallarap references: