This repository has been archived by the owner. It is now read-only.

Replace or remove non-free code (Numerical Recipes etc.) #37

Open
wants to merge 13 commits into
base: master
from

Conversation

Projects
None yet
1 participant
@olebole
Contributor

olebole commented May 5, 2017

Here is a list of files that contain code from Numerical Recipes:

  • pkg/xtools/numrecipes.x (Currently unused; may be removed)
    • GAMMLN: natural log of gamma function
    • POIDEV: Poisson deviates for a given mean
    • GASDEV: normally distributed deviate of zero mean and unit var
    • TWOFFT: complex FFTs of two input real arrays
    • REALFT: FFT of a set of 2N real valued data points
    • FOUR1: forward or inverse FFT of the input array (removed)
    • LUDCMP: LU decomposition of a square matrix
    • LUBKSB: backsubstitution to solve a system
  • noao/onedspec/splot/spdeblend.x
    • GASDEV: normally distributed deviate of zero mean and unit var
  • noao/rv/numrep.x
    • FOUR1: forward or inverse FFT of the input array (removed)
    • REALFT: FFT of a set of 2N real valued data points
    • TWOFFT: complex FFTs of two input real arrays
  • noao/artdata/numrecipes.x
    • GAMMLN: natural log of gamma function
    • POIDEV: Poisson deviates for a given mean
    • GASDEV: normally distributed deviate of zero mean and unit var
  • noao/imred/ccdred/ccdtest/t_mkimage.x
    • MKSIGMA: Sequence of random numbers of the specified sigma
  • sys/mwcs/mwlu.x
    • MW_LUDECOMPOSE: LU decomposition of a square matrix
    • MW_LUBACKSUB: backsubstitution to solve a system
  • pkg/utilities/nttools/stxtools/
    • ludcmd.x ludcmd: lower-upper decomposition
    • ludcmp.x ludcmp: lower-upper decomposition
    • lubksd.f LUBKSD: Solves a matrix equation AX = B
    • lubksb.f LUBKSB: Solves a matrix equation AX = B
    • savgol.x SAVGOL: Savitzky-Golay filtering (removed)
  • noao/astutil/asttools/asttimes.x
    • AST_JULDAY_TO_DATE: Convert Julian date to calendar date
  • noao/digiphot/daophot/daolib/
    • daoran.x DAORAN: random number generator RAN2 (removed; replaced by URAND())
    • ran3.x RAN3: uniform random deviate between 0.0 and 1.0 (removed)
  • noao/imred/vtel/mrqmin.x (removed)
    • MRQMIN: Levenberg-Marquard nonlinear chi square minimization
    • MRQCOF: Evaluate linearized matrix coefficients
    • GAUSSJ: Linear equation solution by Gauss-Jordan elimination
    • COVSRT: Sort covariance matrix
  • pkg/utilities/nttools/trebin/
    • tucspl.f tucspl: copied from Numerical Recipes SPLINE
    • tuispl.f tuispl: copied from Numerical Recipes SPLINT
    • tuiep3.f tuiep3: copied from Numerical Recipes POLINT
    • tuhunt.f tuhunt: copied from Numerical Recipes HUNT
  • vendor/cfitsio/eval.y, vendor/cfitsio/eval_y.c (backported from newer cfitsio)
    • gammln: natural log of gamma function
    • poidev: Poisson deviates for a given mean
    • gasdev: normally distributed deviate of zero mean and unit var

All marked items are handled -- either by replacing with free code, or by removal (which is then marked in the text). This fixes #21.

@olebole

This comment has been minimized.

Show comment
Hide comment
@olebole

olebole Nov 1, 2017

Contributor

The routine MR_SOLVE, and its helpers MR_EVAL and MR_INVERT is not derived from Numerical Recipes code, but instead uses hfti form math$llsq:

# These routines have their origin in Numerical Recipes, MRQMIN, MRQCOF,
# but have been completely redesigned.

Therefore, this code does not need to be replaced:

  • pkg/xtools/numrecipes.x
    • MR_SOLVE: Levenberg-Marquardt nonlinear chi square minimization
      • MR_EVAL: Evaluate curvature matrix.
      • MR_INVERT: Solve a set of linear equations using Householder transforms
  • pkg/images/tv/imexamine/iejimexam.x
    • MR_SOLVE: Levenberg-Marquardt nonlinear chi square minimization
      • MR_EVAL: Evaluate curvature matrix.
      • MR_INVERT: Solve a set of linear equations using Householder transforms
  • noao/onedspec/splot/deblend.x
    • MR_SOLVE: Levenberg-Marquardt nonlinear chi square minimization
      • MR_EVAL: Evaluate curvature matrix.
      • MR_INVERT: Solve a set of linear equations using Householder transforms
  • noao/rv/deblend.x
    • MR_SOLVE: Levenberg-Marquardt nonlinear chi square minimization
      • MR_EVAL: Evaluate curvature matrix.
      • MR_INVERT: Solve a set of linear equations using Householder transforms
  • noao/rv/rvidlines/iddeblend.x
    • ID_MR_SOLVE: Levenberg-Marquardt nonlinear chi square minimization
      • ID_MR_EVAL: Evaluate curvature matrix.
      • ID_MR_INVERT: Solve a set of linear equations using Householder transforms

MR_INVERT calls the routine hfti from math/llsq/hfti.f, which has the following origin notice:

c     c.l.lawson and r.j.hanson, jet propulsion laboratory, 1973 jun 12
c     to appear in 'solving least squares problems', prentice-hall, 1974
c          solve least squares problem using algorithm, hfti.

The llsq package is however also included in scipy, which already clarified the license of the code:

From: Deborah Cotton 
Date: Fri, 14 Sep 2007 12:35:55 -0500
Subject: RE: Algorithm License requested
To: aisaac@american.edu

Prof. Issac,

In that case, then because the author consents to [the ACM] releasing
the code currently archived at http://www.netlib.org/toms/733
under the
BSD license, the ACM hereby releases this code under the BSD license.

Regards,

Deborah Cotton, Copyright & Permissions
ACM Publications
2 Penn Plaza, Suite 701**
New York, NY 10121-0701
permissions@acm.org
212.869.7440 ext. 652
Fax. 212.869.0481
Contributor

olebole commented Nov 1, 2017

The routine MR_SOLVE, and its helpers MR_EVAL and MR_INVERT is not derived from Numerical Recipes code, but instead uses hfti form math$llsq:

# These routines have their origin in Numerical Recipes, MRQMIN, MRQCOF,
# but have been completely redesigned.

Therefore, this code does not need to be replaced:

  • pkg/xtools/numrecipes.x
    • MR_SOLVE: Levenberg-Marquardt nonlinear chi square minimization
      • MR_EVAL: Evaluate curvature matrix.
      • MR_INVERT: Solve a set of linear equations using Householder transforms
  • pkg/images/tv/imexamine/iejimexam.x
    • MR_SOLVE: Levenberg-Marquardt nonlinear chi square minimization
      • MR_EVAL: Evaluate curvature matrix.
      • MR_INVERT: Solve a set of linear equations using Householder transforms
  • noao/onedspec/splot/deblend.x
    • MR_SOLVE: Levenberg-Marquardt nonlinear chi square minimization
      • MR_EVAL: Evaluate curvature matrix.
      • MR_INVERT: Solve a set of linear equations using Householder transforms
  • noao/rv/deblend.x
    • MR_SOLVE: Levenberg-Marquardt nonlinear chi square minimization
      • MR_EVAL: Evaluate curvature matrix.
      • MR_INVERT: Solve a set of linear equations using Householder transforms
  • noao/rv/rvidlines/iddeblend.x
    • ID_MR_SOLVE: Levenberg-Marquardt nonlinear chi square minimization
      • ID_MR_EVAL: Evaluate curvature matrix.
      • ID_MR_INVERT: Solve a set of linear equations using Householder transforms

MR_INVERT calls the routine hfti from math/llsq/hfti.f, which has the following origin notice:

c     c.l.lawson and r.j.hanson, jet propulsion laboratory, 1973 jun 12
c     to appear in 'solving least squares problems', prentice-hall, 1974
c          solve least squares problem using algorithm, hfti.

The llsq package is however also included in scipy, which already clarified the license of the code:

From: Deborah Cotton 
Date: Fri, 14 Sep 2007 12:35:55 -0500
Subject: RE: Algorithm License requested
To: aisaac@american.edu

Prof. Issac,

In that case, then because the author consents to [the ACM] releasing
the code currently archived at http://www.netlib.org/toms/733
under the
BSD license, the ACM hereby releases this code under the BSD license.

Regards,

Deborah Cotton, Copyright & Permissions
ACM Publications
2 Penn Plaza, Suite 701**
New York, NY 10121-0701
permissions@acm.org
212.869.7440 ext. 652
Fax. 212.869.0481

@olebole olebole changed the title from WIP: Replace code from Numerical Recipes with free counterparts to WIP: Replace or remove code non-free code (Numerical Recipes etc.) Nov 2, 2017

@olebole olebole changed the title from WIP: Replace or remove code non-free code (Numerical Recipes etc.) to WIP: Replace or remove non-free code (Numerical Recipes etc.) Nov 2, 2017

@olebole

This comment has been minimized.

Show comment
Hide comment
@olebole

olebole Nov 10, 2017

Contributor

The subpackage noao$imred/vtel was used for the data reduction of the Kitt Peak Vacuum Telescope. After 29 years of solar imaging, the vacuum telescope was decommissioned in 2002, which is 15 years ago.

The subpackage uses a the MRQMIN routine which was copied from the Numerical Recipes book. This code is not licensed for public redistribution and must be removed from the public IRAF package. Since the subpackage is specialized for data from an instrument that is out of operation for a long time, it is not worth replacing the NR code with free code.

Contributor

olebole commented Nov 10, 2017

The subpackage noao$imred/vtel was used for the data reduction of the Kitt Peak Vacuum Telescope. After 29 years of solar imaging, the vacuum telescope was decommissioned in 2002, which is 15 years ago.

The subpackage uses a the MRQMIN routine which was copied from the Numerical Recipes book. This code is not licensed for public redistribution and must be removed from the public IRAF package. Since the subpackage is specialized for data from an instrument that is out of operation for a long time, it is not worth replacing the NR code with free code.

@olebole

This comment has been minimized.

Show comment
Hide comment
@olebole

olebole Nov 14, 2017

Contributor

Some comments on routines that were removed in the Astroconda distribution (@jehturner) of IRAF due to NR copyright issues:

  • math/slalib/svd.f, math/slalib/svdcov.f, math/slalib/svdsol.f:
    The singular value decomposition method used there is (probably) not copied from the Numerical Recipes, but from EISPACK. The book probably also copied that code; therefore the similarity. EISPACK is public domain. SVD is not used in IRAF, however, so in principle this could be just removed anyway.
  • noao/rv/fftutil.x:
    This contains no NR code, but is a wrapper around the code in noao/rv/numrec.x. Since we replace the latter with a API compatible variant, this code should stay.
Contributor

olebole commented Nov 14, 2017

Some comments on routines that were removed in the Astroconda distribution (@jehturner) of IRAF due to NR copyright issues:

  • math/slalib/svd.f, math/slalib/svdcov.f, math/slalib/svdsol.f:
    The singular value decomposition method used there is (probably) not copied from the Numerical Recipes, but from EISPACK. The book probably also copied that code; therefore the similarity. EISPACK is public domain. SVD is not used in IRAF, however, so in principle this could be just removed anyway.
  • noao/rv/fftutil.x:
    This contains no NR code, but is a wrapper around the code in noao/rv/numrec.x. Since we replace the latter with a API compatible variant, this code should stay.

olebole added some commits May 5, 2017

Replace NRs "gasdev" with a clean-room implementation.
The implementation was done by Anastasia Galkin from the original
publication without any reference to the Numerical Recipes book.

This may be distributed under the same license as IRAF itself.
Replace GAMMLN with system provided function lgammaf()
There is no need to re-implement that is already provided in standard
POSIX since 2001.
Replace NRs "poidev" with a clean-room implementation.
The implementation was done by Anastasia Galkin from the original
publication without any reference to the Numerical Recipes book.

This may be distributed under the same license as IRAF itself.
Replace DAORAN with URAND
The IRAF osb library already contains a good random number generator
with the same API as DAORAN, so we can replace it. The URAND random
number generator presented by D. Knuth has generally a very good
reputation in literature.

Reference: D. E. Knuth, The Art of Computer Programming, Vol. 2: Seminumerical
           Algorithms, Addison-Wesley, Reading, MA, 1981.
Remove Numerical Recipes code from cfitsio
This is backported from cfitsio-3370, which is free of NR code.
Replace ast_julday_to_date with a BSD-3-Clause implementation
The implementation is taken from ERFA, which is a relicensed (with
permission) variant of IAU SOFA and adopted to SPP. The license is
also added to local/LICENSES.
Replace NR routines for LU decomposition by lapack calls
Add LU decomposition from LAPACK 3.71
Remove math$ieee package
The code for this package was copied from the book "Programs for
Digital Signal Processing" by the IEEE Press, and has an unknown
license. This code is unused however and therefore can just be
removed.
Remove vtel subpackage from imred
vtel was used for the data reduction of the Kitt Peak Vacuum
Telescope.  After 29 years of solar imaging, the vacuum telescope was
decommissioned in 2002, which is 15 years ago.

The vtel subpackage uses a the "MRQMIN" routine which was copied from
the Numerical Recipes book. This code is not licensed for public
redistribution and must be removed from the public IRAF package. Since
the subpackage is specialized for data from an instrument that is out
of operation for a long time, it is not worth replacing the NR code
with free code.

Therefore, this patch removed the vtel package completely.
Add replacement for spline and polynomial interpolations
Spline and polynomial interpolations is used in the `trebin` task. The
replacement code (in SPP) is based on the standard methods (described
f.e. in Wikipedia)
Add a license notice for the math$llsq package.
This code was re-released under a BSD license by ACM.
Remove unused Savitzky-Golay smoothing code
This code is based on code taken from the Numerical Recipes book.

@olebole olebole changed the title from WIP: Replace or remove non-free code (Numerical Recipes etc.) to Replace or remove non-free code (Numerical Recipes etc.) Nov 15, 2017

@migueldvb migueldvb referenced this pull request Jan 15, 2018

Open

Comments #4

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.