Approximate entropy

[kahaaga]

What is this PR?

This PR is an implementation of the approximate entropy (ApEn) algorithm from Pincus (1991), a widely used measure of the regularity of time series (for a summary, see Delgado-Bonal & Marshak, 2019).

Note: the sample entropy (PR #71) is a proposed improvement of the approximate entropy.

Implementation

A simple tree search is used to locate within-radius-r neighbors. During testing, I found this approach to be more than an order of magnitude faster, both in terms of runtime and allocations, than using a naive loop-based approach.

Tests

Tests are generic, as the original paper lack short, concise examples to test on. However, in the doc example, I reproduce the Henon map example in Pincus (1991) with reasonable accuracy, so I think the implementation should be correct.

References

Pincus, S. M. (1991). Approximate entropy as a measure of system complexity. Proceedings of the National Academy of Sciences, 88(6), 2297-2301.

Delgado-Bonal, A., & Marshak, A. (2019). Approximate entropy and sample entropy: A comprehensive tutorial. Entropy, 21(6), 541.

[codecov]

Codecov Report

Merging #72 (e461492) into main (7ba90f3) will increase coverage by 0.91%.
The diff coverage is 87.09%.

@@            Coverage Diff             @@
##             main      #72      +/-   ##
==========================================
+ Coverage   79.65%   80.56%   +0.91%     
==========================================
  Files          33       34       +1     
  Lines         747      772      +25     
==========================================
+ Hits          595      622      +27     
+ Misses        152      150       -2     

Impacted Files	Coverage Δ
src/complexity.jl	50.00% <ø> (+50.00%)	⬆️
src/complexity_measures/approximate_entropy.jl	83.33% <83.33%> (ø)
src/complexity_measures/sample_entropy.jl	80.55% <100.00%> (+6.26%)	⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[kahaaga]

@Datseris As far as I can see, this should be ready to go now. The docs are here.

I've:

• Updated the PR to use the new API.
• Added tests that compares results for our package with a MATLAB implementation of the same algorithm. I couldn't really come up with any good analytical examples with easily-derived values, because the method involves a difference of scaled and summarized logarithms whose inputs are neighbors counts in m and m+1 dimensional space. I might be able to come up with something in the future, but I think for now, the MATLAB comparison should be enough.
• Updated the example to use Makie for plotting. Within reasonable accuracy, we reproduce values from the original paper (comparison can't be exact, because they don't provide initial conditions).

[Datseris]

Shouldn't there be a small ## Description section in the docstring? For the tests, you could use what you already have in the example plot; compare the value with the value from the table, i.e., that it is close to 0.394 for N = 3000.

[kahaaga]

@Datseris I've now added:

• Fixed your comments.
• Improved notation and added an algorithm description.
• A test based on Pincus' example. I removed the MATLAB tests, because it turns out they do not (according to their documentation) adjust the normalization constant for embedding lags != 1, and they use < instead of <= for neighbor searches. Thus, results would not be comparable in general.
• Added test/Project.toml, so we don't have to make DynamicalSystemsBase.jl a dependency for just installing Entropies.jl. Using a Project.toml file is anyways the standard for Julia 1.3 an onwards.

[kahaaga]

This has now been updated to use the new complexity api (#134). I've also implemented changes based on comments from #71.

[Datseris]

Well now I am confused. I thought we agreed that we do this table thingy and therefore do not provide convenience functions anymore.

[kahaaga]

Well now I am confused. I thought we agreed that we do this table thingy and therefore do not provide convenience functions anymore.

We agreed on having exactly two convenience functions for complexity measures too, for educational purposes, like we do for entropy. Those would be sample entropy and approximate entropy, because they are the most widely used ones. Citing yourself in #134:

Okay let's keep the sample_entropy and approxiamte_entropy as the only "convenience complexity functions".

Remaining complexity measures go in the table.

[Datseris]

I made this comment before the existence of the table was on the table however (pun not intended). If we have a table, which satisfies the "search engine" argument, do we need these convenience methods? Or, we can have a docs section that discusses the library design principles. I have one in Agents.jl as well. Okay. Leave the "convenience method" here. Once we have the table, I'll do a PR that proposes my section.