Skip to content

Commit

Permalink
fix typos
Browse files Browse the repository at this point in the history
  • Loading branch information
anshu-97 committed Jun 1, 2022
1 parent ba2801b commit 738e1ce
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 6 deletions.
8 changes: 4 additions & 4 deletions studies/complexity_structure/analysis.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ bibliography: references.bib

Complexity is an umbrella term for concepts derived from information theory, chaos theory, and fractal mathematics, used to quantify unpredictability, entropy, and/or randomness. Using these methods to characterize physiological signals [a subfield commonly referred to as "fractal physiology", @bassingthwaighte2013fractal] has shown promising results in the assessment and diagnostic of the state and health of living systems [@lau2021brain; @ehlers1995chaos; @goetz2007].

There has been an exponential increase in the number of complexity indices in the past few decades [@yang2013]. Although these new procedures are usually mathematically well-defined and theoretically promising, limited empirical evidence is available to understand their similarities and differences [@lau2021brain; @yang2013]. Moreover, some of these methods are resource-intensive and require long computation times. This complicates their application with techniques that utilise high sampling-rates (e.g., M/EEG) and makes them impractical to implement in real-time settings - such as brain-computer interfaces [@azami2017refined; @manis2018]. As such, having empirical data about the computation time of various complexity indices would prove useful, for instance to objectively guide their selection, especially in contexts where time or computational resourcse are limited.
There has been an exponential increase in the number of complexity indices in the past few decades [@yang2013]. Although these new procedures are usually mathematically well-defined and theoretically promising, limited empirical evidence is available to understand their similarities and differences [@lau2021brain; @yang2013]. Moreover, some of these methods are resource-intensive and require long computation times. This complicates their application with techniques that utilise high sampling-rates (e.g., M/EEG) and makes them impractical to implement in real-time settings - such as brain-computer interfaces [@azami2017refined; @manis2018]. As such, having empirical data about the computation time of various complexity indices would prove useful, for instance to objectively guide their selection, especially in contexts where time or computational resources are limited.

Additionally, the lack of a comprehensive open-source and user-friendly software for computing various complexity indices likely contributes to the limited availability of empirical comparison [@flood2021]. Indeed, most complexity indices are only described mathematically in journal articles, with reusable code seldom made available, therefore limiting their further application and validation [@yang2013; @flood2021]. To address this gap, we added a comprehensive set of complexity-related features to *NeuroKit2*, a Python package for physiological signal processing [@Makowski2021neurokit], to provide users with a software to compute a vast amount of complexity indices. The code is designed to be as fast as possible, while still written in pure Python [though with the help of dependencies such as Numpy or Pandas, @harris2020array; @mckinney2010data] to maximize the re-usability, transparency, and correctness.

Expand Down Expand Up @@ -309,7 +309,7 @@ df |>
theme(axis.text.x = element_text(angle = 45, vjust = 1, hjust = 1))
```

Firstly, one should note that the computation times presented in **Figure 2** are relative and do not correspond to real times, as these would highly depend on the machine used. Rather, the goal here was to convey some intuition on the differences between different classes of indices (using the same machine and the same language of implementation, i.e., Python). While it is possible that computational advances or improvements in the code efficiency might change some of these values, we believe that the "big picture" should remain fairly stable, as it is to a large extend driven by the inherent nature of the algorithms under consideration.
Firstly, one should note that the computation times presented in **Figure 2** are relative and do not correspond to real times, as these would highly depend on the machine used. Rather, the goal here was to convey some intuition on the differences between different classes of indices (using the same machine and the same language of implementation, i.e., Python). While it is possible that computational advances or improvements in the code efficiency might change some of these values, we believe that the "big picture" should remain fairly stable, as it is to a large extent driven by the inherent nature of the algorithms under consideration.

Despite the relative shortness of the signals considered (a few thousand points at most), the fully-parallelized data generation script took about 24h to run on a 48-cores machine. After summarizing and sorting the indices by computation time, the most striking feature is the order of magnitude of difference between the fastest and slowest indices. Additionally, some indices are particularly sensitive to the signal length, a property which combined with computational cost led to indices being 100,000 times slower to compute than others.

Expand Down Expand Up @@ -431,7 +431,7 @@ data <- data |>
cor <- get_cor(data, plot=TRUE)
```

The Pearson correlation analysis revealed that complexity indices, despite their multitude and their conceptual specificities, do indeed share similarities. They form two major clusters that are easily observable (the blue and the red groups in **Figure 2**). That being said, these two anti-correlated groups are mostly revealing of the fact that some indices, by design, index the "predictability", whereas others, the "randomness", and thus are negatively related to one-another. In order to extract finer groupings, further analyses procedures are applied below.
The Pearson correlation analysis revealed that complexity indices, despite their multitude and their conceptual specificities, do indeed share similarities. They form two major clusters that are easily observable (the blue and the red groups in **Figure 2**). That being said, these two anti-correlated groups are mostly revealing of the fact that some indices, by design, index the "predictability", whereas others, the "randomness", and thus are negatively related to one another. In order to extract finer groupings, further analyses procedures are applied below.

### Factor Analysis

Expand Down Expand Up @@ -784,7 +784,7 @@ estimate_means(model, at = c("Index", "Signal", "Noise_Intensity")) |>

## Discussion

As the span and application of complexity science grows, a systematic approach to compare their "performance" becomes necessary to reinforcethe clarity and structure of the field. The term *performance* is here to be understood in a relative sense, as any such endeavor faces the "hard problem" of complexity science: various objective properties of signals [e.g., short-term vs. long-term variability, auto-correlation, information, randomness, @namdari2019; @xiong2017] participate in forming together over-arching concepts such as "complex" and "chaotic". Indices that are sensitive to some of these objective properties are thus conceptually linked through these over-arching framework. However, it remains unclear how these high-level concepts transfer back, in a top-down fashion, into a combination of lower-level features. As such, it is conceptually complicated to benchmark complexity measures against "objectively" complex vs. non-complex signals. In other words, we know that different objective signal characteristics can contribute to the "complexity" of a signal, but there is not a one-to-one correspondence between the latter and the former.
As the span and application of complexity science grows, a systematic approach to compare their "performance" becomes necessary to reinforce the clarity and structure of the field. The term *performance* is here to be understood in a relative sense, as any such endeavor faces the "hard problem" of complexity science: various objective properties of signals [e.g., short-term vs. long-term variability, auto-correlation, information, randomness, @namdari2019; @xiong2017] participate in forming together over-arching concepts such as "complex" and "chaotic". Indices that are sensitive to some of these objective properties are thus conceptually linked through these over-arching framework. However, it remains unclear how these high-level concepts transfer back, in a top-down fashion, into a combination of lower-level features. As such, it is conceptually complicated to benchmark complexity measures against "objectively" complex vs. non-complex signals. In other words, we know that different objective signal characteristics can contribute to the "complexity" of a signal, but there is not a one-to-one correspondence between the latter and the former.

To circumvent the aforementioned consideration, we adopted a paradigm where we generated different types of signals to which we systematically added distinct types - and amount - of perturbations. It is to note that we did not seek at measuring how complexity indices can discriminate between these signal types, nor did we attempt at mimicking real-life signals or scenarios. The goal was instead to generate enough variability to reliably map the relationships between the indices.

Expand Down
4 changes: 2 additions & 2 deletions studies/complexity_structure/manuscript.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,10 @@ affiliation:
authornote: |
Correspondence concerning this article should be addressed to Dominique Makowski, HSS 04-18, 48 Nanyang Avenue, Singapore (dom.makowski@gmail.com) and Annabel Chen (AnnabelChen@ntu.edu.sg).
abstract: |
Complexity quantification, through entropy, information and fractal dimension indices, is gaining a renewed traction in psychopsyiology, as new measures with promising qualities emerge from the computational and mathematical advances. Unfortunately, few studies compare the relationship and objective performance of the plethora of existing metrics, in turn hindering reproducibility, replicability, consistency, and restults clarity in the field. In this study, we systematically compared 125 indices of complexity by their computational weight, their representativeness of a multidimensional space of latent dimensions, and empirical proximity with other indices. We propose that a selection of indices, including *ShanEn (D)*, *MSWPEn*, *CWPEn*, *FuzzyMSEn*, *AttEn*, *NLDFD*, *Hjorth*, *MFDFA (Width)*, *MFDFA (Max)*, *MFDFA (Mean)*, *SVDEn*, *MFDFA (Increment)*, might offer a complimentary choice in regards to the quantification of the complexity of time series.
Complexity quantification, through entropy, information and fractal dimension indices, is gaining a renewed traction in psychophsyiology, as new measures with promising qualities emerge from the computational and mathematical advances. Unfortunately, few studies compare the relationship and objective performance of the plethora of existing metrics, in turn hindering reproducibility, replicability, consistency, and clarity in the field. In this study, we systematically compared 125 indices of complexity by their computational weight, their representativeness of a multidimensional space of latent dimensions, and empirical proximity with other indices. We propose that a selection of indices, including *ShanEn (D)*, *MSWPEn*, *CWPEn*, *FuzzyMSEn*, *AttEn*, *NLDFD*, *Hjorth*, *MFDFA (Width)*, *MFDFA (Max)*, *MFDFA (Mean)*, *SVDEn*, *MFDFA (Increment)*, might offer a complimentary choice in regards to the quantification of the complexity of time series.
keywords : "chaos, complexity, fractal, physiology, noise"
wordcount : "2707"
wordcount : "2709"

bibliography : "bibliography.bib"

Expand Down

0 comments on commit 738e1ce

Please sign in to comment.