[PRE REVIEW]: DataAssimilationBenchmarks.jl: a data assimilation research framework

[whedon]

Submitting author: @cgrudz (Colin Grudzien)
Repository: https://github.com/cgrudz/DataAssimilationBenchmarks.jl
Version: v0.2.0
Editor: @taless474
Reviewers: @peanutfun, @tmigot
Managing EiC: Kristen Thyng

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/478dcc0b1608d2a4d8c930edebb58736"><img src="https://joss.theoj.org/papers/478dcc0b1608d2a4d8c930edebb58736/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/478dcc0b1608d2a4d8c930edebb58736/status.svg)](https://joss.theoj.org/papers/478dcc0b1608d2a4d8c930edebb58736)

Author instructions

Thanks for submitting your paper to JOSS @cgrudz. Currently, there isn't an JOSS editor assigned to your paper.

@cgrudz if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Wordcount for paper.md is 1301

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.17 s (368.2 files/s, 104060.2 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Julia                           24           1353           3628           6036
Python                          27            971            120           3984
Markdown                         3            119              0            582
TOML                             2            108              1            451
TeX                              2             11              0            107
YAML                             3              3              4             39
Bourne Shell                     1              1              0              2
-------------------------------------------------------------------------------
SUM:                            62           2566           3753          11201
-------------------------------------------------------------------------------


Statistical information for the repository '236bb7b6bbac5ce4da3f378c' was
gathered on 2021/12/03.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Colin Grudzien                  18          1572            312           26.23
Colin J Grudzien                 4            32            124            2.17
cgrudz                           7          2460            448           40.48
plinx                           17          2065            170           31.12

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Colin J Grudzien              7           21.9          8.4                0.00
plinx                      5068          245.4          4.4                2.35

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5281/zenodo.2029296 is OK

MISSING DOIs

- 10.1175/2009bams2618.1 may be a valid DOI for title: The data assimilation research testbed: A community facility
- 10.5194/gmd-2021-306 may be a valid DOI for title: A fast, single-iteration ensemble Kalman smoother for sequential data assimilation
- 10.1080/16000870.2018.1445364 may be a valid DOI for title: State-of-the-art stochastic data assimilation methods for high-dimensional non-Gaussian problems
- 10.5194/gmd-13-1903-2020 may be a valid DOI for title: On the numerical integration of the Lorenz-96 model, with scalar additive noise, for benchmark twin experiments
- 10.1002/wcc.535 may be a valid DOI for title: Data assimilation in the geosciences: An overview of methods, issues, and perspectives
- 10.5194/gmd-13-1903-2020 may be a valid DOI for title: On the numerical integration of the Lorenz-96 model, with scalar additive noise, for benchmark twin experiments

INVALID DOIs

- None

[kthyng]

Hi @taless474! Could this be in your wheelhouse?

[kthyng]

@whedon invite @taless474 as editor

[whedon]

@taless474 has been invited to edit this submission.

[kthyng]

@cgrudz in the meantime, could you state here how the GMD paper is different from the JOSS submission? The manuscript for GMD itself certainly doesn't look like an in-depth review of the software like in JOSS, but can you speak to what the reviewers of the paper will be doing as part of their review? From my experience with GMD, there are different "types" of papers which have different levels of rules regarding the software.

[taless474]

Hi @taless474! Could this be in your wheelhouse?

Hi @kthyng . Sure, I will do it.

[taless474]

@whedon assign me as editor

[whedon]

OK, the editor is @taless474

[cgrudz]

@kthyng thanks for handling this so quickly, @taless474, thanks for editing!

@cgrudz in the meantime, could you state here how the GMD paper is different from the JOSS submission? The manuscript for GMD itself certainly doesn't look like an in-depth review of the software like in JOSS, but can you speak to what the reviewers of the paper will be doing as part of their review? From my experience with GMD, there are different "types" of papers which have different levels of rules regarding the software.

Regarding the scope of the different manuscripts, the GMD work is more mathematical where we develop the theory for a novel iterative estimator, the SIEnKS. Particularly, this includes a review of several similar mathematical methods all within a Bayesian estimation framework. We also present a variety of numerical simulations to demonstrate the performance of the scheme versus other similar methods. Because this involved a significant amount of development to rigorously derive the numerics, the DataAssimilationBenchmarks.jl software package grew out of this mathematical study concurrently. We wish submit our manuscript to JOSS to better describe the software implementation aspects which are not discussed in GMD, together giving a comprehensive reference on the study. Writing the JOSS manuscript particularly involved the work of my undergraduate research assistant who was the primary supporting developer, building test cases and developing the overall package structure.

Please let me know if you have any further questions.

Cheers,
Colin

[kthyng]

@cgrudz Thank you!

[taless474]

@cgrudz, thank you for your submission. Would you please suggest a couple of potential reviewers?

[cgrudz]

@taless474, I would like to suggest the following:

John Maclean, Lecturer in Data Science and Statistics, School of Mathematical Sciences, University of Adelaide, john.maclean@adelaide.edu.au

Patrick N. Raanes, Researcher, NORCE Norwegian Research Institute, para@norceresearch.no, @patricknraanes

Artem Moiseev, Researcher, Nansen Environmental and Remote Sensing Center, Artem.Moiseev@nersc.no, @korvinos

Yumeng Chen, Data Assimilation Scientific Programmer (PDRA), National Centre for Earth Observation, University of Reading, yumeng.chen@reading.ac.uk, @yumengch

Please let me know if you would like further suggestions.

[taless474]

@patricknraanes, Are you interested in reviewing this?

[patnr]

@taless474 Hello and thank you for the invitation. I'd be willing, but must disclose that I cannot be a strict reviewer because the author and myself work and worked too closely together.

[taless474]

@patricknraanes thank you for your response. I think we should go forward with other reviewers.
@cgrudz do you have any suggestions from this list?

[cgrudz]

Ah, sorry @taless474, I missed the list at the top, certainly:

Henning Bonart, henning.bonart@tu-berlin.de, bonh, does research in data assimilation and has Julia listed as a secondary language.

Lindsey Heagy, lheagy@eoas.ubc.ca, lheagy, does research in geophysical inverse problems, which is a closely related area to the data assimilation methods studied in this package, and has Julia listed as a secondary language.

Felix Cremer, felix.cremer@dlr.de, felixcremer, does research in remote sensing and earth observation time series analysis, which is a related topic to data assimilation, and has Julia as a preferred language.

Lukas Riedel, lukas.riedel@iwr.uni-heidelberg.de, peanutfun, does research in data assimilation, and although their listed languages does not include Julia, I think they would be familiar with the mathematical methods that underpin the numerics.
Hopefully the software will be easy to understand with this background, as this would represent a target audience.

Please let me know if you need any further information or suggestions.

Cheers,
Colin

[taless474]

@lheagy, Are you interested in reviewing this?

[taless474]

@peanutfun, @bonh are you interested in reviewing this work?

[peanutfun]

Yes, I would like to review! I feel confident in reviewing the data assimilation methods and the repository overall, but I haven't worked with Julia yet. I will agree to review if another reviewer joins who is "fluent" in Julia.

[lheagy]

Thanks for pinging! This looks interesting, but I unfortunately do not have time at the moment to take on the review.

[bonh]

I like the idea, but I am not able to do another review that month.

[taless474]

Yes, I would like to review! I feel confident in reviewing the data assimilation methods and the repository overall, but I haven't worked with Julia yet. I will agree to review if another reviewer joins who is "fluent" in Julia.

Thank you. I will try to find someone fluent in Julia!

[taless474]

@whedon assign @peanutfun as reviewer

[whedon]

OK, @peanutfun is now a reviewer

[taless474]

@felixcremer are you interested in reviewing this work?

[taless474]

@ysimillides are you interested in reviewing this?

[taless474]

@cgrudz do you have any other potential recruiters?

[cgrudz]

Tangi Migot, tmigot, works in related areas and has Julia as a primary language.

Andy Nowacki, anowacki, works in related areas and has Julia as a primary language.

Matthew Wilhelm, mewilhel, works in related areas and has Julia as a primary language.

Luke Gloege, lgloege, works in related areas and has Julia as a secondary language.

Cheers,
Colin

[taless474]

@tmigot are you interested in reviewing this?

[tmigot]

Yes I can do it.

[taless474]

@whedon add @tmigot as reviewer

[whedon]

OK, @tmigot is now a reviewer

[taless474]

@whedon start review

[whedon]

OK, I've started the review over in #4129.