[REVIEW]: CategoricalTimeSeries.jl: A toolbox for categorical time-series analysis

[whedon]

Submitting author: @johncwok (Corentin Nelias)
Repository: https://github.com/johncwok/CategoricalTimeSeries.jl
Version: v1.1.4
Editor: @jbytecode
Reviewer: @bkamins, @felixcremer
Archive: 10.6084/m9.figshare.16917625

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/919a3be5c11e2add7274a8517066361e"><img src="https://joss.theoj.org/papers/919a3be5c11e2add7274a8517066361e/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/919a3be5c11e2add7274a8517066361e/status.svg)](https://joss.theoj.org/papers/919a3be5c11e2add7274a8517066361e)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@bkamins & @felixcremer, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @jbytecode know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @bkamins

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@johncwok) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @felixcremer

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@johncwok) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @bkamins, @felixcremer it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Wordcount for paper.md is 784

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1137/141000671 is OK
- 10.1093/biomet/80.3.611 is OK
- 10.1002/9781119097013 is OK
- 10.1145/369133.369172 is OK
- 10.1162/neco_a_00961 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.10 s (381.9 files/s, 99031.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
SVG                              1              0              0           2671
JavaScript                       6            473           1213           2025
Julia                            8            183             17           1309
HTML                             7            272             18            950
Markdown                         8            134              0            513
CSS                              2             16             67             71
TeX                              1              8              0             70
YAML                             3             11              2             53
TOML                             1              5              0             20
XML                              1              0              0             11
JSON                             1              0              0              1
-------------------------------------------------------------------------------
SUM:                            39           1102           1317           7694
-------------------------------------------------------------------------------


Statistical information for the repository '194e50e4d4a79db82c104c3e' was
gathered on 2021/09/16.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Corentin Nelias                  1          3711              0          100.00

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Corentin Nelias            3711          100.0          0.0               32.69

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[jbytecode]

Dear @bkamins and @felixcremer 👋👋👋

This is the review issue. There are 20 check items for each reviewer. Whenever you finish reviewing the corresponding item, you can set it checked. You can also interact with the author(s), other reviewer(s) and me (editor) during the reviewing process.

Please do not hesitate to ask me anything.

Thank you in advance.

[jbytecode]

Dear @bkamins and @felixcremer, do you need help? Could you please update your status on how is your reviewing going?

[bkamins]

Thank you for pinging. Now I can see that the package got a release 5 days ago that I have requested for earlier. I will go back to the review then.

[bkamins]

I am not fully sure where I should put the comments, so I put them here:

• I would recommend that all packages (also dependencies) get a proper release
• All packages should have explicit [compat] section in their Project.toml (including the [extras] part for e.g. CSV.jl)
• The print_results does not work under current release of DataFrames.jl (I can help fixing it if needed)
• The examples in the tutorial do not work under current release of CSV.jl (I can help fixing it if needed)
• In general before reviewing the whole documentation and manual should be tested against versions of the packages specified in [compat]
• also I would recommend to disambiguate the paths given in the manual (as they are e.g. given in the main package documentation while the files are in the IntegerIB.jl package sources)

Regarding the analytical side of the package:

• the authors mention nominal and ordinal categorical data but do not explain how (or if) the fact that data is ordinal would be used by the package;
• maybe also comment in the documentation how this package could be integrated with CategoricalArrays.jl?

Thank you for working on this.

[jbytecode]

@bkamins it is okay to put your comments here in the thread, thank you.

[CNelias]

Dear @bkamins,

I addressed your comments, specifically:

• made a release for all involved packages.
• updated the code to be compatible with the most recent versions of DataFrames.jl and CSV.jl.
• the files linked in the doc are all contained in the CategoricalTimeSeries.jl repo, and not pointing to other repos anymore.
• I tested the functions with the most recent versions of every packages (as specified in compat), and they are working.
• I adapted the description in the doc. The reason why I wrote about 'nominal' and 'ordinal' was just as a sort of introduction regarding what type of data are referred to as 'categorical'.
  In itself the package is agnostic about the precise type of data since none of the methods are relying on a pre-existing ordering. The only requirement is that the data should be discrete. One would still get results with continuous data, but these would most probably be nonsensical.

I have one question\need help:

• The reason why I didn't have any compat entry is because when I do add them, then the package doesn't pass the test on the travis virtual machine, even though it works perfectly otherwise. I end up having error like

Unsatisfiable requirements detected for package foo [xxxxx]:

 foo [xxxxx] log:

 ├─possible versions are: [0.1.0-0.1.2, 0.2.0-0.2.4, 0.3.0, 1.0.0-1.0.1, 1.1.0-1.1.1, 1.2.0-1.2.4, 1.3.0-1.3.2, 1.4.0-1.4.5] or uninstalled
 ├─restricted to versions 1.4.5-1 by an explicit requirement, leaving only versions 1.4.5
 └─restricted by julia compatibility requirements to versions: [0.1.0-0.1.2, 0.2.0-0.2.4, 0.3.0, 1.0.0-1.0.1, 1.1.0-1.1.1] or uninstalled — no versions left

When I try to fix it, another package will throw this error. For example, if I fix it for FFTW, then Plots errors, and if I fix it for Plots then another one bugs and it is a never ending cycle. I search for a solution but couldn't fix this so I just removed all "problematic" compats in order to have a working package.
Do you know what causes this and how I can solve it? There are already similar issues on the discourse but they are not resolved or do not solve my issue.

[bkamins]

then the package doesn't pass the test on the travis virtual machine, even though it works perfectly otherwise

The reason is the following. You seem not to have done the package releases properly. For instance IntegerIB.jl package has 0.1 tag only made yesterday. However, in Julia Global Registry, here https://github.com/JuliaRegistries/General/blob/master/I/IntegerIB/Versions.toml you see that 0.1 tag was added 13 months ago with git-tree-sha1 786b9878a5889e1feec38510e694016c3bb76c47.

This makes Julia Package manager install the IntegerIB.jl package from 13 months ago not from yesterday when you ask for it being installed (you can check it on a CLEAN environment - without any packages installed in a federated package repository - this is exactly what happens on Travis).

Note that Julia does not look at what is the tag in the GitHub repository for the release but it looks at git-tree-sha1 in https://github.com/JuliaRegistries/General to resolve package versions. You can read how new releases of packages should be registered in https://github.com/JuliaRegistries/Registrator.jl

[CNelias]

So re-registering all the packages to their current version should solve this right?

[bkamins]

No - you have to bump the version number and then register. The problem is exactly the fact that you use the same version number for different states of package GitHub repository (Julia will not allow registering the same version number twice as it would be ambiguous).

Also note that this has to be done sequentially. If A depends on B and B on C then you have to:

• bump version of C
• register new version of C
• change B to depend on new version of C
• bump version of B
• register new version of B
• change A to depend on new version of B
• bump version of A
• register new version of A

[jbytecode]

@bkamins just to inform you, you can send a pull request to the repo under review, if needed. all of the contributions are welcome.

[bkamins]

@jbytecode - it probably requires like 5 PRs in different repos and testing if all works correctly sequentially. The repo maintainer OTOH can probably do each operation in 1 minute (as maintainer has full rights to the repo and knows the packages and their interdependencies).

[CNelias]

@bkamins I took care of it, and it is passing CI now. I don't know how long it will take until the re-registration is complete though.

[whedon]

👋 @bkamins, please update us on how your review is going (this is an automated reminder).

[whedon]

👋 @felixcremer, please update us on how your review is going (this is an automated reminder).

[bkamins]

I don't know how long it will take until the re-registration is complete though.

I have checked your PRs to General Registry. As you can see your packages (for different reasons) fail the registration process.
If you have any questions regarding the registration please ask me and I will try to help.

Otherwise - could you please let me know when the registration is finished - thank you!
(in particular at least one of the packages is missing license and having proper licensing is also a JOSS requirement)

[CNelias]

Do you know why the auto-merge from MotifRecognition.jl failed? The license is there, and the error message is totally incomprehensible to me.
I have added the missing license to SpectralEnvelope.jl.

[bkamins]

Do you know why the auto-merge from MotifRecognition.jl failed?

As you can see in JuliaRegistries/General#45831 (comment) there are two problems:

• your [compat] section is incomplete (missing entry for DataStructures.jl)
• your version numbering is skipping versions. Since currently known to Julia Registry version of this package is 0.1 then it expected that the patch release would have number 0.1.1, while you have made the release with 0.1.11 version, which skips many versions in-between (like 0.1.1, 0.1.2, 0.1.3, ..., 0.1.10)

[CNelias]

I have added the [compat] for DataStructures.jl.

I guess I will just let you know when the re-registration is complete then.

[jbytecode]

@whedon set v1.1.4 as version

[whedon]

OK. v1.1.4 is the version.

[jbytecode]

@whedon set 10.6084/m9.figshare.16917625 as archive

[whedon]

OK. 10.6084/m9.figshare.16917625 is the archive.

[jbytecode]

@whedon recommend-accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1137/141000671 is OK
- 10.1093/biomet/80.3.611 is OK
- 10.1002/9781119097013 is OK
- 10.1145/369133.369172 is OK
- 10.1162/neco_a_00961 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#2723

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#2723, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

[danielskatz]

@johncwok - can you update the title of the figshare archive to match the title of the paper: "CategoricalTimeSeries.jl: A toolbox for categorical time-series analysis"?

[danielskatz]

I've also suggested some small changes in the paper in CNelias/CategoricalTimeSeries.jl#13 - please merge this, or let me know what you disagree with, then we can proceed to acceptance and publishing.

[CNelias]

Dear @danielskatz,

I have no objections to the small changes you made to the paper, thanks for the suggestions.
I changed the title of the figshare archive to match the paper.

[danielskatz]

@whedon recommend-accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1137/141000671 is OK
- 10.1093/biomet/80.3.611 is OK
- 10.1002/9781119097013 is OK
- 10.1145/369133.369172 is OK
- 10.1162/neco_a_00961 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#2724

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#2724, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

[danielskatz]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.03733 joss-papers#2725
1. Wait a couple of minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.03733
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[danielskatz]

Congratulations to @johncwok (Corentin Nelias)!!

And thanks to @jbytecode for editing, and @bkamins and @felixcremer for reviewing!
We couldn't do this without you

[danielskatz]

At least for me right now, the DOI is not resolving - sometimes it takes longer than other times.

Once it does, I will close this issue.

[danielskatz]

the DOI now works for me - again, congratulations & thanks to everyone for making this a smooth and productive process!

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.03733/status.svg)](https://doi.org/10.21105/joss.03733)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.03733">
  <img src="https://joss.theoj.org/papers/10.21105/joss.03733/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.03733/status.svg
   :target: https://doi.org/10.21105/joss.03733

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss