datefixR: Fix Really Messy Dates

[nathansam]

Date accepted: 2022-07-16
Submitting Author Name: Nathan Constantine-Cooke
Submitting Author Github Handle: @nathansam
Other Package Authors Github handles: (comma separated, delete if none)
Repository: https://github.com/nathansam/datefixR
Version submitted: 0.1.4.9000
Submission type: Standard
Editor: @adamhsparks
Reviewers: @AlvesKS, @alstat

Due date for @AlvesKS: 2022-05-30

Due date for @alstat: 2022-07-15
Archive: TBD
Version accepted: TBD
Language: en

• Paste the full DESCRIPTION file inside a code block below:

Package: datefixR
Title: Fix Really Messy Dates
Version: 0.1.4.9000
Authors@R: person("Nathan",
                  "Constantine-Cooke",
                  email = "nathan.constantine-cooke@ed.ac.uk",
                  role = c("aut", "cre"),
                  comment = c(ORCID = "0000-0002-4437-8713"))
Description: Fixes messy dates in data frames such as those entered via text
  boxes. Standardizes / - and whitespace separation, month abbreviations, and
  year first or day first by converting to R's built-in Date class. Imputes
  missing date or month using user-provided values.  
License: GPL (>= 3)
Depends: R (>= 4.0.0)
Imports: 
    lifecycle,
    stringr
Language: en-US
Encoding: UTF-8
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.1.2
URL: https://www.constantine-cooke.com/datefixR/ https://github.com/nathansam/datefixR
BugReports: https://github.com/nathansam/datefixR/issues
Suggests: 
    rmarkdown,
    knitr,
    testthat (>= 3.0.0)
Config/testthat/edition: 3
VignetteBuilder: knitr

Scope

• Please indicate which category or categories from our package fit policies this package falls under: (Please check an appropriate box below. If you are unsure, we suggest you make a pre-submission inquiry.):

  • data retrieval
  • data extraction
  • data munging
  • data deposition
  • data validation and testing
  • workflow automation
  • version control
  • citation management and bibliometrics
  • scientific software wrappers
  • field and lab reproducibility tools
  • database software bindings
  • geospatial data
  • text analysis

• Explain how and why the package falls under these categories (briefly, 1-2 sentences):

datefixR takes date data in which has been stored in many different formats (01/01/2001, 5 April 2020, Dec 2015 etc.) and converts them to R's Date type.

• Who is the target audience and what are scientific applications of this package?

Any researchers using data entered via a questionnaire which (unfortunately) asked for a date as free-text. Given the nature of this data generation, this mainly affects those who work with human subjects.

• Are there other R packages that accomplish the same thing? If so, how does yours differ or meet our criteria for best-in-category?

lubridate::guess_formats() can be used to guess a date format and lubridate::parse_date_time() calls this function when it attempts to parse a vector into a POSIXct date-time object. However:

1. When a date fails to parse in {lubridate} then the user is simply told how many dates failed to parse. In datefixR the user is told the ID (assumed to be the first column by default but can be user-specified) corresponding to the date which failed to parse and reports the considered date: making it much easier to figure out which dates supplied failed to parse and why.
2. When imputing a missing day or month, there is no user-control over this behaviour. For example, when imputing a missing month, the user may wish to impute July, the middle of the year, instead of January. However, January will always be imputed in {lubridate}. In datefixR, this behaviour can be controlled by the month.impute argument.
3. These functions require all possible date formats to be specified in the orders argument, which may result in a date format not being considered if the user forgets to list one of the possible formats. By contrast, datefixR only needs a format to be specified if month-first is to be preferred over day-first when guessing a date.

However, {lubridate} of course excels in general date manipulation and is an excellent tool to use alongside datefixR.

An alternative function is anytime::anydate() which also attempts to convert dates to a consistent format (POSIXct). However {anytime} assumes year, month, and day have all been provided and does not permit imputation. Moreover, if a date cannot be parsed, then the date is converted to an NA object and no warning is raised- which may lead to issues later in the analysis.

• (If applicable) Does your package comply with our guidance around Ethics, Data Privacy and Human Subjects Research?

NA

• If you made a pre-submission inquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

#529

• Explain reasons for any pkgcheck items which your package is unable to pass.

pkgcheck claims: "Package has no continuous integration checks." however Github actions is already used so this is a false positive.

Technical checks

Confirm each of the following by checking the box.

• I have read the rOpenSci packaging guide.
• I have read the author guide and I expect to maintain this package for at least 2 years or to find a replacement.

This package:

• does not violate the Terms of Service of any service it interacts with.
• has a CRAN and OSI accepted license.
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions, created with roxygen2.
• contains a vignette with examples of its essential functions and uses.
• has a test suite.
• has continuous integration, including reporting of test coverage using services such as Travis CI, Coveralls and/or CodeCov.

Publication options

• Do you intend for this package to go on CRAN?
• Do you intend for this package to go on Bioconductor?

(Already available on CRAN)

• Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:

NA

MEE Options

• The package is novel and will be of interest to the broad readership of the journal.
• The manuscript describing the package is no longer than 3000 words.
• You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see MEE's Policy on Publishing Code)
• (Scope: Do consider MEE's Aims and Scope for your manuscript. We make no guarantee that your manuscript will be within MEE scope.)
• (Although not required, we strongly recommend having a full manuscript prepared when you submit here.)
• (Please do not submit your package separately to Methods in Ecology and Evolution)

Code of conduct

• I agree to abide by rOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

[ropensci-review-bot]

Thanks for submitting to rOpenSci, our editors and @ropensci-review-bot will reply soon. Type @ropensci-review-bot help for help.

[ropensci-review-bot]

🚀

Editor check started

👋

[ropensci-review-bot]

Checks for datefixR (v0.1.4.9000)

git hash: 5513135a

• ✔️ Package is already on CRAN.
• ✔️ has a 'codemeta.json' file.
• ✔️ has a 'contributing' file.
• ✔️ uses 'roxygen2'.
• ✔️ 'DESCRIPTION' has a URL field.
• ✔️ 'DESCRIPTION' has a BugReports field.
• ✔️ Package has at least one HTML vignette
• ✔️ All functions have examples.
• ✔️ Package has continuous integration checks.
• ✔️ Package coverage is 100%.
• ✔️ R CMD check found no errors.
• ✔️ R CMD check found no warnings.

Package License: GPL (>= 3)

---

1. Package Dependencies

Details of Package Dependency Usage (click to open)

The table below tallies all function calls to all packages ('ncalls'), both internal (r-base + recommended, along with the package itself), and external (imported and suggested packages). 'NA' values indicate packages to which no identified calls to R functions could be found. Note that these results are generated by an automated code-tagging system which may not be entirely accurate.

type	package	ncalls
internal	base	11
internal	datefixR	2
internal	stats	1
imports	stringr	1
suggests	rmarkdown	NA
suggests	knitr	NA
suggests	testthat	NA
linking_to	NA	NA

Click below for tallies of functions used in each package. Locations of each call within this package may be generated locally by running 's <- pkgstats::pkgstats(<path/to/repo>)', and examining the 'external_calls' table.

base

date (3), for (3), paste0 (2), c (1), new.env (1), nrow (1)

datefixR

fix_date (2)

stats

df (1)

stringr

str_split_fixed (1)

---

2. Statistical Properties

This package features some noteworthy statistical properties which may need to be clarified by a handling editor prior to progressing.

Details of statistical properties (click to open)

The package has:

• code in R (100% in 2 files) and
• 1 authors
• 1 vignette
• no internal data file
• 1 imported package
• 2 exported functions (median 60 lines of code)
• 24 non-exported functions in R (median 11 lines of code)

---

Statistical properties of package structure as distributional percentiles in relation to all current CRAN packages
The following terminology is used:

• loc = "Lines of Code"
• fn = "function"
• exp/not_exp = exported / not exported

All parameters are explained as tooltips in the locally-rendered HTML version of this report generated by the checks_to_markdown() function

The final measure (fn_call_network_size) is the total number of calls between functions (in R), or more abstract relationships between code objects in other languages. Values are flagged as "noteworthy" when they lie in the upper or lower 5th percentile.

measure	value	percentile	noteworthy
files_R	2	13.6
files_vignettes	1	68.4
files_tests	3	75.2
loc_R	287	31.1
loc_vignettes	126	32.9
loc_tests	243	59.4
num_vignettes	1	64.8
n_fns_r	26	35.7
n_fns_r_exported	2	6.8
n_fns_r_not_exported	24	46.2
n_fns_per_file_r	8	80.7
num_params_per_fn	5	69.6
loc_per_fn_r	12	38.9
loc_per_fn_r_exp	60	84.1
loc_per_fn_r_not_exp	12	38.4
rel_whitespace_R	10	22.6
rel_whitespace_vignettes	27	25.8
rel_whitespace_tests	21	56.8
doclines_per_fn_exp	48	60.4
doclines_per_fn_not_exp	0	0.0	TRUE
fn_call_network_size	21	47.5

---

2a. Network visualisation

Click to see the interactive network visualisation of calls between objects in package

---

3. goodpractice and other checks

Details of goodpractice and other checks (click to open)

3a. Continuous Integration Badges

| |

GitHub Workflow Results

name	conclusion	sha	date
pages build and deployment	success	b11eed	2022-04-29
pkgdown	success	551313	2022-04-29
R-CMD-check	success	551313	2022-04-29

---

3b. goodpractice results

R CMD check with rcmdcheck

rcmdcheck found no errors, warnings, or notes

Test coverage with covr

Package coverage: 100

Cyclocomplexity with cyclocomp

The following function have cyclocomplexity >= 15:

function	cyclocomplexity
fix_date	35

Static code analyses with lintr

lintr found the following 4 potential issues:

message	number of times
Avoid 1:nrow(...) expressions, use seq_len.	1
Lines should not be more than 80 characters.	3

---

Package Versions

package	version
pkgstats	0.0.4.30
pkgcheck	0.0.3.11

---

Editor-in-Chief Instructions:

This package is in top shape and may be passed on to a handling editor

[jooolia]

@ropensci-review-bot assign @adamhsparks as editor

[ropensci-review-bot]

Assigned! @adamhsparks is now the editor

[adamhsparks]

Hi @nathansam, could you address the issues lintr found, Avoid 1:nrow(...) expressions, use seq_len. and the lines of code >80 chars, please?

While you're doing that I'll start looking for reviewers.

[adamhsparks]

@ropensci-review-bot seeking reviewers

[ropensci-review-bot]

Please add this badge to the README of your package repository:

[![Status at rOpenSci Software Peer Review](https://badges.ropensci.org/533_status.svg)](https://github.com/ropensci/software-review/issues/533)

Furthermore, if your package does not have a NEWS.md file yet, please create one to capture the changes made during the review process. See https://devguide.ropensci.org/releasing.html#news

[nathansam]

Hi @adamhsparks, thank you! Those lintr issues should now be fixed.

[adamhsparks]

Hi @nathansam, as I've been looking for reviewers and checking over the package some more, I did note some issues with the spelling and tests as well.

When I check the tests, I get this:

Backtrace:
 1. testthat::expect_equal(...)
      at test_fix_dates.R:206:2
 4. datefixR::fix_dates(temp, "date", day.impute = NA)
 9. datefixR::.fix_date(...)
      at datefixR/R/fix_dates.R:81:10

── Warning (test_fix_dates.R:215:3): Imputing month with NA results in NA ──────
NA imputed for subject 1 (date: 1994)

Backtrace:
 1. testthat::expect_equal(...)
      at test_fix_dates.R:215:2
 4. datefixR::fix_dates(temp, "date", month.impute = NA)
 9. datefixR::.fix_date(...)
      at datefixR/R/fix_dates.R:81:10

With the spelling, your DESCRIPTION says "en-US" but you've used British English for "behaviour", also the spellcheck is picking up a few other words that are non-standard but probably correct anyway. I suggest setting up spell checking with usethis::use_spell_check().

[nathansam]

Thanks @adamhsparks ! I have fixed the typo and have now set up {spelling} as you have suggested. Thanks for the suggestion; I will definitely be using it across all R packages I maintain. Thank you for catching the warnings in the tests. The warnings are intended (but were added after the tests were first written), and I have now added explicit expect_warning() statements to those tests.

[adamhsparks]

@ropensci-review-bot add @AlvesKS to reviewers

[ropensci-review-bot]

@AlvesKS added to the reviewers list. Review due date is 2022-05-30. Thanks @AlvesKS for accepting to review! Please refer to our reviewer guide.

[ropensci-review-bot]

@AlvesKS: If you haven't done so, please fill this form for us to update our reviewers records.

[adamhsparks]

@ropensci-review-bot add @arhepworth to reviewers

[ropensci-review-bot]

@arhepworth added to the reviewers list. Review due date is 2022-05-31. Thanks @arhepworth for accepting to review! Please refer to our reviewer guide.

[ropensci-review-bot]

@arhepworth: If you haven't done so, please fill this form for us to update our reviewers records.

[adamhsparks]

Hi @alstat, please don't forget to complete our review template along with the issues that you've opened for datefixR when you're finished with your review. 🙏

[alstat]

Noted @adamhsparks, still reviewing, hoping to complete it within the week. Thanks.

[alstat]

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

• Briefly describe any working relationship you have (had) with the package authors.
• As the reviewer I confirm that there are no conflicts of interest for me to review this work (if you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

• A statement of need: clearly stating problems the software is designed to solve and its target audience in README
• Installation instructions: for the development version of package and any non-standard dependencies in README
• Vignette(s): demonstrating major functionality that runs successfully locally
• Function Documentation: for all exported functions
• Examples: (that run successfully locally) for all exported functions
• Community guidelines: including contribution guidelines in the README or CONTRIBUTING, and DESCRIPTION with URL, BugReports and Maintainer (which may be autogenerated via Authors@R).

Functionality

• Installation: Installation succeeds as documented.
• Functionality: Any functional claims of the software been confirmed.
• Performance: Any performance claims of the software been confirmed.
• Automated tests: Unit tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
• Packaging guidelines: The package conforms to the rOpenSci packaging guidelines.

Estimated hours spent reviewing: 8hrs

• Should the author(s) deem it appropriate, I agree to be acknowledged as a package reviewer ("rev" role) in the package DESCRIPTION file.

Issues opened

• Docs: add format output in the value returned by fix_date #17
• Docs: add datetime input as limitation #18

---

Review Comments

The package provides a straightforward formatting of the dates, and its simplicity makes it a very good alternative to lubridate.

[nathansam]

Thank you both for your reviews, they are gratefully received! I believe I have implemented all of your feedback in the ropensci branch (which I will merge into main if you are both believe the changes I have implemented adequately implement your feedback). Please do let me know if you happy with the changes.

---

@AlvesKS

README

I recommend stating clearly what is the purpose of the package in the README file. The way it is stated in the first paragraph of the README file is okay, but I think it could be more detailed.

Thank you, I have rewritten the purpose of the package which I believe is now clearer.

https://github.com/nathansam/datefixR/blob/6d686729aea7e12dcaf9c66c536137d936fbef0e/README.md?plain=1#L17-L29

Function naming

I think the names of the function can be easily confused. The authors should consider changing the names to fixdate_df() and fixdate_char().

I agree and have changed the function names to fix_date_df() and fix_date_char()

https://github.com/nathansam/datefixR/blob/6d686729aea7e12dcaf9c66c536137d936fbef0e/R/fix_date_df.R#L41-L46
and
https://github.com/nathansam/datefixR/blob/6d686729aea7e12dcaf9c66c536137d936fbef0e/R/fix_date_char.R#L20-L23

As this package is already on CRAN and appears to have some degree of a user base, I have deprecated fix_date() and fix_dates().

https://github.com/nathansam/datefixR/blob/6d686729aea7e12dcaf9c66c536137d936fbef0e/R/fix_dates.R#L57-L72

Interaction with dplyr::mutate()

I notice that it is not possible to use fix_date() in the mutate() function from dplyr package. I recommend adding this functionality.

Thank you for the suggestion. This behaviour was due to fix_date() not supporting vectors greater than length 1. fix_date_char() now uses sapply() so vectors are now supported and fix_date_char()/fix_date() works with dplyr::mutate().

https://github.com/nathansam/datefixR/blob/6d686729aea7e12dcaf9c66c536137d936fbef0e/R/fix_date_char.R#L28-L34

fix_dates() documentation

In the fix_dates() description (documentation), it is stated "Cleans up a dataframe object which...". Consider replacing "Cleans up" with "correct", "fix up", "tidy", "adjust", etc. When I first read "Cleans up", I thought that the function would erase the data from the data frame.

Thank you, the term "tidies" is now used instead.

https://github.com/nathansam/datefixR/blob/6d686729aea7e12dcaf9c66c536137d936fbef0e/R/fix_date_df.R#L6

### Vignette

Provide a function to call the first data frame (bad.dates) provided in the package Vignette. It would be faster to reproduce the examples. After looking at the function documentation, I saw that the code to build the bad.dates data frame (with different data) is provided in the fix_dates() documentation (help material), but I think having a ready to use data frame is better. Another option is providing the code to build the data frame in the Vignette.

Thank you. There is now an example dataset, exampledates, provided by the package which includes examples of dates in different formats some of which have missing date data to demonstrate the imputation feature.
https://github.com/nathansam/datefixR/blob/ropensci/data/exampledates.rda

In the vignette, I still create a data frame, as I want to introduce the data standardisation feature before the imputation feature rather than introducing both at the same time. However, I have now made the data frame creation visible in the vignette by setting echo = TRUE for the code chunk.

https://github.com/nathansam/datefixR/blob/6d686729aea7e12dcaf9c66c536137d936fbef0e/vignettes/datefixR.Rmd#L41-L60

@alstat

I note you have closed the issues you created (thanks for creating them!), so I assume you are satisfied with my additions. Nevertheless, I will include here for the sake of keeping everything together.

Docs: add format output in the value returned by fix_date ropensci/datefixR#17

Thank you! Date Format is now given in the documentation.

https://github.com/nathansam/datefixR/blob/6d686729aea7e12dcaf9c66c536137d936fbef0e/R/fix_date_df.R#L31-L33

Docs: add datetime input as limitation ropensci/datefixR#18

Thank you! Not supporting datetime formats has now been included as a limitation in the readme file.

https://github.com/nathansam/datefixR/blob/6d686729aea7e12dcaf9c66c536137d936fbef0e/README.md?plain=1#L128-L134

[adamhsparks]

@alstat and @AlvesKS, could I ask you to please complete your reviews with the estimated time taken so I can enter them into our database? thank you

[ropensci-review-bot]

📆 @alstat you have 2 days left before the due date for your review (2022-07-15).

[adamhsparks]

@ropensci-review-bot submit review #533 (comment) time 9

[ropensci-review-bot]

Logged review for AlvesKS (hours: 9)

[adamhsparks]

@ropensci-review-bot submit review #533 (comment)

[ropensci-review-bot]

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@ropensci-review-bot help

[alstat]

@adamhsparks I've completed the checklist.

[adamhsparks]

@ropensci-review-bot submit review #533 (comment) time 8

[ropensci-review-bot]

Logged review for alstat (hours: 8)

[adamhsparks]

Thanks, @alstat for finalising that.

@alstat and @AlvesKS, please let us know when you're satisfied with the changes or if you would prefer more.

[alstat]

I'm satisfied with the changes here, both issues I opened were addressed.

[AlvesKS]

Hi @adamhsparks,

I am also satisfied with the changes.
The author has done a great job addressing the suggestions I made.
Congratulations @nathansam!

Kaique

[adamhsparks]

@ropensci-review-bot approve datefixR

[ropensci-review-bot]

Approved! Thanks @nathansam for submitting and @AlvesKS, @alstat for your reviews! 😁

To-dos:

• Transfer the repo to rOpenSci's "ropensci" GitHub organization under "Settings" in your repo. I have invited you to a team that should allow you to do so. You will need to enable two-factor authentication for your GitHub account.
  This invitation will expire after one week. If it happens write a comment @ropensci-review-bot invite me to ropensci/<package-name> which will re-send an invitation.
• After transfer write a comment @ropensci-review-bot finalize transfer of <package-name> where <package-name> is the repo/package name. This will give you admin access back.
• Fix all links to the GitHub repo to point to the repo under the ropensci organization.
• Delete your current code of conduct file if you had one since rOpenSci's default one will apply, see https://devguide.ropensci.org/collaboration.html#coc-file
• If you already had a pkgdown website and are ok relying only on rOpenSci central docs building and branding,
  • deactivate the automatic deployment you might have set up
  • remove styling tweaks from your pkgdown config but keep that config file
  • replace the whole current pkgdown website with a redirecting page
  • replace your package docs URL with https://docs.ropensci.org/package_name
  • In addition, in your DESCRIPTION file, include the docs link in the URL field alongside the link to the GitHub repository, e.g.: URL: https://docs.ropensci.org/foobar, https://github.com/ropensci/foobar
• Fix any links in badges for CI and coverage to point to the new repository URL.
• Increment the package version to reflect the changes you made during review. In NEWS.md, add a heading for the new version and one bullet for each user-facing change, and each developer-facing change that you think is relevant.
• We're starting to roll out software metadata files to all rOpenSci packages via the Codemeta initiative, see https://docs.ropensci.org/codemetar/ for how to include it in your package, after installing the package - should be easy as running codemetar::write_codemeta() in the root of your package.
• You can add this installation method to your package README install.packages("<package-name>", repos = "https://ropensci.r-universe.dev") thanks to R-universe.

Should you want to acknowledge your reviewers in your package DESCRIPTION, you can do so by making them "rev"-type contributors in the Authors@R field (with their consent).

Welcome aboard! We'd love to host a post about your package - either a short introduction to it with an example for a technical audience or a longer post with some narrative about its development or something you learned, and an example of its use for a broader readership. If you are interested, consult the blog guide, and tag @ropensci/blog-editors in your reply. They will get in touch about timing and can answer any questions.

We maintain an online book with our best practice and tips, this chapter starts the 3d section that's about guidance for after onboarding (with advice on releases, package marketing, GitHub grooming); the guide also feature CRAN gotchas. Please tell us what could be improved.

Last but not least, you can volunteer as a reviewer via filling a short form.

[adamhsparks]

Thank you, @AlvesKS and @alstat for your kind and insightful reviews. I hope you found this experience useful and enjoyable. @nathansam, thank you for submitting datefixR. You can transfer it to rOpenSci now.

[nathansam]

Thank you @AlvesKS and @alstat, your reviews were immeasurably helpful! I am of course more than happy to acknowledge you both as reviewers in DESCRIPTION. I have added your details based on the DESCRIPTION files for your own packages. Please let me know if you would like any edits made to your listings in the description.
https://github.com/nathansam/datefixR/blob/3a2b33b657c19fce085f57ac23a831f043838da8/DESCRIPTION

Thank you @adamhsparks for your editorship throughout and for the issues you have opened. I will begin the process of migrating datefixR to the ropensci organization now.

[nathansam]

@ropensci-review-bot finalize transfer of datefixR

[ropensci-review-bot]

Transfer completed.
The datefixR team is now owner of the repository and the author has been invited to the team