Submission: refnet

[aurielfournier]

Summary

• What does this package do? (explain in 50 words or less):
  refnet is a package to read, organize, geocode, analyze, and visualize Clarivate Web of Knowledge/Web of Science, format reference data files for scientometric, social network, and Science of Science analyses.

• Paste the full DESCRIPTION file inside a code block below:

Package: refnet
Type: Package
Title: Thomson Reuters Web of Knowledge/Science and ISI Reference Data Tools
Version: 0.6
Date: 2018-08-26
Authors@R: c(person("Auriel M.V. Fournier", "Developer", role = c("aut"),
                     email = "aurielfournier@gmail.com"),
              person("Forrest R. Stevens", "Developer", role = "aut"),
              person("Matthew E. Boone", "Developer", role = "aut"),
              person("Emilio Bruna", "Developer", role=c("aut","cre"), 
              email="embruna@ufl.edu"))
Maintainer: Emilio Bruna <embruna@ufl.edu>
Description: This function reads Thomson Reuters Web of Knowledge/Science and ISI format reference data files into an R friendly data format and can optionally write the converted data to a friendly CSV format.
License: GPL-3
Imports: maptools, maps, rworldmap, RecordLinkage, Matrix, igraph, network, sna, Hmisc, ggplot2, stringi, stringr, ggmap, Rdpack, tidyr, dplyr, tibble
RoxygenNote: 6.1.0
RdMacros: Rdpack
Suggests: testthat, utils
VignetteBuilder: utils
Encoding: UTF-8

• URL for the package (the development repository, not a stylized html page):
  https://github.com/embruna/refnet

• Please indicate which category or categories from our package fit policies this package falls under *and why(? (e.g., data retrieval, reproducibility. If you are unsure, we suggest you make a pre-submission inquiry.):

Data extraction and munging, since it takes data from one format, and transforms it into something that is useful, and also matches up records among authors.

[Note, the link for the package fit, does not lead to that page anymore, and I couldn't find anything about package fit in the linked policies]

•   Who is the target audience and what are scientific applications of this package?  

Scientists interested in studying the networks of a particular author, subject area or journal.

• Are there other R packages that accomplish the same thing? If so, how does
  yours differ or meet our criteria for best-in-category?

•   If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

#247

Requirements

Confirm each of the following by checking the box. This package:

• does not violate the Terms of Service of any service it interacts with.
• has a CRAN and OSI accepted license.
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions.
• contains a vignette with examples of its essential functions and uses.
• has a test suite.
• has continuous integration, including reporting of test coverage, using services such as Travis CI, Coveralls and/or CodeCov.
• I agree to abide by ROpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

Publication options

• [yes] Do you intend for this package to go on CRAN?
• [yes] Do you wish to automatically submit to the Journal of Open Source Software? If so:
  • The package has an obvious research application according to JOSS's definition.
  • The package contains a paper.md matching JOSS's requirements with a high-level description in the package root or in inst/.
  • [not yet] The package is deposited in a long-term repository with the DOI:
  • (Do not submit your package separately to JOSS)
• [no] Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:
  • The package is novel and will be of interest to the broad readership of the journal.
  • The manuscript describing the package is no longer than 3000 words.
  • You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see MEE's Policy on Publishing Code)
  • (Scope: Do consider MEE's Aims and Scope for your manuscript. We make no guarantee that your manuscript will be within MEE scope.)
  • (Although not required, we strongly recommend having a full manuscript prepared when you submit here.)
  • (Please do not submit your package separately to Methods in Ecology and Evolution)

Detail

• [yes] Does R CMD check (or devtools::check()) succeed? Paste and describe any errors or warnings:

• [yes] Does the package conform to rOpenSci packaging guidelines? Please describe any exceptions:

• If this is a resubmission following rejection, please explain the change in circumstances:

• If possible, please provide recommendations of reviewers - those with experience with similar packages and/or likely users of your package - and their GitHub user names:

Heather Piwowar @hpiwowar

[sckott]

Thanks very much for your submission @aurielfournier - we're discussing now and will get back to you soon

[maelle]

Thanks for your submission @aurielfournier! I see the package doesn't have any test and doesn't have continuous integration yet. I suggest we put the submission on hold while you sort that, unless you can and want to add this within a week or so? There is some guidance in this guide, and I am happy to answer any question here or via Slack!

I was also looking at the dependencies, there are many of them in DESCRIPTION and

• at least one doesn't seem used at all in the package (stringi)
• it seems you have written the NAMESPACE by hand? We'd prefer you to use @importFrom (or even just pkg::fun when calling the function) and @export tags in R files. I wouldn't recommend importing whole packages unless needed. See http://r-pkgs.had.co.nz/namespace.html#imports

[maelle]

@aurielfournier for info I've just added the holding label, please update this thread once you have had time to work on the package, and ask me any question.

[aurielfournier]

Thanks @maelle I and my co-authors are working on it, but its taken a bit longer then we expected. Appreciate your patience!

[maelle]

No problem, and happy to help if I/we can!

[aurielfournier]

@maelle package now has tests and continuous integration.

I removed stringi from the DESCRIPTION file.

I have fixed the issues from the NAMESPACE file.

Huge thanks to my coauthor @birderboone for doing the heavy lifting to get this over the finish line!

I think we are ready for review now. If you have any other things that need to be addressed let me know.

Thanks!

[maelle]

👋 @aurielfournier @birderboone! Awesome, thanks to both of you! A few comments before I do the last editor checks:

• Could you please add the Travis badge at the top of the README?
• The Travis build is failing. To fix this (both WARNING and NOTEs)
  • "Undocumented arguments in documentation object 'references_read'
    ‘include_all’" Update the docs
  • prefix utils functions e.g. utils::flush.console()
  • For some variables you might need to create such a file: https://github.com/ropensci/opencage/blob/master/R/globalVariables.R to define them as global variables (the ones that appear in NOTEs). Unless they come up in dplyr function in which case you can write e.g. mutate(df, y = .data$a + .data$x) to make the NOTE disappear see this vignette
  • Reg JOSS in rtimicropem I put the paper in a paper/ folder https://github.com/ropensci/rtimicropem/tree/master/paper and buildignored it usethis::use_build_ignore("paper/") or something like that.
• Once the Travis build passes please add a code coverage report. Run usethis::use_coverage("codecov") which will give you stuff to add to the Travis config file, browse codecov maybe (I can't remember) and give you the code to paste in the README to get a badge.
• I see two issues about postal codes (leading zeros, Brazil code), do they need to be fixed before review?
• Regarding the other issues I admire the naming scheme. Just in case you ignored that, GitHub also has a milestone feature: https://help.github.com/articles/about-milestones/
• Please add a code of conduct and contributing guide (this can be extremely short) https://ropensci.github.io/dev_guide/collaboration.html#friendlyfiles No need to add the PR and issue templates yet.

[aurielfournier]

Hi @maelle

moved the badge

Addressed the Travis warnings/notes

I closed the two open issues. thanks for pointing out the milestones, I had forgotten about that.

I added the CoC and contribution guides.

Thank you so much for all the links and tips on how to address these issues, it is greatly appreciated.

Build is now passing!!

[maelle]

Yay, green badge! Can you also add a coverage badge? Run usethis::use_coverage("codecov") which will give you stuff to add to the Travis config file, browse codecov maybe (I can't remember) and give you the code to paste in the README to get a badge.

[aurielfournier]

Done! Sorry I missed that.

[maelle]

Thank you! A few more things before I search for reviewers (then they and you have less work 😉).

• Fit: The package meets criteria for fit and overlap
• Automated tests: Package has a testing suite and is tested via Travis-CI or another CI service.
• License: The package has a CRAN or OSI accepted license
• Repository: The repository link resolves correctly
• Archive (JOSS only, may be post-review): The repository DOI resolves correctly
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

---

Editor comments

• goodpractice output

✖ write short and simple functions. These
    functions have high cyclomatic complexity:authors_clean
    (68).

Maybe you can split it in several helper functions?

  ✖ omit "Date" in DESCRIPTION. It is not required
    and it gets invalid quite often. A build date will be
    added to the package when you perform `R CMD build` on it.

  ✖ add a "URL" field to DESCRIPTION. It helps users
    find information about your package online. If your
    package does not have a homepage, add an URL to GitHub, or
    the CRAN package package page.
  ✖ add a "BugReports" field to DESCRIPTION, and
    point it to a bug tracker. Many online code hosting
    services provide bug trackers for free,
    https://github.com, https://gitlab.com, etc.

Run usethis::use_github_links().

  ✖ avoid long code lines, it is bad for
    readability. Also, many people prefer editor windows that
    are about 80 characters wide. Try make your lines shorter
    than 80 characters

    R\authors_clean.R:24:1
    R\authors_clean.R:26:1
    R\authors_clean.R:29:1
    R\authors_clean.R:35:1
    R\authors_clean.R:36:1
    ... and 162 more lines

It's the complicated function, one more reason to try and simplify it?

  ✖ omit trailing semicolons from code lines. They
    are not needed and most R coding standards forbid them

    R\authors_refine.R:20:198

  ✖ avoid sapply(), it is not type safe. It might
    return a vector, or a list, depending on the input data.
    Consider using vapply() instead.

    R\authors_clean.R:97:17
    R\authors_clean.R:132:19
    R\authors_clean.R:152:19
    R\authors_clean.R:186:17
    R\authors_clean.R:223:22
    ... and 14 more lines

  ✖ avoid 1:length(...), 1:nrow(...), 1:ncol(...),
    1:NROW(...) and 1:NCOL(...) expressions. They are error
    prone and result 1:0 if the expression on the right hand
    side is zero. Use seq_len() or seq_along() instead.

    R\authors_clean.R:56:15
    R\authors_clean.R:71:79
    R\authors_clean.R:188:15
    R\authors_clean.R:209:21
    R\authors_clean.R:422:12
    ... and 9 more lines

  ✖ avoid 'T' and 'F', as they are just variables
    which are set to the logicals 'TRUE' and 'FALSE' by
    default, but are not reserved words and hence can be
    overwritten by the user.  Hence, one should always use
    'TRUE' and 'FALSE' for the logicals.

    R/authors_clean.R:NA:NA
    R/authors_clean.R:NA:NA
    R/authors_clean.R:NA:NA
    R/authors_clean.R:NA:NA
    R/authors_clean.R:NA:NA
    ... and 38 more lines

• Can you please re-trigger a Travis build so that the coverage badge indicate a coverage? We're aiming at a minimal coverage of 75% see https://github.com/ropensci/dev_guide/pull/94/files (brand-new official guidance)

• I'd recommend putting all badges on a single line.

• Could you also use Appveyor CI for Windows? usethis::use_appveyor(). This will add another badge.

• You can add the in-review badge

[![](https://badges.ropensci.org/256_status.svg)](https://github.com/ropensci/onboarding/issues/256)

It'll turn green when your package is approved.

• Could you please add examples in the documentation of the functions?

• Running devtools::spell_check() shows a few typos among the false negatives: querrying, nunmbers etc.

---

Reviewers: @njahn82 @bmkramer
Due date: 2018-12-12

[aurielfournier]

Hi @maelle

We are going to pause, and redo authors_clean() to be simpler/broken down into several functions. This will probably take ~ 1 week.

Thanks!

[maelle]

Ok, thank you!

[aurielfournier]

Alright! After some fighting with Travis the past 24 hours, we are good to go.

@birderboone split up authors_clean into three smaller internal functions, that should make review of the code easier. We've also addressed the other comments from @maelle

If I missed something, let me know.

Thanks!

[maelle]

Thanks @aurielfournier and @birderboone!

A few more things from goodpractice to tackle before I look for reviewers

It is good practice to

  
  ✖ add a "BugReports"
    field to DESCRIPTION, and point
    it to a bug tracker. Many
    online code hosting services
    provide bug trackers for free,
    https://github.com,
    https://gitlab.com, etc.

Simply run usethis::use_github_links()

  ✖ use '<-' for
    assignment instead of '='. '<-'
    is the standard, and R users
    and developers are used it and
    it is easier to read your code
    for them if you use '<-'.

    tests\testthat\test_authors_match.R:4:5

The styler package might help.

  ✖ avoid long code
    lines, it is bad for
    readability. Also, many people
    prefer editor windows that are
    about 80 characters wide. Try
    make your lines shorter than 80
    characters

    R\authors_address.R:12:1
    R\authors_address.R:14:1
    R\authors_address.R:17:1
    R\authors_address.R:20:1
    R\authors_address.R:41:1
    ... and 174 more lines

  ✖ avoid sapply(), it is
    not type safe. It might return
    a vector, or a list, depending
    on the input data. Consider
    using vapply() instead.

    R\plot_net_address.R:32:26
    R\plot_net_address.R:33:26
    tests\testthat\test_references_read.R:10:17

  ✖ avoid 1:length(...),
    1:nrow(...), 1:ncol(...),
    1:NROW(...) and 1:NCOL(...)
    expressions. They are error
    prone and result 1:0 if the
    expression on the right hand
    side is zero. Use seq_len() or
    seq_along() instead.

    R\authors_georef.R:55:25
    R\authors_georef.R:71:15
    R\authors_georef.R:113:17
    R\plot_net_address.R:34:35
    R\plot_net_address.R:123:22
    ... and 1 more lines

  ✖ fix this R CMD check
    NOTE: Namespaces in Imports
    field not imported from:
    'Rdpack' 'maps' 'stringr' All
    declared Imports should be
    used.
  
  ✖ avoid 'T' and 'F', as
    they are just variables which
    are set to the logicals 'TRUE'
    and 'FALSE' by default, but are
    not reserved words and hence
    can be overwritten by the user.
    Hence, one should always use
    'TRUE' and 'FALSE' for the
    logicals.

    R/authors_address.R:NA:NA
    R/authors_address.R:NA:NA
    R/authors_address.R:NA:NA
    R/authors_georef.R:NA:NA
    R/authors_georef.R:NA:NA
    ... and 15 more lines

And from me: could you please add a coverage badge? usethis::use_coverage() should help you with that.

Thanks in advance and thanks for all your work until now! 😸

[aurielfournier]

Hi @maelle

Thanks as always for your kind patience. It is greatly appreciated.

I've addressed all of the above, and I finally downloaded goodpractice for myself to check things.

The one issue that I don't totally understand, but isn't throwing an issues in goodpractice is this one

fix this R CMD check NOTE: Namespaces in Imports field not imported from: 'Rdpack' 'maps' 'stringr' All declared Imports should be used.

I thought that meant that I needed to remove Rdpack, maps and stringr from the DESCRIPTION file. So I did, but then the build failed, and it did not pass unless I put Rdpack back in.

But otherwise I think we're ok. :D

[maelle]

Thanks a lot @njahn82 @bmkramer for accepting to review this package! 😺 Your reviews are due on 2018-12-12.

As a reminder, our reviewer guide can be found here and the review template here.

[embruna]

Hi @maelle, I think we got all of these taken care of.

• A few of the lines of code are at 81 characters, but I couldn't reduce them further because of the required formatting. I hope that's ok.

• The description on the repo has been changed to eliminate the "update to refnet text"

• Coverage 91% ! :)

[maelle]

Hi @embruna, thanks! Np reg the remaining long lines.

Reg DESCRIPTION could you have a look at dependencies ‘Rdpack’ ‘gdtools’ ‘mapproj’ ‘vdiffr’ as noted in my previous comment? Dependencies only used in tests should be under Suggests. by the way the line for Imports is very long, I'd suggest running desc::desc_normalize() to have one line per dependency (like what you have for Suggests), which would be easier to browse (that function also orders dependencies alphabetically).

Reg Travis what I meant is that I recommend undoing this commit ropensci/refsplitr@92e9fe5 since your package now is in very good shape, it'd be good for Travis to flag new WARNINGs as they appear.

After that we'll be done. Thanks for your patience and thanks again for your work!

[embruna]

OK, drum roll...

1. I moved the packages to the Suggests, and ran desc::desc_normalize() on the DESCRIPTION. The packages are also listed in alphabetical order.

2. travis.yml now has warnings_are_errors: true

We're the ones who are grateful, both for your patience and the attention to detail. The review of this package has been incredibly helpful and detailed....I wish my papers were reviewed this thoroughly!

[maelle]

Thank you! 😁 And thanks again to both reviewers!

I'm sorry but I now realize these packages are maybe not even used at all in your package, or am I missing something? I've searched for their name in the package code (e.g. https://github.com/embruna/refsplitr/search?q=mapproj&unscoped_q=mapproj&type=Code)

• mapproj
• Rdpack I suppose you used to use it for package docs, but now you no longer use it?
• vdiffr I see https://github.com/embruna/refsplitr/blob/3ba804f62014aee01c370e5e14209f8c2817cf73/tests/figs/deps.txt but right now there doesn't seem to be any test using vdiffr, the tests in https://github.com/embruna/refsplitr/blob/master/tests/testthat/test_plots.R don't use it.

If the packages are really not used, could you please remove them from Suggests?

[maelle]

You don't need covr either since it's run on Travis. I've now found a way to find unneeded dependencies using the attachment package from inside the refsplitr folder.

desc_deps <- attachment::att_from_description()
scripts_deps <- attachment::att_from_rscripts(".")
namespace_deps <- attachment::att_from_namespace()
vignette_deps <- attachment::att_from_rmds()
desc_deps[!desc_deps %in% c(scripts_deps, namespace_deps, vignette_deps)]

[maelle]

Note that you could still have false positives since you could use Suggests packages for other things.

[maelle]

Happy New Year!

@embruna Once you've responded to my comment about Suggests unneeded dependencies the package will be approved. 🙂

[embruna]

Hi all, happy New Year.

Deleted the 4 packages from the Suggests....thanks @maelle!

[embruna]

Well, it was too good to be true...just got this email:

I'm going to try adding the four packages to suggests one at a time and seeing which of them actually do need to be there. I'll update shortly.

[maelle]

Ah indeed sorry mapproj seems needed https://travis-ci.org/embruna/refsplitr/jobs/633881379#L6325

[embruna]

6324
Error: processing vignette 'refsplitr-vignette.Rmd' failed with diagnostics:
6325
there is no package called 'mapproj' ```

[embruna]

Yes, I just reinserted to the Suggests. Fingers crossed!

[maelle]

Cool! I'll come back to this thread tomorrow, it's getting late here.

[maelle]

Approved! Thanks @embruna @aurielfournier @birderboone for submitting and @njahn82 @bmkramer for your reviews! 😁

To-dos:

• Transfer the repo to rOpenSci's "ropensci" GitHub organization under "Settings" in your repo. I have invited you to a team that should allow you to do so. You'll be made admin once you do.
• Add the rOpenSci footer to the bottom of your README
  " [![ropensci_footer](https://ropensci.org/public_images/ropensci_footer.png)](https://ropensci.org)"
• Fix all links to the GitHub repo to point to the repo under the ropensci organization.
• If you already had a pkgdown website, fix its URL to point to https://docs.ropensci.org/package_name and deactivate the automatic deployment you might have set up, since it will not be built centrally like for all rOpenSci packages, see https://devguide.ropensci.org/ci.html#even-more-ci-ropensci-docs. In addition, in your DESCRIPTION file, include the docs link in the URL field alongside the link to the GitHub repository, e.g.: URL: https://docs.ropensci.org/foobar (website) https://github.com/ropensci/foobar
• Add a mention of the review in DESCRIPTION via rodev::add_ro_desc().
• Fix any links in badges for CI and coverage to point to the ropensci URL. We no longer transfer Appveyor projects to ropensci Appveyor account so after transfer of your repo to rOpenSci's "ropensci" GitHub organization the badge should be [![AppVeyor Build Status](https://ci.appveyor.com/api/projects/status/github/ropensci/pkgname?branch=master&svg=true)](https://ci.appveyor.com/project/individualaccount/pkgname).
• We're starting to roll out software metadata files to all ropensci packages via the Codemeta initiative, see https://github.com/ropensci/codemetar/#codemetar for how to include it in your package, after installing the package - should be easy as running codemetar::write_codemeta() in the root of your package.
• Activate Zenodo watching the repo
• Tag and create a release so as to create a Zenodo version and DOI
• Submit to JOSS at https://joss.theoj.org/papers/new, using the rOpenSci GitHub repo URL. When a JOSS "PRE REVIEW" issue is generated for your paper, add the comment: This package has been reviewed by rOpenSci: https://LINK.TO/THE/REVIEW/ISSUE, @maelle

Should you want to acknowledge your reviewers in your package DESCRIPTION, you can do so by making them "rev"-type contributors in the Authors@R field (with their consent). More info on this here.

Welcome aboard! We'd love to host a blog post about your package - either a short introduction to it with one example or a longer post with some narrative about its development or something you learned, and an example of its use. If you are interested, review the instructions, and tag @stefaniebutland in your reply. She will get in touch about timing and can answer any questions.

We've put together an online book with our best practice and tips, this chapter starts the 3d section that's about guidance for after onboarding. Please tell us what could be improved, the corresponding repo is here.

[embruna]

Hi All, a quick update:

The following have been COMPLETED

• Transfer the repo to rOpenSci's "ropensci" GitHub

• Add the rOpenSci footer to the bottom of your README

• Fix all links to the GitHub repo to point to the repo under the ropensci organization.

• Fix any links in badges for CI and coverage to point to the ropensci URL.

I'm finishing the following as soon as I am done with class:

• Add a mention of the review in DESCRIPTION via rodev::add_ro_desc().
• Including Metadata
• I'm awaiting approval from one last reviewer

At that point I will create the Zeonodo 1.0 version and mint the DOI, and submit to JOSS. Should be done shortly.

[embruna]

Hi @noamross and @maelle, a couple quick questions:

1. I've written some code that helps users more quickly narrow down the names to review after doing the disambiguation, but we decided not to include it in the package. I have it on my own github (link here) and point to it in the vignette. Is that ok, or should this also be transferred over to the ROpenSci repo?

2. It's currently versioned as v0.6. Should it be posted as v1.0?

3. I'm just waiting to hear back from a reviewer. I'll wait until Monday noon, at which time I'll do the Zenodo stuff and submit to JOSS.

[maelle]

1. Maybe it could become a tech note (https://ropensci.org/technotes), or a vignette in refsplitr?

2. That's up to you. 🙂

3. Ok!

[embruna]

OK, @noamross and @maelle: refsplitr should be ready for Zendo v1.0.0

The only unexpected thing that came up was that our title (or rather, the text after the colon in refsplitr: --- needed to be fixed. It was inconsistent within the repo (different in readme and description) and far longer than CRAN-style. So it is now as follows:

refsplitr: author name disambiguation, author georeferencing, and mapping of coauthorship networks with Web of Science data.

I was a actually a bit concerned because Clarivate's webpage is clear:

Do not incorporate any Clarivate Analytics product names into your company's product names. link

However, a quick search on CRAN reveals several other packages wth "Web of Science" in the title, so I just left it in. If you prefer it be removed let me know. Otherwise, I will freeze v1.0.0 with Zenodo at the end of the day.

[maelle]

I think the package isn't a company product? But I'm not a lawyer.

In any case I think you'll need to write it 'Web of Science' in DESCRIPTION.

[embruna]

Submitted to JOSS, and Zenodo archived as v0.9.0 with doi 10.5281/zenodo.3608285. Once the JOSS review is complete and I can include the details in the paper.rm file. At that point I'll freeze v1.0. A minor thing, and maybe unnecessary. But I wanted 1.0 to be a complete package with no details to be filled in.

[maelle]

Ok! I don't see the submission at https://joss.theoj.org/papers yet.

[embruna]

Yes, still listed as "pending"...I'll check on it periodically throughout the day.

[maelle]

I've just checked again. Please ping me if you see it appear. 😉

[maelle]

Paper accepted at JOSS 🎉 so closing this issue.