`fastMatMR`: Fast Matrix Market I/O

[HaoZeke]

Date accepted: 2023-10-30
Submitting Author Name: Rohit Goswami
Submitting Author Github Handle: @HaoZeke
Repository: https://github.com/HaoZeke/fastMatMR
Version submitted:
Submission type: Standard
Editor: @maelle
Reviewers: @osorensen, @czeildi

Archive: TBD
Version accepted: TBD
Language: en

---

• Paste the full DESCRIPTION file inside a code block below:

Package: fastMatMR
Title: "fastMatMR: High-Performance Matrix Market File Operations in R"
Version: 1.0.0.0
Authors@R:
    person("Rohit", "Goswami", , "rgoswami@ieee.org", role = c("aut", "cre"),
           comment = c(ORCID = "0000-0002-2393-8056"))
Description: "fastMatMR is an R package offering high-performance read and write operations for Matrix Market files. It acts as a thin wrapper around the 'fast_matrix_market' C++ library, offering speed and extended support for Matrix Market formats. Unlike other R packages, fastMatMR supports not just sparse matrices but also dense vectors and matrices. This makes it a versatile choice for dealing with .mtx files in R."
License: MIT + file LICENSE
SystemRequirements: C++17
Encoding: UTF-8
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.2.3
LinkingTo:
    cpp11
Suggests:
    ggplot2,
    knitr,
    Matrix,
    microbenchmark,
    rmarkdown,
    testthat (>= 3.0.0)
URL: https://github.com/HaoZeke/fastMatMR
BugReports: https://github.com/HaoZeke/fastMatMR/issues
Config/testthat/edition: 3
VignetteBuilder: knitr

Scope

• Please indicate which category or categories from our package fit policies this package falls under: (Please check an appropriate box below. If you are unsure, we suggest you make a pre-submission inquiry.):

  • data retrieval
  • data extraction
  • data munging
  • data deposition
  • data validation and testing
  • workflow automation
  • version control
  • citation management and bibliometrics
  • scientific software wrappers
  • field and lab reproducibility tools
  • database software bindings
  • geospatial data
  • text analysis

• Explain how and why the package falls under these categories (briefly, 1-2 sentences):

The matrix market exchange formats are crucial to much of the ecosystem. The fastMatMR package focuses on high-performance read and write operations for Matrix Market files, serving as a key tool for data extraction in computational and data science pipelines.

• Who is the target audience and what are scientific applications of this package?

Data scientists who might want to load and test the NIST matrices which include:

comparative studies of algorithms for numerical linear algebra, featuring nearly 500 sparse matrices from a variety of applications, as well as matrix generation tools and services.

Additionally, this makes its simpler to interface to scipy and the rest of the data science ecosystem. This also includes working with the Tensor Algebra Compiler (TACO).

• Are there other R packages that accomplish the same thing? If so, how does yours differ or meet our criteria for best-in-category?

The Matrix package in R can perform similar operations but only for sparse matrices. The fastMatMR package not only provides enhanced performance but also extends support to dense matrices and vectors in base R, thus offering a more versatile solution.

We have both read and write performance vignettes backing up the claims made.

• (If applicable) Does your package comply with our guidance around Ethics, Data Privacy and Human Subjects Research?

• If you made a pre-submission inquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

• Explain reasons for any pkgcheck items which your package is unable to pass.

The package passes pkcheck: ropensci/fastMatMR#18, though the review bot disagrees :)

Technical checks

Confirm each of the following by checking the box.

• I have read the rOpenSci packaging guide.
• I have read the author guide and I expect to maintain this package for at least 2 years or to find a replacement.

This package:

• does not violate the Terms of Service of any service it interacts with.
• has a CRAN and OSI accepted license.
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions, created with roxygen2.
• contains a vignette with examples of its essential functions and uses.
• has a test suite.
• has continuous integration, including reporting of test coverage.

Publication options

• Do you intend for this package to go on CRAN?

• Do you intend for this package to go on Bioconductor?

• Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:

MEE Options

• The package is novel and will be of interest to the broad readership of the journal.
• The manuscript describing the package is no longer than 3000 words.
• You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see MEE's Policy on Publishing Code)
• (Scope: Do consider MEE's Aims and Scope for your manuscript. We make no guarantee that your manuscript will be within MEE scope.)
• (Although not required, we strongly recommend having a full manuscript prepared when you submit here.)
• (Please do not submit your package separately to Methods in Ecology and Evolution)

Code of conduct

• I agree to abide by rOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

[ropensci-review-bot]

Thanks for submitting to rOpenSci, our editors and @ropensci-review-bot will reply soon. Type @ropensci-review-bot help for help.

[ropensci-review-bot]

🚀

Editor check started

👋

[HaoZeke]

Also, not part of the issue template, but the package passes pkcheck: ropensci/fastMatMR#18

[ropensci-review-bot]

Checks for fastMatMR (v1.0.0.0)

git hash: a910f6be

• ✔️ Package name is available
• ✔️ has a 'codemeta.json' file.
• ✔️ has a 'contributing' file.
• ✔️ uses 'roxygen2'.
• ✔️ 'DESCRIPTION' has a URL field.
• ✔️ 'DESCRIPTION' has a BugReports field.
• ✔️ Package has at least one HTML vignette
• ✔️ All functions have examples.
• ✔️ Package has continuous integration checks.
• ✖️ Package coverage is 68.1% (should be at least 75%).
• ✔️ R CMD check found no errors.
• ✔️ R CMD check found no warnings.

Important: All failing checks above must be addressed prior to proceeding

Package License: MIT + file LICENSE

---

1. Package Dependencies

Details of Package Dependency Usage (click to open)

The table below tallies all function calls to all packages ('ncalls'), both internal (r-base + recommended, along with the package itself), and external (imported and suggested packages). 'NA' values indicate packages to which no identified calls to R functions could be found. Note that these results are generated by an automated code-tagging system which may not be entirely accurate.

type	package	ncalls
internal	fastMatMR	6
imports	NA	NA
suggests	ggplot2	NA
suggests	knitr	NA
suggests	Matrix	NA
suggests	microbenchmark	NA
suggests	rmarkdown	NA
suggests	testthat	NA
linking_to	cpp11	NA

Click below for tallies of functions used in each package. Locations of each call within this package may be generated locally by running 's <- pkgstats::pkgstats(<path/to/repo>)', and examining the 'external_calls' table.

fastMatMR

fmm_to_mat (1), fmm_to_vec (1), mat_to_fmm (1), release_questions (1), sparse_to_fmm (1), vec_to_fmm (1)

NOTE: No imported packages appear to have associated function calls; please ensure with author that these 'Imports' are listed appropriately.

---

2. Statistical Properties

This package features some noteworthy statistical properties which may need to be clarified by a handling editor prior to progressing.

Details of statistical properties (click to open)

The package has:

• code in C++ (7% in 3 files), C/C++ Header (91% in 15 files) and R (2% in 3 files)
• 1 authors
• 3 vignettes
• no internal data file
• 1 imported package
• 7 exported functions (median 3 lines of code)
• 8 non-exported functions in R (median 3 lines of code)
• 217 C/C++ functions (median 5 lines of code)

---

Statistical properties of package structure as distributional percentiles in relation to all current CRAN packages
The following terminology is used:

• loc = "Lines of Code"
• fn = "function"
• exp/not_exp = exported / not exported

All parameters are explained as tooltips in the locally-rendered HTML version of this report generated by the checks_to_markdown() function

The final measure (fn_call_network_size) is the total number of calls between functions (in R), or more abstract relationships between code objects in other languages. Values are flagged as "noteworthy" when they lie in the upper or lower 5th percentile.

measure	value	percentile	noteworthy
files_R	3	21.5
files_src	3	84.3
files_inst	15	99.6
files_vignettes	3	92.4
files_tests	3	75.2
loc_R	45	4.3	TRUE
loc_src	202	27.6
loc_inst	2545	81.7
loc_vignettes	276	60.9
loc_tests	68	31.5
num_vignettes	3	94.2
n_fns_r	15	21.2
n_fns_r_exported	7	34.0
n_fns_r_not_exported	8	18.0
n_fns_src	217	89.5
n_fns_per_file_r	3	46.2
n_fns_per_file_src	8	68.9
num_params_per_fn	2	11.9
loc_per_fn_r	3	1.1	TRUE
loc_per_fn_r_exp	3	1.5	TRUE
loc_per_fn_r_not_exp	3	1.5	TRUE
loc_per_fn_src	5	5.0	TRUE
rel_whitespace_R	29	11.6
rel_whitespace_src	20	31.5
rel_whitespace_inst	22	82.8
rel_whitespace_vignettes	41	68.6
rel_whitespace_tests	24	30.6
doclines_per_fn_exp	18	10.4
doclines_per_fn_not_exp	0	0.0	TRUE
fn_call_network_size	480	95.1	TRUE

---

2a. Network visualisation

Click to see the interactive network visualisation of calls between objects in package

---

3. goodpractice and other checks

Details of goodpractice checks (click to open)

3a. Continuous Integration Badges

GitHub Workflow Results

id	name	conclusion	sha	run_number	date
6056018810	pages build and deployment	success	c3523b	8	2023-09-02
6056006940	pkgcheck	success	fe999d	21	2023-09-02
6056006937	pkgdown	success	fe999d	23	2023-09-02
6056006936	pre-commit	success	fe999d	49	2023-09-02
6056006938	R-CMD-check	success	fe999d	39	2023-09-02

---

3b. goodpractice results

R CMD check with rcmdcheck

R CMD check generated the following note:

1. checking installed package size ... NOTE
   installed size is 11.1Mb
   sub-directories of 1Mb or more:
   libs 10.2Mb

R CMD check generated the following check_fail:

1. rcmdcheck_reasonable_installed_size

Test coverage with covr

Package coverage: 68.1

The following files are not completely covered by tests:

file	coverage
inst/include/fast_matrix_market/chunking.hpp	63.83%
inst/include/fast_matrix_market/fast_matrix_market.hpp	18.92%
inst/include/fast_matrix_market/header.hpp	59.69%
inst/include/fast_matrix_market/read_body_threads.hpp	67.44%
inst/include/fast_matrix_market/read_body.hpp	42.29%
inst/include/fast_matrix_market/write_body.hpp	50%
R/misc.R	0%

Cyclocomplexity with cyclocomp

No functions have cyclocomplexity >= 15

Static code analyses with lintr

lintr found the following 1 potential issues:

message	number of times
Avoid library() and require() calls in packages	1

---

Package Versions

package	version
pkgstats	0.1.3.8
pkgcheck	0.1.2.2

---

Editor-in-Chief Instructions:

Processing may not proceed until the items marked with ✖️ have been resolved.

[HaoZeke]

FWIW there's .covignore which is not being picked up by the bot here (works on the repo, ropensci/fastMatMR#18). Package coverage is:

> covr::package_coverage()
fastMatMR Coverage: 94.69%
src/from_file.cpp: 93.88%
src/to_file.cpp: 94.23%
R/fastMatMR-package.R: 100.00%

[mpadge]

@HaoZeke is indeed right there - the failing coverage check is a bug on our side, and not this submission.

[noamross]

@ropensci-review-bot assign @maelle as editor

[ropensci-review-bot]

Assigned! @maelle is now the editor

[maelle]

👋 @HaoZeke! Nice to see you as an author this time around, thank you for your submission! 😀

A few comments/questions before we proceed to the reviewer search:

• Why add the bundled files to .covrignore, if they contribute to the functionality of the package?
• I see you use conventional commits, nice. How is the changelog generated? I see DOC: Fix news fastMatMR#17 Disclaimer: I contribute to https://github.com/cynkra/fledge/
• In the example where you use -> I'd suggest using <- instead since the former is more rare.
• In the README you write "writing and reading from other R objects". Could you please elaborate, adding a few sentences about this to the README?
• To me the second part of the README, "Development", belongs to the contributing guide, so you could move the content and instead point contributors to the contributing guide.
• In the contributing guide could you please add a link to pixi? I for instance do not know what it is.
• In the issue tracker there are a few enhancement issues. What's the timeline for them, are they "nice to have" or do you intend to tackle them soon? If so should the review wait?

Thank you!

[HaoZeke]

👋 @HaoZeke! Nice to see you as an author this time around, thank you for your submission! 😀

^_^

A few comments/questions before we proceed to the reviewer search:

* Why add the bundled files to .covrignore, if they contribute to the functionality of the package?

This is because there is no need to expose bindings of the inner workings of the fast_matrix_market C++ library to R users. This approach is also taken by other libraries which bundle C++ libraries, like RCppEigen.

The user-facing C++ functions are in src/ and covered by tests.

* I see you use conventional commits, nice. How is the changelog generated? I see [DOC: Fix news HaoZeke/fastMatMR#17](https://github.com/HaoZeke/fastMatMR/issues/17) Disclaimer: I contribute to https://github.com/cynkra/fledge/

That's a very cool library, I am currently using towncrier and tbump which are fairly generic tools.

* In the example where you use `->` I'd suggest using `<-` instead since the former is more rare.

* In the README you write "writing and reading from other R objects". Could you please elaborate, adding a few sentences about this to the README?

* To me the second part of the README, "Development", belongs to the contributing guide, so you could move the content and instead point contributors to the contributing guide.

* In the contributing guide could you please add a link to pixi? I for instance do not know what it is.

Yup these are great suggestions, updated the package with them.

* In the issue tracker there are a few enhancement issues. What's the timeline for them, are they "nice to have" or do you intend to tackle them soon? If so should the review wait?

These are more version 2 issues and shouldn't block the review I think. It would take a while for me to get to them, the current functionality covers R matrices, vectors and sparse matrices from Matrix which cover most of the use-cases. Other sparse matrix formats are on the issue list but they're very optional "nice to have" issues since users can always convert to Matrix formats / dense types before using the library.

Thank you!

You're welcome :)

[maelle]

This is because there is no need to expose bindings of the inner workings of the fast_matrix_market C++ library to R users. This approach is also taken by other libraries which bundle C++ libraries, like RCppEigen.

To clarify, the question was about tests specifically. Part of the code in inst/ is covered by tests anyway, so why not include them (and remove the unused code)?

We do not have strict requirements on testing coverage for bundled code yet, but we can improve our guidance based on cases like this submission, so this discussion is important.