Skip to content

contentvalidR 0.9.0

Choose a tag to compare

@JUhalt JUhalt released this 27 Sep 14:11
· 24 commits to master since this release
c0d0696

Ninth public release. v0.9.0 is about how the package presents itself before
1.0. Every reference list is in APA 7 form; printed output is shorter and says
what it means; the component functions and report tables print in APA style;
the figures show the criterion and the uncertainty; and the README is a front
page rather than a manual.

The package continues to declare Imports: stats only.

No computed value changes. Across 25 analyses spanning every workflow,
every value 0.8.0 returns is identical in 0.9.0. Some wording changes: the
scale-level summaries of the item-sort and construct-rating workflows name the
Colquitt band of the weaker index, counts read "1 item" or "3 items", and the
note when judge severity cannot be estimated no longer blames near-total
agreement when one dissenting judge causes it.

Breaking changes.

  • content_report() writes an APA table by default. For the numeric table
    that used to be the default, pass format = "data.frame"; its p values
    now keep three decimals whatever digits is.
  • The component functions' results (compute_psa(), aikens_v(), cvr(),
    and eleven others) gain a class, so they print as formatted tables. Every
    value is unchanged and as.data.frame() returns the plain data frame; only
    code that tests class(x) == "data.frame" exactly would notice.

Deprecated. agreement_summary() warns and will be removed in 1.0.0; use
panel_agreement(), which takes raters in rows like every other ratings
function.

New: content_report(format = "apa"), and csv_binom_test() returns its
inputs (n_target, N, p0, alpha) beside its result.

Published on GitHub and R-universe. 0.4.0 is still in CRAN's review queue, and
CRAN policy asks that no further version be submitted while one is pending.

A README that is a front page

  • The README is a front page rather than a manual. It ran to about 1,400
    lines of printed output: every workflow in full, the component functions,
    and six copies of the key. It now shows one first analysis (the printout,
    its APA table, and its plot), a table matching each question put to judges
    or experts to its workflow, the sources it is built on, and its guide, then
    how to read the output, what every workflow returns, and what you can rely
    on. The worked examples it held live in the vignettes.
  • vignette("design-and-reporting") gains the planning examples the README
    held: expert_power(), with why a fourth or fifth expert lowers the chance
    of clearing Lynn's criterion, and gtheory_content(), with how many judges a
    design would need. Its method list now names all six workflows, and it has a
    reference list.

Plots show the criterion and the uncertainty

No computed value changes; this is about what the figures draw.

  • The item-sort Psa plot draws each item's interval, and a dashed mark at
    the share of judges the exact test needs for that item
    , so an item is
    retained when its point reaches its mark. It used to show bare points.
  • The expert relevance plot shows Aiken's V and I-CVI side by side, each
    with its interval
    , and a dashed line at the I-CVI criterion when every
    item had the same number of experts ("I-CVI criterion (5 of 6)"). Before,
    only V had an interval, and an I-CVI point could hide behind V's.
  • Legends list only what the figure draws and sit in headroom above the
    data. The item plots offered a "No data" symbol no item used, the profile
    plot a "Review gap" line it had not drawn, and the essentiality key sat on
    its own zero line. The content map's key is titled "Cluster" or "Blueprint
    cell", where it showed bare numbers.
  • Axis labels say what is measured ("Psa: share of judges choosing the
    target") in place of "Psa correspondence", and axes for statistics that
    cannot exceed 1 are labeled as APA prints them (.25, .50). No figure leaves
    an empty band for a title unless main is given, and each restores the
    graphics settings it changes. Item rows run top to bottom in the order of
    the results table, and scale means are triangles, clearly distinct from
    retained items.
  • Every figure in the README and the vignettes has alt text describing
    what it shows; 17 of 19 had none.

Shorter keys, formatted components, and APA report tables

No computed value changes; this is about what is printed.

  • The key under each result explains only what it shows. It ran 12 to 36
    lines and was often longer than the results. Each column now gets one line
    after its full name, and "What the decisions mean" lists only the decision
    words that appear, saying what each means in that workflow ("Retain --
    met the exact target-assignment criterion"). The old legend listed all
    four shared statuses and left readers to match Retain or Typical to them.
    contentvalid_glossary() keeps the full definitions and now lists every
    workflow's decision words. The item-sort example in the README prints 58
    lines instead of 71.
  • The component functions print as formatted tables. compute_psa(),
    compute_csv(), htc(), htd(), anova_content(), aikens_v(), cvr(),
    ioc(), interpret_colquitt(), colquitt_benchmarks(), csv_binom_test(),
    signal_detection(), reproducibility_phi(), and similarity_from_sort()
    printed raw data frames and lists, such as a p of 1.941920e-16 beside
    seven-digit decimals. They now print as cvi() already did: a titled table
    in APA style with a short note. anova_content() reports its tests as
    F(1.71, 32.57) = 108.55. The values are unrounded; each result only gains a
    class, as.data.frame() returns the plain data frame, and workflow results
    are unaffected. csv_binom_test() also returns its inputs.
  • Breaking: content_report() writes APA tables by default. The new
    default format = "apa" gives readable headings, counts such as 18/20,
    p to three decimals, and [LL, UL] intervals, and prints without row
    names; format = "markdown" writes the same table and prints as the
    table. For the numeric table that used to be the default, pass
    format = "data.frame"; its p values now keep three decimals whatever
    digits is. The construct-rating table shows the contrast p beside the
    omnibus p, since a Review item can have an omnibus p below .001.

agreement_summary() is deprecated

  • Deprecated in 0.9.0, to be removed in 1.0.0. Use panel_agreement().
    agreement_summary() is the only function that takes items in rows; every
    other ratings function, panel_agreement() included, takes raters in rows,
    so one matrix passed to both was read two different ways.
    panel_agreement() also reports an interval and needs no other package.
    agreement_summary() still works in 0.9.0 and warns when called. Removing
    it at 1.0.0 leaves nothing deprecated past 1.0, as the stability policy in
    ?contentvalidR sets out.

References follow APA 7 everywhere

  • One reference list, installed in BibTeX form. REFERENCES.bib held 13
    of the works the package cites, and one it no longer cited; it now holds all
    53, generated from the reference list in the README, with author names and
    details checked against the Crossref record for each DOI. A test keeps the
    two in step. The README calls it a reference list rather than a
    bibliography.
  • Every reference list is in alphabetical order, as APA 7 requires, in the
    README, every help page, and every vignette. They had been ordered by topic.
    Page ranges use en dashes.
  • Corrections found by the check. De Boeck and Wilson (2004) are the
    editors of Explanatory item response models, not its authors. Book titles
    are in sentence case. Four works gain the DOI they lacked (Brennan, 2001;
    De Boeck & Wilson, 2004; Newcombe, 1998; Vach & Gerke, 2023). Colquitt et
    al. (2014) and the irr package, both cited in help pages, join the list in
    the README.
  • citation("contentvalidR") prints the package reference in APA form, with
    its version and "[Computer software]".

Printed output says what it means

  • The expert-panel print opens with its verdict, as the other workflows'
    prints do: how many items met the criterion, then the flagged items by name.
    0.8.0's release notes said every print did this; the expert-panel print
    showed only a tally. Items rated by fewer than three experts are named too,
    where the tally left them out.
  • The key no longer says modified kappa runs from 0 to 1. It falls below 0
    whenever fewer experts agree than chance predicts: with four experts, an
    item none of them rates relevant has a modified kappa of -.07.
  • An essentiality panel with no usable ratings no longer prints "an item needs
    at least NA rating it essential".
  • The scale-level summaries of the item-sort and construct-rating workflows
    name the band the weaker index falls in ("The weaker of Psa and Csv falls in
    the Moderate band of published scales"), where they spoke of "normative
    standing". The construct-rating print names its benchmark set the way the
    item-sort print does.
  • Every summary heading has the same form and rule; counts read "1 item" or
    "3 items" rather than "item(s)"; the handoff says held-back items are listed
    only when some are; and the note when judge severity cannot be estimated no
    longer attributes it to near-total agreement when one judge rejecting the
    items the others accept causes it.

No computed value changes.