-
Added experimental metadata-based dispersion analysis with
dispersionAnalysis()and interactive plots withhc_dispersion()to compare distributions across sources, countries and years, accounting for corpus-part sizes. -
Added interactive and static frequency bar charts with
hc_freq_bar_ci()andgeom_freq_bar_ci(), including confidence intervals. The ggplot2 time-series helper now also supports curved connections between annual observations. -
Running searches can now be cancelled without entering the error debugger. Retrieval also handles server rate limits and failed page downloads more reliably, with warnings for incomplete results and support for resuming them.
-
Internal: Tests now run offline using synthetic response fixtures and local test servers. Live-server checks remain opt-in.
-
follows the DeReKo-KorAP-2026-II release of the German Reference Corpus, the default corpus of the default KorAP instance, which replaced the
textClassmetadata field bydmozDomainandwikiDomain.corpusQuery()now fetches both by default, so that the topic domain of matches is available again (#31). -
long running searches are no longer aborted after 10 minutes. The underlying curl library gave up on transfers delivering less than one byte per second for 600 seconds, which is just what a search does while the server is still counting, regardless of the
timeoutof the connection. Only that timeout applies now -
failed and incomplete searches are reported:
corpusQuery()andfrequencyQuery()now warn when a request failed or when the KorAP server cut a search short (timeExceeded), instead of silently reporting 0 or too few hits. Their data frame results carry the newqueryDurationcolumn, the time in seconds a request took on the client side -
clearer verbose output: a series of queries or virtual corpora is logged as an aligned table, one row per item with its hits or tokens, the time it took and whether it came from the cache, followed by a one-line summary. A virtual corpus definition shared by all rows is named once in the header, e.g.
textType = /Zeit.*/ & pubDate in …, and the rows show only what varies. The estimated remaining time no longer counts the query just finished as pending, which made it about twice too long in the beginning, and ignores cached queries. Numbers beyond 2^31, e.g. the token count of the whole of DeReKo, are no longer printed asNA, and no colour codes are written where the output does not go to a terminal or the RStudio console, e.g. to a log file
-
cacheAsnow consistently covers all query functions –frequencyQuery(),corpusStats(),collocationScoreQuery()andtextMetadata(). The cache file records its relevant production context: parameters, KorAP instance, index revision and package version.cacheAsInfo()prints that record,blessCacheAs()vouches for a file that is sound regardless, andwithCachedResults()takes the files as they are for one expression,mode = "offline"refusing to compute anything that is not in one already -
the names of a named
vcvector are now consistently used as labels byfrequencyQuery(),corpusStats()andcollocationScoreQuery(), ascollocationAnalysis()already did. Where a vector carries no names, nothing changes: nolabelcolumn appears that was not there before -
collocationAnalysis()now discards collocates that occur less often than expected by chance.logDice, by which it ranks and thresholds, expresses how salient a pair is rather than how surprising, so a frequent word could appear among the top collocates although the node does not attract it at all: for Grund in a 5+5 window, Berlin reaches a logDice of 3.84, close to triftiger at 3.96, while co-occurring 1.74 bits less often than expected. The newminObservedExpectedRatioparameter defaults to 1 and keeps such pairs out. Raise it to demand a stronger contrast, e.g. 2 for collocates occurring at least twice as often as expected, or set it to 0 for the unfiltered result of earlier versions, e.g. to study repulsion.collocationScoreQuery()is unaffected, as there the pairs to score are given explicitly -
changed
logDicevalues:logDice()is now computed as defined by Rychlý (2008),14 + log2(2 * O / (O1 + O2)), so that its values are comparable to those of Sketch Engine and other tools. It previously multiplied the node frequency by the window size,14 + log2(2 * O / (w * O1 + O2)), which added a count of window positions to a count of word tokens and made the coefficient asymmetric, so that swapping node and collocate changed the score. Becausew * O1dominated the denominator for a frequent node, rare collocates were penalised: for triftiger as a collocate of Grund in a 5+5 window, logDice was 0.64, below Berlin at 2.03, although Berlin co-occurs with Grund less often than chance predicts. The values are now 3.96 and 3.84. Scores rise by up tolog2(w), that is by up to 3.32 for the default context of 5 left and 5 right, socollocationAnalysis()with the defaultthresholdScore = "logDice"andthreshold = 2is now somewhat more permissive when recursing. Results computed with a total window size of 1, as in the light verb construction example of the Readme, are unaffected. The other association scores are unchanged: they take the window size into account through the expected frequencyE, which is correct -
changed
ll()values: the contingency table ofll()scaled only its row total by the window size, an inconsistency spotted by Tim Feldmüller, leaving cells that do not add up to one sample. Following Evert (2004), the sample consists of co-occurrence tokens, so the sample size and both marginals scale with the window: an occurrence of either word takes part inwindow_sizepairs. The expected co-occurrence frequency is unchanged atwindow_size * O1 * O2 / N, which is whypmi,mi2andmi3are unaffected, but log-likelihood values differ, by 0.1% to well over 100% depending on the frequencies and the window. This also removes the case whereN - window_size * O1turned negative and the score becameNaN, sincewindow_size * (N - O1)cannot: the warning added earlier in this development version is therefore gone again, having treated a symptom of this -
improved coverage of the doc-prompting tests, which now range from a frequency query over time to comparing collocates across virtual corpora, keeping a result in a
cacheAsfile and labelling corpora by name. They prompt current LLMs with the Readme and check the code written from it, so that a gap in the documentation shows up as a failing test. This guards the quality of the Readme and improves vibe coding results. The approach is briefly described in Kupietz et al. (2026) -
fixed collocation analysis dropping snippets whose markup did not have one particular shape, which cost about 15% of the hits of a
contains(<base/s=s>, ...)query: those are cut at the sentence boundary and carry a<span class="cutted">inside the match, and a match filling the whole sentence leaves an empty context span. The two context spans are now read one by one and stripped of their markup, whatever it contains (#14). This only concerns servers that do not deliver tokenized matches, where collocation analysis falls back to parsing the KWIC markup -
findExample()no longer aborts with "replacement has length zero" when a failed request leaves the query without any snippet to take an example from (#14) -
snippet2FreqTable()andmatches2FreqTable()work with their own default of an empty stopword list again, which used to leave the table without the column the stopwords are joined on -
dropped the
PTXQCdependency, which was imported for two small string functions (lcpCount()andlcsCount(), used byqueryStringToLabel()) but pulled inrmzqc,jsonvalidateandV8, and with them the only dependency requiring alibv8installation.
- added
cacheAsparameter tocollocationAnalysis()for transparent result caching: if the specified RDS file exists, the cached result is returned immediately; otherwise the analysis runs and the result is saved to the file (.rdsextension is added automatically if omitted) - fixed score threshold in recursive CA
- focus is now injected into webUIRequestUrls in collocationAnalysis results, when possible
- added experimental support for comparing collocation analyses across multiple vcs (
max_delta_<score>,winner<score>,loser_score<score>columns etc.), including explicit winner/loserwebUIRequestUrlcolumns for association scores, ranks, and percentile ranks. Missing per-label concordance URLs are now derived by replacing thecqparameter of an available row URL with the target label's vc, and unsuffixed consensuswinner_webUIRequestUrl/loser_webUIRequestUrlcolumns are populated when score-based URL choices agree. The names and semantics of these columns may still change without a deprecation cycle - collocates that are not attested in every compared vc get their missing scores imputed. Such rows are now marked by the new
imputed,n_imputedandimputed_<label>columns, and reported in verbose mode, so that presence/absence artifacts can be told apart from measured contrasts (filter(!imputed)). See the new "Interpreting multi-VC comparisons" section in?collocationAnalysisfor how to read the comparison columns collocationAnalysis()now warns when repeated node/collocate/label rows are reduced to their first occurrence for the comparison columns, instead of dropping them silently- added support for passing condition labels when comparing multiple vcs by allowing for named vc lists
collocationScoreQuery()now accepts a vector of collocates and queries every combination of collocate and virtual corpusKorAPConnection()is now a regular constructor function instead of the S4 class generator, so that its manual page documents how it is actually called.new("KorAPConnection", ...)and all existing calls keep working unchanged
- fixed
fetchAnnotations()morphology so MarMoT and other foundries keep all features from nested spans (#30) - reimplemented annotation parsing with
xml2, retaining multiple lemma/POS values and improving robustness of token alignment - changed format of annotation_snippet column for compatibility with Python client
fetchAnnotations()method added toKorAPQueryclass, to fetch annotations for all collected matches- regional demo updated and extended with interactive highcharter plot
- automatic documentation tests introduced (via copromted Readme.md and different LLMs)
- allow overriding
KorAPConnection(verbose=)default via env varKORAP_VERBOSEor R optionrkorap.verbose(explicit argument still wins)
- improved documentation, which should be more use case oriented now
- fixed bug with fetching result pages with an offset >= 10,000 (=1e+05 ...) issue #25
- timed out corpus queries are no longer cached (see issue #7)
- improved stability of
cifunction - improved error handling
- improved logging
- added ETAs to logging in verboose mode
- Simplified authorization process for accessing restricted data via the new
auth()function (see?auth) - Switched from
httrtohttr2for all API requests - Minimally required R version is now 4.1.0 (released 2021-05-18)
- Fixed compatibility with upcoming
ggplot23.6.0 (thanks @teunbrand) - Fixed issues with tokenized matches in
corpusQueryresults - Fixed smoothing constant in
mergeDuplicateCollocatesfunction - Fixed missing suggested packages in DESCRIPTION (for demos)
- added citation hint, see
citation("RKorAPClient") - added
matchStartandmatchEndcolumns tocorpusQueryresults, that contain the start and end position of the match in the text - added function
mergeDuplicateCollocatesto merge collocation analysis results for different context positions - added a column
queryto collocation analysis results - fixed CRAN check notes on Rd
\link{}targets missing package - fixed possible problems in
corpusStatsmethod - improved documentation for
spanparameter in collocation analysis functions - uses new metadata fields API in
textMetadatamethod, if available, to retrieve also custom metadata for a text based on its sigle - uses new metadata fields API in
corpusQuerymethod, if available
- fixed R 4.4 compatibility of collocationAnalysis function
- replaced deprecated legend.position (ggplot2)
- fixed rare frequencyQuery incompatibility with outdated KorAP instances (see KorAP/Kustvakt#668)
- added
textMetadataKorAPConnection method to retrieve all metadata for a text based on its sigle - added
webUiRequestUrlcolumn also to corpusStats results, so that also virtual corpus definitions can be linked to / tested directly in the KorAP UI - added on-click action to open KorAP query also to highcharter scatter plots
- if not
metadataOnlyis set, also tokenized snippets are now retrieved in corpus queries (stored incollectedMatches$tokens) - uses server side tokenized matches in collocation analysis, if supported by KorAP server
- fixed regional demo (will be updated again soon)
- fixed R 4.3 compatibility (see #12)
- added demo that plots plural gender variant frequencies over time
- added runtime OAuth2 browser flow demos with httr and httr2
- fixed error in recursiveCA demo
- added documentation on authorization using the OAuth browser flow
- fixed graceful fail on invalid json responses from API requests (CRAN policy violation)
- updated Roxygen to 7.2.3
- added context parameter to corpusQuery
- updated GitHub workflows
- in collocationAnalysis: given withinSpan parameters are now correctly passed to queries for examples
- replaced our log.info function with log_info to avoid name clashes
- resolved CRAN policy violation (writing to user's home filespace)
- resolved CRAN requests:
- documentation completed and improved for hc_add_onclick_korap_search, hc_freq_by_year_ci, KorAPConnection class
- proper cache directory used in regional demo
- fixed and improved path handling in collocation analysis and light verb construction demos
- fixed documentation for reexported magrittr::
%>%(pipe function)
- replaced head with dplyr::slice_head when used on data frames or tibbles
- updated Roxygen to 7.2.1 (fixes bug that caused CRAN removal of RKorAPClient)
- updated shiny demo
- added some demos for comparisons by country of publication
- fixed collocation scores for lemmatized node or collocate queries
- now fails gracefully with an informative message if the API server is not accessible
- fixed shiny demo
hc_add_onclick_korap_search()now also works with bar, column and pie highcharts- new parameters added to
collocationAnalysisfunction:maxRecurse- apply collocation analysis recursivelymaxRecursetimesaddExamples- IfTRUE, examples for instances of collocations will be added in a columnexample. This makes a difference in particular if 'node' is given as a lemma query.thresholdScore- association score function to use for computing the threshold that is applied for recursive collocation analysis callsthreshold- minimum value ofthresholdScorefunction call to apply collocation analysis recursively
- documentation migrated to roxygen2md
- Some examples in the documentation are now wrapped with:
This was necessary to meet the time limits of automatic CRAN checks under all circumstances and has otherwise no significance,
## Not run ... ## End(Not run)
The change of the minor version from 6 to 7 automatically invalidates your cached query results. However, since the API has not changed, you can continue using the old cached results by copying or renaming your cache folder.
On linux, for example, this can look like this:
mv ~/.cache/R/R.cache/RKorAPClient_0.6 ~/.cache/R/R.cache/RKorAPClient_0.7
or like this:
mv ~/.Rcache/RKorAPClient_0.6 ~/.Rcache/RKorAPClient_0.7
To find the RKorAPClient cache directory for your environment, you can use:
R.cache::getCacheRootPath()
- experimental new
collocationAnalysismethod (client-sided) - new parameter
randomizePageOrderto fetch result pages in randomized order - new parameter
ignoreCollocateCaseincollocationScoreQuery - new parameter
withinSpan(default:base/s=s) incollocationScoreQuery - number of hits logged during queries, if
verbose=TRUE
- fixed umlaut queries on windows
- fixed retrieval of access token when multiple access tokens are stored
- fixes calculation of logDice coefficient
- updated Readme.md
- collocationScoreQuery method added
- hc_add_onclick_korap_search function added
- shiny web application demo added
- support for orphaned plotly package dropped
- improved documentation structure
- The local cache is now notified of new corpus index revisions when new connections are established. As a side effect, with v0.5.9 existing caches are invalidated.
- New boolean parameter
smoothadded tohc_freq_by_year_ci
- Fixed issues with dplyr 1.0
- Introduced continuous integration tests via gh-actions, pulled from highcharter (thanks @pachamaltese)
- Fixed some links in Readme.md
- Removed redundant curl import
- Fixed handling of single query term queries in highcharter-helper that had an empty plot and a warning
as consequences.
Unknown or uninitialised column: 'condition'.
- Fix some links in Readme.md
- First release on CRAN
- Use TRUE and FALSE instead of T and F
- Be more specific about authors and rights holder
- Elaborate Description field in DESCRIPTION file
- Add forthcoming LREC paper to Readme.md
- Fix DESCRIPTION file.
- Further minimization of tests to make CRAN happy.
- Make authors and maintainers machine readable.
- Further minimization of tests to make CRAN happy. Safety shouldn't suffer.
- Make license specification github and CRAN compliant.
- Fixed many typos.
- Removed redundant and too long running tests.
- Initial release on CRAN.
- Merge highcharter branch providing the new helper function
hc_freq_by_year_ci. - Fix reporting cached results in verbose mode.
- Add demo for comparing frequencies in spoken vs. written virtual corpora. See
demo("writtenVsSpoken") - Don't invalidate cache on patch level increments.