Add Chinese error message translations (#3935)

* initial exploration of adding a chinese translation of messages working version of chinese translation * adding another .mo file that was untracked * trying to get the .pot for src/ files * adding warning()s from C * C messages with DTPRINT or Rprintf * adding STOP messages * DTWARN messages * Error in forder * odds&ends more messages * cannot use _() with \r * update .pot for src files * moving pot file to po/ * update header metadata for src .pot * run update_pkg_po on the src to get mo files * update_pkg_po after merge * add template message in onAttach() & clear typos/PYDT issue along the way * remove some strings * improve arg checking&message in groupingsets; add some messages * get tests working (fix some bad quotes, adjust to new messages) * update po with latest message edits * Finished the translation assigned to Chun-Hui Gao (#3983) * Finished the translation of line 3058-3231 in po/zh_CN.po. Totally 34 messages. * Finished the translation assigned to Chun-Hui Gao in R-zh_cn.po. Lines 1500-1598 in original R-zh_CN.po. Totally 26 messages. * Resolved reviewer's comments. * Resolved reviewer's comments. * (hopefully) apply encoding fix & check compilation * improve internal error & regenerate messages * Finish translation to zh_CN assigned to group 2 (Hongyuan Jia) (#3986) * Finish translation. * Resolve review comments. * balance quotes * extra nocov * some coverage of fmelt * complete coverage of fmelt * fix test error * update message per DeLuns feedback * update message again; some minor tweaks to nearby code * Translation to zh_CN assigned to group 3 (#3985) * translate assigned to group 3 * translate assigned to group 3 * translate assigned to group 3 * remove merge conflict vestiges * remove merge conflict vestiges * plonk * update translation * minor update * added new message caught by xianghui * Translation haogao gu (#4002) * R-zh_CN_2019-10-26 * 10-26 * restore previous * updated with feedback * regenerate with small fix * regenerate .mo files * Translation finished (soappp) (#4023) * readme link examples, rm old link to a talk (#4001) * Homepage additions (#4006) * pkgdown, keep manual tab, link cranlike page, minor rename (#4008) * fix Rd warnings (#4011) * use Rdevel for revdeps due to bioc-devel (3.11) now needing R 4+ * (hopefully) apply encoding fix & check compilation * improve internal error & regenerate messages * Finish translation to zh_CN assigned to group 2 (Hongyuan Jia) (#3986) * Finish translation. * Resolve review comments. * balance quotes * extra nocov * some coverage of fmelt * complete coverage of fmelt * fix test error * update message per DeLuns feedback * update message again; some minor tweaks to nearby code * translatation completed * fixed somme message 1 * fix format tags for two translations * done some translation! * Finish translation to zh_CN assigned to group 5 (Leo Lee) (#3995) * chinese translation by Leo * Update R-zh_CN.po fix a typo * Update zh_CN.po * add missing quotes * number out-of-order format strings * apply numbering to formatters * Update R-zh_CN.po & Change author * Fix the problems of missing %d in zh_CN.po & Polish translation * Translation dracodoc (#4034) * 895 - 939 * 945 - 1007 * To the real 1007 line msg * convert enconding from GBK to UTF-8 * Revert "convert enconding from GBK to UTF-8" This reverts commit 6e65d0e. * translate start and end of assigned msgs. * till 1899 * two entries then found possible problem of missing messages * 10 missing message not captured? * till 1962 * finished assignment * review and edit * update meta info * macro-within-macro removed * syntax error -- split ternary operator * roll back use of translation in ASSERT internal errors * Update according to @shrektan's comments * re-run update_pkg_po * update binaries * Translation caiquanyou (#4039) * finish translation * Update R-zh_CN.po * Update zh_CN.po * update mo files * translation(partial) into zh_CN by Yuanchen Xie (#4010) * translation(partial) into zh_CN by Yuanchen Xie * Add new translation results, based on messages * update mo file * update transitions as renkun-ken's suggestions * update binary mo * update vs recent master * Translation to zh_CN - Group 7 (#4019) * translate R msgs * src error messages * missing quote * missing quote * remove extra quote * fix-up of merge * update po after merge * update fuzzy translations * update mistaken PRIu64->PRId64 * caught a few more failed merges * line # update * rerun update_pkg_po * Translation shawn (#4050) * done some translation! * done some translation! * done some translation * revised the translation * small tweak * Translation yilei (#3989) * messsage * messsage * messsage * messsage * messsage * done some translation! * done it. * updating some small misses from last pull * Translation to zh_CN assigned to group 12 (#3987) * Update translations * Minor refine * Minor reformatting * no longer using %llu, other small tidying * Translation fengqifang (#3990) * First version, finished all message * 2nd one, multiple lines of msgstr * 3rd, finished * zh_CN.po 1st * after zh_CN.po reviewed * fix encoding issue * manually editing per GH UI submission * run update_pkg_po on new merge * done some translation (#4069) * done some translation * do some revisions * touch-up * Translation to zh_CN assigned to group 4 (#3997) * finish R-zh_CN.po * finish zh_CN.po * finalize two files * change author * make the package * finalize the messages * change the format of numbering * install the pacakge * delete excessive src folders * manually revert coding issues (open to a better solution... `iconv(bad_str, from = 'GBK'|'GB18030', to = 'UTF-8')` both returned something gibberish-y. * update mo files * update .mo * add in suggestions from gaospecial * update mo * My part of translation has been completed. It's so happy! (#4051) * messsage * Update R-zh_CN.po * Update R-zh_CN.po * Update zh_CN.po * some revisions due to foreign char, touch-up, remove fuzzy * Translation yuliang li (#4055) * completed yuliang's translation * yuliang's work * Update zh_CN.po Co-authored-by: Michael Chirico <michaelchirico4@gmail.com> * tweaks for syntax mistakes * Translation to zh_CN - Group 17 (#4049) * finish the Chinese message translation for group 17 * change as KingdaShi suggests Co-authored-by: Michael Chirico <michaelchirico4@gmail.com> * update mo, fix missing field * Translation yc0802 (#4054) * message * message * added hongyuans review Co-authored-by: Michael Chirico <michaelchirico4@gmail.com> * adding "leftover" translations (#4143) * done some translation!-kingda (#4046) * done some translation! * done some translation * messsage * messsage Co-authored-by: Michael Chirico <michaelchirico4@gmail.com> * Translation leftover pt2 (#4146) * covering trivial translations from among the leftovers * progress on paring down leftover translations * completed translations!!! * adjustments * missing dot * add note for proceeding to CRAN_Relase * %f needed for > integer max Co-authored-by: Chun-Hui Gao <gaospecial@gmail.com> Co-authored-by: Hongyuan Jia <hongyuan.jia@bears-berkeley.sg> Co-authored-by: biobai <biobai@zju.edu.cn> Co-authored-by: Koohoko <koohoko@gmail.com> Co-authored-by: soappp <soappp9527@gmail.com> Co-authored-by: KingdaShi <50854702+KingdaShi@users.noreply.github.com> Co-authored-by: Guangzheng Li <leolee0828@outlook.com> Co-authored-by: dracodoc <dracodoc@gmail.com> Co-authored-by: caiquanyou <48861819+caiquanyou@users.noreply.github.com> Co-authored-by: Yuanchen Xie <yuanchen.gm@gmail.com> Co-authored-by: Amy Tzu-Yu Chen <amy17519@gmail.com> Co-authored-by: ShawnChen1996 <shawnchen1996@outlook.com> Co-authored-by: Yilei <30611492+Zachary-Wu@users.noreply.github.com> Co-authored-by: Kun Ren <mail@renkun.me> Co-authored-by: fengqifang <nclxin@gmail.com> Co-authored-by: Shayebuhuii <32659317+sunshine1126@users.noreply.github.com> Co-authored-by: Zhi Yang <zhiyang@usc.edu> Co-authored-by: Xueliang24 <56901274+Xueliang24@users.noreply.github.com> Co-authored-by: JulianYlli12 <43525270+JulianYlli12@users.noreply.github.com> Co-authored-by: Xianying Tan <shrektan@126.com> Co-authored-by: yc0802 <55459478+yc0802@users.noreply.github.com>
Rdatatable · Dec 31, 2019 · ff076a7 · ff076a7
1 parent d52b0d8
commit ff076a7
Show file tree

Hide file tree

Showing 60 changed files with 14,327 additions and 1,003 deletions.
diff --git a/.dev/CRAN_Release.cmd b/.dev/CRAN_Release.cmd
@@ -1,3 +1,69 @@
+###############################################
+#  Updating translations
+###############################################
+
+# 1) Update messages for new release
+## (a) Update C template file: src/data.table.pot
+##     ideally, we are including _() wrapping in
+##     new PRs throughout dev cycle, and this step
+##     becomes about tying up loose ends
+## Appending _() char array wrapping to all messages
+##   that might be shown to the user. This step is slightly
+##   too greedy, as it includes too many msg, some of which
+##   need not be translated [more work to do here to make
+##   this less manual] some things to watch out for:
+##     * quote embedded (and escaped) within message [could be fixed with smarter regex]
+##     * multi-line implicit-concat arrays (in C, `"a" "b"` is the same as `"ab"`) should be wrapped "on the outside" not individually
+##     * `data.table` shares some of its `src` with `pydatatable`, so the requirement to `#include <R.h>` before the `#define _` macro meant we need to be careful about including this macro only in the R headers for these files (hence I created `po.h`)
+##     * Can't use `_()` _inside_ another functional macro. Only wrap the string passed to the macro later.
+for MSG in error warning DTWARN DTPRINT Rprintf STOP Error;
+  do for SRC_FILE in src/*.c;
+    # no inplace -i in default mac sed
+    do sed -E "s/$MSG[(]("[^"]*")/$MSG(_(\1)/g" $SRC_FILE > out;
+    mv out $SRC_FILE;
+  done
+done
+
+## checking for other lines calling these that didn't get _()-wrapped
+for MSG in error warning DTWARN DTPRINT Rprintf STOP Error;
+  do grep -Er "\b$MSG[(]" src --include=*.c | grep -v _ | grep -Ev "(?://|[*]).*$MSG[(]"
+
+## similar, but a bit more manual to check snprintf usage
+
+## look for char array that haven't been covered yet
+grep -Er '"[^"]+"' src --include=*.c | grep -Fv '_("' | grep -v "#include" | grep -v '//.*".*"'
+
+## look for lines starting with a char array (likely continued from prev line & can be combined)
+grep -Er '^\s*"' src/*.c
+
+## Now extract these messages with xgettext
+cd src
+xgettext --keyword=_ -o data.table.pot *.c
+cd ..
+
+## (b) Update R template file: src/R-data.table.pot
+## much easier, once the update_pkg_po bug is fixed
+R --no-save
+## a bug fix in R still hadn't made the 2019-12-12 release,
+##   so run the following to source the corrected function manually
+STEM='https://raw.githubusercontent.com/wch/r-source/trunk/src/library/tools/R'
+source(file.path(STEM, 'utils.R'))
+source(file.path(STEM, 'xgettext.R'))
+source(file.path(STEM, 'translations.R'))
+## shouldn't be any errors from this...
+update_pkg_po('.')
+q()
+
+# 2) Open a PR with the new templates & contact the translators
+#   * zh_CN:
+## Translators to submit commits with translations to this PR
+##   [or perhaps, if we get several languages, each to open
+##    its own PR and merge to main translation PR]
+
+## 3) Check validity
+##   update_pkg_po('.') to be run again for the PR
+##     [can this be done via Travis?]
+
 ###############################################
 #  Basic checks
 ###############################################

diff --git a/R/between.R b/R/between.R
@@ -1,6 +1,6 @@
 # is x[i] in between lower[i] and upper[i] ?
 between = function(x, lower, upper, incbounds=TRUE, NAbounds=TRUE, check=FALSE) {
-  if (is.logical(x)) stop("between has been x of type logical")
+  if (is.logical(x)) stop("between has been passed an argument x of type logical")
   if (is.logical(lower)) lower = as.integer(lower)   # typically NA (which is logical type)
   if (is.logical(upper)) upper = as.integer(upper)   # typically NA (which is logical type)
   is.px = function(x) inherits(x, "POSIXct")

diff --git a/R/data.table.R b/R/data.table.R
@@ -183,7 +183,7 @@ replace_dot_alias = function(e) {
     }
     return(x)
   }
-  if (!mult %chin% c("first","last","all")) stop("mult argument can only be 'first','last' or 'all'")
+  if (!mult %chin% c("first","last","all")) stop("mult argument can only be 'first', 'last' or 'all'")
   missingroll = missing(roll)
   if (length(roll)!=1L || is.na(roll)) stop("roll must be a single TRUE, FALSE, positive/negative integer/double including +Inf and -Inf or 'nearest'")
   if (is.character(roll)) {
@@ -820,7 +820,7 @@ replace_dot_alias = function(e) {
           if (!typeof(byval[[jj]]) %chin% ORDERING_TYPES) stop("column or expression ",jj," of 'by' or 'keyby' is type ",typeof(byval[[jj]]),". Do not quote column names. Usage: DT[,sum(colC),by=list(colA,month(colB))]")
         }
         tt = vapply_1i(byval,length)
-        if (any(tt!=xnrow)) stop("The items in the 'by' or 'keyby' list are length (",paste(tt,collapse=","),"). Each must be length ", xnrow, "; the same length as there are rows in x (after subsetting if i is provided).")
+        if (any(tt!=xnrow)) stop(gettextf("The items in the 'by' or 'keyby' list are length(s) (%s). Each must be length %d; the same length as there are rows in x (after subsetting if i is provided).", paste(tt, collapse=","), xnrow, domain='R-data.table'))
         if (is.null(bynames)) bynames = rep.int("",length(byval))
         if (length(idx <- which(!nzchar(bynames))) && !bynull) {
           # TODO: improve this and unify auto-naming of jsub and bysub
@@ -874,7 +874,7 @@ replace_dot_alias = function(e) {
             # attempt to auto-name unnamed columns
             for (jj in which(nm=="")) {
               thisq = q[[jj + 1L]]
-              if (missing(thisq)) stop("Item ", jj, " of the .() or list() passed to j is missing") #3507
+              if (missing(thisq)) stop(gettextf("Item %d of the .() or list() passed to j is missing", jj, domain="R-data.table")) #3507
               if (is.name(thisq)) nm[jj] = drop_dot(thisq)
               # TO DO: if call to a[1] for example, then call it 'a' too
             }

diff --git a/R/devel.R b/R/devel.R
@@ -13,7 +13,7 @@ dcf.repo = function(pkg, repo, field, type){
   idx = file(file.path(contrib.url(repo, type=type),"PACKAGES"))
   on.exit(close(idx))
   dcf = read.dcf(idx, fields=c("Package",field))
-  if (!pkg %in% dcf[,"Package"]) stop("There is no ", pkg, " package in provided repository.")
+  if (!pkg %in% dcf[,"Package"]) stop(gettextf("There is no package %s in provided repository.", pkg, domain='R-data.table'))
   dcf[dcf[,"Package"]==pkg, field][[1L]]
 }
 

diff --git a/R/groupingsets.R b/R/groupingsets.R
@@ -60,10 +60,10 @@ groupingsets.data.table = function(x, j, by, sets, .SDcols, id = FALSE, jj, ...)
     stop("All columns used in 'sets' argument must be in 'by' too. Columns used in 'sets' but not present in 'by': ", brackify(setdiff(sets.all.by, by)))
   if (id && "grouping" %chin% names(x))
     stop("When using `id=TRUE` the 'x' data.table must not have a column named 'grouping'.")
-  if (!all(sapply(sets, function(x) length(x)==uniqueN(x))))
+  if (any(sapply(sets, anyDuplicated)))
     stop("Character vectors in 'sets' list must not have duplicated column names within a single grouping set.")
-  if (!identical(lapply(sets, sort), unique(lapply(sets, sort))))
-    warning("Double counting is going to happen. Argument 'sets' should be unique without taking order into account, unless you really want double counting, then get used to that warning. Otherwise `sets=unique(lapply(sets, sort))` will do the trick.")
+  if (length(sets) > 1L && (idx<-anyDuplicated(lapply(sets, sort))))
+    warning("'sets' contains a duplicate (i.e., equivalent up to sorting) element at index ", idx, "; as such, there will be duplicate rows in the output -- note that grouping by A,B and B,A will produce the same aggregations. Use `sets=unique(lapply(sets, sort))` to eliminate duplicates.")
   # input arguments handling
   jj = if (!missing(jj)) jj else substitute(j)
   av = all.vars(jj, TRUE)

diff --git a/R/last.R b/R/last.R
@@ -35,7 +35,7 @@ last = function(x, n=1L, ...) {
     }
   } else {
     if (!requireNamespace("xts", quietly=TRUE))
-      stop("'xts' class passed to data.table::last function but 'xts' is not available, you should have 'xts' installed already") # nocov
+      stop(gettextf("'xts' class passed to %s function but 'xts' is not available, you should have 'xts' installed already", "data.table::last", domain="R-data.table")) # nocov
     if (verbose)
       cat("last: using xts::last: is.xts(x)\n")
     xts::last(x, n=n, ...)
@@ -76,7 +76,7 @@ first = function(x, n=1L, ...) {
     }
   } else {
     if (!requireNamespace("xts", quietly=TRUE))
-      stop("'xts' class passed to data.table::first function but 'xts' is not available, you should have 'xts' installed already") # nocov
+      stop(gettextf("'xts' class passed to %s function but 'xts' is not available, you should have 'xts' installed already", "data.table::first", domain="R-data.table")) # nocov
     if (verbose)
       cat("first: using xts::first: is.xts(x)\n")
     xts::first(x, n=n, ...)

diff --git a/R/onAttach.R b/R/onAttach.R
@@ -20,6 +20,8 @@
   if (!isTRUE(getOption("datatable.quiet"))) {   # new option in v1.12.4, #3489
     packageStartupMessage("data.table ", v, if(dev)paste0(" IN DEVELOPMENT built ",d,g),
                           " using ", getDTthreads(verbose=FALSE), " threads (see ?getDTthreads).  Latest news: r-datatable.com")
+    if (gettext("TRANSLATION CHECK", domain='R-data.table') != "TRANSLATION CHECK")
+      packageStartupMessage(gettext("**********\n", "Running data.table in English; package support is available in English only. When searching for online help, be sure to also check for the English error message. This can be obtained by looking at the po/R-<locale>.po and po/<locale>.po files in the package source, where the native language and English error messages can be found side-by-side"))
     if (dev && (Sys.Date() - as.Date(d))>28L)
       packageStartupMessage("**********\nThis development version of data.table was built more than 4 weeks ago. Please update: data.table::update.dev.pkg()\n**********")
     if (!.Call(ChasOpenMP))

diff --git a/R/setkey.R b/R/setkey.R
@@ -251,7 +251,7 @@ setorder = function(x, ..., na.last=FALSE)
 # na.last=FALSE here, to be consistent with data.table's default
 # as opposed to DT[order(.)] where na.last=TRUE, to be consistent with base
 {
-  if (!is.data.frame(x)) stop("x must be a data.frame or data.table.")
+  if (!is.data.frame(x)) stop("x must be a data.frame or data.table")
   cols = substitute(list(...))[-1L]
   if (identical(as.character(cols),"NULL")) return(x)
   if (length(cols)) {
@@ -357,7 +357,7 @@ CJ = function(..., sorted = TRUE, unique = FALSE)
     }
   }
   nrow = prod( vapply_1i(l, length) )  # lengths(l) will work from R 3.2.0
-  if (nrow > .Machine$integer.max) stop("Cross product of elements provided to CJ() would result in ",nrow," rows which exceeds .Machine$integer.max == ",.Machine$integer.max)
+  if (nrow > .Machine$integer.max) stop(gettextf("Cross product of elements provided to CJ() would result in %.0f rows which exceeds .Machine$integer.max == %d", nrow, .Machine$integer.max, domain='R-data.table'))
   l = .Call(Ccj, l)
   setDT(l)
   l = setalloccol(l)  # a tiny bit wasteful to over-allocate a fixed join table (column slots only), doing it anyway for consistency since

diff --git a/R/test.data.table.R b/R/test.data.table.R
@@ -43,7 +43,7 @@ test.data.table = function(script="tests.Rraw", verbose=FALSE, pkg=".", silent=F
     # nocov start
     fn2 = paste0(fn,".bz2")
     if (!file.exists(file.path(fulldir, fn2)))
-      stop("Neither ",fn," or ",fn2," exist in ",fulldir)
+      stop(gettextf("Neither %s nor %s exist in %s",fn, fn2, fulldir, domain="R-data.table"))
     fn = fn2
     # nocov end
     # sys.source() below accepts .bz2 directly.

diff --git a/inst/po/en@quot/LC_MESSAGES/R-data.table.mo b/inst/po/en@quot/LC_MESSAGES/R-data.table.mo
diff --git a/inst/po/en@quot/LC_MESSAGES/data.table.mo b/inst/po/en@quot/LC_MESSAGES/data.table.mo
diff --git a/inst/po/zh_CN/LC_MESSAGES/R-data.table.mo b/inst/po/zh_CN/LC_MESSAGES/R-data.table.mo
diff --git a/inst/po/zh_CN/LC_MESSAGES/data.table.mo b/inst/po/zh_CN/LC_MESSAGES/data.table.mo