[help] Dynamic branch names on different systems

[neilstats]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

I'm trying to run a targets pipeline on both a Windows PC and a Linux HPC system. I am getting different names (and hence also seeds) for dynamic branches. For example, running the following code:

library(targets)
tar_script({
      tar_option_set(format = "qs")
      list(tar_target(vals, 1),
           tar_target(res, 1, pattern = map(vals)))
      },
      ask = FALSE)
tar_make()
tar_meta(fields = c("name", "data", "command", "depend", "seed"))

On the Windows PC (and also on a mac, and posit.cloud) this gives:

  name         data             command          depend                  seed
  <chr>        <chr>            <chr>            <chr>                  <int>
1 vals         4ebb285df3c00f5e 0e2890e1b1d6be9c ef46db3751d8e999 -1582794411
2 res_fc68408c 4ebb285df3c00f5e 0e2890e1b1d6be9c ef46db3751d8e999  -650145132
3 res          64f5b505f55269f8 0e2890e1b1d6be9c <NA>             -1086954299

But on the Linux HPC I get:

  name         data             command          depend                  seed
  <chr>        <chr>            <chr>            <chr>                  <int>
1 vals         4ebb285df3c00f5e 0e2890e1b1d6be9c ef46db3751d8e999 -1582794411
2 res_b3bb9178 4ebb285df3c00f5e 0e2890e1b1d6be9c ef46db3751d8e999  -474200353
3 res          5f43e261705d7d46 0e2890e1b1d6be9c NA               -1086954299

Running something like targets:::pattern_name_branches("res", list()) interactively returns the same value on both systems.

Using targets_1.4.1 and R 4.2.2 in both cases.

Any suggestions for what could be causing the discrepancy (which means the pipeline is not portable between the two) or how to fix would be much appreciated.

[neilstats]

After some investigations, I think that naming dynamic targets at some point(s) uses digest_obj32(). So, the difference could be down to the serialization as per this discussion: #956 (comment)

Update: After further investigation, I found that harmonising locale across systems (and so I assume fixing the encoding and serialization issue above) fixed the example I gave. Additionally, in my real data analysis I had to make sure the same version of the stringfish package (used by qs) was used (version 0.16.0 of stringfish added ALTREP serialization).

[wlandau]

Thanks for figuring this out.

Locale was also a hashing issue in #1108 (reply in thread). Temporarily setting LC_COLLATE = "C" before sorting seems to have solved that problem. Unfortunately, I do not know if it is feasible to lock down all aspects of locale to harmonize hashes in the general case. Sys.setlocale(LC_ALL = "...") only applies to a subset of locale settings, and I am not sure if the others are platform-dependent, or if new locale settings will be added in future versions of R.

Next time someone asks, I will try to remember to ask for the value of Sys.getlocale() on both machines.