Documents no longer buildable with 'koch docs'.

[ci4ic4]

Description

I follow Nim's development and often rebuild locally the documentation using 'koch docs' and 'koch pdf'. My last successful run was on the 26th of April; I tried today and got a curious problem:
`

/home/xci/src/Nim/lib/pure/times.nim(1537) times_examples_51
/home/xci/src/Nim/lib/std/assertions.nim(41) failedAssertImpl
/home/xci/src/Nim/lib/std/assertions.nim(36) raiseAssert
/home/xci/src/Nim/lib/system/fatal.nim(53) sysFatal
Error: unhandled exception: /home/xci/.cache/nim/times_d/runnableExamples/times_examples_51.nim(9, 3) initDateTime(21, mApr, 2018, 0, 0, 0) == initDateTime(dSat, 16, 2018.IsoYear, 0, 0, 0) [AssertionDefect]
Error: execution of an external program failed: '/home/xci/.cache/nim/times_d/runnableExamples/times_group0_examples'
[runnableExamples] failed: generated file: '/home/xci/.cache/nim/times_d/runnableExamples/times_group0_examples.nim' group: 'rdoccmd:
docCmd:
code: # autogenerated by docgen
source: /home/xci/src/Nim/lib/pure/times.nim

`

Separately running times_example_51.nim indeed shows that the first and the third asserts fail. This happens to be run under NetBSD-current amd64.

The same failure was also observed under
Linux www.ci4c.tk 6.2.0-1003-oracle #3-Ubuntu SMP Thu Apr 6 10:27:48 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux.

A separate issue exists under NetBSD-current AArch64 - bitops_d/runnableExamples/@mbitops_examples_33.nim.c fails as it tries to include an obviously non-existent x86intrin.h...

Nim Version

$ nim --version
Nim Compiler Version 1.9.3 [NetBSD: amd64]
Compiled at 2023-06-09
Copyright (c) 2006-2023 by Andreas Rumpf

git hash: f091014
active boot switches: -d:release

Current Output

No response

Expected Output

No response

Possible Solution

No response

Additional Information

No response

[metagn]

Can you run:

import times

let a = initDateTime(21, mApr, 2018, 0, 0, 0)
let b = initDateTime(dSat, 16, 2018.IsoYear, 0, 0, 0)
echo a
echo b
echo a == b

There was a similar issue dealt with in #20193, presumably related to #17223. @capocasa Do you have any idea?

[ci4ic4]

./testtimes
2018-04-21T00:00:00+01:00
2018-04-21T01:00:00+01:00
false

[Araq]

For me it fails with:

external program failed, retrying serial work queue for logs!
bin/nim md2html --errormax:3 --hint:Conf:off --hint:Path:off --hint:Processing:off --hint:XDeclaredButNotUsed:off --warning:UnusedImport:off -d:boot --putenv:nimversion=1.7.3 -d:nimExperimentalLinenoiseExtra  --doc.plausibleAnalytics:nim-lang.org --git.url:https://github.com/nim-lang/Nim -o:web/upload/1.7.3/apis.html --index:on doc/apis.md
bin/nim md2html --errormax:3 --hint:Conf:off --hint:Path:off --hint:Processing:off --hint:XDeclaredButNotUsed:off --warning:UnusedImport:off -d:boot --putenv:nimversion=1.7.3 -d:nimExperimentalLinenoiseExtra  --doc.plausibleAnalytics:nim-lang.org --git.url:https://github.com/nim-lang/Nim -o:web/upload/1.7.3/backends.html --index:on doc/backends.md
bin/nim md2html --errormax:3 --hint:Conf:off --hint:Path:off --hint:Processing:off --hint:XDeclaredButNotUsed:off --warning:UnusedImport:off -d:boot --putenv:nimversion=1.7.3 -d:nimExperimentalLinenoiseExtra  --doc.plausibleAnalytics:nim-lang.org --git.url:https://github.com/nim-lang/Nim -o:web/upload/1.7.3/contributing.html --index:on doc/contributing.md
bin/nim md2html --errormax:3 --hint:Conf:off --hint:Path:off --hint:Processing:off --hint:XDeclaredButNotUsed:off --warning:UnusedImport:off -d:boot --putenv:nimversion=1.7.3 -d:nimExperimentalLinenoiseExtra  --doc.plausibleAnalytics:nim-lang.org --git.url:https://github.com/nim-lang/Nim -o:web/upload/1.7.3/destructors.html --index:on doc/destructors.md
bin/nim md2html --errormax:3 --hint:Conf:off --hint:Path:off --hint:Processing:off --hint:XDeclaredButNotUsed:off --warning:UnusedImport:off -d:boot --putenv:nimversion=1.7.3 -d:nimExperimentalLinenoiseExtra  --doc.plausibleAnalytics:nim-lang.org --git.url:https://github.com/nim-lang/Nim -o:web/upload/1.7.3/docgen.html --index:on doc/docgen.md
/mnt/c/Users/rumpf/projects/nim/doc/docgen.md(13, 1) Error: cannot open '/mnt/c/Users/rumpf/projects/nim/web/upload/1.7.3/compiler/docgen.idx'
lineinfos.nim(312)       raiseRecoverableError
Error: unhandled exception: cannot open '/mnt/c/Users/rumpf/projects/nim/web/upload/1.7.3/compiler/docgen.idx' [ERecoverableError]

Very strange, where is the version 1.7.3 coming from?

[ringabout]

I cannot reproduce the issue on windows, probably related to OS.

Make sure follow the instructions below:

nim c koch.nim
./koch boot -d:release
./koch docs

As to testtimes

2018-04-21T00:00:00+08:00
2018-04-21T00:00:00+08:00
true

[metagn]

This is the full relevant proc:

Nim/lib/pure/times.nim

Lines 1529 to 1545 in 21d941c

	proc initDateTime*(weekday: WeekDay, isoweek: IsoWeekRange, isoyear: IsoYear,
	hour: HourRange, minute: MinuteRange, second: SecondRange,
	nanosecond: NanosecondRange,
	zone: Timezone = local()): DateTime {.since: (1, 5).} =
	## Create a new `DateTime <#DateTime>`_ from a weekday and an ISO 8601 week number and year
	## in the specified timezone.
	##
	## .. warning:: The ISO week-based year can correspond to the following or previous year from 29 December to January 3.
	runnableExamples:
	assert initDateTime(21, mApr, 2018, 00, 00, 00) == initDateTime(dSat, 16, 2018.IsoYear, 00, 00, 00)
	assert initDateTime(30, mDec, 2019, 00, 00, 00) == initDateTime(dMon, 01, 2020.IsoYear, 00, 00, 00)
	assert initDateTime(13, mSep, 2020, 00, 00, 00) == initDateTime(dSun, 37, 2020.IsoYear, 00, 00, 00)
	assert initDateTime(2, mJan, 2021, 00, 00, 00) == initDateTime(dSat, 53, 2020.IsoYear, 00, 00, 00)
	# source https://webspace.science.uu.nl/~gent0113/calendar/isocalendar.htm
	let d = isoweek * 7 + weekday.int - initDateTime(4, mJan, isoyear.int, 00, 00, 00).weekday.int - 4
	initDateTime(1, mJan, isoyear.int, hour, minute, second, nanosecond, zone) + initDuration(days=d)

The 1 hour difference makes me think it could be a timezone quirk, that may or may not also be related to the OS.

[ci4ic4]

I get the discrepancy on Windows, Linux (aarch64) and NetBSD (both amd64 and aarch64), so it is not OS-related, but most likely timezone quirk.
On Windows I get:

$ nim c -r .\testtimes.nim
Hint: used config file 'C:\Users\xci\source\repos\Nim\config\nim.cfg' [Conf]
Hint: used config file 'C:\Users\xci\source\repos\Nim\config\config.nims' [Conf]
................................................................................................
C:\Users\xci\source\repos\Nim\testtimes.nim(3, 9) Warning: use `dateTime`; initDateTime is deprecated [Deprecated]
CC: lib/system/exceptions.nim
CC: lib/std/private/digitsutils.nim
CC: lib/std/assertions.nim
CC: lib/system/dollars.nim
CC: lib/std/exitprocs.nim
CC: lib/std/syncio.nim
CC: lib/system.nim
CC: lib/pure/math.nim
CC: lib/pure/strutils.nim
CC: lib/pure/dynlib.nim
CC: lib/windows/winlean.nim
CC: lib/pure/times.nim
CC: testtimes.nim
Hint:  [Link]
Hint: mm: orc; threads: on; opt: none (DEBUG BUILD, `-d:release` generates faster code)
46145 lines; 5.072s; 71.121MiB peakmem; proj: C:\Users\xci\source\repos\Nim\testtimes.nim; out: C:\Users\xci\source\repos\Nim\testtimes.exe [SuccessX]
Hint: C:\Users\xci\source\repos\Nim\testtimes.exe [Exec]
2018-04-21T00:00:00+01:00
2018-04-21T01:00:00+01:00
false

---

BTW I haven't been able to complete a Nim build under Windows for a few days now, I am getting

....
bin\nim.exe c -o:bin\vccexe.exe --skipUserCfg --skipParentCfg --hints:off tools/vccexe/vccexe.nim
C:\Users\xci\source\repos\Nim\tools\vccexe\vcvarsall.nim(36, 115) Warning: sdk_version, vctoolset all have default value '""', this may be unintentional, either use ';' (semicolon) or explicitly write each default value [ImplicitDefaultValue]
bin\nim.exe c -o:bin\nimpretty.exe -d:release --skipUserCfg --skipParentCfg --hints:off nimpretty/nimpretty.nim
bin\nim.exe c -o:bin\testament.exe -d:release --skipUserCfg --skipParentCfg --hints:off testament/testament.nim
C:\Users\xci\source\repos\Nim\testament\testament.nim(123, 59) Warning: workingDir, input all have default value '""', this may be unintentional, either use ';' (semicolon) or explicitly write each default value [ImplicitDefaultValue]
bin\nim.exe c -o:bin\nim_dbg.exe --opt:speed --stacktrace -d:debug --stacktraceMsgs -d:nimCompilerStacktraceHints --skipUserCfg --skipParentCfg --hints:off compiler/nim.nim
deps.cmd: git checkout -q 168416290e49023894fc26106799d6f1fc964a2d
deps.cmd: git checkout -q b4c73320253f78e3a265aec6d9e8feb83f97c77b
bin\nim.exe c -o:bin\nimble.exe -d:release --mm:refc --noNimblePath --skipUserCfg --skipParentCfg --hints:off dist/nimble/src/nimble.nim
C:\Users\xci\source\repos\Nim\dist\nimble\src\nimblepkg\developfile.nim(796, 3) Warning: target type is larger than source type
  target type: 'uint' (8 bytes)
  source type: 'set[ValidationErrorKind]' (1 byte) [CastSizes]
deps.cmd: git checkout -q #head
error: pathspec '#head' did not match any file(s) known to git
C:\Users\xci\source\repos\Nim\koch.nim(742) koch
C:\Users\xci\source\repos\Nim\koch.nim(163) bundleAtlasExe
C:\Users\xci\source\repos\Nim\tools\deps.nim(38) cloneDependency
C:\Users\xci\source\repos\Nim\tools\deps.nim(10) exec
C:\Users\xci\source\repos\Nim\lib\std\assertions.nim(41) failedAssertImpl
C:\Users\xci\source\repos\Nim\lib\std\assertions.nim(36) raiseAssert
C:\Users\xci\source\repos\Nim\lib\system\fatal.nim(53) sysFatal
Error: unhandled exception: C:\Users\xci\source\repos\Nim\tools\deps.nim(10, 3) `status == 0` git checkout -q #head [AssertionDefect]

but this is a separate problem, obviously.

[metagn]

Does it work if you change initDateTime(4, mJan, isoyear.int, 00, 00, 00) to initDateTime(4, mJan, isoyear.int, 00, 00, 00, zone)?

[ci4ic4]

Yes.

[ci4ic4]

This did not fix it for me at all, the test runs exactly as before - unless there is some obj cache I am not aware of:

 times_examples_51.nim
#[
autogenerated by docgen
loc: /home/xci/src/Nim/lib/pure/times.nim(1537, 3)
rdoccmd:
]#
import std/assertions
import "/home/xci/src/Nim/lib/pure/times.nim"
{.line: ("/home/xci/src/Nim/lib/pure/times.nim", 1537, 3).}:
  echo initDateTime(21, mApr, 2018, 00, 00, 00)
  echo initDateTime(dSat, 16, 2018.IsoYear, 00, 00, 00)
  echo initDateTime(30, mDec, 2019, 00, 00, 00)
  echo initDateTime(dMon, 01, 2020.IsoYear, 00, 00, 00)
  echo initDateTime(13, mSep, 2020, 00, 00, 00)
  echo initDateTime(dSun, 37, 2020.IsoYear, 00, 00, 00)
  echo initDateTime(2, mJan, 2021, 00, 00, 00)
  echo initDateTime(dSat, 53, 2020.IsoYear, 00, 00, 00)

diff --git a/lib/pure/times.nim b/lib/pure/times.nim
index 61971ba3a..3685fdb20 100644
--- a/lib/pure/times.nim
+++ b/lib/pure/times.nim
@@ -1541,7 +1541,7 @@ proc initDateTime*(weekday: WeekDay, isoweek: IsoWeekRange, isoyear: IsoYear,
     assert initDateTime(2, mJan, 2021, 00, 00, 00) == initDateTime(dSat, 53, 2020.IsoYear, 00, 00, 00)

   # source https://webspace.science.uu.nl/~gent0113/calendar/isocalendar.htm
-  let d = isoweek * 7 + weekday.int - initDateTime(4, mJan, isoyear.int, 00, 00, 00).weekday.int - 4
+  let d = isoweek * 7 + weekday.int - initDateTime(4, mJan, isoyear.int, 00, 00, 00, zone).weekday.int - 4
   initDateTime(1, mJan, isoyear.int, hour, minute, second, nanosecond, zone) + initDuration(days=d)

 proc initDateTime*(weekday: WeekDay, isoweek: IsoWeekRange, isoyear: IsoYear,

$ ./times_examples_51
2018-04-21T00:00:00+01:00
2018-04-21T01:00:00+01:00
2019-12-30T00:00:00+00:00
2019-12-30T00:00:00+00:00
2020-09-13T00:00:00+01:00
2020-09-13T01:00:00+01:00
2021-01-02T00:00:00+00:00
2021-01-02T00:00:00+00:00

[metagn]

How come it worked before? Can you use --forceBuild to recompile times_examples_51?

[ci4ic4]

~/src/nimbits ❯ nim c --forceBuild -r times_examples_51.nim
Hint: used config file '/home/xci/src/Nim/config/nim.cfg' [Conf]
Hint: used config file '/home/xci/src/Nim/config/config.nims' [Conf]
Hint: used config file '/home/xci/nim.cfg' [Conf]
..............................................................................................
/home/xci/src/nimbits/times_examples_51.nim(9, 8) Warning: use `dateTime`; initDateTime is deprecated [Deprecated]
/home/xci/src/nimbits/times_examples_51.nim(11, 8) Warning: use `dateTime`; initDateTime is deprecated [Deprecated]
/home/xci/src/nimbits/times_examples_51.nim(13, 8) Warning: use `dateTime`; initDateTime is deprecated [Deprecated]
/home/xci/src/nimbits/times_examples_51.nim(15, 8) Warning: use `dateTime`; initDateTime is deprecated [Deprecated]
/home/xci/src/nimbits/times_examples_51.nim(6, 11) Warning: imported and not used: 'assertions' [UnusedImport]
CC: ../Nim/lib/system/exceptions.nim
CC: ../Nim/lib/std/private/digitsutils.nim
CC: ../Nim/lib/std/assertions.nim
CC: ../Nim/lib/system/dollars.nim
CC: ../Nim/lib/system.nim
CC: ../Nim/lib/pure/math.nim
CC: ../Nim/lib/pure/strutils.nim
CC: ../Nim/lib/posix/posix.nim
CC: ../Nim/lib/pure/times.nim
CC: times_examples_51.nim
Hint:  [Link]
Hint: mm: orc; threads: on; opt: none (DEBUG BUILD, `-d:release` generates faster code)
47585 lines; 1.352s; 71.277MiB peakmem; proj: /home/xci/src/nimbits/times_examples_51.nim; out: /home/xci/src/nimbits/times_examples_51 [SuccessX]
Hint: /home/xci/src/nimbits/times_examples_51 [Exec]
2018-04-21T00:00:00+01:00
2018-04-21T01:00:00+01:00
2019-12-30T00:00:00+00:00
2019-12-30T00:00:00+00:00
2020-09-13T00:00:00+01:00
2020-09-13T01:00:00+01:00
2021-01-02T00:00:00+00:00
2021-01-02T00:00:00+00:00

$ nim --version
Nim Compiler Version 1.9.3 [NetBSD: amd64]
Compiled at 2023-06-13
Copyright (c) 2006-2023 by Andreas Rumpf

git hash: 606e9941d0c42a361b519038dfff3af061eba381
active boot switches: -d:release

[metagn]

I assumed when you responded "yes" to "does this change work" that it fixed the issue.

The offending commit is very likely to be 9c2d277 (based on you saying "last success on april 26" and the commit history of times), which subtly changes initTimeInterval to initDuration in the current code we have. Reverting this part should fix it (and makes more sense on my surface level understanding).

(Also the other procs called initDateTime are deprecated aliases for dateTime but this isn't deprecated)

[ci4ic4]

It is definitely the timezone. On NetBSD it is driven by /etc/localtime, which I set usually during the OS installation as a soft link to /usr/share/zoneinfo/Europe/London. My aarch64 NetBSD system was installed differently and had /etc/localtime point to /usr/share/zoneinfo/UTC; in this case the test program executed as expected. When I set /etc/localtime to point to Europe/London, it fails:

➜  Nim git:(devel) ✗ ls -l /etc/localtime
lrwxr-xr-x 1 root wheel 33 Jun 13 22:48 /etc/localtime -> /usr/share/zoneinfo/Europe/London
➜  Nim git:(devel) ✗ ./t
2018-04-21T00:00:00+01:00
2018-04-21T01:00:00+01:00
2019-12-30T00:00:00+00:00
2019-12-30T00:00:00+00:00
2020-09-13T00:00:00+01:00
2020-09-13T01:00:00+01:00
2021-01-02T00:00:00+00:00
2021-01-02T00:00:00+00:00
➜  Nim git:(devel) ✗ rm /etc/localtime && ln -s /usr/share/zoneinfo/UTC /etc/localtime
➜  Nim git:(devel) ✗ ./t
2018-04-21T00:00:00+00:00
2018-04-21T00:00:00+00:00
2019-12-30T00:00:00+00:00
2019-12-30T00:00:00+00:00
2020-09-13T00:00:00+00:00
2020-09-13T00:00:00+00:00
2021-01-02T00:00:00+00:00
2021-01-02T00:00:00+00:00

The Windows 11 host is also set to Europe/London and fails; all this is using the latest Nim sources.

[ci4ic4]

FYI the test succeeds under Linux after making /etc/localtime point to UTC.

On my NetBSD-current amd64 installation 'koch doc' and 'koch pdf' finish OK after the same, so there do not appear to be other regressions with respect to this particular problem.

[metagn]

Can you try the patch in #22091

[ci4ic4]

Tried it, works for me on all my test builds - NetBSD, Ubuntu and Windows 11.

[metagn]

As in the original test passes and there is no hour difference?

[kaushalmodi]

I see this issue on CentOS 7.9.2009 in GMT-4 time zone.

Steps to reproduce the issue:

cd lib/pure
nim doc times.nim

Output:

---

Update: This is with the latest commit from devel branch.

[Araq]

Why can't we disable the test? It's not that important that it should prevent the docs from building.

[metagn]

I really think #22091 fixes it. It would be nice if someone gave hard confirmation that it does because it's hard to test.

[ci4ic4]

​As I said, I tested it under Ubuntu 2304 aarch64, NetBSD-current amd64 and aarch64 and Windows 11 and it works for me.

I also ran, as suggested above, 'nim doc times.nim' without a problem, no matter the value of localtime.

[kaushalmodi]

@metagn I also confirm that #22091 fixes this issue. I rebased your branch on top of the latest devel and reran the whole Nim build + tests + koch docs.