Clock calibration problems after rebooting VAX cpu from the sim> prompt

[agn453]

• Context

Idling and clock calibration for OpenVMS VAX V7.3 using SIMH microvax3900 seem to go awry when a reboot is done without exiting and restarting the simulator. Just doing a @sys$system:shutdown gets to the sim> prompt as expected. If I do a boot cpu again intending to reboot OpenVMS, the console becomes unresponsive following the test output. Even a Ctrl-E (to return to the sim> prompt) isn't acted upon for a few minutes.

Examining the clock status (I suspect clock recalibration) shows the following.

Should the instruction time be a large negative number?

        SYSTEM SHUTDOWN COMPLETE - use console to halt system
Infinite loop, PC: 833492E2 (BRB 833492E2)
sim> b cpu


KA655-B V5.3, VMB 2.7
Performing normal system tests.
40..39..38..37..36..35..34..33..32..31..30..29..28..27..26..25..
24..23..22..21..20..19..18..17..16..15..14..13..12..11..10..09..
08..07..06..05..04..03..
Tests completed.
>>>

No reponse from typing the b command to the >>> prompt, so I enter a few Ctrl-E and wait.

After a minute or so, I get

Simulation stopped, PC: 20043740 (MFPR #20,R0)
sim> ^E^E
sim> sh clock
Minimum Host Sleep Time:        1 ms (1000Hz)
Host Clock Resolution:          1 ms
Execution Rate:                 44,177,083,300 instructions/sec
Idling:                         Enabled
Time before Idling starts:      0 seconds
Calibrated Timer:               CLK
Pre-Calibration Estimated Rate: 10,162,601
Calibration:                    Always
Asynchronous Clocks:            Available

MicroVAX 3900 clock device is CLK
Calibrated Timer 0:
  Running at:                100 Hz
  Tick Size:                 10 msecs
  Ticks in current second:   5
  Seconds Running:           11 (11 seconds)
  Calibration Opportunities: 11
  Instruction Time:          -4018774930
  Real Time:                 1173248
  Virtual Time:              1165418
  Next Interval:             500
  Base Tick Delay:           883,541,666
  Initial Insts Per Tick:    64,160
  Current Insts Per Tick:    441,770,833
  Initializations:           4
  Ticks:                     1,105
  Total Ticks:               51,950
  Peak Clock Skew:           7.212838 seconds fast
  Total Ticks Acked:         47,401
  Tick Time:                 11.05 seconds
  Total Tick Time:           8:39.49 minutes
  Initialize Base Time:      08:18:48.920
  Tick Start Time:           08:18:49.060
  Wall Clock Time Now:       08:20:14.443
  Total Time Idled:          5:19.112 minutes
sim> sh ver
MicroVAX 3900 simulator V4.0-0 Current
    Simulator Framework Capabilities:
        64b data
        64b addresses
        Threaded Ethernet Packet transports:PCAP:NAT:UDP
        Idle/Throttling support is available
        Virtual Hard Disk (VHD) support
        RAW disk and CD/DVD ROM support
        Asynchronous I/O support (Lock free asynchronous event queue)
        Asynchronous Clock support
        FrontPanel API Version 12
    Host Platform:
        Compiler: Microsoft Visual C++ 19.00.24215.01
        Simulator Compiled as C arch: x86 (Debug Build) on Jan 22 2022 at 09:13:38
        Build Tool: simh-Visual-Studio-Project
        Memory Access: Little Endian
        Memory Pointer Size: 32 bits
        Large File (>2GB) support
        SDL Video support: SDL Version 2.0.20, PNG Version 1.6.37, zlib: 1.2.11
        PCRE RegEx (Version 8.43 2019-02-23) support for EXPECT commands
        OS clock resolution: 1ms
        Time taken by msleep(1): 1ms
        Ethernet packet info: WinPcap version 4.1.3 (packet.dll version 4.1.0.2980), based on libpcap version 1.0 branch 1_0_rel0b (20091008)
        OS: Microsoft Windows [Version 10.0.19044.1466]
        Architecture: x86 on AMD64, Processors: 8
        Processor Id: Intel64 Family 6 Model 30 Stepping 5, GenuineIntel, Level: 6, Revision: 1e05
        tar tool: bsdtar 3.5.2 - libarchive 3.5.2 zlib/1.2.5.f-ipp
        curl tool: curl 7.79.1 (Windows) libcurl/7.79.1 Schannel
        simh git commit id: 370bfe00
        simh git commit time: 2022-01-16T07:44:15-08:00
sim>

• the output of "sim> SHOW VERSION" while running the simulator which is having the issue

See above. Compiled last week using Visual Studio 2015 under Windows 10 Pro 21H2 and all recent Microsoft updates are installed.

My SIMH .ini file has

set cpu 64m
set cpu idle
set cpu idle=VMS
set cpu conhalt
;set cpu model=VAXstation

and I'm using the internal compiled-in CPU microcode.

Happy to provide more details should you not be able to reproduce these symptoms.

Tony

[markpizz]

I have seen this, but not been able to reproduce it in the context of trying to track it down.

The excessive numbers and negative instruction time suggest we've got some sort of overflow issue in the calibration computations.

I just got a new laptop which absolutely doesn't see this issue. It happened much more often on my older box.

[sgcyphers]

I get the same issue when testing as well.

I have compiled under:
Visual Studio 2022 -- Windows 10 Pro 21H2
Visual C/C++ 2008 -- Windows 7 Pro 32-bit
Visual C/C++ 2008 -- Windows 7 Pro 84-bit

SIMH commit id is 370bfe0 for all three compiles. (Same commit id as Tony)

My VAX reboot's don't hang like Tony's but I always get the Infinite loop error:

Operator _GRISOM$OPA0: has been disabled, username SYSTEM

        SYSTEM SHUTDOWN COMPLETE - use console to halt system
Infinite loop, PC: 9E4044D3 (BRB 9E4044D3)
sim>

So I went back to the last official release:

git commit id: 0912a92
git commit time: 2020-06-09T20:07:41-07:00

And I got the same error!

Tony, what date might you be putting in at the VMS Date/Time prompt???

Something like:

%WBM-I-WBMINFO Write Bitmap has successfully completed initialization.
PLEASE ENTER DATE AND TIME (DD-MMM-YYYY  HH:MM)  27-JAN-2021 12:28
$!  Copyright 2001 Compaq Computer Corporation.

Because the Hobbyist licenses expired at the end of last year?

Just a hunch...

[markpizz]

My VAX reboot's don't hang like Tony's but I always get the Infinite loop error:

Operator _GRISOM$OPA0: has been disabled, username SYSTEM

    SYSTEM SHUTDOWN COMPLETE - use console to halt system

Infinite loop, PC: 9E4044D3 (BRB 9E4044D3)
sim>

What you're seeing here is not an error. It is expected behavior.

These systems didn't have a software means of powering off the system, so when you shut it down the shutdown logic displays the message you see and sits in an infinite loop awaiting the operator to turn off the computer. As it turns, out this infinite loop is easy to detect (it loops with all interrupts disabled) so the VAX simulator detects this now otherwise useless behavior and stops the simulation and returns to the sim> prompt. Even if they did have a software means of powering off the system, the result of a power off command would also return to the sim> prompt.

[sgcyphers]

Makes perfect sense. That mean's I can't reproduce Tony's error either. Sorry I can't be of much help on this one.

[agn453]

My VAX reboot's don't hang like Tony's but I always get the Infinite loop error:

The infinite loop is normal and the way SIMH detects that the system has shutdown (it goes into a tight loop).

I might also add that if I choose the reboot option with OpenVMS's shutdown command, I don't see the issue. It's only when I do a second boot cpu from the sim> prompt. Maybe since the clock calibration work for system idling was already done upon the first boot - it should be skipped for subsequent restarts.

In my case there had been a power failure overnight, so I had booted an alternative system disk to perform disk maintenance (an $ analy/disk/repair), then shut this down and wanted to resume normal operation from the normal system disk (a shutdown reboot won't work here).

Tony, what date might you be putting in at the VMS Date/Time prompt???

My system is not using a hobbyist license.

I don't get prompted for or need to enter the date or time as I keep it running with regular reboots/disk image file backups once a month (co-inciding with the monthly Microsoft update regime for the host running SIMH).

For reference, I've just entered Ctrl-E on the running instance from yesterday to check the clock stats. This is what they look like -

Simulation stopped, PC: 827ADB3A (MOVAL 827B48B0[R0],R5)
sim> sh clock
Minimum Host Sleep Time:        1 ms (1000Hz)
Host Clock Resolution:          1 ms
Execution Rate:                 6,237,600 instructions/sec
Idling:                         Enabled
Time before Idling starts:      0 seconds
Calibrated Timer:               CLK
Pre-Calibration Estimated Rate: 10,288,065
Calibration:                    Always
Asynchronous Clocks:            Available

MicroVAX 3900 clock device is CLK
Calibrated Timer 0:
  Running at:                100 Hz
  Tick Size:                 10 msecs
  Ticks in current second:   39
  Seconds Running:           76,116 (21:08:36 hours)
  Calibration Opportunities: 76,116
  Instruction Time:          523604069866
  Real Time:                 87926798
  Virtual Time:              87926759
  Next Interval:             961
  Base Tick Delay:           64,908
  Initial Insts Per Tick:    5,000
  Current Insts Per Tick:    62,376
  Initializations:           3
  Ticks:                     7,608,944
  Total Ticks:               7,611,639
  Peak Clock Skew:           13.909151 seconds fast
  Ticks Acked:               7,608,824
  Tick Time:                 21:08:09.439999 hours
  Total Tick Time:           21:08:36.379999 hours
  Initialize Base Time:      11:16:45.701
  Tick Start Time:           11:16:45.852
  Wall Clock Time Now:       08:24:57.407
  Catchup Tick Time:         08:25:07.937
  Catchup Base Time:         11:16:58.497
  Total Time Idled:          18:40:17.735 hours
sim> g

[markpizz]

I might also add that if I choose the reboot option with OpenVMS's shutdown command, I don't see the issue. It's only when I do a second boot cpu from the sim> prompt. Maybe since the clock calibration work for system idling was already done upon the first boot - it should be skipped for subsequent restarts.

That suggestion was already part of my thinking. I'm exploring where/how to do that specific thing. I'm sure that part of the problem here is that shutdown followed by a subsequent new boot runs the POST diagnostics and this code runs from ROM. Memory references running from ROM are specifically designed to get the CPU to run at about 1 MIP since there are timing loops in the diagnostics which assume a particular instruction execution rate relative to wall clock time. Running in this speed bounded mode will certainly produce wonky results relative to what was previously observed by the calibration logic when the system was running before the shutdown. Re-initializing the calibration parameters on a new boot is a good suggestion and probably a good fix, BUT such a change will mask the overflow behaviors we see now and which may come back at any time depending on other load on the host system. Finding and fixing this has a higher priority than a fix that masks it. :-(

[racingmars]

I believe I'm seeing the same problem on a couple of my systems (both Linux systems, Debian and Arch, building with GCC 10.2.1 and 12.2.0, on fast processors -- Intel i9-9900K and i5-10600K), with the vax, vax780, and vax8600 simulators. If I run the simh boot command a second time without quitting and starting simh again, each input character takes a long time to appear (it varies...sometimes it takes >10 seconds and sometimes it takes only 1 second or less).

I did notice that if I don't set SET CPU IDLE=VMS in my configuration, it appears to avoid (or at least hide) the problem.

[meltdown03]

I believe I'm seeing the same problem on a couple of my systems (both Linux systems, Debian and Arch, building with GCC 10.2.1 and 12.2.0, on fast processors -- Intel i9-9900K and i5-10600K), with the vax, vax780, and vax8600 simulators. If I run the simh boot command a second time without quitting and starting simh again, each input character takes a long time to appear (it varies...sometimes it takes >10 seconds and sometimes it takes only 1 second or less).

I did notice that if I don't set SET CPU IDLE=VMS in my configuration, it appears to avoid (or at least hide) the problem.

I found the same thing, but with a 4th Gen i5-4300U. Noidle seems to fix it. Happens on all PDP11 too and with other simh variants.