Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"non expandable" container refused to attach read-only #1065

Open
al20878 opened this issue Aug 7, 2021 · 11 comments
Open

"non expandable" container refused to attach read-only #1065

al20878 opened this issue Aug 7, 2021 · 11 comments

Comments

@al20878
Copy link
Contributor

al20878 commented Aug 7, 2021

  • Context

  • the output of "sim> SHOW VERSION" while running the simulator which is having the issue

current version of simh in git

  • the simulator configuration file (or commands) which were used when the problem occurred.

  • the expected behavior and the actual behavior

$ simh
sim> set cpu 11/45
sim> att -r rk dos_rk.dsk
RK0: unit is read only
RK0: non expandable  disk container 'dos_rk.dsk' is smaller than simulated device (1228.800KW < 1247.232KW)
File open error
sim> att -r rk system.dsk
RK0: unit is read only
RK0: 'system.dsk' Contains an ODS1 File system
RK0: Volume Name:       SYSTEM Format: DECFILE11A   Sectors In Volume: 4800
RK0: non expandable  disk container 'system.dsk' is smaller than simulated device (1228.800KW < 1247.232KW)
File open error

Above, I was trying to attach RK05 images read-only to an RK: disk drive:

sim> dir dos_rk.dsk
 Directory of /home/systems/dos11

08/06/2021  10:38 PM         2,457,600 dos_rk.dsk
               1 File(s)         2,457,600 bytes
               0 Dir(s)
sim> dir system.dsk
 Directory of /home/systems/dos11

09/13/2005  06:28 PM         2,457,600 system.dsk
               1 File(s)         2,457,600 bytes
               0 Dir(s)

Both files are correctly-sized containers. Moreover, in case of "system.dsk" the filesystem was recognized by simh, and so it should not matter if the container is non-expandable -- the file system is sized exactly to occupy all of it, and it's unlikely that the OS will access anything beyond that boundary. Besides, it is a READ-ONLY access, which is requested, so nothing could be changed / screwed up, in principle. Note that if I remove the -r flag, then the attachment works.

  • you may also need to provide specific pointers to data files that may be necessary to demonstrate the problem

An RK11-D (aka RK05) disk cartridge is documented to have a 1,228,800 words storage capacity (looking at the pdp11 peripherals handbook (1976 version) right now, and a 1978 version agrees with that too), so the containers were actually sized quite perfectly right.

Yet, simh shows 1247KW for the capacity:

sim> sho rk
RK      address=17777400-17777417, vector=220, BR5, 8 units
  RK0   1247KW, not attached, write enabled
        RK05, AUTO detect format
  RK1   1247KW, not attached, write enabled
        RK05, AUTO detect format
  RK2   1247KW, not attached, write enabled
        RK05, AUTO detect format
  RK3   1247KW, not attached, write enabled
        RK05, AUTO detect format
  RK4   1247KW, not attached, write enabled
        RK05, AUTO detect format
  RK5   1247KW, not attached, write enabled
        RK05, AUTO detect format
  RK6   1247KW, not attached, write enabled
        RK05, AUTO detect format
  RK7   1247KW, not attached, write enabled
        RK05, AUTO detect format

These drives do have 3 spare cylinders (in addition to 200 accessible by software), but the true usable
capacity is 200 tracks/surface x 2 surfaces x 12 sectors/track x 256 words/sector = 1,228,800 words,
per the hardware handbooks.

Looks like simh is using the full drive capacity (203x2x12x256) in the report above.

@al20878
Copy link
Contributor Author

al20878 commented Aug 7, 2021

BTW, sim_disk.c:

                const char *container_dtype = ctx->footer ? (const char *)ctx->footer->DriveType : "";

                sim_switches = SWMASK ('R');
                uptr->capac = (t_addr)(container_size/(ctx->capac_factor*((dptr->flags & DEV_SECTORS) ? 512 : 1)));
                sim_printf ("%s: non expandable %s disk container '%s' is %s than simulated device (%s %s ",
                            sim_uname (uptr), container_dtype, cptr, (container_size < current_unit_size) ? "smaller" : "larger", sprint_capac (dptr, uptr), (container_size < current_unit_size) ? "<" : ">");

if footer is NULL (which it can perfectly be in R/O attachment of an image, which wasn't modded by simh yet per #1059 -- and it does not append a footer in R/O access, thankfully) -- then that creates an extra (double) space in the error message, because contater_dtype is "":

RK0: non expandable  disk container 

Note it between "expandable" and "disk": looks irregular, like a bug, if something was meant to be there but is missing...

@markpizz
Copy link
Member

markpizz commented Aug 7, 2021

Hmmm... That pdp11 handbook on page 4-441 clearly describes the hardware/software interface for cylinder which says the maximum cylinder is 312 (octal) which is 202 decimal, so counting cylinder 0 there are 203 cylinders that software can access.

If software chooses to not access beyond cylinder 199 decimal, that is merely a convention used by the OS that happens to be currently running. The DEC standard 144 convention of storing a bad block table on the last track of the drive combined with the OS option of not using the last cylinders of the drive would be a good way to preserve the bad block data across independent initializations of the drive. Had the initial disk image been created with the DEC 144 bad block data recorded, then the size would have been the full expected size of the drive and this complaint wouldn't be happening, right? If you attach the drive without the -R flag, it will be extended to full size as the meta data is written beyond the software accessible data area of the container and things would just work, right? After doing this, If you remove the meta data, the full sized drive will then attach read only without any problem, right?

@al20878
Copy link
Contributor Author

al20878 commented Aug 8, 2021

That pdp11 handbook on page 4-441 clearly describes the hardware

The handbook says (my page here is 4-443) that the drive capacity is 1,228,800 words (which is 4800 blocks), 200. cylinders (+ 3 spare) -- that's what it says. The OS/software did not have to use the spare cylinders (even if they were there and addressable).

The "systems.dsk" disk (that I showed in my initial post), is a DEC image for the RSX11M system (v3.2 install media, IIRC).
It does have a bad block file in it, and the file lists only the last block as "bad". Meaning, there was no last-track bad block data written on that drive.

Moreover, both RSX-11 and DOS-11 (I'm not sure about other OSes, so not talking about them) used no cylinders past 200. -- this is the design capacity of the drive (while it could address 6 more tracks, they were considered spare and unused).

Like I also said, simh could figure out that the filesystem occupied only 4800 blocks (exactly 200. cylinders as the image capacity), so why the fuss about "not being able to open the container"? Even if OS would "access" any data beyond 200. cylinders (which it won't BTW), how is that simh business? Besides, simh can return 0's in those "phantom" blocks missing from the image, regardless of the capacity -- to preserve backward compatibility with how it was handled in the previous versions of this very simulator.

BADBLK.SYS;1        (3,3)          1./1.              15-DEC-1977 14:07:23 [1,1]    [RWED,RWED,RWED,R]

SOFTWARE BAD BLOCK DESCRIPTOR @ LBN 4799. (VBN 1.)
    -- empty list --
BAD BLOCK FILE SUMMARY
    Bad block(s)
    SIZE    LBN
    1.      H:000 L:011277 = 4799.-4799.

DEC STD144 seems to apply to RK06/07 and future drives, not to RK05 drives (RK05 predates STD144).

http://bitsavers.org/pdf/dec/standards/DEC_STD_144.pdf

After doing this, If you remove the meta data, the full sized drive will then attach read only without any problem, right?

No, it is a problem, because that won't be the original image, anymore.

Also, don't you think that your workaround "recipe" has unreasonably many actions for a simple read-only attachment of an image?

markpizz added a commit that referenced this issue Aug 8, 2021
- Fix 'non expandable' error message to avoid potential extra space as
   mentioned in #1065
- Reasonably report open errors as mentioned in #1064
@al20878
Copy link
Contributor Author

al20878 commented Aug 8, 2021

BTW, HELP does not mention STD144 properly, instead it says "standard 044":

simh/pdp11
$ grep 'last track' *.c
pdp11_hk.c:        &hk_set_bad, NULL, NULL, "write bad block table on last track" },
pdp11_hk.c:fprintf (st, "044 compliant bad block table on the last track:\n\n");
pdp11_io_lib.c:   last track of the specified unit as described in:
pdp11_rl.c:        &rl_set_bad, NULL, NULL, "Write bad block table on last track" },
pdp11_rl.c:fprintf (st, "044 compliant bad block table on the last track:\n\n");
pdp11_rp.c:        &rp_set_bad, NULL, NULL, "write bad block table on last track" },
pdp11_rp.c:fprintf (st, "standard 044 compliant bad block table on the last track.\n\n");

A typo?

@al20878
Copy link
Contributor Author

al20878 commented Aug 8, 2021

At any rate, I was thinking how is it (logically) possible for simh to complain that a container is non-expandable and larger than the drive it's being attached to, at the same time? Reading the code, the condition can realize, but from a logical POV, that does not make any sense: rather, the container must be complained about as "non-shrinkable" in this case... LOL

Anyways, I think it's reasonable to expect if somebody has a read-only disk image, they should be able to attach it to a device, which is larger, without having to jump through the hoops with image modification with simh resizing it first.

BTW, RK05 cannot have STD144 track written into it, even according to the very same simh code (and that's correct because like I mentioned, the standard was adopted later): the pdp11tracksize parameter is passed as 0 to sim_disk_attach_ex() called from pdp11_rk.c. So simh cannot actually "correct" the size of the image. Finally, the "autosizing" cannot be changed for the RK drive, either, so there seems to be no way to attach a read-only smaller (actually, a standard capacity) RK05 image to a drive...

@markpizz
Copy link
Member

markpizz commented Aug 8, 2021

At any rate, I was thinking how is it (logically) possible for simh to complain that a container is non-expandable and larger than the drive it's being attached to, at the same time? Reading the code, the condition can realize, but from a logical POV, that does not make any sense: rather, the container must be complained about as "non-shrinkable" in this case... LOL

All containers are non-shrinkable. :-) I'll look at that.

Anyways, I think it's reasonable to expect if somebody has a read-only disk image, they should be able to attach it to a device, which is larger, without having to jump through the hoops with image modification with simh resizing it first.

I'm thinking about this. Please wait. I'm waiting for some feedback from Bob who's out of town for at least the next week.

BTW, RK05 cannot have STD144 track written into it, even according to the very same simh code (and that's correct because like I mentioned, the standard was adopted later): the pdp11tracksize parameter is passed as 0 to sim_disk_attach_ex() called from pdp11_rk.c. So simh cannot actually "correct" the size of the image. Finally, the "autosizing" cannot be changed for the RK drive, either, so there seems to be no way to attach a read-only smaller (actually, a standard capacity) RK05 image to a drive...

Right you are and along with the misspelled help text (which actually originally came from the pdp11_doc.doc and/or vax780_doc.doc) I've reviewed the subject of the DEC 144 support and thus changed the pdp11_rp to only support that on the appropriate drives and explicitly added RP05 support which had been skipped probably due to it being the same size as the RP04.

markpizz added a commit that referenced this issue Aug 8, 2021
- Added a specific drive type RP05 which is the same size as the RP04
  but can be distringuished by OS software.
- Restrict SET rpn BADBLOCK to only disk types which actually supported
  the DEC Standard 144 bad block table.
- Fixed typo's in help and comments about DEC 044 vs DEC 144 and also
  in pdp11_doc and vax780_doc documentation files.

As discussed in #1065
@al20878
Copy link
Contributor Author

al20878 commented Aug 9, 2021

Oh cool, while you are at updating the docs, please note that 2.10.3 of pdp11_doc.doc calls TQK50 as a disk controller (twice)... I don't think it's correct, as it is a tape controller:

2.10.3	TQK50 TMSCP Disk Controller (TQ)

The TQ controller simulates the TQK50 TMSCP disk controller

markpizz added a commit that referenced this issue Aug 9, 2021
@ghost
Copy link

ghost commented Aug 15, 2021

SimH simulates hardware. The RK05 has 203 cylinders x 12 sectors x 512 words. Therefore, its hardware capacity is 1,247,232 words.

If DEC software reserved three cylinders for remapping, that has nothing to do with hardware or the simulator. Non-DEC software used the full capacity.

The error is a V4 problem. Attaching a 200 cylinder image to the 203 cylinder RK05 works just fine in V3.X - the "extra" 3 cylinders are implicitly 0.

DEC's bad block standard postdates the RK05. There is no bad block table on an RK05.

@markpizz
Copy link
Member

@al20878 will you please attach a zip of a RK container file that demonstrates the problem that you've reported here.

Thanks.

@al20878
Copy link
Contributor Author

al20878 commented Sep 9, 2021

SimH simulates hardware.

@archivisth Thank you! I cannot agree with you more! But unfortunately simh has been stepping too much onto the software soil lately, e.g. #1059, and that's upsetting... This issue here is somewhat of the same sort, actually.

will you please attach a zip of a RK container file

@markpizz Any dsk image sized at 2,457,600 will do (even if you attach an empty container with all 0's inside, created by "dd").

$ dd if=/dev/zero of=zero.dsk count=4800
4800+0 records in
4800+0 records out
2457600 bytes (2.5 MB, 2.3 MiB) copied, 0.0328812 s, 74.7 MB/s
sim> att -r rk2 zero.dsk
RK2: Unit is read only
RK2: non expandable disk container 'zero.dsk' is smaller than simulated device (1228.800KW < 1247.232KW)
File open error

But you can also find these in here: http://www.bitsavers.org/bits/DEC/pdp11/discimages/rk05/
Take, for example, rt54f.dsk, and you'll get:

sim> att -r rk2 rt54f.dsk
RK2: Unit is read only
RK2: 'rt54f.dsk' Contains RT11 partitions
1 valid partition, Type: V05, Sectors On Disk: 4800
RK2: non expandable disk container 'rt54f.dsk' is smaller than simulated device (1228.800KW < 1247.232KW)
File open error

@al20878
Copy link
Contributor Author

al20878 commented Oct 31, 2021

As noted in #1059 today, it's still the issue as of the current codebase:

sim> att -r rk rt54f.dsk
RK0: Unit is read only
RK0: 'rt54f.dsk' Contains RT11 partitions
1 valid partition, Type: V05, Sectors On Disk: 4800
RK0: non expandable disk container 'rt54f.dsk' is smaller than simulated device (1228.800KW < 1247.232KW)
File open error

markpizz added a commit that referenced this issue Oct 31, 2021
leobru pushed a commit to leobru/simh that referenced this issue Jan 23, 2022
- Fix 'non expandable' error message to avoid potential extra space as
   mentioned in simh#1065
- Reasonably report open errors as mentioned in simh#1064
leobru pushed a commit to leobru/simh that referenced this issue Jan 23, 2022
- Added a specific drive type RP05 which is the same size as the RP04
  but can be distringuished by OS software.
- Restrict SET rpn BADBLOCK to only disk types which actually supported
  the DEC Standard 144 bad block table.
- Fixed typo's in help and comments about DEC 044 vs DEC 144 and also
  in pdp11_doc and vax780_doc documentation files.

As discussed in simh#1065
leobru pushed a commit to leobru/simh that referenced this issue Jan 23, 2022
leobru pushed a commit to leobru/simh that referenced this issue Jan 23, 2022
markpizz added a commit that referenced this issue Feb 8, 2022
… meta data

At this time:
- The ZAP command exists to remove meta data from containers that
   have it.  Container files processed by the ZAP command will generally
   be restored to the size it was before the addition of the meta data
   and the file time stamps will be unchanged.
- Newly created containers get meta data.
- At attach time, containers that don't have meta data, but have
   recognized file systems that fit within the drive it is being
   attached to get meta data added without changing the file timestamps.
- Containers that don't have meta data and don't have a recognized
   file system that is <= the drive size are attached without comment
   and without adding meta data as long as the drive is NOT set to
   autosize (controllers that support multiple drive types all default to
   autosize, which can explicitly be disabled on a drive by drive basis).
- Containers that don't have meta data which are > the drive size can
   only be attached read only.
- Containers with meta data can not be attached to a different
   controller at all if the container size is smaller than the drive on the
   other controller.
- Containers that have meta data can be freely be attached to the
   controller that they were attached to when they got the meta data.
   If a file system is detected, it will be reported.  Otherwise if no
   recognized file system is found, the attach will be silent.  File
   system detection reporting can be suppressed with -Q on the attach
   command.
- Containers with meta data can only be attached read only to a different
   controller if the container is larger than the drive it is being
   attached to.
In the future:
- In general, containers with meta data (or recognized file systems)
   will be attachable to MSCP and SCSI controllers, as long as reasonable
   sector sizes and file system not requiring interleaving have been found.
- Containers without meta data will only be attachable if autosize
   is disabled and the container is <= the size of the drive.
- Explicitly setting a drive type on a unit will implicitly disable
   autosizing.  If a user wants to set the default drive for a unit
   and still allow autosizing they must explicitly set the unit to
   autosize after setting the drive type.

Relevant to: #1065, #1059, #1094, #1100, #1118, #1117
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants