[trd] access to cylinder>80

[Volutar]

Working with .trd images.
I've beein using ordinary FDDs (5.25") and all of them were formatted for 84 tracks. TR-DOS supports that, there were no problems accessing these track numbers. But F-F has problems, since it reads trd "info" sector9 and treats it either 40 or 80 tracks single or doublesided. This info is necessary for physical drives, since 40 means it should make double step to position next track (if it's 80track drive). There is no reason to limit track number with 80!
Actual disk size can be calculated by taking "last free trk/sector" (offset 0xE1=sec 0xE2=trk) and adding "free sectors" (0xE5/0xE6), and adding 16 sectors (system sectors of track0), so for my disks it will be 2688 sectors, or 168 tracks (84 cyls + 2 sides). For ordinary images it will be 2560 sectors or 160 tracks (80 cyls +2 sides).
For example:
offset 0x8e1 = 0x01 (first free sec)
offset 0x8e2 = 0xa5 (first free trk)
offset 0x8e5 = 0x1f (free sectors l)
offset 0x8e6 = 0x00 (free sectors h)
0xa5*0x10+0x01+0x001f +0x10 = 0xa80

Also, ordinary 80 cyls trd images have size 655360 bytes, and 84 cyls images are 688128 bytes.

So could you please fix that at least for trd (but I'm sure lots of other systems also can have these "oversized" images).
I can upload images if they are needed, empty or filled.

Thanks.

[keirf]

Please attach some examples to this ticket and I will fix this next week.

[Volutar]

024-Music.trd.zip
109-GAME-021.trd.zip
EMPTY_TRD.zip
Here you are.
Thanks in advance!

[keirf]

Can I use 0x8e3 (Disk Type, 0x16..0x19) to determine number of sides, even if I calculate nr cylinders/tracks by this new means?

[Volutar]

Sure. They are still legit. The only thing that needs to be fixed, is the maximum reachable cylinder number. Instead of 80 it should be 84 (and 40 is still 40). By the way, lots of flux capturing hw/sw like kryoflux are limited with exactly 84th. So this might be fixed not just for TRD, but for lots of other formats.

[keirf]

Max reachable is not generically limited to 80. It's just that assumption was made for TRD, until we fix it now.

[keirf]

Okay I have made changes and tested them on your supplied TRD images. It does not seem to be the case that you need to add 16 sectors for track0: That is already accounted for in the "first free trk/sector" field (see your Blank images for example).

Actually 024-Music.trd is interesting -- it seems to not use the second side of last cylinder! I have made the most changes in the code to cleanly deal with that (it now properly reads as an empty track).

Anyway, please test the attached firmware with a range of TRD images including the ones you supplied, and let me know if it works okay. Then I will merge in for the next v3.x release:

ff_260_1.zip

[Volutar]

Tested, it seems that it works just fine. Thank you! :)

About extra 16 sectors, well, yes," first free trk/sector" + "free sectors" works fine to determine entire disk image size.
About 024-Music - most probably it was captured somehow wrong, still it doesn't stop it from working ok in emuls (it simply doesn't have to reach 83:1). But what about the case when you try to write to this "out of reach" empty track? Does it try to expand the image?

[keirf]

My point is that for all your supplied images, first_free + nr_free adds up precisely to the total image size. None of these image files is truncated. Not even 024-Music: It really does seem to only use one side of cylinder 83. Why? I don't know. But TR-DOS will never write to 83:1 because nr_free does not extend that far.

Does that make sense?

[keirf]

Further evidence for this interpretation of 024-Music: The first fully zero (ie. empty) 256-byte block in the image file is at 0xa5100. That corresponds to the reported first_free_trk and first_free_sector, with no need to add 16 sectors for the system sectors (trk0).

[Volutar]

OK, I've researched all of my disks in sense of these values. And figured that some images have modified "free sectors" value, to correspond image size.
There was special formatting utility which figures max cyl, and format "extended" disks. After doing this it adds special bytes in sector 8 (counting from 0), offset 0xDF/0xE0 - initial free sectors. All these disks have value 0x0A70 in it (so "original" image size is 0xa8000). But actual image sizes are different, though free sectors do correspond the size. So it must be some sort of post-capture correction, or something else. Found one image with first trk/sec+ free sec less than image file. It can be edited or even messed, and in some cases can't be reliable (though, it can be fixed).
No big deal really. It does work as supposed in any case :)
Thanks once more!

[keirf]

Yes should be fine. If image is too short for reported free sectors (ie. image is truncated) then FlashFloppy will automatically extend it :)

[Volutar]

There's a side effect of this "fix"

Now it causes *ERR*31* for some images, having wrong "free sectors" value (zero in my case). First free trk/sec = 1/0 and free sectors=0. And voila - F-F doesn't recognize this image :(

Most of emulators I know simply using image file size for that matter. 655360 bytes = 2x80x16x256, 688128 bytes = 2x84x16x256, 679936 bytes =2x83x16x256 (type 0x16, 2s/80), image size 327680 means either 2x40x16x256 (type 0x17, 2s/40) or 1x80x16x256 (type 0x18, 1s/80).
Free sectors+first free sector is only necessary for image expanding

[keirf]

So what's the answer? Base on file size, header info, both? Basing on file size is hard to recommend imo because TRD files are often truncated at end of in-use area. Are the problematic images just badly formed?

[keirf]

Okay let's stick with handling this in this original ticket, please.

Is there some canonical recognised good emulator geometry detection code I should be using for reference? If so, point me at it.

Should I special-case a few well-known image sizes? Perhaps with a look at the header for auto image expansion?

Or should I stick with the new code I have (which seems generally to work well, right?) and fall back to image-size-based detection if the new code fails (because of bad "free sectors" value for example)?

I quite like that last possibility :)

[Volutar]

I think there is no canonical detection code. Each emulator author was writting the code the way he felt right.
For instance in UnrealSpeccy it's exactly the image size which is used:
https://bitbucket.org/djdron/unrealspeccyp/src/a49cc983bec56631efb52e5f2da7f598009b3c53/devices/fdd/fdd_trd.cpp#lines-98
int max_cyl = data_size / (256 * 16 * 2);
But image type (0x16-0x18) for byte 0xe3 of sector 9 is geometry identifying. Hardware simply doesn't refer to second side in case of single side, but in image there are not empty tracks of 2nd side (in HFE image there's empty 2nd side because of format purposes). For 40track images it makes 2 steps instead of 1, but in image it doesn't store "empty" even track.

I think the code should work with the image size, as in US, but fallback to "free" in case of error when writing beyond of file (by extending image).

Some disks are pretty messy, may even use wrong image type values. The only thing that is fixed in format is byte 0xe7 of 9th sector of track 0. It should always be 0x10, otherwise betadisk won't even recognize the disk.

Some people with emulators are using "overexpanded" images with 127 tracks, to handle more files on single image. Some emulators check max cyl, some don't, but hardware itself allows head moving beyond 127 (tho I don't know the real hardware capable of that). But logically there's no problem of using such disks (or images, through the disk emulator) on real hardware. BTW, F-F shows track 99 even beyond 99th, but works fine (and I hope it will stay the same). TRDOS format uses logical coordinate byte as track, so track 255 is practical maximum.

I think there's no need to even know first free track/sector and free sectors from the image until writing beyond image size wants to happen. There are some alternative disk formats, even with folder support (which is not supported by BetaDisk). At the moment only HFE can be used to handle them. Though HFE is not supported by emulators, ...
/slight offtopic/
But there is the UDI format for disk with protections and different low-level things (alternatives to TR-DOS), and used with most of speccy emulators. It's pretty straight forward, and can be realized with F-F (I dont' know about HxC support). Version 1.0 of this format is wide-spreaded for protected images (along with TD0). It uses MFM decoded byte stream for each track, and sync-code "bitmap" (where bit=1 means it's sync A1 or C2 MFM variation). It doesn't aware of any high level sector/CRC stuff, same as HFE.
I've recently written HFE2UDI converter (in python), and now at the proccess of making UDI2HFE.
This image type can be used as alternative to HFE, and not only for Spectrum.

Falling back from current scheme only when "wrong" free sectors is pretty flimsy method - it's hard to define "wrong". Free sector can be 1 sector less than "legal" value, or "31" sectors less. Like, it can be image with 84 cylinders (with size of 688128 bytes), but free sectors can show 1 sector, with last free - 159:15 (so logically it will be detected as 80 cylinders), and it WON'T allow for track 81 read, while actually it can have real data beyond track 80.
Something like that.

[keirf]

I'll come up with a revised firmware for you to try out.

[keirf]

Here is a revised firmware:

ff_260_2.zip

[Volutar]

Disks with >80 cyls are working fine. Disks with wrong "free sectors" are working fine, ...but not all of them. investigating.

I didn't check image expanding thingy tho (I actually use device as R/O). Does it need to be checked?

[Volutar]

Some disks have disk type (t0:s9:0xE3)==0 and that causes why ERR31 still happens.
So I guess if illegal disk type is set - it should fallback to default 80/DS.

[keirf]

Here you go:

ff_260_3.zip

[Volutar]

Yup. Now it looks like handles every image I have :)
Thanks!