Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Single density version of "SD HxC Floppy Emulator Direct Access mode" #179

Closed
richard-broadhurst opened this issue Nov 4, 2018 · 89 comments

Comments

@richard-broadhurst
Copy link

Please add a single density version of "SD HxC Floppy Emulator Direct Access mode".
I think you suggested using track 254 and leaving the rest the same.

@keirf
Copy link
Owner

keirf commented Nov 4, 2018

Yes sorry I had totally forgotten about this. I will work on it and post progress (and firmware to test) here on this ticket.

@keirf
Copy link
Owner

keirf commented Nov 4, 2018

Question: I assume a 512-byte sector size would be okay? 256-byte is more normal for FM formats, but I don't think 512-byte is precluded. It makes things simpler if the logical sector format and overall protocol is identical to MFM (except for essential track-framing differences between FM and MFM). Only difference will be defaults to 5x512-bytes sectors (FM) vs 9x512-byte sectors (MFM).

@Simone2018
Copy link

I think Direct Access Mode is already in "single density" if you intend MFM DD (250 kBit/s), on track 255 or I'm wrong?

@keirf
Copy link
Owner

keirf commented Nov 6, 2018

Okay here is a firmware for you to test. The source is in Github branch fm_da.

ff_179_1.zip

@richard-broadhurst
Copy link
Author

I think the beeb OS only supports 256 byte sectors without driving all the different controllers directly, but I see how far I get.

@keirf
Copy link
Owner

keirf commented Nov 7, 2018

If you need changes just let me know. It's mostly easy to switch to 256b sectors. What were you planning to use the Direct Access mode for by the way? I don't think you're writing an image selector, right?

@richard-broadhurst
Copy link
Author

The primary use is to select disk images.

@keirf
Copy link
Owner

keirf commented Nov 7, 2018

That may require me to implement a new command. But get the protocol going on BBC first.

@richard-broadhurst
Copy link
Author

Well, I can read the status, which gives the "HxCFEDA" string and "FF-v0.1".
I don't know if writing the control data is working, doing a NOP seems to work, but it might just be being ignored.
The command is supposed to have a checksum, but I can't find any details of it.
I had assumed that command 4 was all I needed, but it doesn't seem to do anything, including giving an error.
I don't know if this is because my checksum (=0) is invalid, or because 512 byte sector writes don't work (on the BBC, but they should), or maybe it requires the config where files have a sequential number or who knows!
The "Menu" disc would be built on the PC, so I don't need to be able to scan the stick.

@keirf
Copy link
Owner

keirf commented Nov 7, 2018

Command 4 isn't implemented! Why not try a set lba command? You could check it worked by seeing if contents of sector 2 changes (that's part of the window into lba space)?

Ps I can implement command 4. Or a variant with semantics of our choice.

@richard-broadhurst
Copy link
Author

seems to read different sectors, got to go now, but I'll try setting a new LBA base tomorrow.

@richard-broadhurst
Copy link
Author

Cmd 1 set LBA base seems to return a consistent set of sectors, so it looks like we have two way comms.
Selecting disc image by name might be a nice to have, but by index would be great.
Selecting image, would also open up a way to mount a seconds or even third drive, if FF added support for mounting multiple images at once - maybe it already does, I haven't checked.

keirf added a commit that referenced this issue Nov 9, 2018
Identical to the original DD/MFM layout except for data rate,
FM framing, and default # sectors (5 vs 9).

Refs #179
@keirf
Copy link
Owner

keirf commented Nov 9, 2018

Okay the single-density FM access is now in master branch.

I will implement CMD_SELECT_IMAGE (cmd 4) and notify you here when that's done. That simply takes a 16-bit slot number/index. Very simple.

@richard-broadhurst
Copy link
Author

Thanks

@richard-broadhurst
Copy link
Author

Will this work with indexed mode?
DSKA9999.ssd

@keirf
Copy link
Owner

keirf commented Nov 26, 2018

DSKA9999.ssd won't work because only slots 000-999 are supported. However that means that DSKA0000.ssd up to DSKA0999.ssd will work.

@keirf
Copy link
Owner

keirf commented Nov 26, 2018

I will get on with CMD_SELECT_IMAGE soon but I wanted to rework some FF internals at the same time to give more flexibility to direct-access mode.

@richard-broadhurst
Copy link
Author

No problem, I think I am ready at my end, 2897 games/apps over 792 single sided disk (ssd) images indexed in 25KB with a menu system, just need to test with CMD_SELECT_IMAGE.
I am grateful for all the effort that you put in for us.

keirf added a commit that referenced this issue Nov 27, 2018
@keirf
Copy link
Owner

keirf commented Nov 27, 2018

CMD_SELECT_IMAGE is implemented in the latest FF version (v0.12).

What mode are you running FF in for your tool, is it indexed mode (DSKA0000 etc)? I have supported CMD_SELECT_IMAGE for all navigation modes, but I might restrict it to Indexed and Autoboot-Selector modes as it doesn't make a great deal of sense for direct-navigation mode.

@richard-broadhurst
Copy link
Author

Thanks, Indexed mode as it made the most sense, but I was thinking of supporting the other modes so as not to rename the files although this would be very fragile.

@keirf
Copy link
Owner

keirf commented Nov 28, 2018

Perhaps for Direct-Navigation mode it will make more sense to provide a different command, eg allow you to specify a full filename (or even full path, perhaps via a sequence of chdir() type invocations). Could provide a FS-type interface... opendir, getnext, chdir, openfile, etc

@keirf
Copy link
Owner

keirf commented Nov 28, 2018

Assuming you don't want to go straight at the block layer and do FAT munging yourself. I'm thinking... not ;)

@keirf
Copy link
Owner

keirf commented Nov 28, 2018

Anyway let me know how CMD_SELECT_IMAGE goes for you at the mo. I didn't really actually test it...

@richard-broadhurst
Copy link
Author

It's not going well, I can't update to 0.12 instead of UPD, I get EJE followed by rdo.
The same thing happens on several sticks, ones that were fine for going to 0.11.

@keirf
Copy link
Owner

keirf commented Nov 28, 2018

Noone else has reported this so it's probably you, and doubly so if you did not update the bootloader to 0.11 (if you're not sure, then you didn't). Are you definitely holding both buttons at power on?

@richard-broadhurst
Copy link
Author

I didn't update the bootloader to 0.11, I'll go and RTM!

@richard-broadhurst
Copy link
Author

richard-broadhurst commented Nov 28, 2018 via email

@keirf
Copy link
Owner

keirf commented Nov 28, 2018

You don't need to update the bootloader. But it is the bootloader that runs first, checks buttons, and displays UPD. So that should behave just as before as the bootloader hasnt changed. It must be user error. Be sure buttons are pressed during power on.

@sonicthewedge
Copy link

sonicthewedge commented Dec 10, 2018 via email

@keirf
Copy link
Owner

keirf commented Dec 10, 2018

You'd probably find it useful to make a debug build of FlashFloppy and find out in the serial log what happens when the USB disconnects. You may also need to add extra logging to the USB stack.

@sonicthewedge
Copy link

sonicthewedge commented Dec 10, 2018 via email

@keirf
Copy link
Owner

keirf commented Dec 10, 2018

“debug=y make -j8“ will leave debug hex file in the root build dir. Adjust baud rate at the top of src/console.c

Connect a serial ttl adapter just as you would for serial firmware programming. Actually I have one permanently connected and Gotek jumpered in programming mode. I then "make flash start serial" to program my new firmware, start it running, and connect a dumb serial terminal.

@sonicthewedge
Copy link

A consideration when creating the filesystem with 42GB of space:

sudo dd bs=1M if=/dev/zero of=/piusb.bin count=43008
sudo mkdosfs /piusb.bin -F 32 -I

The file system Samba share shows as NTFS on Windoze.

Perhaps there is a problem with drives over 32GB?

Reference:
https://manpages.debian.org/wheezy/dosfstools/mkdosfs.8.en.html

@sonicthewedge
Copy link

So, frustratingly, the following setting needed to be reordered and appended on the Pi:

sudo modprobe g_mass_storage file=/piusb.bin removable=1 ro=0 stall=0

I have no idea when or why this was required to change but was the cause of the write problems.

In other news, I got FlashFloppy firmware to debug build but I encountered a problem building HxC_FF_File_Selector:

m68k-amigaos-strip -s FFManager
make bootblock.bin
make[1]: Entering directory '/home/pi/dev/HxC_FF_File_Selector/amiga'
m68k-amigaos-objcopy -O binary bootblock.o bootblock.bin
m68k-amigaos-objcopy:bootblock.bin: invalid bfd target
Makefile:29: recipe for target 'bootblock.bin' failed
make[1]: *** [bootblock.bin] Error 1
make[1]: Leaving directory '/home/pi/dev/HxC_FF_File_Selector/amiga'
Makefile:18: recipe for target 'AUTOBOOT.HFE' failed
make: *** [AUTOBOOT.HFE] Error 2

@keirf
Copy link
Owner

keirf commented Dec 13, 2018

I will try upgrading to latest tool chain and see if I can repro the build failure.

@sonicthewedge
Copy link

sonicthewedge commented Dec 24, 2018 via email

@keirf
Copy link
Owner

keirf commented Dec 24, 2018

I didn't have time, and then I forgot. Could you please open a new ticket for this amiga-gcc build issue?

@keirf
Copy link
Owner

keirf commented Jan 22, 2019

FYI I just implemented CMD_SELECT_NAME which allows to select a new image by name.

@richard-broadhurst
Copy link
Author

I'm sorry, I'm being a bit lazy here, but has something changed with this option in recent FW?
It locks up while trying to seek to a new image.
I have my gotek HW packed away and can't test but recent testing on two new GOTEKs, one you small one (I think) lockup, whilst my code hasn't changed since I first added support.
I'm happy to support both, but need to find out what has changed.
Off to trawl the commits.

@keirf
Copy link
Owner

keirf commented Feb 26, 2023

Which recent firmware version are you testing?

@richard-broadhurst
Copy link
Author

To be honest, I have forgotten as it wasn't in my PC, but I believe they downloaded the new version in January. I think that the problem has been around for quite a while though - maybe even a year.
I'll just have to dig my kit out and install the Debug build.

@keirf
Copy link
Owner

keirf commented Feb 26, 2023

Yes I'm interested if I get more details especially if this is on latest 3.x stable firmware.

@richard-broadhurst
Copy link
Author

I finally got some hardware out and can confirm that this feature works correctly on my old first ge GOTEK with v3.39 loader and firmware with and without logging. I don't have any new HW, so will have to wait until someone with new HW can check what is going on, assuming that logging is fairly detailed for error conditions :)
Thanks again for all your effort on this great project.

@richard-broadhurst
Copy link
Author

If you need changes just let me know. It's mostly easy to switch to 256b sectors. What were you planning to use the Direct Access mode for by the way? I don't think you're writing an image selector, right?

Did this end up as 256 byte sectors?
I've been using as 256 byte sectors and it has worked for many people, but a couple of setsups do not work and someone suggested that it might be because FF was expecting 512B sectors.

@keirf
Copy link
Owner

keirf commented Jan 28, 2024

No, it is 5 * 512-byte sectors. That is 1 * command/response sector (sector ID 0) + 4 * "LBA window" sectors (sector IDs 1-4).

@richard-broadhurst
Copy link
Author

Thanks, I released my menu program for the BBC Micro on the day you added the SD option on track 254, sending a single sector of 256 bytes with only the first 11 set and whatever bytes were in code memory for the CRC and other params and it has worked on everything!
I can't see how from the code as it seems to check the CRC in the code that has been there for 6 years but no failures were logged using the logging FW.
Now it also seems to fail on at least one system if the disc isn't "spinning", but it has just done a seek to track 254, so I don't really understand this either.
I'm not asking you to do anything, just letting you know :)

@keirf
Copy link
Owner

keirf commented Jan 29, 2024

Yes that's a couple of things that don't make sense.

If you are only writing a 256-byte sector then it's unlikely you would write enough bytes following the sync pattern to satisfy the check here, let alone the CRC check: https://github.com/keirf/flashfloppy/blob/master/src/image/da.c#L372

Is it possible that the FDC interprets the IDAM as reported by FlashFloppy, and does "sane" things to read/write a 256-byte prefix of a 512-byte sector?

FlashFloppy doesn't generally care about motor on/off. Even if configured to emulate motor, only the READY signal line is affected. Not read/write of data.

@richard-broadhurst
Copy link
Author

The call that I am using doesn't care about READY iirc.
I can follow the code easily enough, but not knowing what any of the variables mean, understanding it is tricky!
I'm only sending CMD 4 and 2 bytes, plus whatever is lying around to make a 256 byte sector.
I'd really like to try to understand why what I have works for the majority of people so that I don't break it for them with any changes.
I'm very tight on space as you might expect on an 8bit machine and with a possibly unnecessary constraint of position independent code, data takes up extra space.
It isn't helped that I don't have a gtek that I can access at the moment!

@keirf
Copy link
Owner

keirf commented Jan 30, 2024

The easiest way to see what is happening will be to connect a Gotek running debug firmware, and gather serial logs via a USB-TTL adapter. Then FlashFloppy can be modified to show what it receives on writes, and is serving up on reads. For the latter, actually I can probably take a dump of the track using Greaseweazle...

Really it's impossible though to see how FlashFloppy could be serving up CRC-qualified 256-byte sectors, or accepting same for writes. So the question is what happens between your code and what "hits the wire".

@keirf
Copy link
Owner

keirf commented Jan 30, 2024

Here is a dump of track 254.0 from FlashFloppy v3.42. If you look at it in HxC tools (for example) you will see 5 512-byte sectors as expected. The only complaint I have of the layout is that a bit more gap post-index would be nice (and perhaps an Index Address Mark).

da_fm.zip

@keirf
Copy link
Owner

keirf commented Jan 31, 2024

For comparison, an SCP with both the FM and MFM direct-access tracks.

da.zip

@keirf
Copy link
Owner

keirf commented Feb 1, 2024

By the way, on digging into this some more for another issue, on Greaseweazle, it seems to me that the sector size issued in say a READ DATA command is not checked against the sector header's N field off disk. Hence mismatches between those sector sizes don't stop READ/WRITE operations. It seems to me that in the command-requested size is smaller, reads would be truncated and writes would be nul-padded. Which is probably what you're seeing.

I'm not sure this behaviour is documented though! Or necessarily consistent across FDC clones.

@richard-broadhurst
Copy link
Author

That sounds right, including the not working on the odd piece of new HW.
Writing 2.5K takes a while when all you want is to send is one command of 3 bytes ;)
Even the code to calc the command hash, which evidently isn't used would be hard to squeeze in and I've already used my 16 byte budget!
I understand that FF has to sit above a HW layer that it can't control.

@keirf
Copy link
Owner

keirf commented Feb 3, 2024

FF could choose a smaller sector size, however 512 bytes is natural for LBA addressing the Flash drive. It's also the usual MFM sector size. I could allow the sector size to be adjusted, but that would probably require at least one command to be transferred at the default 512-byte sector size.

@keirf
Copy link
Owner

keirf commented Feb 3, 2024

Even the code to calc the command hash, which evidently isn't used would be hard to squeeze in and I've already used my 16 byte budget!

What do you mean by command hash? Do you mean CRC? That is not usually calculated by the host. It resides outside the sector data/payload, and gets automatically generated/verified by the FDC.

@richard-broadhurst
Copy link
Author

Sorry, haven't been well.
I mean the 8th param or whatever it is that is supposed to be the hash of the params, but as I think you said, it isn't checked.
I can make it write 5 sectors of 512 bytes, it just takes 10 times longer than 1 sector of 256 bytes to execute and feels wasteful when I am only sending CMD_4 + two bytes and not actually writing data to the disc (maybe it does and that is how it works!) ;)
As you say, the disc controller is doing the heavy lifting on the sector hashes.
The spinning up the disc is also a bit of a pain, although I haven't checked that this is what is stopping one of the GOTEK clones although spinning the disc immediately before issuing the write does seem to make it work on that machine - not one I can access.
I will make up some test apps that do different things, including what seems to be necessary as well as see if I can fit the code into the space poor exes that I also need.

@keirf
Copy link
Owner

keirf commented Feb 9, 2024

Oh yes, the original HxC spec does have a checksum byte. FlashFloppy never implemented it; the contents of that byte are not checked.

You don't need to write 5 sectors. You only need to write sector 0 to issue a command. Which is what you're already doing, and successfully (most of the time!).

It might be interesting to know the model and configuration of the GOTEK which needs to be "spun up". Is it one which can be configured to respect motor signal w.r.t. generating the READY signal? It could be in that case that no spin up means no READY and the FDC is aware of this and fails the write. Just a theory. Apart from this FF really doesn't care about spinning up.,

@richard-broadhurst
Copy link
Author

I mean the eighth command parameter or whatever number it is that you said was ignored :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants