New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow read/write speeds ST/HFE on Atari ST (WD1772) #44

Closed
TzOk83 opened this Issue Jan 18, 2018 · 44 comments

Comments

Projects
None yet
2 participants
@TzOk83

TzOk83 commented Jan 18, 2018

I have already mentioned about this problem, but now I have done the measurements and it is really bad :(
I was testing on Atari 1040STE using low-level disk copy tool called FastCopy III. I was formatting and then copying an empty disk onto itself with a fast format option. Disk copy routine of FC III first reads the whole disk into RAM, and then writes it onto the destination disk. My ST has 4MB of RAM, so no disk switching is required. Step rate has been set to 3ms (default).

Real FDD (80TRK/9SEC):
FORMAT: 0:34
READ: 0:34
FMT+WRITE: 1:06

Real FDD (80TRK/11SEC):
FORMAT: 1:06
READ: 1:12
FMT+WRITE: 2:10

FF/ST (80TRK/9SEC):
FORMAT: 1:06
READ: 1:06
FMT+WRITE: 5:45 <- that's way too long

FF/ST (80TRK/11SEC):
FORMAT: 1:05 <- like on real FDD 80/11 disk
READ: 1:07 <- like on real FDD 80/11 disk
FMT+WRITE: 2:14 <- like on real FDD 80/11 disk

FF/HFE (80TRK/9SEC):
FORMAT: 1:08
READ: 0:34 <- like real FDD
FMT+WRITE: 5:44 <- that's way too long

FF/HFE (80TRK/11SEC):
FORMAT: 1:06
READ: 1:12 <- again, like real FDD
FMT+WRITE: 7:31 <- that's even worse than in ST

I also think there is something wrong in side switching, as my favorite disk copy program - FastCopy PRO (a slightly newer and enhanced version of of FC III) reports "Drive B: is one-sided!" on any attempt to read or write any disk image on Gotek/FF.

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 19, 2018

Owner

Side change takes too long, definitely. That probably explains the 2x slow-down in reading 10-sector images. 11-sector images do not suffer because they have a sector interleave.

The horrendous slow writes could be verify failures and retries? Because those are slow as hell. Do you get any indication of problems in FastCopy III?

Interesting that Format is not that slow, probably it does not verify though!

I don't suppose you have a logic analyser? I may need to set this up and repro it myself and see what's going on on the super slow writes.

Owner

keirf commented Jan 19, 2018

Side change takes too long, definitely. That probably explains the 2x slow-down in reading 10-sector images. 11-sector images do not suffer because they have a sector interleave.

The horrendous slow writes could be verify failures and retries? Because those are slow as hell. Do you get any indication of problems in FastCopy III?

Interesting that Format is not that slow, probably it does not verify though!

I don't suppose you have a logic analyser? I may need to set this up and repro it myself and see what's going on on the super slow writes.

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 19, 2018

No errors at all, however during whole read and write process on 11 sector images it happens 2-3 times that one track takes noticeable more time to read/write. I don't know if in case of retry there would be more "ticks" from the piezo-speaker, but there is just a single one per track... just very slow.

Actually I have Saleae-clone 8-Ch logic analyzer and also a 2-Ch 25MHz 500MSa/s DSO.

TzOk83 commented Jan 19, 2018

No errors at all, however during whole read and write process on 11 sector images it happens 2-3 times that one track takes noticeable more time to read/write. I don't know if in case of retry there would be more "ticks" from the piezo-speaker, but there is just a single one per track... just very slow.

Actually I have Saleae-clone 8-Ch logic analyzer and also a 2-Ch 25MHz 500MSa/s DSO.

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 19, 2018

Owner

If not too tricky to set up could be worth grabbing a Saleae trace of the floppy bus: say IDX, SEL, WGATE, WDATA, RDATA, STEP, DIR, SIDE for one of the super-long fmt+write ops.

Owner

keirf commented Jan 19, 2018

If not too tricky to set up could be worth grabbing a Saleae trace of the floppy bus: say IDX, SEL, WGATE, WDATA, RDATA, STEP, DIR, SIDE for one of the super-long fmt+write ops.

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 19, 2018

These are 30s samples @ 1MSa/s in Logic format. Hope this will be helpful.
ATARI.zip

TzOk83 commented Jan 19, 2018

These are 30s samples @ 1MSa/s in Logic format. Hope this will be helpful.
ATARI.zip

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 19, 2018

Owner

Thank you kindly. I will take a look but possibly not before next week.

Owner

keirf commented Jan 19, 2018

Thank you kindly. I will take a look but possibly not before next week.

@keirf keirf added the bug label Jan 21, 2018

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 21, 2018

Here are some screenshots, from most interesting (I think) perspective:
.ST 9 sector/track:
st_write_9
.HFE 9 sector/track:
hfe_write_9
.ST 11 sector/track:
st_write_11

TzOk83 commented Jan 21, 2018

Here are some screenshots, from most interesting (I think) perspective:
.ST 9 sector/track:
st_write_9
.HFE 9 sector/track:
hfe_write_9
.ST 11 sector/track:
st_write_11

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 21, 2018

Owner

I will take a closer look at the trace myself but I'm pretty sure the problem is that the copy is writing a sector at a time (after doing the whole-track format) and that, by the time FF has written a sector back to USB, the next sector has passed under the virtual "drive head" and the copier has to wait a whole revolution (200ms) to find that sector header again, to write the data.

The fix will be to pretend that no time passed during writeback, and to restart the read stream exactly where the write left off. I will implement this tomorrow and supply you a patched firmware for testing.

Owner

keirf commented Jan 21, 2018

I will take a closer look at the trace myself but I'm pretty sure the problem is that the copy is writing a sector at a time (after doing the whole-track format) and that, by the time FF has written a sector back to USB, the next sector has passed under the virtual "drive head" and the copier has to wait a whole revolution (200ms) to find that sector header again, to write the data.

The fix will be to pretend that no time passed during writeback, and to restart the read stream exactly where the write left off. I will implement this tomorrow and supply you a patched firmware for testing.

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 21, 2018

You're probably right, take a look at this:
st_write_9_zoom
After each write there are 7 bursts of read, but 1 of them is not complete.

I also don't like the INDEX pattern... I think it should have constant frequency. After writing track header one INDEX pulse is missing. It should occur about the time that the SIDE1 pulse occurs.

TzOk83 commented Jan 21, 2018

You're probably right, take a look at this:
st_write_9_zoom
After each write there are 7 bursts of read, but 1 of them is not complete.

I also don't like the INDEX pattern... I think it should have constant frequency. After writing track header one INDEX pulse is missing. It should occur about the time that the SIDE1 pulse occurs.

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 22, 2018

Owner

Actually the gaps in RDATA are a concern, that could be a real bug in ST/IMG handling.

The irregular INDEX pattern is because INDEX gets suppressed during writeback to USB. This avoids FDC timeouts on long writebacks on some systems. The strict regularity of INDEX is not actually a concern in just about all cases, and we're about to make this worse on writes as we are going to resync the read stream to restart where the write ended. Given that there will be a wallclock delay between end of write and start of read (while the data is drained to USB stick) this will necessarily resync the INDEX pulse too. But nothing will care :)

I did not get round to making patches today, or indeed look at this at all. But it is the first job for tomorrow.

Owner

keirf commented Jan 22, 2018

Actually the gaps in RDATA are a concern, that could be a real bug in ST/IMG handling.

The irregular INDEX pattern is because INDEX gets suppressed during writeback to USB. This avoids FDC timeouts on long writebacks on some systems. The strict regularity of INDEX is not actually a concern in just about all cases, and we're about to make this worse on writes as we are going to resync the read stream to restart where the write ended. Given that there will be a wallclock delay between end of write and start of read (while the data is drained to USB stick) this will necessarily resync the INDEX pulse too. But nothing will care :)

I did not get round to making patches today, or indeed look at this at all. But it is the first job for tomorrow.

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 22, 2018

It is a pity that there is no external RAM on Gotek board to buffer read/write operations :( 64kB of internal SRAM is not much.

TzOk83 commented Jan 22, 2018

It is a pity that there is no external RAM on Gotek board to buffer read/write operations :( 64kB of internal SRAM is not much.

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 23, 2018

Owner

The gaps in RDATA I cannot reproduce. Also there are gaps in RDATA in your HFE traces too, it is merely that the period is much smaller (approx 300us as opposed to approx 30ms in the ST traces).

And now I look, there are similar dropouts in WDATA too. So I think your logic analyser is dropping batches of samples.

You might want to dig a bit further and confirm whether you can get dropout-free traces from your LA setup -- when WGATE is asserted (LOW) then WDATA should be continuously busy, with edges every few microseconds. Similarly, when the floppy bus is otherwise quiescent (no SEL, SIDE, STEP, WGATE changes) then RDATA should be continuously busy, with edges every few microseconds.

Owner

keirf commented Jan 23, 2018

The gaps in RDATA I cannot reproduce. Also there are gaps in RDATA in your HFE traces too, it is merely that the period is much smaller (approx 300us as opposed to approx 30ms in the ST traces).

And now I look, there are similar dropouts in WDATA too. So I think your logic analyser is dropping batches of samples.

You might want to dig a bit further and confirm whether you can get dropout-free traces from your LA setup -- when WGATE is asserted (LOW) then WDATA should be continuously busy, with edges every few microseconds. Similarly, when the floppy bus is otherwise quiescent (no SEL, SIDE, STEP, WGATE changes) then RDATA should be continuously busy, with edges every few microseconds.

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 23, 2018

Owner

Please test this patched firmware which restarts the read stream exactly where the write ended. Hence the host should immediately find the next sector and writes will be many times faster.

Please note the version number within this zip file is not modified, so it looks like 0.9.6a but it is not!

ff_44.zip

Owner

keirf commented Jan 23, 2018

Please test this patched firmware which restarts the read stream exactly where the write ended. Hence the host should immediately find the next sector and writes will be many times faster.

Please note the version number within this zip file is not modified, so it looks like 0.9.6a but it is not!

ff_44.zip

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 23, 2018

It is better now, but still much slower than the real FDD.
FF/ST (80TRK/9SEC):
READ: 1:06
FMT+WRITE: 1:20

FF/HFE (80TRK/9SEC):
READ: 0:34
FMT+WRITE: 2:13

Atari.zip

TzOk83 commented Jan 23, 2018

It is better now, but still much slower than the real FDD.
FF/ST (80TRK/9SEC):
READ: 1:06
FMT+WRITE: 1:20

FF/HFE (80TRK/9SEC):
READ: 0:34
FMT+WRITE: 2:13

Atari.zip

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 24, 2018

I've found this during HFE write:
write_lag

TzOk83 commented Jan 24, 2018

I've found this during HFE write:
write_lag

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 24, 2018

Owner

Ok that's interesting, I should look at your Saleae traces. :) Theory about that long gap is simply that the USB stick takes a long time to commit a write. Many USB sticks do that (random 'outage' of 100s of milliseconds).

Try your measurements with another USB stick!

Owner

keirf commented Jan 24, 2018

Ok that's interesting, I should look at your Saleae traces. :) Theory about that long gap is simply that the USB stick takes a long time to commit a write. Many USB sticks do that (random 'outage' of 100s of milliseconds).

Try your measurements with another USB stick!

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 24, 2018

Owner

Having looked at the traces, I conclude the following:

FF/HFE and FF/ST FMT + WRITE RESULTS:
The longer time vs a real FDD drive is the extra latency to write each sector to USB stick and the FlashFloppy to start serving up RDATA again (so that the host can see the next sector and so do the next write). Another USB stick may perform better. I don't envisage otherwise improving this in the near future.

FF/ST READ RESULT:
The cause is less obvious here, but one theory is that side switching and head stepping is taking too long and the host misses the first sector on the new track and has to wait one revolution. It may make sense to have a new mode which starts read stream on the new track at the same rotational position it ended on the previous track. I will sort out a patch for that for you to test. (If we're lucky that might help fastcopy pro's double-sided-drive test too.)
My problem with this theory is that I don't know why it would affect FF/ST but not FF/HFE.
This one is also worth just trying another USB stick and see how that performs...

Another thing I notice on the FF/ST Read result is that the side change happens some milliseconds later in the track than on FF/HFE. I will have to look and see if that could be due to some bug in FF/ST handling.

Owner

keirf commented Jan 24, 2018

Having looked at the traces, I conclude the following:

FF/HFE and FF/ST FMT + WRITE RESULTS:
The longer time vs a real FDD drive is the extra latency to write each sector to USB stick and the FlashFloppy to start serving up RDATA again (so that the host can see the next sector and so do the next write). Another USB stick may perform better. I don't envisage otherwise improving this in the near future.

FF/ST READ RESULT:
The cause is less obvious here, but one theory is that side switching and head stepping is taking too long and the host misses the first sector on the new track and has to wait one revolution. It may make sense to have a new mode which starts read stream on the new track at the same rotational position it ended on the previous track. I will sort out a patch for that for you to test. (If we're lucky that might help fastcopy pro's double-sided-drive test too.)
My problem with this theory is that I don't know why it would affect FF/ST but not FF/HFE.
This one is also worth just trying another USB stick and see how that performs...

Another thing I notice on the FF/ST Read result is that the side change happens some milliseconds later in the track than on FF/HFE. I will have to look and see if that could be due to some bug in FF/ST handling.

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 25, 2018

Owner

Please try the following patched firmware which fixes a few minor bugs and also starts next track exactly where previous was ended. Please test with default new behaviour, and also try with this line in FF.CFG:
synced-track-changes = no

ff_44_2.zip

Owner

keirf commented Jan 25, 2018

Please try the following patched firmware which fixes a few minor bugs and also starts next track exactly where previous was ended. Please test with default new behaviour, and also try with this line in FF.CFG:
synced-track-changes = no

ff_44_2.zip

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 25, 2018

This time no Saleae/Logic, just a stopwatch, because it seems everything is fine now, at least with .ST images.

FF/ST (80TRK/9SEC, synced-track-changes = no):
READ: 1:05
FMT+WRITE: 1:20

FF/ST (80TRK/9SEC, synced-track-changes = yes):
READ: 0:35
FMT+WRITE: 1:19

The synced-track-changes does not affect HFE and there writing time is still twice as long as it should be. No difference since last "ff_44" firmware. I didn't have chance to test with another flash-drive. The one I'm using (SanDisk Cruzer Fit SDCZ33 32GB) isn't apparently fast (SEQ R/W: 32/14 MB/s; 4K R/W: 4/2 MB/s).

FF/HFE (80TRK/9SEC, synced-track-changes = yes/no):
READ: 0:35
FMT+WRITE: 2:11

FastCopy PRO works ok, it was my fault, it has no ability to detect if drive is single or double sided, it is set in its configuration. I realized that when I was working with swapped drive, and I got the same error, but this time for the internal floppy.

TzOk83 commented Jan 25, 2018

This time no Saleae/Logic, just a stopwatch, because it seems everything is fine now, at least with .ST images.

FF/ST (80TRK/9SEC, synced-track-changes = no):
READ: 1:05
FMT+WRITE: 1:20

FF/ST (80TRK/9SEC, synced-track-changes = yes):
READ: 0:35
FMT+WRITE: 1:19

The synced-track-changes does not affect HFE and there writing time is still twice as long as it should be. No difference since last "ff_44" firmware. I didn't have chance to test with another flash-drive. The one I'm using (SanDisk Cruzer Fit SDCZ33 32GB) isn't apparently fast (SEQ R/W: 32/14 MB/s; 4K R/W: 4/2 MB/s).

FF/HFE (80TRK/9SEC, synced-track-changes = yes/no):
READ: 0:35
FMT+WRITE: 2:11

FastCopy PRO works ok, it was my fault, it has no ability to detect if drive is single or double sided, it is set in its configuration. I realized that when I was working with swapped drive, and I got the same error, but this time for the internal floppy.

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 25, 2018

Owner

I think we are done here and I will push my patches into master to roll into the next release, and close this ticket.

I'm still not sure why HFE Reads were always okay. Could you attach the HFE file you used for read timing tests here? I would like a quick look at it.

After that, I will close this ticket down.

Owner

keirf commented Jan 25, 2018

I think we are done here and I will push my patches into master to roll into the next release, and close this ticket.

I'm still not sure why HFE Reads were always okay. Could you attach the HFE file you used for read timing tests here? I would like a quick look at it.

After that, I will close this ticket down.

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 25, 2018

Here is the .HFE file I was using for read/write tests:
test.zip

TzOk83 commented Jan 25, 2018

Here is the .HFE file I was using for read/write tests:
test.zip

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 25, 2018

Owner

Okay the HFE has the sectors slightly squished together (smaller inter-sector gaps) which gives FlashFloppy more time to get to the next track.
It also, bizarrely has bogus extra IDAM areas (sector headers) for sector 66 at the start and end of the track. Why I do not know :)
Anyway, it looks sufficiently different than a normal 9-sector track, as generated for a ST file, that I think it adequately explains the performance difference.

Owner

keirf commented Jan 25, 2018

Okay the HFE has the sectors slightly squished together (smaller inter-sector gaps) which gives FlashFloppy more time to get to the next track.
It also, bizarrely has bogus extra IDAM areas (sector headers) for sector 66 at the start and end of the track. Why I do not know :)
Anyway, it looks sufficiently different than a normal 9-sector track, as generated for a ST file, that I think it adequately explains the performance difference.

@keirf keirf closed this Jan 25, 2018

keirf added a commit that referenced this issue Jan 25, 2018

keirf added a commit that referenced this issue Jan 25, 2018

floppy: When changing track, restart read stream where previous track…
… ended.

This is conditional on new config option, synced-track-changes, default "yes".
For old behaviour, add this line to FF.CFG:
synced-track-changes = no

See Github issue #44

@keirf keirf reopened this Jan 25, 2018

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 25, 2018

Owner

I have reopened this issue because I think another fix is that ST disks may typically have their sector ordering skewed from track to track. Could you try the attached firmware, again with synced-track-changes=yes and synced-track-changes=no? It only needs to be done for FF/ST, as FF/HFE is not affected. Thanks!

ff_44_3.zip

Owner

keirf commented Jan 25, 2018

I have reopened this issue because I think another fix is that ST disks may typically have their sector ordering skewed from track to track. Could you try the attached firmware, again with synced-track-changes=yes and synced-track-changes=no? It only needs to be done for FF/ST, as FF/HFE is not affected. Thanks!

ff_44_3.zip

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 25, 2018

Now it is always:
READ: 0:41
FMT+WRITE: 1:47

No difference if synced-track-changes is yest or no.

From what I know there is a sector skew between sides, so when switching side head falls in the inter-sector gap just before the sector begin. Tracks were always skewed, so stepping head to next track at the end of one would result on head reaching next track, just before its header begins. That is why gap between the last sector of the track and track header is so huge.

TzOk83 commented Jan 25, 2018

Now it is always:
READ: 0:41
FMT+WRITE: 1:47

No difference if synced-track-changes is yest or no.

From what I know there is a sector skew between sides, so when switching side head falls in the inter-sector gap just before the sector begin. Tracks were always skewed, so stepping head to next track at the end of one would result on head reaching next track, just before its header begins. That is why gap between the last sector of the track and track header is so huge.

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 25, 2018

Owner

I'm not sure whether to do the skew, or not, then. Obviously it slows down this particular test, but it should give more predictable performance overall... That said I'm not sure how prevalent the use of skew actually was on the Atari ST.

Owner

keirf commented Jan 25, 2018

I'm not sure whether to do the skew, or not, then. Obviously it slows down this particular test, but it should give more predictable performance overall... That said I'm not sure how prevalent the use of skew actually was on the Atari ST.

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 25, 2018

MFM drives always had skew, both between tracks and between surfaces/heads. It should "happened" in the inter-track gaps, so it should not add any extra delay, just "consume" the gap.

P.S.
Have you noticed the SIDE pulse on the oscillograms? Shouldn't it be ignored?

TzOk83 commented Jan 25, 2018

MFM drives always had skew, both between tracks and between surfaces/heads. It should "happened" in the inter-track gaps, so it should not add any extra delay, just "consume" the gap.

P.S.
Have you noticed the SIDE pulse on the oscillograms? Shouldn't it be ignored?

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 25, 2018

Owner

The skew does not change the bitcell offset of the first sector after the index, but the numbering of the sectors iyswim, for example (skew=2):
Track 0 Side 0: 1, 2, 3, 4, ...
Track 0 Side 1: 8, 9, 1, 2, ...
Track 1 Side 0: 6, 7, 8, 9, 1, 2, ...
And so on...

Owner

keirf commented Jan 25, 2018

The skew does not change the bitcell offset of the first sector after the index, but the numbering of the sectors iyswim, for example (skew=2):
Track 0 Side 0: 1, 2, 3, 4, ...
Track 0 Side 1: 8, 9, 1, 2, ...
Track 1 Side 0: 6, 7, 8, 9, 1, 2, ...
And so on...

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 25, 2018

It wasn't easy to setup but here you have - oscillograms from a REAL floppy drive:

ATARI_ST.rar.zip

Please delete the .zip extension, as this is a RAR archive, but .rar extension is not allowed here.

80/9 FMT+WRITE:
9_80_w
80/11 FMT+WRITE:
11_80_w

TzOk83 commented Jan 25, 2018

It wasn't easy to setup but here you have - oscillograms from a REAL floppy drive:

ATARI_ST.rar.zip

Please delete the .zip extension, as this is a RAR archive, but .rar extension is not allowed here.

80/9 FMT+WRITE:
9_80_w
80/11 FMT+WRITE:
11_80_w

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 25, 2018

Owner

Thank you for these :)

Owner

keirf commented Jan 25, 2018

Thank you for these :)

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 25, 2018

There are some interesting time relations. Every track gets SYNC only on SIDE 0. Side 0->1 changes before begin of track, so INDEX pulse occurs soon after SIDE change. While changing from SIDE 1 to SIDE 0 there is no INDEX pulse. Single track write cycle takes 3 REVs on SIDE 0 but only 2 on SIDE 1.

untitled

TzOk83 commented Jan 25, 2018

There are some interesting time relations. Every track gets SYNC only on SIDE 0. Side 0->1 changes before begin of track, so INDEX pulse occurs soon after SIDE change. While changing from SIDE 1 to SIDE 0 there is no INDEX pulse. Single track write cycle takes 3 REVs on SIDE 0 but only 2 on SIDE 1.

untitled

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 26, 2018

Owner

Since this specific issue is fixed in master I will close the issue again, but I will keep an open mind for sector skew for ST image files in future.

By the way the ~0.5ms SIDE pulses are weird. I guess they occur within the shadow of USB writeback on FlashFloppy so doesn't really matter much about being clever for them.

Owner

keirf commented Jan 26, 2018

Since this specific issue is fixed in master I will close the issue again, but I will keep an open mind for sector skew for ST image files in future.

By the way the ~0.5ms SIDE pulses are weird. I guess they occur within the shadow of USB writeback on FlashFloppy so doesn't really matter much about being clever for them.

@keirf keirf closed this Jan 26, 2018

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 26, 2018

I spent 12€ to see how it looks in HxC... it looks almost the same, with the difference - there is no direct support for .ST images!?! .HFE writing speed is slightly faster, but still too slow (~1:50 vs ~1:00 on real FDD).

ATARI_HxC.rar.zip
(Please delete the .ZIP extension, as this is a .RAR archive)

TzOk83 commented Jan 26, 2018

I spent 12€ to see how it looks in HxC... it looks almost the same, with the difference - there is no direct support for .ST images!?! .HFE writing speed is slightly faster, but still too slow (~1:50 vs ~1:00 on real FDD).

ATARI_HxC.rar.zip
(Please delete the .ZIP extension, as this is a .RAR archive)

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 26, 2018

Owner

You need to update to the HxC Alpha firmware (v3.1.1.1a). It certainly does directly support ST images, and has many performance improvements too, so you would need to re-measure HFE as well.

Owner

keirf commented Jan 26, 2018

You need to update to the HxC Alpha firmware (v3.1.1.1a). It certainly does directly support ST images, and has many performance improvements too, so you would need to re-measure HFE as well.

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 26, 2018

I was trying on ALPHA v3.1.1.2a... doesn't matter now. I've re-flashed your bootloader, so my HxC license is now gone. Maybe I'll buy another Gotek and another HxC license to do some experiments in future (when Jeff will fix OLED support). But for now I'm staying with FF :)

TzOk83 commented Jan 26, 2018

I was trying on ALPHA v3.1.1.2a... doesn't matter now. I've re-flashed your bootloader, so my HxC license is now gone. Maybe I'll buy another Gotek and another HxC license to do some experiments in future (when Jeff will fix OLED support). But for now I'm staying with FF :)

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Jan 29, 2018

One more finding...
Gotek/FF:
2018-01-29 20 39 07
Internal FDD:
2018-01-29 20 40 01

TzOk83 commented Jan 29, 2018

One more finding...
Gotek/FF:
2018-01-29 20 39 07
Internal FDD:
2018-01-29 20 40 01

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Jan 31, 2018

Owner

@TzOk83 Thank you for your help with this issue by the way. The resulting improvements have had broader impact than just ST, in particular HD images for IBM PC and others are working tonnes better. :)

I'm feeling close to a v1.0 non-alpha release now... I just need to fix back-to-back writes (which is mainly an Amiga disk-copy thing, haven't seen so much evidence of it elsewhere).

Owner

keirf commented Jan 31, 2018

@TzOk83 Thank you for your help with this issue by the way. The resulting improvements have had broader impact than just ST, in particular HD images for IBM PC and others are working tonnes better. :)

I'm feeling close to a v1.0 non-alpha release now... I just need to fix back-to-back writes (which is mainly an Amiga disk-copy thing, haven't seen so much evidence of it elsewhere).

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Mar 1, 2018

I'm afraid we went too far in optimizing it for FastCopy 3. I've prepared a 512kB file with a random content, and was reading this file from disk (to a RAM disk), and writing it to disk (from a RAM disk). Copy operation has been done from the GEM desktop, without any additional software.
On the other hand, I havent noticed any difference between instant and real time track change in this test.
operation | FlF | HxC | FDD |
READ......| 0:49| 0:39| 0:31|
WRITE.....| 1:46| 1:19| 1:19|
For the FDD I have tested TOS formatted disk and FastCopy PRO formatted one (with the "optimized" option enabled). There was no significant difference. Disk image format was .ST.

TzOk83 commented Mar 1, 2018

I'm afraid we went too far in optimizing it for FastCopy 3. I've prepared a 512kB file with a random content, and was reading this file from disk (to a RAM disk), and writing it to disk (from a RAM disk). Copy operation has been done from the GEM desktop, without any additional software.
On the other hand, I havent noticed any difference between instant and real time track change in this test.
operation | FlF | HxC | FDD |
READ......| 0:49| 0:39| 0:31|
WRITE.....| 1:46| 1:19| 1:19|
For the FDD I have tested TOS formatted disk and FastCopy PRO formatted one (with the "optimized" option enabled). There was no significant difference. Disk image format was .ST.

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Mar 1, 2018

Owner

Are these disks 720k (9 sectors per track)? Do you have a dump or HFE of a TOS formatted real disk (and perhaps even a FastCopy formatted one too), so that interleave and skew can be determined?

Read perf would be the easiest thing to go for first. That would probably fix write too, actually.

Owner

keirf commented Mar 1, 2018

Are these disks 720k (9 sectors per track)? Do you have a dump or HFE of a TOS formatted real disk (and perhaps even a FastCopy formatted one too), so that interleave and skew can be determined?

Read perf would be the easiest thing to go for first. That would probably fix write too, actually.

@keirf keirf added optimise and removed bug labels Mar 1, 2018

@keirf keirf reopened this Mar 1, 2018

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Mar 1, 2018

Yes, they were standard DD TOS disks - 80 tracks, 9 sectors/track. I'll make a HFE images of them, but not today. I guess, it will be better to make HFE images on HxC? Unfortunately I don't have the KryoFlux.

TzOk83 commented Mar 1, 2018

Yes, they were standard DD TOS disks - 80 tracks, 9 sectors/track. I'll make a HFE images of them, but not today. I guess, it will be better to make HFE images on HxC? Unfortunately I don't have the KryoFlux.

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Mar 1, 2018

Owner

Oh yes format an HFE image using standard TOS tools sounds good :) It should work on FlashFloppy as well as HxC, if it doesn't that's a bug really.

Owner

keirf commented Mar 1, 2018

Oh yes format an HFE image using standard TOS tools sounds good :) It should work on FlashFloppy as well as HxC, if it doesn't that's a bug really.

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Mar 3, 2018

Here are the HFE images formatted using TOS 1.62/2.06, FastCopy 3 and FastCopy PRO, under recent version of HxC firmware.
HFE_TEST.zip

Images contain one 512kB file. On HxC firmware writing to FC3 formatted image was 30sec faster, than writing to a TOS or FCPRO formatted one, so I guess only FC3 differs.

TzOk83 commented Mar 3, 2018

Here are the HFE images formatted using TOS 1.62/2.06, FastCopy 3 and FastCopy PRO, under recent version of HxC firmware.
HFE_TEST.zip

Images contain one 512kB file. On HxC firmware writing to FC3 formatted image was 30sec faster, than writing to a TOS or FCPRO formatted one, so I guess only FC3 differs.

@keirf keirf added the priority label Mar 4, 2018

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Mar 5, 2018

Owner

TOS and FCPRO formatted disks have track skew of 2. FC3 format has no skew.

I will prepare you a firmware which applies the track skew to *.ST files and you can test that :)

Owner

keirf commented Mar 5, 2018

TOS and FCPRO formatted disks have track skew of 2. FC3 format has no skew.

I will prepare you a firmware which applies the track skew to *.ST files and you can test that :)

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Mar 5, 2018

Owner

Here you go, this gives ST files the correct TOS skew:

ff_44_1.zip

Owner

keirf commented Mar 5, 2018

Here you go, this gives ST files the correct TOS skew:

ff_44_1.zip

@TzOk83

This comment has been minimized.

Show comment
Hide comment
@TzOk83

TzOk83 Mar 5, 2018

Nice - ST image: 512kB file read - 0:30, write - 1:28.

TzOk83 commented Mar 5, 2018

Nice - ST image: 512kB file read - 0:30, write - 1:28.

@keirf

This comment has been minimized.

Show comment
Hide comment
@keirf

keirf Mar 5, 2018

Owner

Excellent, write performance will improve a bit when I implement a universal buffer cache after v1.0 is out. But this is good for now. I will implement a proper fix to put into master and then close this ticket. The fix will be in next release (v0.9.15a).

Owner

keirf commented Mar 5, 2018

Excellent, write performance will improve a bit when I implement a universal buffer cache after v1.0 is out. But this is good for now. I will implement a proper fix to put into master and then close this ticket. The fix will be in next release (v0.9.15a).

@keirf keirf closed this in 0458965 Mar 6, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment