ICD host adapter compatibility for Atari MegaSTE, internal drive connector

[uweseimet]

Atari's ACSI protocol supports a subset of SCSI commands, but not all SCSI command classes. There are host adapters that permit to connect SCSI peripherals to the ACSI connector. Most host adapters (the ICD compatible ones) and some drivers support sending all SCSI command classes. In order to achieve that they use a custom SCSI opcode ($1F). This opcode is followed by the full regular SCSI command to be sent.
While it it not possible to directly connect RaSCSI to the Atari's ACSI port, one can implement the ICD extension. This will enable users of an Atari MegaSTE, which has an internal SCSI connector but nevertheless is limited to 1 GB drive capacity, to use images of more than 1 GB. No host adapter hardware would be needed.

[uweseimet]

@anodynesoftware I would appreciate a quick comment from your side on whether this could really work. I just thought about this, but may be missing something. I happen to get a MegaSTE soon, and it would be nice if one could use RaSCSI as an internal hard disk drive (even several) without any capacity restriction.

[anodynesoftware]

On 1 Feb 2022 at 10:31, Uwe Seimet wrote: @anodynesoftware I would appreciate a quick comment from your side on whether this could really work. I just thought about this, but maybe missing something. I happen to get a MegaSTE soon, and it would be nice if one could use RaSCSI as an internal hard disk drive without any capacity restriction.

I'm not a hardware guy, but I looked at my copy of the MegaSTe schematics and I can't find anything that looks like a SCSI connector. I can see the ACSI connector (a DB19). And there's no SCSI chip as far as I know. How were you planning to physically connect the RaSCSI to the MegaSTe? Roger

[uweseimet]

@anodynesoftware I'm also not a hardware guy :). I thought the stock MegaSTE has an internal SCSI host adapter? Also see https://www.atari-forum.com/viewtopic.php?t=20133, for instance.

[czietz]

There is an internal (and optional(!), i.e., not included in every MegaSTE) ACSI-SCSI host adapter. I am, however, uncertain about its concrete limitations. Certainly, as already pointed out by @uweseimet it only allows SCSI opcodes up to 0x1F, which could be worked around by implementing the ICD extension on the RaSCSI. But maybe there are other parts missing for full SCSI compatibility.

While I have a MegaSTE and even an (albeit never tested) ACSI-SCSI host adapter, I don't have a RaSCSI to simply test this.

[uweseimet]

@czietz As far as I know this adapter only supports a single device (one SCSI ID), but till now I never really cared. But provided there is no general issue with connecting RaSCSI (e.g. with SCSI parity), one SCSI ID does not really matter, because you can stil have several LUNs, so that you can add more than one internal device with just one RaSCSI board.
I hope I will get my MegaSTE soon, so that I can test whether the internal adapter works with RaSCSI at all. But I am confident it will. RaSCSI and ICD host adapters work fine, by the way.
For now I was just wondering whether this could work at all. I like the idea a lot, but when you like something too much you might miss obvious problems ;-).

[czietz]

The MegaSTE adapter does not support parity. But I hope that is not mandatory for the RaSCSI? Nor does the adapter support "advanced" SCSI features such as initiator identification.

[uweseimet]

@akuker Can you comment on whether the RaSCSI hardware requires parity?

@czietz RaSCSI does not require initiator identification. Otherwise it would not work with the ICD adapter, which does not support initiator identification. (Since RaSCSI is mostly software-controlled if there was a need one could change the C++ code in order to also work without initiator identification. Parity might also just be a software issue, but I don't know this for sure.)

[uweseimet]

A Megafile44 with RaSCSI inside should also work, because it also has an internal host adapter. You would not need a MegaSTE in this case. Quite an exotic setup, though, and I guess there are not that many Megafile44.

[akuker]

@akuker Can you comment on whether the RaSCSI hardware requires parity?

For output - RaSCSI will calculate parity and set the "DP" pin accordingly. For input, RaSCSI doesn't check parity. It should work fine without the parity signal driven.

[uweseimet]

@akuker Sounds good!

[uweseimet]

It works! :-) My MegaSTE arrived today (in exchange for some bottles of wine), and when connecting RaSCSI to the internal SCSI connector I can make use of all SCSI command classes. This means, for instance, that one can make use of 1 GB capacity, even though the internal SCSI adapter does not support this on its own. But with built-in ICD support with RaSCSI this limitation is removed.
In addition, even though the internal adapter is limited to a single SCSI ID, with RaSCSI one can now "connect" up to 8 devices to this adapter, by making use of LUNs. The boot drive must be LUN 0, but other devices can have any LUN.

https://www.hddriver.net/forum/viewtopic.php?t=253 has the details and a screenshot.

[czietz]

That's impressive 👍!

I wonder if this would even work inside my SH204 harddrive. It contains a SCSI host adapter, too. Well, I guess I might have to purchase a RaSCSI one day to find out...

[uweseimet]

@czietz It may work, indeed, as long as there is no incompatibility with the host adapter and RaSCSI.

@anodynesoftware Does your Daynaport driver work with a Daynport device that has a LUN > 0? I guess not, but this would probably be useful. Since TOS requires LUN 0 for the boot drive, if one wants to add a RaSCSI Daynaport adapter it would have to use a LUN > 0.
Yes, I know, the real Daynaport device most likely does not support to be configured as LUN > 0 ;-). But from a user's perspective, being able to add an emulated Dynaport to the internal host adapter of a MegaSTE and still being able to boot from an emulatedhard disk would be very useful.

[anodynesoftware]

On 3 Feb 2022 at 3:55, Uwe Seimet wrote: @anodynesoftware Does your Daynaport driver work with a Daynport device that has a LUN > 0? I guess not, but this would probably be useful. Since TOS requires LUN 0 for the boot drive, if one wants to add a RaSCSI Daynaport adapter it would have to use a LUN > 0. Yes, I know, the real Daynaport device most likely does not support to be configured as LUN > 0 most likely ;-). But from a user's perspective, being able to add an emulated Dynaport to the internal host adapter of a MegaSTE and still being able to boot from hard disk would be very useful.

I don't disagree, but please don't think I'm prepared to implement this. I wrote the original STinG driver a long time ago mostly for my own benefit. I then added a FreeMiNT driver, partly out of curiosity, and then donated it to FreeMiNT. It was, of course, written for real Daynaport SCSI/Link devices. It seems to work fine. But given the limitations of the SCSI implementation on Atari systems, I think that Ethernet support over SCSI is a bit of a dead-end anyway. For the last month I have been doing some debugging as a courtesy to Steven. So far all I have found (after spending many hours on something I'm really not interested in) is what I believe to be a shortcoming in RaSCSI Daynaport support (I know you have a different opinion). I'm continuing to look at Steven's problem out of courtesy to him, but unless I discover something clearly wrong in the driver, I won't be doing any updates to the code. But that shouldn't be a problem: since it's all open-source, anyone can fix any problems they perceive in the current code or add any features they wish.

[uweseimet]

@anodynesoftware Since your driver is open source, can you please add a link to the source repository to this ticket? This will make things easier if somebody wants to follow up. Maybe this should also be pointed out in the MiNT mailing list, in order to at least make transparent that there is room for improvements and that RaSCSI is prepared to support them.
If somebody wants to add support for LUNs > 0 I guess only a minor change is required. Making the LUN number configurable may be more work than a more flexible LUN support as such.

I just created PR #641, which is going to lift the LUN restriction for the Daynaport emulation. Currently for the Daynaport RaSCSI only supports LUN 0, but this restriction is rather pointless. If there should ever be a driver with Daynaport support for LUNs > 0, with this change it should work out of the box with RaSCSI.

[uweseimet]

@akuker Does RaSCSI also work with the Zero 2 W?

[anodynesoftware]

On 3 Feb 2022 at 6:51, Uwe Seimet wrote: @anodynesoftware Since your driver is open source, can you please add a link to the source repository to this ticket?

It's part of the FreeMiNT repository, exactly where you'd expect it to be :-).

If somebody wants to add support for LUNs > 0 I guess only a minor change is required. Making the LUN number configurable may be more work than a more flexible LUN support as such.

I doubt there's work to do to add LUN support, since it uses the SCSIDRV interface. Probably just parsing the setup args and changing the contents of the data passed to the various SCSIDRV functions.

[uweseimet]

@anodynesoftware That's what I assumed. The SCSI Driver interface unfortunately does not have explicit LUN support (otherwise up to 32 LUNs would be possible), but the LUN has to be coded (legacy) in the CDB, where only 3 bits are available. (Since SCSI 2 the correct method is to pass the LUN to the device with an IDENTIFY message.)
The required change would have to add the LUN information to the CDB.

[akuker]

@akuker Does RaSCSI also work with the Zero 2 W?

@uweseimet - Yes. it works quite well!! I even made a custom RaSCSI board for it....
https://github.com/akuker/RASCSI/tree/develop/hw/rascsi_zero_1p0

[uweseimet]

@akuker Sounds good! Looks as if I need an additional RaSCSI board :). I assume the non-custom board also works for the Zero 2 W? What's the advantage of the custom board.
Looks as if because of the chip shortage it's easier these days to get a RaSCSI board than a Zero 2 W ;-).

[uweseimet]

Users who would like to connect RaSCSI to their Atari can find more details on https://www.hddriver.net/en/rascsi.html. I Intend to extend this webpage with more information on topics in particular interesting when using Ataris with RaSCSI.

[uweseimet]

@anodynesoftware There may be a solution to support LUNs > 0 for old drivers even without changing these drivers, at least for the Atari: An AUTO folder program could map SCSI IDs to LUNs. This program would have to implement the SCSI Driver interface in order to intercept SCSI Driver calls and would (based on a configuration file, for instance) modifiy the CDBs for the In() and Out() commands according to the configured mapping. The LUNs are only relevant for the outgoing commands, i.e. for data returned by an SCSI command the original device or LUN is not relevant.
Do I miss anything or would tihs work?

[anodynesoftware]

On 4 Feb 2022 at 12:12, Uwe Seimet wrote: @anodynesoftware There may be a solution to support LUNs > 0 for old drivers even without changing these drivers, at least for the Atari: An AUTO folder program could map SCSI IDs to LUNs. This program would have to implement the SCSI Driver interface in order to intercept SCSI Driver calls and would (based on a configuration file, for instance) modifiy the CDBs for the In() and Out() commands according to the configured mapping. The LUNs are only relevant for the outgoing commands, i.e. for data returned by an SCSI command the original device or LUN is not relevant. Do I miss anything?

That would not work for SCSI-3-compatible devices, of course, which select the LUN via an IDENTIFY message. So it theoretically wouldn't work for CD/DVD players for example. But few devices use multiple LUNs anyway, so I'm guessing that your idea is: . driver calls SCSIDRV, passing SCSI id N and pointer to CDB . termres program modifies scsi ID to 0 and CDB to contain LUN = N . RaSCSI sees CDB for id 0 and selects emulated device based on LUN in CDB (then ignores LUN) I think that should work. Roger

[akuker]

Sounds good! Looks as if I need an additional RaSCSI board :).

You do!!! If you email me your address, I'll drop one in the mail when I get my next batch.

I assume the non-custom board also works for the Zero 2 W? What's the advantage of the custom board.

Ya, both boards are interchangeable. The RaSCSI Zero is missing the ribbon cable (for space reasons). It also has parts on both sides of the board, so a little more difficult to assemble. The only advantage of the board is that it is smaller.

Looks as if because of the chip shortage it's easier these days to get a RaSCSI board than a Zero 2 W ;-).

It really is!! I can't wait for this chip shortage to be over.

[uweseimet]

@anodynesoftware Thank you for your feedback.

The IDENTIFY message may need special care, indeed. I would have to verify the current method HDDRIVER uses to evaluate the CDB. Essentially HDDRIVER's SCSI Driver extracts the LUN from the CDB and sends an IDENTIFY message for the that LUN. (One can see the message in the rascsi trace logs.) This is only done for SCSI, because ACSI does not support messaging.

In practice, the only relevant use case for re-mapping IDs would IMO be the internal adapter of the MegaSTE with RaSCSI connected to it. Messaging would not be an issue here. RaSCSI takes the LUN from the CDB, and the IDENTIFY message just overrides it, if the message is sent at all.

For an ordinary user it would probably be too complicated to configure mappings of this kind, though. And the LUNs have to be consecutive in order to satisfy the SCSI standard. (This is enforced by RaSCSI.) Otherwise REPORT LUNS, for instance, would not work.

Anyway, this approach may be an option to lift the restrictions of the MegaSTE's adapter even more.

[uweseimet]

@akuker Very cool, thank you! I might need to mention that the board would have to be assembled, because I have absolutely no experience with soldering.
My address is public anyway, just see https://www.hddriver.net/en/contact.html.

[uweseimet]

@czietz Any idea if an (external) SCSI host adapter for the Atari without ICD support would be easier/cheaper to produce than an adapter with ICD support? Most likely it does not really make a difference?
A RaSCSI board that can be directly connected to the ACSI port would be nice ;-).

[czietz]

@czietz <https://github.com/czietz> Any idea if an (external) SCSI host adapter for the Atari /without/ ICD support would be easier/cheaper to produce than an adapter with ICD support? Most likely it does not really make a difference?

Maybe Wolfgang Förster is better qualified to answer this, as he actually builds and sells ACSI-SCSI-adapters. However, nowadays you would put all the logic inside a CPLD and it would not make much of a difference to add ICD support, I guess. FYI: Atari published a reference implementation (schematic and PAL equations) for an adapter *without* ICD support: https://docs.dev-docs.org/htm/search.php?find=Atari+ACSI%2FDMA+Integrati Regards Christian

[uweseimet]

@akuker Looks as if this is going to be your next hardware project, a RaSCI board for Atari's ACSI bus :-).