Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DaynaPort not working in branch develop #191

Closed
pfuentes69 opened this issue Aug 15, 2021 · 93 comments · Fixed by #184
Closed

DaynaPort not working in branch develop #191

pfuentes69 opened this issue Aug 15, 2021 · 93 comments · Fixed by #184

Comments

@pfuentes69
Copy link

Current develop branch fails for DaynaPort and auto mount. While accessing the DP from the Mac, RASCSI log shows a warning "Received a CmdRetrieveStats command for a non-daynaport unit SCDP". Probably all related to the new device management.

Branch daynaport3 still works fine, not having these 2 issues.

If I may do a suggestion, it would be good to have a dedicated branch named "experimental" that is stable enough (at least supporting DP and auto mount) and not subject to ongoing works.

uweseimet added a commit that referenced this issue Aug 15, 2021
@uweseimet
Copy link
Contributor

uweseimet commented Aug 15, 2021

@pfuentes69 Most likely fixed in the uweseimet_develop branch. Would be great if you could test whether the problem is gone with this branch. In case you test with this branch please read the information on major changes at the beginning of #184.
In general, it's the nature of a develop branch that it is not necessarily stable and free of issues.

@uweseimet
Copy link
Contributor

@pfuentes69 By the way, do you have any instructions how to use the DP on a 68K Mac (Performa 630)? I'm not a Mac user, and only use this Mac for testing, but I don't know how to install the DP with it.

@pfuentes69
Copy link
Author

@uweseimet Hi!
I won't be able to test today, but certainly I'll do it ASAP.

The main issue is that the instructions to setup the DaynaPort (https://github.com/akuker/RASCSI/wiki/Dayna-Port-SCSI-Link) tell people to use the develop build, which logically is not meant to be stable, so this is tricky.

That's why I suggested to create an "experimental" branch, which is kind of an "stable develop"... at least until DaynaPort is part of the master branch.

@pfuentes69
Copy link
Author

pfuentes69 commented Aug 15, 2021

@pfuentes69 By the way, do you have any instructions how to use the DP on a 68K Mac (Performa 630)? I'm not a Mac user, and only use this Mac for testing, but I don't know how to install the DP with it.

Please check the link in the above comment. These are the best instructions so far, but at least until yesterday would only work with the daynaport3 branch, not with the develop branch as stated in the wiki

@pfuentes69
Copy link
Author

BTW... should the auto mount work again too with the latest develop?

@uweseimet
Copy link
Contributor

uweseimet commented Aug 15, 2021

@pfuentes69 Now I remember actually reading these instructions, but it's inconvenient for me to use my Pi based on these instructions, because for my Mac I use a Pi that is usually not connected by ethernet by only by Wifi.

@pfuentes69
Copy link
Author

@pfuentes69 Now I remember actually reading these instructions, but it's inconvenient for me to use my Pi based on these instructions, because for my Mac I use a Pi that is usually not connected by ethernet by only by Wifi.

A guy posted instructions in Discord yesterday to setup on wireless.

https://docs.google.com/document/d/1I4iFgQSYBCAi0xo9GqFOMDDFmt683b2TNaBw9j1fkb4/edit#

I didn't test myself though...

@uweseimet
Copy link
Contributor

I'm afraid I don't know. I have no idea when the daynaport branch was merged into develop the last time, and which changes have only been applied to the daynaport branch in the meantime.
The problem with the daynaport code is that this code inherits from the disk code, even though it is not a disk, and that there are hard-coded daynaport checks in other classes. This is prone to side-effects.

@rdmark
Copy link
Member

rdmark commented Aug 15, 2021

Thanks for raising this issue in the tracker, pfuentes69. (I'm the guy who wrote the wifi setup guide :) )
I'm compiling the latest revision of uweseimet_develop (ec9216a) on my Pi now, and will report back how well it works with my Macs.

@uweseimet
Copy link
Contributor

@rdmark Cool! Note that if there are is an issue it might have been there even earlier than my commits to the develop branch. Anyway, by comparing the DP code (it's not that much) with the daynaport3 branch one should be able to identify any relevant divergency.

@rdmark
Copy link
Member

rdmark commented Aug 15, 2021

Agreed, I was looking at a diff between develop and daynaport3 this morning, and the changeset didn't look too overwhelming. Should be possible to figure it out relatively quickly, unless there is a ton of coupling elsewhere in the codebase.

@uweseimet
Copy link
Contributor

The fact that different device types (hard disk, bridge, daynaport) are not cleanly separated is one of the issues I have been working on. But a clean separation still requires a lot of changes, and I don't think it will be possible very soon.

@pfuentes69
Copy link
Author

Thanks for raising this issue in the tracker, pfuentes69. (I'm the guy who wrote the wifi setup guide :) )

I'm compiling the latest revision of uweseimet_develop (ec9216a) on my Pi now, and will report back how well it works with my Macs.

Hi "guy" ;)

I'll be out from home for several days, but I'll try to test too when back. I'll check for your results!

For me the basic need is auto mount and DaynaPort setup... let's see if Uwe could put that on tracks again.

Pedro

@rdmark
Copy link
Member

rdmark commented Aug 15, 2021

@uweseimet So I tested on two of my vintage Macs, one using the legacy MacTCP 2.1 networking stack, one using the more modern Open Transport 1.3 stack. Unfortunately, when trying to ping any IP address, it fails with this message in the logs:

Aug 15 23:20:16 raspberrypi RASCSI[12722]: [2021-08-15 23:20:16.359] [info] The DaynaPort interface has been ENABLED.
Aug 15 23:20:16 raspberrypi kernel: rascsi_bridge: port 1(ras0) entered blocking state
Aug 15 23:20:16 raspberrypi kernel: rascsi_bridge: port 1(ras0) entered forwarding state
Aug 15 23:20:17 raspberrypi RASCSI[12722]: [2021-08-15 23:20:17.087] [warning] Received 'Mode Select'
Aug 15 23:20:17 raspberrypi RASCSI[12722]: [2021-08-15 23:20:17.087] [warning] Operation Code: [0A]
Aug 15 23:20:17 raspberrypi RASCSI[12722]: [2021-08-15 23:20:17.087] [warning] Logical Unit 0, PF 0, SP 0 [00]
Aug 15 23:20:17 raspberrypi RASCSI[12722]: [2021-08-15 23:20:17.087] [warning] Reserved: 00
Aug 15 23:20:17 raspberrypi RASCSI[12722]: [2021-08-15 23:20:17.088] [warning] Parameter List Len 2A
Aug 15 23:20:17 raspberrypi RASCSI[12722]: [2021-08-15 23:20:17.088] [warning] Reserved: 80
Aug 15 23:20:17 raspberrypi RASCSI[12722]: [2021-08-15 23:20:17.088] [warning] Ctrl Len: 00000000
Aug 15 23:20:17 raspberrypi RASCSI[12722]: [2021-08-15 23:20:17.088] [warning] Error occured while processing Mode Select command 0A

On a positive note, the interface is able to get initialized and it can resolve its own IP address, and the Daynaport hardware test passes. So it seems to work 'better' than on the latest develop branch code. But as soon as you try to ping any IP (such as the bridge on the Pi, or a public Google DNS server) you get timeouts or error 500s, while the above "Error occured while processing Mode Select command 0A" warning block appears over and over in the logs.

@rdmark
Copy link
Member

rdmark commented Aug 15, 2021

As a side note, my 600MB hard drive image that I created as part of the initial setup script (default settings) doesn't work well on this latest code in uweseimet_devel, when mounted as a non-system disk. Mac OS (7.6.1) keeps complaining that the file system is damaged over and over until I dismount it. The very same image worked fine with develop and daynaport3 code. Perhaps something's you're already on top of, but I thought I'd mention it.

@uweseimet
Copy link
Contributor

uweseimet commented Aug 16, 2021

@rdmark Thank you for testing. I also ran more tests today and fixed some (not only new) issues, e.g. that "Received Mode Select", was reported on write/verify commands and not only on mode select itself (see the
uweseimet_develop diffs if you are interested).
There was also an issue with the new write protection feature for regular hard disk media, which may have been the cause for your Mac reporting a damaged file system.

With the latest uweseimet_develop codebase my Mac is fine. I tested with a 600 MB hard disk, copied applications to it and could successfully start them.
If you still have issues, a logfile excerpt on "trace" log level would be great. Just start rascsi with the "-g trace" option to set the log level, or use "rasct -g trace" to change the log level for a running instance of rascsi just before you start the operation which should be logged in detail.

Regarding the daynaport, I minimized the differences in scsi_daynaport.cpp compared to the daynaport3 branch (only a few lines differ now, but they should not matter), and fixed a potential issue but (at least with my current setup) cannot test it. I also verified the (unfortunately) hard-coded references to the daynaport device in sasi_ctrl.cpp and scsi_ctrl.cpp.
If possible, please give it a try with the current uweseimet_develop. At least the bogus "Received Mode Select" should be gone.

@uweseimet
Copy link
Contributor

uweseimet commented Aug 16, 2021

@rdmark In the meantime I installed OpenTransport and the DaynaPort drivers and set up the network as explained on your page. rascsi displays a lot of DaynaPort-related trace output, but I'm afraid I don't know how to finalize my setup. I'm not familiar with Mac networking, AppleTalk etc. I'm afraid.
The SendEcho test shows a device name "Jupiter" (no idea where this name comes from) and reports success with transmitting packets.

@rdmark
Copy link
Member

rdmark commented Aug 16, 2021

@uweseimet I've tried building 21c726e and can confirm that the "Received Mode Select" messages have been suppressed. I'm still unable to ping any IP from the Mac, including the bridge interface itself. Each request times out almost instantly.

Checked out & building 0f0c657 now (which takes forever on a Pi Zero -- I think it overheats very quickly and slow down when compiling code) to see if that makes a difference. If not, I'll play around with the log levels and see if I can get some insights where the requests get hung up.

Thank for trying to set up a Mac environment to test yourself. Did you choose the right Ethernet interface in the TCP/IP control panel? The emulated DaynaPORT interface is usually called "Alternative Ethernet". If you have wired Ethernet on your Pi, you can also try the 'official' method.

@uweseimet
Copy link
Contributor

uweseimet commented Aug 16, 2021

@rdmark I just succeeded in getting my Mac network with DP to work, after switching from DHCP to manual configuration. I somehow figured out, just as you said, that I need the alternative ethernet. Using the Netscacpe Communicator I can now access hosts in my local network. External addresses also work, but whenever I enter a nameserver address in the configuration dialog the system hangs up immediately.
But except for this nameserver configuration issue I don't have any obvious issues.

Yes, building on a Pi Zero is slow. I usually use the Eclipse IDE on a regular Linux x86 PC for time-consuming changes (this also gives you refactoring support etc.) and then check out on the Pi (usually a Pi 4) for compling and testing on the real hardware. And I usually run rasctl on my x86 PC and connect remotely to the Pi with "rasctl -h HOSTNAME". All in all one can do a lot of the development work in a powerful environment this way.

0f0c657 should be fine, at least it is for me.

Any idea what might cause my nameserver issue?

By the way, I really appreciate that you set up the instructions for using the network interface of the Pi Zero. This helped me a lot.

@rdmark
Copy link
Member

rdmark commented Aug 16, 2021

I'm back online again, so I think 0f0c657 is a winner (as far as DP is concerned)!

IMG_3693

@uweseimet
Copy link
Contributor

:-) Great news! Can you please tell me how your TCP/IP configuration looks like on the Mac? I would like to try to resolve my nameserver problem.

@rdmark
Copy link
Member

rdmark commented Aug 16, 2021

@uweseimet This is what my MacTCP configuration looks like. Give me a few min to switch over to my OT setup. I should make sure this code works over there too.

@rdmark
Copy link
Member

rdmark commented Aug 16, 2021

IMG_3694

@uweseimet
Copy link
Contributor

@rdmark Looks as if you are using a different tool for setting up the nameserver IP. I will try that one later most likely.

@pfuentes69
Copy link
Author

@rdmark Looks as if you are using a different tool for setting up the nameserver IP. I will try that one later most likely.

It seems you're using OpenTransport.
To see that up config screen you'd have to use the network selection tool in "Apple extras" to switch the networking to "Classic Networking"

@uweseimet
Copy link
Contributor

@pfuentes69 Can you also confirm that with the current uweseimet_develop branch your DP is working again?

@rdmark
Copy link
Member

rdmark commented Aug 16, 2021

@uweseimet You should only really use MacTCP / Classic Networking on really old machines that can't use OT (68000 Macs, System 7.1 or earlier). Here's what my OT setup looks like. It works great too!
IMG_3695

@pfuentes69
Copy link
Author

@pfuentes69 Can you also confirm that with the current uweseimet_develop branch your DP is working again?

Sorry but I'm in a business trip for part of the week. I could try Thursday probably.

@rdmark
Copy link
Member

rdmark commented Aug 16, 2021

@uweseimet BTW, the 'damaged file system' errors seem to have been resolved as well. I can copy files back and forth just fine (although just cursory testing so far.)

@rdmark
Copy link
Member

rdmark commented Aug 19, 2021

@uweseimet I use their webapp client, but there are standalone clients for computers and smartphones too. Go to https://github.com/akuker/RASCSI/wiki and follow the Discord link under "Want to chat?". It will take you through the account creation process if you don't have one.

@uweseimet
Copy link
Contributor

@rdmark OK, I'm in. Which channel would be best?

@uweseimet
Copy link
Contributor

Ah, rascsi-dev you said.

@rdmark
Copy link
Member

rdmark commented Aug 19, 2021

@uweseimet rascsi-general, rascsi-dev, and rascsi-support are three useful channels depending on purpose.

@rdmark
Copy link
Member

rdmark commented Aug 19, 2021

BTW, I am 'hulksmash' on Discord :)

@uweseimet
Copy link
Contributor

I more or less kept my real name, i.e. uweseimet :)

@uweseimet
Copy link
Contributor

For the record: The reason I could not reproduce the issue is that I compiled for debug with -O0. When compiling with -O3 I also observe this error.

@pfuentes69
Copy link
Author

Just in case this wasn't said before @rdmark confirmed that the las working build he tested before the issue was the cdb2890

@uweseimet
Copy link
Contributor

uweseimet commented Aug 19, 2021

@pfuentes69 I can reproduce the issue when compiling an optimized binary (see above). For testing I don't optimize, and the code behavior was different.
I just de-inlined two methods, and now also the optimized uweseimet_develop branch works for me.

If you are interested in what needed to be changed see the diffs berween e3bc353 and b32f324. I have no idea why inlining was causing a problem in the optimized code. My guess is that this is a compiler bug. Or is it a documented feature of C++ that inlining code that accesses a nested data structure causes undefined behavior? What actually happened was quite well-defined: GetBlockCount() returned disk.size instead of disk.blocks.

@pfuentes69 I tested the optimized build and the problem is gone in the latest uweseimet_develop codebase.

@pfuentes69
Copy link
Author

@uweseimet Just a quick message to tell you that I tested the latest build and it looks good!
Disk attachments with auto mount, DaynaPort (only tested web)... all working like a charm apparently.

Thanks a lot!

@uweseimet
Copy link
Contributor

@pfuentes69 Good to hear that it also works for you now!

For the record: With gcc 10.3.0 on Ubuntu 32 bit there is exactly the same issue. With gcc 8.3.0 on Raspberry Pi OS 64 bit there is no issue with the inlined code.

@rdmark
Copy link
Member

rdmark commented Aug 19, 2021

Tested both my HD images, all five known good CD images, and daynaport on my SE. No errors, warnings or unexpected behaviors observed so far with b32f324

@rdmark
Copy link
Member

rdmark commented Aug 20, 2021

One marginal bug that I found is that the webui isn't able to attach a daynaport device:
The logs say:

Aug 20 04:32:14 raspberrypi RASCSI[3361]: [2021-08-20 04:32:14.140] [info] Validating: cmd=ATTACH, id=4, unit=0, type=UNDEFINED, filename='', device name='', params=''
Aug 20 04:32:14 raspberrypi RASCSI[3361]: [2021-08-20 04:32:14.143] [info] Executing: cmd=ATTACH, id=4, unit=0, type=UNDEFINED, filename='', device name='', params=''

@uweseimet
Copy link
Contributor

@rdmark Fixed with latest uweseimet_develop:

[2021-08-20 08:22:14.879] [info] Validating: cmd=ATTACH, id=6, unit=0, type=SCDP, filename='', device name='', params=''
[2021-08-20 08:22:14.879] [info] Executing: cmd=ATTACH, id=6, unit=0, type=SCDP, filename='', device name='', params=''
[2021-08-20 08:22:14.882] [info] Note: rascsi_bridge already created
[2021-08-20 08:22:14.887] [info] Tap device ras0 created
[2021-08-20 08:22:14.889] [info] Added new SCDP device, ID: 6 unit: 0

@rdmark
Copy link
Member

rdmark commented Aug 20, 2021

The webui can attach and detach daynaport devices now. Great fix! 910d330

@pfuentes69
Copy link
Author

Hi @uweseimet @rdmark
On my side I could also test the Web UI and I'm pretty OK. I can't test today AppleTalk, but having seen the previous behaviour, with IP working fine as it does, seems to me that the current build is good to go.
I think that this issue can be closed, but I strongly recommend to "freeze" this build in a branch that is not affected by additional changes, so other people interested in DaynaPort can use it.
Thanks guys to make my SE/30 smile again!

@uweseimet
Copy link
Contributor

uweseimet commented Aug 20, 2021

@rdmark @pfuentes69 Except for bug fixes my intention is to not add new features to the uweseimet_develop branch, so that it can safely be merged into develop.

akuker pushed a commit that referenced this issue Aug 21, 2021
…d error handling, 64 bit OS support, fixed issues (#184)

* Device type unification, support of removable media

* Added support for .hdr extension

* Removable flag cleanup

* Manpage update

* Enriched PbOperation with PbDevice

* Added file size to PbImageFile

* Added device list support

* Set image_file

* Make remote interface more robust by ignoring SIGPIPE

* Return status only once

* Fixed typo

* Error handling update

* When starting rascsi parse everything before attaching devices

* Added dry run mode

* Comment update

* Updated logging

* Added Device base class, Disk class inherits from it

* Renaming

* Use vectors for controllers and disks, as preparation for using maps

* Updated file support handling

* Comment update

* DaynaPort and Bridge inherit from Device instead of Disk

* ProcessCmd() now works with devices instead of disks

* Renaming

* Added DeviceFactory

* Improved factory

* Comment update

* protected disk_t

* Code cleanup, added translations

* Device name can be set for rascsi

* rasctl can set device name

* Manpage update

* Manpage update

* Formatting update

* Check for missing name

* Initialize fd

* Initialize type

* Fixed string length issue

* Updated capacity formatting

* Fixed typo

* Split PbDevice into device and device definition

* Added TODO

* Renaming

* Renaming

* Device types can be explicitly specified with -t (no FILE:TYPE syntax anymore)

* Fixed compile-time issue

* Removed unused Append mode, updated read-only handling

* Type handling and manpage update

* Cleanup

* rasctl parser cleanup

* Review

* Constructor update

* Added .hdr (SCRM) support to web interface, tested web interface

* Default folder can be set remotely

* Removed deprecated operation

* DETACH supports all parameters in order to detach all devices

* include cleanup

* Logging should not depend on NDEBUG, for RaSCSI it is not peformance-critical

* INFO is default log level

* Exception renaming

* Updated GetPaddedName()

* Inheritance update

* Added BlockDevice class

* Removed unused code

* Updated typedefs

* Revert "Updated typedefs"

This reverts commit 546b462.

* Removed unused code

* Fixed warnign

* Use standard C++ integer types, use streams to resolve printf data type issues

* Added TODOs

* Added TODO

* Renaming

* Added TODO

* Added TODO

* Improved dry-run

* Code cleanup

* Updated handling of unknown options, code review and cleanup

* Manpage update

* Added PrimaryDevice

* Include cleanup

* Added pure virtual methods

* Comment updates

* Split rasutil

* Replaced some occurrences of BOOL

* Removed obsolete RASCSI definition in xm6.h

* Removed unused code, updated TODOs, replaced BOOL

* Added capacity check (issue #192)

* Fixed (most likely) #191

* Fixed wrong error messages

* For root the default image folder is /home/pi/images, updated error handling

* Dynaport code review

* Improved error handling

* Implemented READ CAPACITY(16)

* Comment update

* Commands can be 16 bytes long

* Implemented READ/WRITE/VERIFY(16)

* Comment update

* Renamed method to reflect the name of the respective SCSI command

* Do not created devices during dryRun

* Fixed padding of SCSIHD_APPLE vendor and product

* Initial implementation

* Updated ReportLuns

* Byte count update

* Fixed typo

* Finalized REPORT LUNS

* Removed TODO

* Updated TODO

* TODO update

* Updated device factory

* Comment update

* 64 bit update, tested on Ubuntu 64 bit system

* Removed assertion

* SCSI hard disks always have Apple specific mode pages (resolves issue #193)

* Error messsage update, 64 bit cleanup

* Reduced streams usage

* Updated handling of device flags

* MOs are protectable

* Removed duplicate error code handling

* Removed duplicate code

* Fixed CmdReadToc buffer overflow (#194)

* Added naive implementation of GET EVENT STATUS NOTIFICATION to avoid wranings

* HD must set removable device bit if the media is removable

* Removed duplicate logging

* Updated daynaport additional length

* Removed broken daynaport REQUEST SENSE. Successfully tested with my Mac.

* EnableInterface should not always return TRUE

* Updated Inquiry

* Updated LUN handling

* Replaced incorrect free by delete

* Updated comments and write-protection handling

* Made default HD name consistent

* STATUS_NOERROR is default

* Fixed Eject

* More eject handling updates

* Manpage updates

* Logging update

* Changed debug level

* Logging update

* Log capacity of all media types

* Logging update

* Encapsulated disk.blocks

* Encapsulated sector size

* Added overrides

* Added more overrides

* Fixed error message

* Fixed typos

* Fixed logging

* Added logging

* Use PrimaryDevice when calling Inquiry

* Comment update

* Changed default buffer size for testing

* Reverted last change

* Removed debug output

* De-inlined methods because optimized code did not work with them inlined

* Web interface can attach Daynaport again

* Improved handling of read-only hard disks

* Fixed issue with "all" semantics of DETACH

* rasctl supports adding removable media devices without providing a filename

* Removed unused flag in PbDeviceDefinition

* Updated rasctl output for ecjected media (resolves issue #199)

* Validate default folder name when changing default folder
@uweseimet
Copy link
Contributor

@pfuentes69 uweseimet_develop has been merged into develop. If possible, please double check whether develop works as expected. If it does, please close this ticket.

@rdmark
Copy link
Member

rdmark commented Aug 22, 2021

Develop 0bd12e9 works great on my SE. Tested daynaport networking with MacPing and MacWeb.

@pfuentes69
Copy link
Author

Habemus papam!

Happily closing the issue.

Thanks Uwe!

@uweseimet
Copy link
Contributor

@rdmark I would like to get back to the bridge change for wlan0 we talked about above. Do you see any issue with merging this change, even though there are still open questions about how rascsi uses the bridge in general? I don't think these question can be answered soon.

@rdmark
Copy link
Member

rdmark commented Aug 23, 2021

@uweseimet I have no objections. It would be a great utility to have.

@rdmark
Copy link
Member

rdmark commented Aug 23, 2021

BTW this is related #204

@uweseimet
Copy link
Contributor

@rdmark I just merged this change (only ctapdriver.cpp differs) to uweseimet_develop.

@rdmark
Copy link
Member

rdmark commented Aug 23, 2021

@uweseimet Super neat! I tested it in a2c2362, removing the lines from /etc/rc.local and now rascsi brings up both ras0 and rascsi_bridge automatically when attaching a DP device. Someone else should definitely test over eth0 too since I only have a wlan0 environment. Also worth considering is a setup with multiple ethX wlanX interfaces on one Pi?

@uweseimet
Copy link
Contributor

@rdmark Currently the code only deals with eth0 and wlan0. For multiple interface it might be best to make the interface names configurable, with eth0 and wlan0 being the default interfaces. Would that make sense?

@rdmark
Copy link
Member

rdmark commented Aug 23, 2021

@uweseimet yep I agree with that approach; eth0/wlan0 are the overwhelmingly most common interfaces on a Pi I think, but there will be exceptions.

@uweseimet
Copy link
Contributor

@rdmark Please see #207 as a suggestion for how this configuration option should work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants