Support for multiple MCU chaining via I2C

[darenschwenke]

The architecture of Klipper is split in such a way as to lend itself to using SPI or I2C communication to 'smart' stepper drivers located on or near the individual motors.

These drivers could be purpose built such as the Mechaduino boards, or by using a low cost dedicated Ardunio + stepstick per axis. These devices could then simply be daisy chained which would further extend the capabilities of Klipper to more complicated control scenarios such as robotic control, hexapods, or multiple filament extrusion.

Further dividing the step generation workload among several Arduino's should also allow for higher step rates to be achieved.
At the very least, utilizing SPI/I2C instead of serial would enable an order of magnitude faster hardware based communication between a single host and slave device and without the timing jitter of a USB to serial adapter.

Please consider adopting a long term goal of extending the existing serial interface to support SPI/I2C.

[hg42]

I like the idea of daisy chaining especially with I2C which uses less wires.
(btw. the underlying link url is something on github)

However, Klipper is designed to calculate moves ahead of time and transfer this schedule to the controllers. So it will not work in real time out of the box.

But this is also the reason why USB jitter does not matter. The host transfers the planned moves ahead of time and the controller outputs those steps exactly at the times they are planned, using queues and time synchronization.

It may be possible to add real time commands (may be via some special time value of "now").
But I guess there will be several things in the design to be changed. E.g. things like getting an error if the schedule is missed, which would be the case for "now".

And at least some bigger parts of the host software would have to be rewritten.

[darenschwenke]

I'm not suggesting implementing real-time. The idea was multiple arduinos, each with it's own schedule, so the existing non real-time behavior can be maintained. The price point of the Mechaduino boards is still a bit high for my taste, but it's was a good example of chainable steppers each with an onboard arduino which could probably be updated to support the architecture of Klipper. For the short term, just need to allow different bits of the schedule on the host to be routed to different SPI/I2C client addresses. No route = all to one via serial. Route = to one, via some bus. The arduino code, minus utilizing the alternate bus and adding a byte for selecting the client address, could likely stay the same.

…

On Fri, Jan 12, 2018 at 9:42 AM, Harald ***@***.***> wrote: I like the idea of daisy chaining especially with I2C which uses less wires. (btw. the underlying link url is something on github) However, Klipper is designed to calculate moves *ahead of time* and transfer this schedule to the controllers. So it will not work in real time out of the box. But this is also the reason why USB jitter does not matter. The host transfers the planned moves ahead of time and the controller outputs those steps exactly at the times they are planned, using queues and time synchronization. It may be possible to add real time commands (may be via some special time value of "now"). But I guess there will be several things in the design to be changed. E.g. things like getting an error if the schedule is missed, which would be the case for "now". And at least some bigger parts of the host software would have to be rewritten. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#119 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADCSQmeBtJaP2frnxeGv1b_KMbCJMOmWks5tJ29xgaJpZM4RbajD> .

[hg42]

I'm not suggesting implementing real-time

I concluded that from

robotic control

so, of which advantages of SPI or I2C over USB are you thinking?

[hg42]

so something like:

[stepper_x]
step_pin: arduino1:ar54
...

[stepper_y]
step_pin: arduino2:ar54
...

[mcu arduino1]
serial: i2c:<channel1>:<address1>
...

[mcu arduino2]
serial: i2c:<channel1>:<address2>
...

[darenschwenke]

Instead of a single arduino handling the queue for all axis and generating the pulse trains for all axis, the work can be divided up so each arduino only handles the part of the queue it needs to. Should result in raising the maximum step rate proportionally.

I also really like the idea of chaining them.

[hg42]

you can already delegate each stepper to a different MCU. And it handles only it's own queue, as you said with maximum performance.

Only chaining isn't possible (but you can probably use USB hubs). And you cannot use SPI/I2C for that.

I am not sure, if a shared bus is a good idea, but it would probably work if the total bandwidth is sufficient.

[hg42]

for example I recently used:

[stepper_x]
step_pin: ramps:ar54
...

[stepper_y]
step_pin: tronxy:PC6
...

[mcu ramps]
serial: /dev/serial/by-id/usb-Arduino__www.arduino.cc__Arduino_Mega_2560_752323334353513081D0-if00
...

[mcu tronxy]
serial: /dev/serial/by-id/usb-1a86_USB2.0-Serial-if00-port0
...

(note: I use a patch, to use the first mcu as main clock sync instead of needing a nameless [mcu])

[dragonnn]

If you need something controlled in real-time connected it to Rassberry Pi directly, Rassberry Pi have SPI and I2C too, so for example handling LCD-s over SPI will work much better if it is connected directly to Pi.

[darenschwenke]

Ok, I'm speaking imprecisely. :) Shared bus, not daisy chaining. I suppose you could also do daisy chaining like the Dotstar modules over SPI by acting like a bit shifter. Probably not worth it.

I2C is a true shared bus with 127 devices at up to 2Mhz with your average Arduino capable of 1Mhz.

SPI is also a shared bus, but each added device consumes another pin on the master so the benefits of chaining decrease a lot. I have sent uncompressed video over it, but the Arduino maxes out at 8Mhz.

[darenschwenke]

@hg42 Yes, your original addressing scheme looks like it would work. Another layer of abstraction might be clearer but it's your party.

[hg42]

USB can be much faster (though the Pi is not that fast, on the other end the serial adapter limits the single connection) and hubs and cables are cheap.
May be I2C can be made more stable, but that depends.

[darenschwenke]

I suppose you are right, but I'll explain my vision.
An arduino pro mini + step stick will both nicely fit on the back of a nema 17 stepper. Creative placement of pin headers lets you stack them directly with no pcb, gives you software control over microstepping/enable, and only needs one passive: a bypass cap for the stepstick.
The stepstick itself has a solderable backplane which I've used in the past for cooling and mounting them. Provided you are satisfied with running your steppers at 12v, that's just 12v, ground, and your two I2C lines required and can be chained from stepper to stepper.

[hg42]

@darenschwenke I already had a similar vision of distributed controllers beforehand. You added I2C to the picture which I appreciate.

It may not be fast enough, but if not, there could be one bus for XYZ (I eventually need 3xZ = 3...5) and one for the extruders (max. 3 colors + black + white + ... = 1...5++).

Lets estimate: I2C=2MHz / serial=250000 is about 8 controllers fully saturated. In my tests a controller had no problem with 3 steppers.
So 8x3 = 24 steppers should work.

[darenschwenke]

Confining this enhancement to I2C comes with the added benefit that a master write to channel 0 is received by all slaves simultaneously. Perhaps this could simplify slave synchronization?

[hg42]

that would be nice (however the sync works very well already).

One disadvantage:
the boot loader does not support I2C, right? Then you need a second connection via USB or ISP.
The same is bothering me on LPC176x where I also need two interfaces because update only works over another serial port (DFU).

[jonathanperret]

There seem to exist I2C AVR bootloaders, though.

[hg42]

Thanks for the info, I didn't search for it yet, too man things to do...

[darenschwenke]

I think I've been unusually lucky with I2C.
Per the specification, the higher data rates should reduce the maximum cable length to the point where mounting an Arduino per axis shouldn't work. So perhaps this is a dead end.

[dragonnn]

I2C is designed for inter PCB communication. It is not designed to run over wires and doesn't have any like differential signal to protect again EMI. A much better idea would be RS485, chaining can be easy implementing via adding to every packet an 'address' bit.

[darenschwenke]

RS485 would work, but needs another chip and some passives. At that point I might as well use a nano and USB hub based communication which already works.

[dragonnn]

USB isn't so immune to EMI as RS485. One of my prints failed because of EMI breaking down the communication. I would not feel save wiring USB directly to a stepper motor.

[hg42]

@dragonnn did the print fail on Klipper?
The crc check should prevent this as long as the repeated transmission doesn't take too much time.
Even then Klipper would pause the print (I think, early prints with too much segments were pausing after printing several holes), otherwise it would be considered as a bug.
I think USB also retransmits.
In a noisy environment the whole communication should still be safe, but slowed down or paused.

Even prints with usual firmware are crc checked. There usually are a sequence number ("N") and a checksum ("*") while printing, depending on the host software.

From my experience EMI with 3D printing was always caused by power spikes which resetted some part of the electronics, probably the CPU. This happend on Arduino and SMOOTHIE with different cables. The event was always triggered by switching on a device like a bigger motor or lighting which happened to use the same power strip. Using another mains connection some meters away always worked. This was reproducible (ABAB test).
Though, I cannot be 100% sure about the exact reasons.

[dragonnn]

@hg42 no this wasn't on Klipper, regular Marlin + Octoprint but CRC doesn't help in USB because what happens - the device disappears form the system and in dmesg you find this:
[170522.353802] usb 1-1-port2: disabled by hub (EMI?), re-enabling...
Probably the time need to reinitialize USB and reconnect to it would already trigger a timeout.

[dragonnn]

I don't think that even klipper can handle an USB disconnect during printing without aborting it.

[hg42]

would be a nice test :-) (I currently have a broken setup, so anyone else? pull and push the cable)

I assume a simple USB transfer error (= bit error = crc error) would be handled by retransmission.

I once used USB extensively with cameras (uEye, on Windows) and found that EMI on the signal lines simply caused retransmission. But the power lines are much more sensitive, because they are often used as power of the device. We found that using a separate power supply for the device made it more stable.
I guess spikes on the power line trigger the reset circuit or make the CPU loop or halt in some way, especially for firmware that tests certain conditions (like max temperatures) and doesn't skip short issues and then goes into a safe state (you might see heaters switched off etc.).
You can test this easily for devices that show some led blinking or similar while working (e.g. smoothieware toogles a LED in idle loop).

I also remember my EMI failures (some years ago) made the USB device disappear. I am not sure if it came back without intervention from myself (which would be a reset). Since then I always use a different power cable for the printer and didn't have another failure. My cheap 5m USB cable (amazon basics) from the workstation to the printer often ran along the motor cables but didn't make any problems (using printrun). Any communication problems were reproducible (e.g. Cura), definitely not EMI.

[KevinOConnor]

I don't foresee any issues with using different communication protocols between mcu and host - for example, the PRU already uses a "rpmsg" channel for communication. It will, of course, require the hardware and some minor host and micro-controller code to use that channel.

I'm closing this issue as I don't think github issues are a good way to track these types of long-term ideas - I fear they'll linger in an open state forever.