Plugin For Jogging the Machine w/o Sender active

[5ocworkshop]

I often find myself in the shop and needed to jog the machine to get the gantry out of the way, but if I'm cutting on the machine I don't take my laptop out with me.

I'd like a way to move the machine in a limited way from buttons or a joystick etc. without having to get my laptop and fire up the sender.

On the grblHAL2000 board there is a RaspberryPi header that includes connecting the UART on the Pi to Serial2 on the Teensy. Originally I was thinking that the Teensy behaved like the arduino and that the USB serial would be replicated on the serial port by default, so I could just send some basic jog commands from the Pi when the Laptop wasn't connected, and move the machine.

It turns out the Teensy does not mirror the USB serial communications on any physical serial pins by default.

I'm thinking of using the Pi on serial2 on the Teensy and a plugin, perhaps, to allow jogging but I'm unsure of the best way to do it. I would like to be able to use something like a USB IR remote (that I have a few of) on the Pi side and maps those key presses to various GCODE and grbl commands. Does this sound like a reasonable way to do this? Or would it make more sense to somehow enable bidirectional communication on serial2 when the USB isn't active and just talk directly to the grblHAL core that way as if it was just another sender?

[terjeio]

Originally I was thinking that the Teensy behaved like the arduino and that the USB serial would be replicated on the serial port by default, so I could just send some basic jog commands from the Pi when the Laptop wasn't connected, and move the machine.

Teensy has the USB stack on the MCU, not on an external chip as you have found out. However, a plugin can do all kinds of stuff by maniuplating function pointers including modifying how streaming is handled. Replicating the output should be easy, in the plugin init function save the (relevant) HAL stream pointers, call the serial init function (this returns a struct with pointers to the serial stream functions) and save away this too. Add output functions to the plugin and update the HAL with those, in these call the functions in the saved away pointers and you are good to go:

#include "grbl/hal.h"
#include <string.h>

static io_stream_t usb;
static io_stream_t serial;

static void my_write (const char *data)
{
    usb.write(data);
    serial.write(data);
}

static void my_write_all (const char *data)
{
    usb.write_all(data);
    serial.write_all(data);
}

void my_plugin_init (void)
{
    memcpy(&usb, &hal.stream, sizeof(io_stream_t));
    memcpy(&serial, serialInit(115200), sizeof(io_stream_t));

//    hal.stream.write = my_write;
    hal.stream.write_all = my_write_all;

    serial.write("Hello!" ASCII_EOL);
}

Handling input is not that easy, IMO intermingling usb and serial data is not a good idea. Attaching a function to the foreground process (via grbl.on_execute_realtime) is better. From that read the serial input and submit it for processing.
There are two methods you can use for that:

1. Buffer the input up to a newline and submit it via grbl.enqueue_gcode(). This is a relatively safe way to do it as commands are only accepted when the controller is in Idle, Jog or Tool change states.
2. Enable the keypad plugin and send single character commands to it, many commands are supported including jogging. Send input to the keypad_enqueue_keycode() function. For jogging a jog cancel command, 0x85, must be sent on key up to terminate jogging.

Note that real-time commands are handled differently from regular input, you'll have to redirect those separately by calling serial.set_enqueue_rt_handler() with the function that is to receive them.

Does this sound like a reasonable way to do this? Or would it make more sense to somehow enable bidirectional communication on serial2 when the USB isn't active and just talk directly to the grblHAL core that way as if it was just another sender?

The issue here is to detect "USB not active" - it can be done by listening in to real time commands from it and if quiet for a certain amount of time let serial take over control? IMO the methods above is better as it allows jogging even when USB is in control.

It can be noted that I have made a driver level protocol that allows stream switching via handshaking over a pin, the downside to this is that it requires sender cooperation. BTW the Teensy driver does not yet support this.

Another option available for the Teensy is to connect via the second USB port on Phils Pro board and add code support for that. It would be nice if someone would tackle that challenge...

[5ocworkshop]

Terje,

As always, thanks for the detailed suggestions.

Before I saw this, I was poking around looking at your MPG code. Would it also be an option for me to use the signal line that shifts the serial input to the pins used by the MPG controller when I want to use my IR remote?

I have the IR remote talking to the Pi, and I've written a little python stub to generate basic jog commands and output them on the serial port. Right now I've just plugged the USB link to the Pi directly in to the Teensy but I think if I use the UART on the Pi instead of the USB serial, and I connect a pin to the MPG input trigger pin, I should be able to send them over serial pins, even if the laptop is connected?

(EDIT) I'm just seeing now that you have a reference to it requiring "sender cooperation". Where can I learn more about that? I'm guessing IOSender has this today but it isn't widely deployed elsewhere? Ah, I also somehow glossed over you saying that the Teensy driver does not yet support this. Which driver can I study for an example today?

[5ocworkshop]

I just did a fresh pull to a clean project and I'm still having issues getting the UART serial communications to work.

I have UART8 TX and RX pins from the teensy connected to my Pi. I do not have CTS/RTS pins connected.

If I run a basic sketch on the teensy board I get the expected output at 115200 on those pins at the Pi, so know the pins are good. I tried setting these in my_machine.h:

> #define UART_PORT           5 // See hardware definition in uart.c
> #define USB_SERIAL_CDC      0 // 1 for Arduino class library and 2 for PJRC C library. Comment out to use UART communication. 

I also updated the pins in uart.c for the correct pins for uart8:

     // .pin = 21,
        .pin = 34, // JAC For Pi Header
        .mux_val = 2,
        .select_reg = &IOMUXC_LPUART8_RX_SELECT_INPUT,
        .select_val = 1
    },
    .tx_pin = {
        // .pin = 20,
        .pin = 35, // JAC for Pi Header

No luck. Nothing at the pins.

I also tried commenting out the USB_SERIAL config, and left the UART_PORT. That also did not work.

Any ideas on what to try next?

[5ocworkshop]

I think it has to do with using Serial8. Looking at the details for UART5 in the uart.c source, file I notice that some of the values differ from this from provided by pjrc:

https://github.com/PaulStoffregen/cores/blob/master/teensy4/HardwareSerial8.cpp
CCM_CCGR3, CCM_CCGR3_LPUART5(CCM_CCGR_ON),
Jumped out at me in particular but I'm in way over my head at this point, so I'm only guessing.

[5ocworkshop]

I am happy to report the dragon has been slayed:

Port /dev/ttyAMA0, 21:38:24

Press CTRL-A Z for help on special keys


GrblHAL 1.1f ['$' or '$HELP' for help]

Here is the entry I created for Serial8 on the Teensy 4.1. I apologize for not submitting a patch but I think it best someone who really understands it and how to add it to the desired structure make the necessary changes so it can coexist with various board/pin combinations:

// JAC adjusted per https://github.com/PaulStoffregen/cores/blob/master/teensy4/HardwareSerial8.cpp
// Applies to grblHAL2000 board.  Serial8 is routed to the 40 pin Pi format expansion header on the board

static const uart_hardware_t uart5_hardware =
{
    .port = &IMXRT_LPUART5,
    .ccm_register = &CCM_CCGR3,
    .ccm_value = CCM_CCGR3_LPUART5(CCM_CCGR_ON),
    .irq = IRQ_LPUART5,
    .irq_handler = uart_interrupt_handler,
    .rx_pin = {
        // .pin = 21,
        .pin = 34, // JAC For connection to Pi Header
        .mux_val = 1,
        .select_reg = &IOMUXC_LPUART5_RX_SELECT_INPUT,
        .select_val = 1
    },
    .tx_pin = {
        // .pin = 20,
        .pin = 35, // JAC for connection to Pi Header
        .mux_val = 1,
        .select_reg = &IOMUXC_LPUART5_TX_SELECT_INPUT,
        .select_val = 1
    }
};

[5ocworkshop]

Now that I've got the serial working, I just tested a sketch that confirm that I can pull the QEI-SELECT pin high from the Pi while I have the serial link open. That should, in theory, allow me to take over as the sender and send jogging and other gcode from he Pi just like the MPG project.

What is involved in completing support for that in the Teensy driver and is there a way I can help make that happen?

[terjeio]

I apologize for not submitting a patch but I think it best someone who really understands it and how to add it to the desired structure make the necessary changes so it can coexist with various board/pin combinations:

That is ok, I'll have to check the datasheet - it is likely that UART5 can be routed to more than one set of pins.

What is involved in completing support for that in the Teensy driver and is there a way I can help make that happen?

You may try to port the code from the MSP432 driver. Most of the code is guarded by MPG_MODE_ENABLE.
See the "Improved startup sequence" section in the MPG&DRO readme for details about how the MPG pin is used.

[5ocworkshop]

Thanks, I'll take a look now.

[5ocworkshop]

I quite like the idea of replacing the data read functions in the keypad plugin with equivalent serial functions, but that's more than I know how to do at the moment. More learning to do but making progress. Very pleased to have the second serial port operating now, as it will improve debug ability without interfering with the main grbl console.

[5ocworkshop]

It is correct that by defining UART=5 and ensuring the port is setup, I can also go back enabling serial over USB for the main grbl console and the UART port will still be exposed to grblHAL for separate debug or application use?

I'm trying to understand what the HAL commands would be to send and receive data from the alternate port, or if I should just be using direct Arudino like calls to it?

[terjeio]

I quite like the idea of replacing the data read functions in the keypad plugin with equivalent serial functions, but that's more than I know how to do at the moment.

It is quite simple:

#include "grbl/hal.h"
#include "keypad/keypad.h"
#include "uart.h"

#include <string.h>

static io_stream_t usb;
static io_stream_t serial;

static void my_write (const char *data)
{
    usb.write(data);
    serial.write(data);
}

static void my_write_all (const char *data)
{
    usb.write_all(data);
    serial.write_all(data);
}

static bool forward_char (char c)
{
    keypad_enqueue_keycode(c);

    return true; // "claim" character - nothing will be entered into the input buffer.
}

void my_plugin_init (void)
{
    memcpy(&usb, &hal.stream, sizeof(io_stream_t));
    memcpy(&serial, serialInit(115200), sizeof(io_stream_t));

    serial.set_enqueue_rt_handler(forward_char);

//    hal.stream.write = my_write;
    hal.stream.write_all = my_write_all;

    my_write("Hello!" ASCII_EOL);
}

Enable the keypad plugin in my_machine.h, all incoming characters will be delivered directly to the plugin. Then single jog character commands can be sent. Terminate a jog by CTRL-X (or by sending 0x85). Toggle between fast, slow and step mode with h. There are $-settings for jog distances and feed rates.

It is correct that by defining UART=5 and ensuring the port is setup, I can also go back enabling serial over USB for the main grbl console and the UART port will still be exposed to grblHAL for separate debug or application use?

Yes, you can also enable in-build debug mode by uncommenting #define DEBUGOUT in grbl/config.h - this exposes debug_write() that can be used to write strings to the UART port. IIRC this is currently only supported by the iMXRT1062 driver. The downside to this is that it claims the port.

I'm trying to understand what the HAL commands would be to send and receive data from the alternate port, or if I should just be using direct Arudino like calls to it?

Use the serial "HAL", in the example above you interact with the port via the calls (function pointers) available in the "serial" io_stream_t struct. This exposes many functions for handling the stream. E.g. call serial.write("hi!"); to write a string. A bit like you would do with C++...

[5ocworkshop]

Terje, I guess I was over thinking it. I'll go test the above code now, if this starts reading from the serial port as is I'll be estatic!

Thanks again for all your time and effort on grblHAL. It's really elegant the way you've structured it, and I greatly appreciate being able to work "around the edges" on things like this without touching the core code.

[terjeio]

if this starts reading from the serial port as is I'll be estatic!

Well, it is working for me here ;-)

It's really elegant the way you've structured it, and I greatly appreciate being able to work "around the edges" on things like this without touching the core code.

Yep - the whole ting has become like a specialized OS, problem is how to get users to grasp what is possible. Part of it is following up requests such as this, I appreciate that you have chosen to spend time on it yourself and not dump it on me...

[5ocworkshop]

It worked great for reading. I've got a copy of the console output scrolling up my screen on the Pi serial terminal program.

I can't seem to get it to accept an of the keypresss though. I've got IOSender open and I've unlocked and homed the machine from IOSender so I am Idle. When I press 'h' or 'H' in the serial terminal, nothing happens though. Is there a second character I need to send (I've tried enter) after the key to "send" it?

[5ocworkshop]

Terje,

I agree with the specialized OS analogy. I've actually been thinking about this a lot and wondering how I can help. I was discussing this last night with Drew Marles who also contributes here. I was wondering if there is some way I could help create a series of flow diagrams that would visually represent your vision of how the various components interact and how to hook in to the process to add features etc.

Every time I have a new idea or question your structure already seems to be built to accommodate it. I'm happy to volunteer time to try and create a visual map of the relationships between elements of the grblHAL environment.

Having worked my way through grblMega to shoehorn in some basic RGB alarm code and now the RGB plugin, I have a much better picture than when I started. If you think this would be of value, I'll take a swing at it and come back to you for review and correction.

[5ocworkshop]

Yep - the whole ting has become like a specialized OS, problem is how to get users to grasp what is possible. Part of it is following up requests such as this, I appreciate that you have chosen to spend time on it yourself and not dump it on me...

One idea may be to take a few examples, like the RGB plugin and what I am hoping to achieve with the IR remote control, and showcase how the modular HAL nature of the system allows even those fairly new to programming to experiment with plugins and develop useful applications without having to understand the deep core of grblHAL.

Prior to doing my early RGB stuff on grblMega5x, I had never touched an Arduino and virtually all off my programmatic (if you can even call it that) experience was in bash scripts. It is a testament to the quality of the abstraction and support you provide, supported by the power of the PlatformIO IDE to rapidly write, build, flash test and debug, that I could contribute to the project so far.

The Pi header on the grblHAL2000 board is part of that enthusiasm. Getting this serial link working will open up a whole new world for folks to use things like python, lirc, bash, node-red etc. to do plugin style projects and add features and functions to their machines. The header as intended to support a Pi Zero mounted via the 40 pin header, right on the back of the board. With the announcement the other day of the Pi Zero 2 W, and it's quad core processor, there is even more accessible potential there.

There are a lot of brilliant people in the CNC community who are not on the device programming side of things, but I think if they understood just how accessible it has become, they may start bringing interesting and innovative ideas and plugins to the project. I'll do what I can to help develop materials to convey the grblHAL story.

[5ocworkshop]

I just realized, I need to be sending the hex for the keypresses, rather than just pressing keys in the terminal, is that right?

[terjeio]

When I press 'h' or 'H' in the serial terminal, nothing happens though.

'h' toggles the jog mode, 'H' should home the machine. There is no feedback provided per default. There is an "event" that can be hooked into to provide feedback, on_jogmode_changed, I use this on my CO2 laser to write the mode to an OLED.

If you try 'F' what happens then? For me the controller enters jog mode in Y direction, I then press CTRL-X to stop it.
Tip: if you add serial.write_char(c); in the forward_char() function you will get an echo of what you have typed.

One idea may be to take a few examples, like the RGB plugin and what I am hoping to achieve with the IR remote control, and showcase how the modular HAL nature of the system allows even those fairly new to programming to experiment with plugins and develop useful applications without having to understand the deep core of grblHAL.

I'll do what I can to help develop materials to convey the grblHAL story.

Great if you will do this - it is a plus when others join in the effort.

I just realized, I need to be sending the hex for the keypresses, rather than just pressing keys in the terminal, is that right?

Nope, it is plain keypresses. 0x85 is a bit difficult to send from the keyboard (perhaps possible via some ALT plus numeric keys combination?), and is the reason I added CTRL-X as an alternative.

[5ocworkshop]

Tip: if you add serial.write_char(c); in the forward_char() function you will get an echo of what you have typed.

Funny, I just added that before I saw your note and I'm getting a bit of garbage on the terminal. Sounds like maybe wiring. Checking now.

[5ocworkshop]

I am getting strange characters and then I seem to have a stability issue. Trying to capture the character now.

On reboot, IOSender can connect and home, but if I disconnect it can not seem to reconnect. So something isn't quite right I think. The problem seems to start when I issue a keypress.

[5ocworkshop]

As background (while I reboot things again), I am usng Putty from windows to ssh to the Pi and then running minicom. I have minicom in VT102 mode, and I think Putty is in vt100 mode, in case we're left had scratching about what exact character we see.

Wiring appears to be solid.

Pressing 'F' in the terminal displays H¯ó

[terjeio]

On reboot, IOSender can connect and home, but if I disconnect it can not seem to reconnect. So something isn't quite right I think. The problem seems to start when I issue a keypress.

That is odd, I have no such problems here.

[5ocworkshop]

AHA! 'H' just homed for the first time. I had exited IOsender so the console lines wouldn't keep flooding. Restarting IO Sender and trying again to confirm it still works. And yes it does.

[5ocworkshop]

'F' still giving that strange character when echoed back to the screen.

'R' echos as Hh¹½K and nothing happens.

[5ocworkshop]

Terminal emulation maybe? I'm VT102 in minicom on the serial port from the P to the teensy. Putty is the default on windows, and I'm ssh'd to the Pi. I think it's vt100 for default.

[5ocworkshop]

I stopped IOSender to make it easier to capture the output. After three or four of the unusual strings we got back, I'm not getting an echo and now IOSender can't open the port again so I think we have hung the Teensy.

Aha, that could be my mistake. I was using serial.write(c) not serial.write_char(c) in forward_char(c).

[5ocworkshop]

Ok no I am seeing F and L and R in the console correctly.

H worked to home the machine, and IOSender says it is Idle and Ready but the other keys aren't triggering anything. Odd.

[5ocworkshop]

I just noticed that 'H' is defined directly in keypad.c but that all the others come from the #define's in keypad.h.

keypad.h appears to be correctly included at the top of the plugin but I wonder if that is somehow the problem?

The path looks correct and I can right click it and "Go To Definition" and it opens the header file correctly.

[terjeio]

H worked to home the machine, and IOSender says it is Idle and Ready but the other keys aren't triggering anything. Odd.

Maybe not, jog commands could be ignored if outside machine limits. Try with setting $40=1, this limits jogging to be within limits. Also try M and C, should toggle mist and coolant.

[5ocworkshop]

M & C work correctly.

I tried it with $40=0 and $40=1, no change.

I also moved to the middle of the table to make sure I had clearance in all directions.

[5ocworkshop]

AHA! I thought "I wonder if regular keyboard jogging works within IOSender?":

error:22 - Feed rate has not yet been set or is undefined.

[terjeio]

Hmm, maybe the jog defaults are wrong ($50-55), try $RST=& if so.

AHA! I thought "I wonder if regular keyboard jogging works within IOSender?":

It should, I have no problems with that.

error:22 - Feed rate has not yet been set or is undefined.

Yep, that could be from jog settings beeing wrong.

[5ocworkshop]

Yes, they are zeroed, I can see that now in the IOSender UI. They were definitely populated previously. Is this a case of the driver loading them from storage, except they didn't exist, so it set everything to zero?

Yes, once I did the reset they reappeared (although not sure those were my values before, but I think I haven't written my latest values to my long term storage lately.

I'm jogging! Woohoo.

[5ocworkshop]

Thank you so much. My not using the correct function call for the serial.write_char aside, this was incredibly easy.

The code is so straight forward to do it that I sometimes second guess whether you are providing pseudo code to "show the way" and the rest is "left as an exercise for the reader" or it is all the code.

So now all I need to do is add equivalent keymaps to my python script on the Pi so each command you accept maps to an appropriate button and the project is basically ready to go. Then I can look at providing feedback/confirmation of jog speed changes etc using the RGB lights, if available. Perfect.

This makes the Pi header especially powerful now, especially once the UART support gets added to the mainline.

[5ocworkshop]

PS - The final thing I was pondering was whether or not it is possible to send two keys at once with this method, or otherwise trigger diagonal movement.

EDIT - Nevermind, I see you already accounted for that!

[terjeio]

Yes, they are zeroed, I can see that now in the IOSender UI. They were definitely populated previously. Is this a case of the driver loading them from storage, except they didn't exist, so it set everything to zero?

They keypad plugin adds these settings. I rely on a 8-bit checksum for determining if data are valid, if not a reset to defaults is performed. I want to change to a 16-bit checksum as 8-bits are not very sensitive to errors..

.> PS - The final thing I was pondering was whether or not it is possible to send two keys at once with this method, or otherwise trigger diagonal movement.

You'll have to add N_KEY rollover handling on the sending side and send the respective single (lowercase) character commands for diagonal jogging. Or the keypad plugin has to be updated, but that could be tricky?

[5ocworkshop]

There are already definitions for most of the combinations of diagonal movement in keypad.h. I can add anything else I want I suspect.

One thing i can see that I'd like to do is make the Z feedrate independent of the overall jog rates. My Z moving up and down rapidly is always a bit unnerving. I realize this will mean compound moves with X/Y & Z won't be possible, but I think its probably safer to move Z to clearance height before hand.

[5ocworkshop]

The Microsoft MCE remote has a large set of default keys, so I will likely just use the arrow keys for individual axis moves, and do something like use the number pad or the media control buttons for diagonal moves.

This is the remote I'm using: https://www.amazon.ca/Pinnacle-Remote-Windows-Media-Center/dp/B000WR2E00 so lots of options.

[terjeio]

The Microsoft MCE remote has a large set of default keys

Does it have key modifiers similar to keyboard CTRL & SHIFT that could be used to select the jog mode? If so jog mode could be part of the single character commands. 0, 1 and 2 can be used to set it as well.

[5ocworkshop]

Sorry I disappeared. We lost power at the house shortly after my last message.

I think I can send key modifiers from the python script I'm writing. Right now I have the numerics sending 1, 2, 3 and successfully changing modes. Just testing all the combinations now. Working brilliantly.

I did find a typo/bug in the code. B wasn't working and I noticed there appears to be a stray 1 character in this statement n keypad.c:

case JOG_YB: // Jog -Y
jog_command(command, "1Y-?F");
break;

I rremoved it and B works as expected.

I've got to go out, unfortunately, as I'd love to tidy this up and make you a little video to see it all in action, but that will have to wait until a later time I'm afraid.

[5ocworkshop]

For the benefit of anyone who finds this thread in the future via Google, lirc is still an option for IR control but apparently since around 4.14 of the kernel, IR support has been moved in to the kernel and is exposed via the:
ir-keytable

driver. This should make it fairly fast to get mainstream remotes working with a Linux machine and behaving as a keyboard. I am going to look at removing the overhead of lirc and switching to this today. My reading suggests this should be feasible and will make it faster for initial setup by users in the future.

[terjeio]

I did find a typo/bug in the code. B wasn't working and I noticed there appears to be a stray 1 character in this statement n keypad.c

This will be fixed in the next commit.

I agree with the specialized OS analogy. I've actually been thinking about this a lot and wondering how I can help. I was discussing this last night with Drew Marles who also contributes here. I was wondering if there is some way I could help create a series of flow diagrams that would visually represent your vision of how the various components interact and how to hook in to the process to add features etc.

Interesting idea, not sure how easy it will be to draw/describe. A discussion for this would be nice. FYI Phil has an overview on his website, is it something like this you would like to expand upon?

[5ocworkshop]

I'll take a look at Phil's site again, it's been a few months since I read through it.

[terjeio]

FYI I have now added and option to enable the keypad plugin without it claiming the keypad strobe pin by defining KEYPAD_ENABLE 2 instead of KEYPAD_ENABLE 1.