Allow access to custom message headers

[pschichtel]

See: https://github.com/linux-can/can-utils/blob/c45a17e96ba307ad752ac4551513559daefbefbd/candump.c#L723-L743

This could e.g. be used for flow control by monitoring dropped frame count

[pschichtel]

Descoped as this is not very relevant right now.

[pschichtel]

I don't think this is feasible right now

[splatch]

Maybe its worth to leave it open with low priority and feature request flag?

I think we could utilize this, at some point, in Apache PLC4X project. Currently PLC4X does not have support for metadata at certain levels (read, write or subscription callbacks), but if it gets that part then definitely providing a time when message touched the bus would be welcome.
Other thing is that gs_usb dongles (based on STM32Fx) I used so far didn't support that socketcan option. As I recently switched to new single board computers with CAN onboard based on different CAN driver so I am gaining a testing capacity. I think that if I get more people on @ConnectorIO end we could contribute also the JNI part.

[pschichtel]

Sure. I've looked at this a bit in the past. I think I'd provide a bit if a different API for applications that want to utilize this, so to not make the normal flow more involved.

[pschichtel]

@splatch 256e2d4 contains a first design of an API that would provide message headers. Would something like that work for your use cases?

[pschichtel]

@splatch I implemented message headers based on the approach I used for the J1939 headers, which is a lot more memory friendly. Is this something you could work with?

[splatch]

Thank you for remembering this feature request!

The request was motivated to improve plc4x-can integration. For now there is no unified API in it which could be integrated, yet I have now an argument to bring a message level and transport level metadata (i.e. hardware level publish time).
I've got a USB adapter from (as far I remember) PEAK. I have to verify if its possible to use it through socketcan, cause firmware for adapter based on gs_usb/STM32F4 I started with did not support this.

[pschichtel]

Do you happen to have a link to that adapter? It would be nice to have something USB-based so I don't have to pull out my raspberry pi to test with real hardware.

[splatch]

I’ve got https://www.peak-system.com/PCAN-USB.199.0.html, mainly to test it under windows but under linux with peak_usb module you can interact with it through socketcan.
There are other vendors you can pick, based on ros wiki: https://wiki.ros.org/socketcan_interface all major interface producers have USB option.

[pschichtel]

Good to know. The PEAK device looks like exactly what I was thinking of, but they are a little more pricy than I'd like given my needs, especially the FD variant. Maybe some company using this lib is willing to gift one at some point.

[splatch]

I got a second hand unit (without FD), cause it is expensive for me as well.

Out of curiosity, are you able to provide an invoice/receipt or you are bound to github sponsors?

[pschichtel]

I'm not bound to anything really. I enabled github sponsors mainly as an attempt to lower resistance. I can write invoices/receipts if necessary, that would not be an issue.

[splatch]

I was able to confirm that timestamps can be fetched using headers with current develop. Thank you!

What I was not able to confirm so far was whether PEAK usb adapter populates them. When I read headers of peak interface I keep getting 1970, meaning its not putting epoch time in there.

Hardware I confirmed hardware time being ok is https://ucandevices.github.io/uccb.html - with this firmware https://github.com/UsbCANConverter-UCCbasic/UCCB_GS_Embedded/releases/tag/0004 (it is talking through gs_usb kernel module)

[pschichtel]

Do you see timestamps with candump ? Also can you share the test program (or the relevant sections of it) ?

[splatch]

Below is whole program, quite basic. I have two interfaces, one with uccb, other with peak, peak does not report timestamps.

public class Main2 {

    public static void main(String[] args) throws Exception {
        RawCanChannel iface = CanChannels.newRawChannel("can0");

        final CanSocketOptions.TimestampingFlagSet sendFlags = CanSocketOptions.TimestampingFlagSet.of(SOFTWARE, RAW_HARDWARE, TX_SOFTWARE, TX_HARDWARE, TX_SCHED, OPT_TSONLY, OPT_ID);
        final CanSocketOptions.TimestampingFlagSet receiveFlags = CanSocketOptions.TimestampingFlagSet.of(SOFTWARE, RAW_HARDWARE, RX_SOFTWARE, RX_HARDWARE, OPT_TSONLY, OPT_ID);

        iface.setOption(CanSocketOptions.SO_TIMESTAMPING, sendFlags);
        iface.setOption(CanSocketOptions.SO_TIMESTAMPING, receiveFlags);

        RawReceiveMessageHeaderBuffer headerBuffer = new RawReceiveMessageHeaderBuffer();
        while (true) {
            CanFrame frame = iface.receive(headerBuffer);
            System.out.println(frame + " " + headerBuffer.getTimestamp());
            System.out.println("\n");
        }

    }

}

Output for uccb:

CanFrame(ID=701, FLAGS=0, LEN=1, DATA=[05]) 2024-03-12T18:39:27.226135355Z
CanFrame(ID=701, FLAGS=0, LEN=8, DATA=[05, 8A, 01, 01, 00, 00, 00, 00]) 2024-03-12T18:39:27.229413355Z

For peak:

CanFrame(ID=701, FLAGS=0, LEN=1, DATA=[05]) 1970-01-05T02:13:32.456082149Z
CanFrame(ID=701, FLAGS=0, LEN=8, DATA=[05, 8A, 01, 01, 00, 00, 00, 00]) 1970-01-05T02:13:32.459367149Z
CanFrame(ID=705, FLAGS=0, LEN=1, DATA=[05]) 1970-01-05T02:13:33.173052149Z
CanFrame(ID=705, FLAGS=0, LEN=8, DATA=[05, 87, 01, 00, 9D, 9B, 9D, 91]) 1970-01-05T02:13:33.176167149Z

[pschichtel]

Interesting, the timestamps are not quite 1970-01-01 as you would expect for the zero value. The memory is zeroed before use while reading the extended headers¹ into it, so it seems the kernel provided values close to zero, not sure why. This SO_TIMESTAMPING interface is kinda weird, maybe this logic doesn't make sense.

Footnotes

1. https://github.com/pschichtel/JavaCAN/blob/f1cd630447048f009dacbd0d7147d66b271c5d89/core/src/main/c/javacan_raw.c#L188 ↩

[pschichtel]

the idea there was: if there is a non-zero hardware timestamp, then take it, otherwise take the software timestamp it. maybe it's not that simple. problem is I think, that this parsing function is not aware of any of the options have actually been enabled on the socket. Maybe I should probe the socket options for the SO_TIMESTAMPING configuration.

[splatch]

I had a look on peak_usb Kernel module and it has some time related operations, I just can't digest its logic: https://github.com/torvalds/linux/blob/master/drivers/net/can/usb/peak_usb/pcan_usb_core.c#L155.

I believe that I had 0 or 1970 values before for uccb devices with old firmware. I will test behavior with uccb dongle and earlier firmware (I have one with mpcie form factor), to see if it will be consistent with peak.
I think its better to have 0 rather than software timestamp, it might lead to confusion of what timestamp is. Having zero (or near zero) clearly indicates that hardware timestamp is not available and caller have to deal with this, not JavaCAN.

[splatch]

It could be that PEAK has some config options which I haven't specified. I plugged it for first time yesterday. ;-)

[pschichtel]

on latest master I separated software and hardware timestamps, would be interesting to see the PEAK timestamps again

[splatch]

Looks way better now, I've changed just one line:

 System.out.println(frame + " " + headerBuffer.getHardwareTimestamp() + " " + headerBuffer.getSoftwareTimestamp());

uccb:

CanFrame(ID=705, FLAGS=0, LEN=1, DATA=[05]) 2024-03-12T23:00:47.621278105Z 2024-03-12T23:00:47.635612307Z

CanFrame(ID=705, FLAGS=0, LEN=8, DATA=[05, 87, 01, 00, 9D, 9B, 9D, 91]) 2024-03-12T23:00:47.624456105Z 2024-03-12T23:00:47.638770462Z

PEAK

CanFrame(ID=701, FLAGS=0, LEN=1, DATA=[05]) 1970-01-05T03:54:12.742399840Z 2024-03-12T22:59:39.646466228Z

CanFrame(ID=701, FLAGS=0, LEN=8, DATA=[05, 8A, 01, 01, 00, 00, 00, 00]) 1970-01-05T03:54:12.745684840Z 2024-03-12T22:59:39.649771050Z

Seems that PEAK over socketcan layer reports time as software, while gs_usb reports both, and they even differ. ;-)