Kvaser can_filter not working.

[dhje0ng]

I did the filtering by specifying a range to filter out specific CAN ID.
As a result, it was impossible to obtain by filtering CAN messages within the desired range.

python-can doesn't seem to do that well.
The filtering range of CAN message I want is 0x600 ~ 0x6FF.
However, even if the filtering is implemented as Another out-of-range CAN message is being received.

can_filters = [{"can_id": 0x600, "can_mask": 0x6FF, "extended": False}]
bus = can.ThreadSafeBus(bustype=driver, channel=channel, bitrate=bitrate, data_bitrate=data_bitrate, \
                fd=True, accept_virtual=False, can_filters=can_filters)

but, This is not the result I want. out of range.
If filtering is applied, I think the desired result should be output within the range of 0x600 ~ 0x6FF

I'm using it Kvaser Driver.

Can i solve this problem?

[dhje0ng]

this problem occurs in a Windows OS(Windows 10 PRO) environment.
In Linux(Ubuntu 20.04), filtering works normally.

[dhje0ng]

@zariiii9003 update content.

Test Code

import can

can_filters = [{"can_id": 0x200, "can_mask": 0x7ff, "extended": False}]

canbus = can.ThreadSafeBus(bustype="kvaser", channel=0, bitrate=500000, data_bitrate=2000000, accept_virtual=False, fd=True, can_filters=can_filters)

while True:
    res = canbus.recv(timeout=5)
    print(res)

Output

Windows 10

Linux(Ubuntu 20.04)

[dhje0ng]

@zariiii9003 hello. We were able to find the cause of this bug by analyzing it.
I use Kvaser hardware, so I use the KvaserBus() implementation.

The _apply_filters() function is called when filter options are included in the Bus.
In that function, the canSetAcceptanceFilter() function is called, which seems to be a problem when referring here.

interfaces/kvaser/canlib.py

def _apply_filters(self, filters): # ****** call
    if filters and len(filters) == 1:
        can_id = filters[0]["can_id"]
        can_mask = filters[0]["can_mask"]
        extended = 1 if filters[0].get("extended") else 0
        try:
            for handle in (self._read_handle, self._write_handle):
                canSetAcceptanceFilter(handle, can_id, can_mask, extended) # ****** here
        except (NotImplementedError, CANLIBError) as e:
            self._is_filtered = False
            log.error("Filtering is not supported - %s", e)
        else:
            self._is_filtered = True
            log.info("canlib is filtering on ID 0x%X, mask 0x%X", can_id, can_mask)

    else:
        self._is_filtered = False
        log.info("Hardware filtering has been disabled")
        try:
            for handle in (self._read_handle, self._write_handle):
                for extended in (0, 1):
                    canSetAcceptanceFilter(handle, 0, 0, extended)
        except (NotImplementedError, CANLIBError) as e:
            log.error("An error occured while disabling filtering: %s", e)

canSetAcceptanceFilter doesn't seem to be implemented as a function but refers to another library.
An example would be calling via __get_canlib_function()

interfaces/kvaser/canlib.py

canSetAcceptanceFilter = __get_canlib_function(
    "canSetAcceptanceFilter",
    argtypes=[c_canHandle, ctypes.c_uint, ctypes.c_uint, ctypes.c_int],
    restype=canstat.c_canStatus,
    errcheck=__check_status_operation,
)

The cause is unclear, but the problem seems to occur when referencing the implementation of this function.
As a temporary workaround for this issue, modify the code as follows:

interfaces/kvaser/canlib.py

canSetAcceptanceFilter = __get_canlib_function(
    "", # fixed code
    argtypes=[c_canHandle, ctypes.c_uint, ctypes.c_uint, ctypes.c_int],
    restype=canstat.c_canStatus,
    errcheck=__check_status_operation,
)

This way I'm not referencing that function, but I've debugged that the filtering is in effect.
I need some more confirmation.

Windows Test (fixed code)

I can't reference the library, but I was able to get it by filtering only the CAN IDs I wanted.
This seems to be the problem with the canSetAcceptanceFilter function in the canlib library

This has been tested equally on the Linux(as Ubuntu 20.04) operating system.
The library cannot be referenced, but filtering is possible normally.

[dhje0ng]

When I checked, using KvaserBus() seems to use the filter function _apply_filters()

To make it clear that this function is the cause, if you change the return of the function arbitrarily and test it, it can work normally.

interfaces/kvaser/canlib.py

def _apply_filters(self, filters):
    if filters and len(filters) == 1:
        can_id = filters[0]["can_id"]
        can_mask = filters[0]["can_mask"]
        extended = 1 if filters[0].get("extended") else 0
        try:
            for handle in (self._read_handle, self._write_handle):
                canSetAcceptanceFilter(handle, can_id, can_mask, extended)
        except (NotImplementedError, CANLIBError) as e:
            self._is_filtered = False
            log.error("Filtering is not supported - %s", e)
        else:
            return 0 # test code
            self._is_filtered = True # bug occured!!
            log.info("canlib is filtering on ID 0x%X, mask 0x%X", can_id, can_mask)

    else:
        self._is_filtered = False
        log.info("Hardware filtering has been disabled")
        try:
            for handle in (self._read_handle, self._write_handle):
                for extended in (0, 1):
                    canSetAcceptanceFilter(handle, 0, 0, extended)
        except (NotImplementedError, CANLIBError) as e:
            log.error("An error occured while disabling filtering: %s", e)

If you set self._is_filtered = True in your code, the problem seems to be happening.
If filtered is treated as True in your code, unexpected filtering will lead to unexpected behavior.

Therefore, if this code is temporarily modified, it should be modified as follows.

  try:
      for handle in (self._read_handle, self._write_handle):
          canSetAcceptanceFilter(handle, can_id, can_mask, extended)
  except (NotImplementedError, CANLIBError) as e:
      self._is_filtered = False
      log.error("Filtering is not supported - %s", e)
  else:
      self._is_filtered = False # bug fix
      log.info("canlib is filtering on ID 0x%X, mask 0x%X", can_id, can_mask)

Now the problem with filtering is solved!
However, you can refer to this function in other code and use it.
If the code is modified, sufficient testing is required.

[zariiii9003]

canSetAcceptanceFilter sets the hardware filtering in the kvaser driver. If you set self._is_filtered = False then the python-can software filtering is used instead as a fallback.

But i cannot say why the hardware filtering fails, i don't have any kvaser devices.

[christiansandberg]

It could perhaps be due to FD frames are not handled correctly.

[RagnvaldIV]

I ran into the same issue with Kvaser interface, and this fix worked for me on Win10.

I am not using CAN FD, just classic 500k baud.

[dhje0ng]

Yes, that's correct. The same issue may occur with both CAN and CAN-FD, and it could be due to a problem with the Kvaser hardware driver. To solve this problem, I modify the code for the self._is_filtered when using the Kvaser library on the Windows OS.

For example, the modification could look like the following.

if platform.system() == "Windows":
    from lib.can.interfaces.kvaser.canlib import * # windows only
else:
    from can.interfaces.kvaser.canlib import * # is other platform

For platforms other than Windows, we use the pre-installed libraries by default. However, for Windows, we use the modified library to fix the bug. Since we cannot patch it yet, this is generally a good idea as an alternative solution.

[bobataylor]

Not sure if this will fix the problem you are having, but I ran into a very similar issue with the Kvaser filters while using a U100-X3 and the canlib package.

The 11-bit standard filter and 29-bit extended filter are actually 2 separate filters. If you only want to receive standard messages you also have to set the extended filter to block all IDs. Something like id=0x00000000 mask=0x1FFFFFFF should do the trick.

Additionally, the Kvaser starts storing messages into its' internal receive buffer as soon as you call busOn(). So anything sent on the bus between busOn() and you setting the filters will show up when you call read(). Adding a call to iocontrol.flush_rx_buffer() after setting the filters solves this problem.

Hope that helps!

[grant-allan-ctct]

I wonder if it might be useful to have another function added to the BusABC class called flush_rx_buffer, and then overrides could be provided for those hardware devices that offer it.

For my ValueCAN device, I achieve receive buffer flushing by importing a lower-level python library (viz. ics) to help me do it. It would be kinda nice to have receive buffer flushing capability at the python-can library level though, I think.

Code along these lines seems to do the trick for my device:

def flushReceiveBuffer(serialNumber):
  # This will get rid of any historical messages.
  device = ics.open_device(serialNumber)
  if device is not None:
    try:
      _, _ = ics.get_messages(device)
    finally:
      ics.close_device(device)

[hardbyte]

@bobataylor - so we could fix Kvaser's implementation by setting the hardware filters in the Bus.__init__ before calling busOn and telling users to pass can_filters to the Bus initializer?

@grant-allan-ctct I think you're onto the right idea to solve this for the general case. We can add a call to flush_rx_buffer after calling Bus.set_filters (with an optional argument to disable).

[bobataylor]

@hardbyte maybe? I don't see why that wouldn't work, but I prefer just calling iocontrol.flush_rx_buffer() after setting the filters. Any time you change the filters the problem is going to re-appear and need flushed.

[zariiii9003]

Could you try PR #1796 ?