Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flir camera only starting after Spinview #492

Open
rogerartigaspursals opened this issue Mar 20, 2021 · 16 comments
Open

Flir camera only starting after Spinview #492

rogerartigaspursals opened this issue Mar 20, 2021 · 16 comments
Labels
1. Device support 4. Hardware wanted The issue is hardware specific. Donation welcome ! 5. Usb3 Issue in USB3Vision implementation 7. ARM

Comments

@rogerartigaspursals
Copy link

I have succesfully integrated aravis with Baumer and Sentech USB3 cameras. Recently, I started to work with a Flir Camera, model BFS-U3-16S2. The camera is recognized, but no images are streamed neither, on arv-viewer or my software. After executing SpinView from FLIR, the camera works, and it can stream in arv-viewer and my software.

I have gone through all the installation process, and the aravis.rules files.

Any suggestion to avoid goring through the SpinView everythime I restart the computer?

Just for additional information: I’m using a Jetson Xavier NX.

This is the output after executing "arv-camera-test -d all" just after restarting the Jetson (Still no Spinview executed)

Looking for the first available camera
Found 1 USB3Vision device (among 7 USB devices)
[UvDevice::new] Vendor = FLIR
[UvDevice::new] Product = Blackfly S BFS-U3-16S2M
[UvDevice::new] S/N = 0138C343
[UvDevice::new] Using control endpoint 2, interface 0
[UvDevice::new] Using data endpoint 1, interface 2
Get genicam
MANUFACTURER_NAME = 'FLIR'
MAX_DEVICE_RESPONSE_TIME = 0x000000c8
DEVICE_CAPABILITY = 0x0000000000000f0d
SRBM_ADDRESS = 0x0000000000010000
MANIFEST_TABLE_ADDRESS = 0x0000000000040000
U3VCP_CAPABILITY = 0x00000003
MAX_CMD_TRANSFER = 0x00000400
MAX_ACK_TRANSFER = 0x00000400
SIRM_OFFSET = 0x0000000000020000
SI_INFO = 0x03000000
SI_CONTROL = 0x00000000
SI_REQ_PAYLOAD_SIZE = 0x0000000000100000
SI_REQ_LEADER_SIZE = 0x00000034
SI_REQ_TRAILER_SIZE = 0x00000024
SI_MAX_LEADER_SIZE = 0x00000000
SI_PAYLOAD_SIZE = 0x00000000
SI_PAYLOAD_COUNT = 0x00000000
SI_TRANSFER1_SIZE = 0x00000000
SI_TRANSFER2_SIZE = 0x00000000
SI_MAX_TRAILER_SIZE = 0x00000000
MANIFEST_N_ENTRIES = 0x0000000000000001
MANIFEST ENTRY
00000000 00 00 00 01 00 04 01 01 00 00 00 60 00 00 00 00 ...........`....
00000010 1b 0a 01 00 00 00 00 00 17 35 1a d8 06 b4 91 73 .........5.....s
00000020 d8 08 e8 e4 41 73 cd b8 4c eb 67 08 00 00 00 00 ....As..L.g.....
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

genicam address = 0x0000000060000000
genicam size = 0x0000000000010a1b
zip file = public_camxml.xml
[GcFeature::set_attribute] Unknown attribute 'xmlns:xi'
[GcFeature::set_attribute] Unknown attribute 'xmlns:xsi'
[GcFeature::set_attribute] Unknown attribute 'xmlns'
[GcFeature::set_attribute] Unknown attribute 'StandardNameSpace'
[GcFeature::set_attribute] Unknown attribute 'ToolTip'
[GcFeature::set_attribute] Unknown attribute 'ProductGuid'
[GcFeature::set_attribute] Unknown attribute 'VersionGuid'
[GcFeature::set_attribute] Unknown attribute 'xsi:schemaLocation'
[Genicam::create_element] Unknown tag (Visibility)
[Genicam::create_element] Unknown tag (ImposedAccessMode)
[Genicam::create_element] Unknown tag (Representation)
[Genicam::create_element] Unknown tag (Visibility)
[Genicam::create_element] Unknown tag (ImposedAccessMode)
[Genicam::create_element] Unknown tag (Representation)
[Genicam::create_element] Unknown tag (Visibility)
...
...
...
ArvDevice::set_status] Status changed ('[ArvDevice::write_memory] Timeout')
[ArvDevice::set_integer_feature_value] Node 'DeviceLinkThroughputLimitMode' is not an integer
vendor name = FLIR
model name = Blackfly S BFS-U3-16S2M
device id = 20497219
image width = 1024
image height = 1024
horizontal binning = 1
vertical binning = 1
payload = 1048576 bytes
exposure = 2051 µs
gain = 0 dB
uv bandwidth limit = 380000000 [0..500000000]
SI_INFO = 0x03000000
SI_REQ_PAYLOAD_SIZE = 0x0000000000100000
SI_REQ_LEADER_SIZE = 0x00000034
SI_REQ_TRAILER_SIZE = 0x00000024
Required alignment = 8
SI_PAYLOAD_SIZE = 0x00010000
SI_PAYLOAD_COUNT = 0x00000010
SI_TRANSFER1_SIZE = 0x00000000
SI_TRANSFER2_SIZE = 0x00000000
SI_MAX_LEADER_SIZE = 0x00000038
SI_MAX_TRAILER_SIZE = 0x00000028
USB transfer error: LIBUSB_ERROR_TIMEOUT
USB transfer error: LIBUSB_ERROR_IO
USB transfer error: LIBUSB_ERROR_IO
USB transfer error: LIBUSB_ERROR_IO
USB transfer error: LIBUSB_ERROR_IO
...
...
...

AFTER EXECUTING SPINVIEW:

Looking for the first available camera
Found 1 USB3Vision device (among 7 USB devices)
[UvDevice::new] Vendor = FLIR
[UvDevice::new] Product = Blackfly S BFS-U3-16S2M
[UvDevice::new] S/N = 0138C343
[UvDevice::new] Using control endpoint 2, interface 0
[UvDevice::new] Using data endpoint 1, interface 2
Get genicam
MANUFACTURER_NAME = 'FLIR'
MAX_DEVICE_RESPONSE_TIME = 0x000000c8
DEVICE_CAPABILITY = 0x0000000000000f0d
SRBM_ADDRESS = 0x0000000000010000
MANIFEST_TABLE_ADDRESS = 0x0000000000040000
U3VCP_CAPABILITY = 0x00000003
MAX_CMD_TRANSFER = 0x00000400
MAX_ACK_TRANSFER = 0x00000400
SIRM_OFFSET = 0x0000000000020000
SI_INFO = 0x03000000
SI_CONTROL = 0x00000001
SI_REQ_PAYLOAD_SIZE = 0x0000000000100000
SI_REQ_LEADER_SIZE = 0x00000034
SI_REQ_TRAILER_SIZE = 0x00000024
SI_MAX_LEADER_SIZE = 0x00000038
SI_PAYLOAD_SIZE = 0x00000000
SI_PAYLOAD_COUNT = 0x00000000
SI_TRANSFER1_SIZE = 0x00100000
SI_TRANSFER2_SIZE = 0x00000000
SI_MAX_TRAILER_SIZE = 0x00000028
MANIFEST_N_ENTRIES = 0x0000000000000001
MANIFEST ENTRY
00000000 00 00 00 01 00 04 01 01 00 00 00 60 00 00 00 00 ...........`....
00000010 1b 0a 01 00 00 00 00 00 17 35 1a d8 06 b4 91 73 .........5.....s
00000020 d8 08 e8 e4 41 73 cd b8 4c eb 67 08 00 00 00 00 ....As..L.g.....
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

genicam address = 0x0000000060000000
genicam size = 0x0000000000010a1b
zip file = public_camxml.xml
[GcFeature::set_attribute] Unknown attribute 'xmlns:xi'
[GcFeature::set_attribute] Unknown attribute 'xmlns:xsi'
[GcFeature::set_attribute] Unknown attribute 'xmlns'
[GcFeature::set_attribute] Unknown attribute 'StandardNameSpace'
[GcFeature::set_attribute] Unknown attribute 'ToolTip'
[GcFeature::set_attribute] Unknown attribute 'ProductGuid'
[GcFeature::set_attribute] Unknown attribute 'VersionGuid'
[GcFeature::set_attribute] Unknown attribute 'xsi:schemaLocation'
[Genicam::create_element] Unknown tag (Visibility)
[Genicam::create_element] Unknown tag (ImposedAccessMode)
[Genicam::create_element] Unknown tag (Representation)
[Genicam::create_element] Unknown tag (Visibility)
[Genicam::create_element] Unknown tag (ImposedAccessMode)
[Genicam::create_element] Unknown tag (Representation)
[Genicam::create_element] Unknown tag (Visibility)
...
...
...

[ArvDevice::set_status] Status changed ('[ArvDevice::write_memory] Timeout')
[ArvDevice::set_integer_feature_value] Node 'DeviceLinkThroughputLimitMode' is not an integer
vendor name = FLIR
model name = Blackfly S BFS-U3-16S2M
device id = 20497219
image width = 1024
image height = 1024
horizontal binning = 1
vertical binning = 1
payload = 1048576 bytes
exposure = 2051 µs
gain = 0 dB
uv bandwidth limit = 380000000 [0..500000000]
SI_INFO = 0x03000000
SI_REQ_PAYLOAD_SIZE = 0x0000000000100000
SI_REQ_LEADER_SIZE = 0x00000034
SI_REQ_TRAILER_SIZE = 0x00000024
Required alignment = 8
SI_PAYLOAD_SIZE = 0x00010000
SI_PAYLOAD_COUNT = 0x00000010
SI_TRANSFER1_SIZE = 0x00000000
SI_TRANSFER2_SIZE = 0x00000000
SI_MAX_LEADER_SIZE = 0x00000038
SI_MAX_TRAILER_SIZE = 0x00000028
Frame rate = 262 Hz
Frame rate = 244 Hz
Frame rate = 238 Hz
Frame rate = 239 Hz
Frame rate = 238 Hz
^CFrame rate = 238 Hz
Completed buffers = 1459
Failures = 0
Underruns = 0
[UvStream::finalize] n_completed_buffers = 1459
[UvStream::finalize] n_failures = 0
[UvStream::finalize] n_underruns = 0
[Stream::finalize] Flush 49 buffer[s] in input queue
[Stream::finalize] Flush 1 buffer[s] in output queue
@EmmanuelP
Copy link
Contributor

Hi Roger,

Thanks for the report.

Could you checkout this branch:

https://github.com/AravisProject/aravis/tree/issue/492%2Fcheck-stream-enable

And attach the output of arv-camera-test -d all:3 before and after spinview please ?

@EmmanuelP EmmanuelP added 1. Device support 5. Usb3 Issue in USB3Vision implementation labels Mar 21, 2021
@rogerartigaspursals
Copy link
Author

rogerartigaspursals commented Mar 21, 2021 via email •
edited by EmmanuelP
Loading

@EmmanuelP
Copy link
Contributor

EmmanuelP commented Mar 21, 2021 •
edited
Loading

Thanks for the informations, but the output is to much truncated to be really useful. Unfortunately, I did not spot anything obvious in what you sent.

The last thing we can try is a capture of the USB data using wireshark. A capture of aravis before and after spinview, as well as during spinview could may be help to find what is missing or wrong in aravis.

@rogerartigaspursals
Copy link
Author

rogerartigaspursals commented Mar 22, 2021 via email •
edited by EmmanuelP
Loading

@eusebi
Copy link

eusebi commented Jun 21, 2021

Hi, rogerartigaspursals,
have you found a solution to this problem ? I am running into the same problem and any hint is appreciated

@rogerartigaspursals
Copy link
Author

rogerartigaspursals commented Jun 22, 2021 via email •
edited by EmmanuelP
Loading

@EmmanuelP EmmanuelP added the 4. Hardware wanted The issue is hardware specific. Donation welcome ! label Sep 16, 2021
@boilerbots
Copy link

I can say that I have been using Aravis with BlackFly USB3 cameras for years and they do start and will work if you configure them properly, however if you program any registers outside of their "legal" values you can lock up the camera or get garbage out. The cameras can be recovered using a script such as this:

arv-tool-0.6 | while read -r line; do
    serial=`echo $line | awk '{print $1}'`
    echo "Resetting $serial"
    arv-tool-0.6 -n $serial control DeviceReset=1
done

@timohueser
Copy link

I'm running into the same problem on the Jetson Orin developement Kit, using a Flir BlackFly S camera. LibUSB times out during the call to libusb_bulk_transfer as soon as I try to acquire images from the camera. This happens both in the arv-viewer and the 02-multiple-acquisition.c from the aravis-c-examples. Interestingly I sometimes get few frames to show up before it times out, but this happens irregularly and most of the time it's just a single frame.
The issue completely goes away as soon as the camera is initialized using the Spinnaker SDK. All that is needed is a call to the init() function of the camera thorugh the Spinnaker SDK and Aravis starts streaming the images as expected.

This issue does not occur when I use the exact same camera on my Ubuntu 22.04 desktop PC, so I suspect the problem is Jetson/ARM related. I tried examining the USB traffic between the camera and the host using wireshark, and the traffic looks identical on my desktop and the Jetson Orin.
I also tried upgrading libUSB to the newest release (1.0.26) without any luck.

One last thing that might be worth noting is that even after the image streaming gets stuck on the timeout all camera settings are still accessible and get applied to the camera correctly. I tried a bunch of combinations of framerates, pixel formats and resolutions, all with pretty much the same result. Very small resolutions (16x16 in Mono8) did stop the libusb_bulk_transfer function from timing out, but there were still no images transferred to the buffers.

I would really appreciate any help or suggestions!

@EmmanuelP
Copy link
Contributor

EmmanuelP commented Feb 24, 2023 •
edited
Loading

Hi Timo,

Like I said in #492 (comment) , some wireshark captures would help to understand the issue.

@timohueser
Copy link

Hi Emmanuel,

I will try to do some captures this afternoon! Since I'm new to using wireshark, what format (e.g. csv or raw capture etc.) is most useful to you? And do I need to configure it in any special way to give you the data you need?

Thanks for your help!

@timohueser
Copy link

Hi again,

Here is a collection of Wireshark captures:

  • 02-multiple-acquisition: Running the example from the aravis-c-examples, both with and without having the cameras initialized using the Spinnaker SDK (success/fail respectively).
  • arv_camera_test: Running the arc-camera-test utility, both with and without having the cameras initialized using the Spinnaker SDK (success/fail respectively). I also included the logs produced at debug level 3, like you suggested in an earlier comment.
  • FLIR_Spinview_Init: Capture during launch of the SpinView software and selecting the camera in the camera list. This is enough to initialize the camera and make it work with aravis.
  • FLIR_Minimal_Init_Deinit: Capture of the Spinnaker SDK being used to init and deinit the camera. No settings are changed on the cameras and no Acquisition is started. This also is sufficient to make the camera work with aravis afterwards.
  • FLIR_Acquisition_Example: Capture of the unedited Acquisition sample from the Spinnaker SDK as a refernce.

I hope this is helpful, please let me know if you need anything else.
Wireshark_Captures.zip

@timohueser
Copy link

Some additional observations I made this weekend:

  • Using the asynchronous mode (e.g. with --usb-mode=async in arv-camera-test) makes it to work for 1000-10000 frames, after which the framerate abruptly drops to 0. Curiosuly there are no libusb errors reportet in the async mode, it just silently stops working. After initializing the camera using the Spinnaker SDK it works indefinitely, both in sync and async mode.
  • It seems like the only write command that the Spinnaker API issues over the U3V protocoll to the camera during the init() call is setting the EI_Control register. As far as I can tell the value it sets is the default value that camera is initialized with anyways, so this should have no effect. The only other non-parameter-related thing I could spot are some URB_BULK transfers to interface 3 of the camera (this interface is not used with aravis as far as I can tell). I have no idea what these do though and if they are related to the problem at all.
  • To make sure it the Spinnaker init() call changes the configuration on the camera, and not some USB related settings on the jetson itself I tried powering the camera externally, initializing it with the Spinnaker SDK on a second computer and then plugging it into the Jetson. And indeed this has the same effect as initializing it on the Jetson directly, so whatever the Spinnaker SDK does it has to be related to the camera itself.

If anyone has any suggestions on what else I could try to troubleshoot this please let me know!

@EmmanuelP
Copy link
Contributor

Hi Timo,

Thanks for the capture. In the FLIR_Spinview_Init.pcapng file, I have noticed this command:

No.	Time	Source	Destination	Protocol	Length	Info
1313	39.785737	host	2.5.2	U3V	88	> WRITEMEM_CMD [EI_Control]: 4 bytes
1314	39.786836	2.5.2	host	USB	64	URB_BULK out
1315	39.786901	host	2.5.2	USB	64	URB_BULK in
1316	39.787015	2.5.2	host	U3V	80	< WRITEMEM_ACK [EI_Control]

Frame 1313: 88 bytes on wire (704 bits), 88 bytes captured (704 bits) on interface usbmon0, id 0
USB URB
USB 3 Vision
    CCD: WRITEMEM_CMD
    SCD: WRITEMEM_CMD
        Address: 0x0000000000028000 [EI_Control]
        Event Interface Control: 1

Which enables device events. Aravis does not implement event support yet, and does nothing about them. I'm not sure how this command could make aravis work though.

@timohueser
Copy link

Hi Emmanuel,

Thanks for looking through the captures! I tried adding the EI_Control command to aravis in the _bootstrap method and confirmed that it issues the same write command as the Spinnaker SDK using wireshark. Unfortunately it has no effect on the behaviour of the camera. Do you thin the issue could still somehow be related to device events?

Another thing I noticed is that I also get LIBUSB_ERROR_IO errors when I change the length parameter in the libusb_bulk_transfer call that is responsible for reading the frame data from the camera (as far I can tell). This is just a wild guess, but could it be that the camera sends some packets with a length different from the expected one?

I also contacted the FLIR support about the issue in the hopes of getting some more info about what happens during the Spinnaker init() call. I'll update this issue if I get any more info.

@boilerbots
Copy link

Just a wild guess but try disable USB3 low power mode on ports and hubs that you connect the FLIR cameras to.

@eusebi
Copy link

eusebi commented May 10, 2023

@timohueser Have you found the solution to this spinview magic initialization ? This detail still plagues our code...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
1. Device support 4. Hardware wanted The issue is hardware specific. Donation welcome ! 5. Usb3 Issue in USB3Vision implementation 7. ARM
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@EmmanuelP @boilerbots @eusebi @timohueser @rogerartigaspursals