Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query on Pi Camera v3 CSI clock rates #43

Closed
darksidelemm opened this issue Feb 18, 2023 · 64 comments
Closed

Query on Pi Camera v3 CSI clock rates #43

darksidelemm opened this issue Feb 18, 2023 · 64 comments

Comments

@darksidelemm
Copy link

I've been testing using a Pi Camera v3 for a high-altitude balloon application, where there is a GPS receive in proximity to the raspberry pi. I've previously used Pi Camera v2's with only minimal GPS interference issues, however now I'm testing a Pi Camera v3, I find that as soon as image capture starts, the GPS loses lock.

My suspicion is that the larger sensor size of the v3 results in higher CSI data lane rates, which may have harmonic issues at GPS L1 frequencies (1575 MHz).
How would I determine what CSI rates are in use? I'm capturing full-resolution stills.
Is there any way to force the CSI rate? Again, I'm only interested in capturing the occasional (every few second) still image, not high FPS video...

I've switched back and forth between the Pi Camera v2 and v3, and am fairly certain the issue is more prevalent on the v3 camera.

@darksidelemm
Copy link
Author

Just as confirmation of what's going on here, here's some measurements taken with a HackRF showing the RFI around the GPS L1 frequency when the v3 camera has started:

Screen Shot 2023-02-18 at 18 26 26

Here's the same measurement, but with a v2 camera:
Screen Shot 2023-02-18 at 18 30 11

In both cases, I use the following python commands to start up the camera:

from picamera2 import Picamera2
cam = Picamera2()
config = cam.create_still_configuration()
cam.configure(config)
cam.start()

@6by9
Copy link
Collaborator

6by9 commented Feb 18, 2023

All V4L2 sensor subdev drivers should advertise the CSI-2 link frequency/frequencies used via V4L2_CID_LINK_FREQ. Those are generally configured via device tree too for the case where the driver allows variation of them.

v4l2-ctl -d /dev/v4l-subdev0 --list-ctrls-menu should give you the list, or
https://github.com/raspberrypi/linux/blob/rpi-5.15.y/drivers/media/i2c/imx708.c#L38

#define IMX708_DEFAULT_LINK_FREQ	450000000

https://github.com/raspberrypi/linux/blob/rpi-5.15.y/drivers/media/i2c/imx219.c#L48

#define IMX219_DEFAULT_LINK_FREQ	456000000

Those are both relying on the driver developer having calculated or been given the correct numbers based on the register settings, but I believe they are correct for both sensors.

In both cases the external oscillator feeding the camera module is 24MHz, but the PLLs may result in different frequencies internally.

@6by9
Copy link
Collaborator

6by9 commented Feb 18, 2023

Checking the information I have from Sony, the link frequency is correct as 450MHz for all modes.

The internal pixel clock varies based on mode, being 1488MHz for full res, and 1464MHz for binned. HDR mode is 1944MHz, so it might be worth seeing if switching HDR on reduces your interference issue.

@darksidelemm
Copy link
Author

OK, I can try this out and see if it moves the EMI somewhere else, however HDR mode loses me the resolution that I'm after.

Is there any mechanism for adjusting these frequencies?

@darksidelemm
Copy link
Author

Unfortunately HDR mode does not solve the problem. The interference changes from being essentially continuous to now being impulsive (presumably with the frame rate?), but is still very much present and causes as much of an issue as it did previously.

Screen Shot 2023-02-19 at 13 56 58

@6by9
Copy link
Collaborator

6by9 commented Feb 19, 2023

Is there any mechanism for adjusting these frequencies?

A chunk of the register settings in the driver configure the PLLs. In theory you can change them, but I'm afraid the documentation we have from Sony is under NDA, and it's not something that we would anticipate doing.

@dp111 Any thoughts? Worth a quick discussion on Monday.

@darksidelemm
Copy link
Author

I should note that putting Type-61 ferrites around each end of the camera cable brings the EMI down to the point that it's not taking out the GPS... but I suspect most users of this camera aren't going to have that kind of thing on-hand like I do.

It would certainly be better if it just didn't emit this in the first place.

I'm not 100% sure it's something within the camera module itself - I still suspect it's related to the data transfer from the camera itself.

@naushir
Copy link
Collaborator

naushir commented Feb 19, 2023

@darksidelemm can you try adjusting your framerate to, say, 15fps (66ms duration) to see if/how that changes your interference? You can do this with something like:

picam2.set_controls({"FrameRate": 15.0})

@lurch
Copy link

lurch commented Feb 19, 2023

This reminds me of https://www.stuffaboutcode.com/2014/01/raspberry-pi-gps-helmet-cam.html where the solution was to wrap the CSI-cable in tinfoil 😃

@naushir
Copy link
Collaborator

naushir commented Mar 1, 2023

@darksidelemm did you have a chance to look at the results with reduced framerates?

@darksidelemm
Copy link
Author

Not yet... the hardware I was testing with has been embedded into a balloon payload box so I haven't had the chance to test it.

On that note, I did fly the PiCamera v3 on a balloon launch. With the ferrites it didn't affect the GPS, unfortunately the lens de-focused during the flight due to the cold (certainly out of specification, but still disappointing).

I'll try and get to this this weekend.

@darksidelemm
Copy link
Author

OK, have tried out a few FPS settings (default, 15fps, 1fps). Clearly the duration of the EMI is dependent on how long the data bus is transmitting imagery, but it's still present. I can't say that going to 1fps is a solution here...

Is there any way the CSI clock rate can be adjusted, to see if that shifts the frequency of the EMI around?

Screenshots of results:
default_fps
15fps
1fps

@naushir
Copy link
Collaborator

naushir commented Mar 2, 2023

Is there any way the CSI clock rate can be adjusted, to see if that shifts the frequency of the EMI around?

There is no way at present to do this. We will have to talk with the Sony sensors team to discuss if/how this would be possible.

@darksidelemm
Copy link
Author

OK, thanks for that. Please do, as I expect this is going to affect more than just me...

@aallan
Copy link

aallan commented Mar 2, 2023

I've added a [NOTE] block in the camera documentation pointing at this issue in case others stumble across it.

@julled
Copy link

julled commented Mar 9, 2023

Is there any way the CSI clock rate can be adjusted, to see if that shifts the frequency of the EMI around?

There is no way at present to do this. We will have to talk with the Sony sensors team to discuss if/how this would be possible.

This problem also prevent us from using the camera in our SearchAndRescue drone searchwing.org .

I would be very much happy if this could be resolved.

@naushir
Copy link
Collaborator

naushir commented Mar 10, 2023

The sensor pixel clock rates used by the various modes are listed as (double these up for DDR/2-lane operation):

Mode Freq
2304x1296 HDR 777.60 MHz
1536x864 566.40 MHz
2304x1296 585.60 MHz
4608x2592 595.20 MHz

We will have a go at measuring interference while tweaking various PLL parameters to see what we can come up with.

@naushir
Copy link
Collaborator

naushir commented Mar 10, 2023

@julled and @darksidelemm, are you able to rebuild the imx708 kernel driver? We have a clocking setup that might possibly eliminate the GPS band interference, but want to do some more testing and validation before we can publish the change.

I can provide the source change here if you are able to test this with a custom kernel.

@julled
Copy link

julled commented Mar 10, 2023

@julled and @darksidelemm, are you able to rebuild the imx708 kernel driver? We have a clocking setup that might possibly eliminate the GPS band interference, but want to do some more testing and validation before we can publish the change.

I can provide the source change here if you are able to test this with a custom kernel.

unforunately i dont have capacity to test this right now

@RolandasRazma
Copy link

are you able to rebuild the imx708 kernel driver?

could you point to resources on how to do that please (for a different reason)

@darksidelemm
Copy link
Author

@julled and @darksidelemm, are you able to rebuild the imx708 kernel driver? We have a clocking setup that might possibly eliminate the GPS band interference, but want to do some more testing and validation before we can publish the change.

I can provide the source change here if you are able to test this with a custom kernel.

I'm not setup to build a custom kernel. If some instructions were provided I could probably do it. I've only for the RPi Zero W I've been testing on to work with and I'm guessing compiling. custom kernel on that is going to take a long time.

@6by9
Copy link
Collaborator

6by9 commented Mar 10, 2023

@naushir If you push the change to Github, then CI will build a kernel that can be picked up through sudo rpi-update pulls/<pull_num>. Saves people needing to do custom kernel builds themselves.

@darksidelemm
Copy link
Author

Also, since I was seeing the same kind of interference in HDR mode (reduced resolution) and full resolution mode, I suspect that the pixel clock is not the issue here.
That the duty cycle of the interference is changing with frame rate suggests that this is likely related to the data transfer from the camera to the Pi.

@dp111
Copy link

dp111 commented Mar 10, 2023 via email

@darksidelemm
Copy link
Author

Sure, which is what I suspected it was. Looking forward to trying out a fix!

@naushir
Copy link
Collaborator

naushir commented Mar 13, 2023

Thanks @6by9, I've published the change to https://github.com/raspberrypi/linux/actions/runs/4403064758 for the build artifacts to be available in rpi-update. Please do not do update a production sdcard image, use a spare one or one that you are happy to scrap if things go wrong.

To get the changes, run sudo rpi-update pulls/5381

To change the clock settings, run the following command (make sure the camera application is not running!):

echo <mult> | sudo tee /sys/module/imx708/parameters/iop_pll_mpy

where <mult> is a clock multiplier value. The default value for <mult> is 300. It would be useful to get feedback on <mult> values 298 and 299. If the interference issues are resolved with these values, we will look to update the driver after some more testing.

@naushir
Copy link
Collaborator

naushir commented Mar 16, 2023

Has anybody been able to try out the new clock settings?

@darksidelemm
Copy link
Author

Not yet, will be able to give this a go this weekend.

@darksidelemm
Copy link
Author

Well, I did an apt-get update and upgrade, then ran your rpi-update command.
Now my RPi Zero W isn't connecting to my home WiFi, so I'm going to have to debug why that's the case before I can continue...

@naushir
Copy link
Collaborator

naushir commented Mar 23, 2023

@julled thank you for your thorough analysis!

For my own (limited!) understanding, what is the difference between a cold and warm gps starttype?

@julled
Copy link

julled commented Mar 24, 2023

@julled thank you for your thorough analysis!

For my own (limited!) understanding, what is the difference between a cold and warm gps starttype?

you can see the differences in wikipedia: https://en.wikipedia.org/wiki/Time_to_first_fix

@julled
Copy link

julled commented Mar 24, 2023

Test hardware

Did another test with a USB gps which advertises itself as "u-blox 7" , but its unclear if there a real ublox hidden within the plastic case:

image

... nevertheless this one works way better

Results

Test id Test Description GPS-starttype Camera PLL TestStart TestEnd TestLength [min] GPS-fixtype GPS-noiselevel GPS-sats
1 Only gps cold   04:36:40 AM 04:39:00 AM   3d 117 7
2 pi without cam cold   04:39:49 AM 04:41:00 AM 1.18 3d 113-115 7
3 pi cam cold 300 04:44:00 AM 04:48:00 AM 4.00 nofix 135-151 2-6
4 pi cam cold 299 04:48:20 AM 04:52:00 AM 3.67 3d 103-110 7
5 pi cam cold 298 04:52:30 AM 04:56:00 AM 3.50 3d 106-117 8
6 pi cam cold 297 04:56:00 AM 05:00:00 AM 4.00 3d 106-113 7

To me now these results look really promising that the 298 setting (test 5) is working out get GPS fixes even if there is a cold start. Also the noise levels look reasonable.
So i think this setting should work for our SAR-drones with GPS in combo.

@julled
Copy link

julled commented Mar 28, 2023

@naushir any plans how to proceed?

I think users would prefer to adjust the values of the PLL, just in case of unwanted noise for specific usecases.

For us (and i guess for any other GPS users) the 298 value looks most promising, so maybe this could be a new default value.

@naushir
Copy link
Collaborator

naushir commented Mar 29, 2023

@naushir any plans how to proceed?

I think users would prefer to adjust the values of the PLL, just in case of unwanted noise for specific usecases.

For us (and i guess for any other GPS users) the 298 value looks most promising, so maybe this could be a new default value.

I will prepare a change to the driver that allows users to modify the clock frequency through the official V4L2 controls.

Note that I will keep the default frequency as-is (mult value of 300), since all our EMC certification has been run with that value and any change would invalidate it.

@darksidelemm
Copy link
Author

I'm surprised this wasn't picked up in your EMC certification?

@6by9
Copy link
Collaborator

6by9 commented Mar 29, 2023

I'm surprised this wasn't picked up in your EMC certification?

EMC testing is looking for spurious emissions, and immunity to disturbance.
I've checked the test reports and we're comfortably below the levels required for FCC, EU, and Korean compliance. The full reports are available under https://pip.raspberrypi.com/categories/799-test-report if you complete the relevant NDA.

GPS receivers are looking for a very weak signal, so it isn't uncommon to have integration issues - those do not fall under compliance requirements.
If you wish to have further discussions with regard compliance, then feel free to email compliance@raspberrypi.com and my colleagues can provide some assistance.

@RolandasRazma
Copy link

I'm lurking here for different reason - we see interference on video recording from external source and warping cable in foil fixes it, so it is not only GPS...

@booo
Copy link

booo commented Mar 30, 2023

We used some foil too. But did someone of you test shielded cables such as:

https://kksb-cases.com/products/dsi-ffc-flexible-flat-cable-15cm

@naushir
Copy link
Collaborator

naushir commented Apr 3, 2023

raspberrypi/linux#5381 has now been merged.

In order to use this functionality, you will need to add a parameter to /boot/config.txt to set the required link frequency to use with the following setup:

  1. Edit the /boot/config.txt file (as sudo).
  2. Find and comment out the camera_auto_detect=1 line.
  3. At the bottom of the file, add dtoverlay=imx708,link-frequency=<freq>.

<freq> can take one of the following values (in Hz): 447000000, 450000000, 453000000. These frequencies correspond to a <mult> value of 298, 300, 302 respectively from the earlier tests. The default (in the absence of the link-frequency param) is 450000000,

Unfortunately the camera does not seem to frame correctly with frequencies lower than 447Mhz. It might work with higher frequencies, but that has not been tested.

@naushir
Copy link
Collaborator

naushir commented Apr 3, 2023

I'll resolve this issue now, but feel free to reopen if you are still encountering problems.

@naushir naushir closed this as completed Apr 3, 2023
@julled
Copy link

julled commented Apr 4, 2023

raspberrypi/linux#5381 has now been merged.

In order to use this functionality, you will need to add a parameter to /boot/config.txt to set the required link frequency to use with the following setup:

1. Edit the `/boot/config.txt` file (as sudo).

2. Find and comment out the `camera_auto_detect=1` line.

3. At the bottom of the file, add `dtoverlay=imx708,link-frequency=<freq>`.

<freq> can take one of the following values (in Hz): 447000000, 450000000, 453000000. These frequencies correspond to a <mult> value of 298, 300, 302 respectively from the earlier tests. The default (in the absence of the link-frequency param) is 450000000,

Unfortunately the camera does not seem to frame correctly with frequencies lower than 447Mhz. It might work with higher frequencies, but that has not been tested.

When / how can i get these changes via official software distribution channels?
Do newly generated pi-gen images (we can do this on our own) include these changes already?

@naushir
Copy link
Collaborator

naushir commented Apr 5, 2023

You can get the changes by running a sudo rpi-update command. Note, this will update you to the latest kernel and firmware, so please do keep a backup of your critical work on the sdcard if you need to before running this command.

Eventually the changes will filter into apt and our official images - probably a week or two for the former.

@darksidelemm
Copy link
Author

It's worth noting that the PiCamera HQ appears to have exactly the same EMI issue as the Pi Camera 3, with strong EMI present on 1575 MHz.

Are there similar settings changes available for the HQ camera to shift the clock rates around?

@naushir
Copy link
Collaborator

naushir commented Apr 24, 2023

@darksidelemm - it's possible, but right now I don't have the resources to do it so will be lower on the priority list. However, can you email info@raspberrypi.com and quote this github issue and perhaps we can look into a quick short-term workaround.

@athtest800
Copy link

Is it possible to also fix the same problem for Camera HQ (imx477 overlay)?
I have serious problems while using this camera and GPS.

@darksidelemm
Copy link
Author

darksidelemm commented Apr 30, 2023

I should note that I've been able to make the Pi Camera HQ usable in my application with a stock RPi Zero camera cable, but using the ferrite sleeves i've mentioned previously (Laird P/N: 28R0898-200), one near each end of the cable. You can see the layout of my high-altitude balloon payload in this image: https://twitter.com/vk5qi/status/1650750546922569733/photo/1

I am using a uBlox 8-series GPS with a patch antenna, with the antenna located approximately 15cm from the camera. The GPS maintains lock outside with a clear view of the sky, but does have issues inside where it would normally gain lock.

Without the ferrites the GPS does not gain lock however, so they seem to be critical to making it work with the current cable.

@SChancey
Copy link

SChancey commented Feb 1, 2024

A belated comment but be aware that the ribbon cable used at high (L1) frequencies is also a transmission line. A transmission line (or practically any cable) will act as an antenna for its full and half EMI wavelengths. The "200 mm" ribbon cable is very close to the L1 wavelength. Ferrite beads will add reactance to the cable but altering the ribbon cable's length (moving it away from the L1 wavelength) should be an effective counter-measure to coupling. Don't shorten the cable to a half wavelength (100 mm in this case). For FCC / EMI /RFI product compliance, I always consider interconnecting cable lengths first and if a given cable length happens to correlate to an interfering wavelength, I will shield and / or add respective ferrite cores at the cable ends.

@RolandasRazma
Copy link

@SChancey very interesting, could you elaborate please? How to select correct lengths? You mentioning not to shorten it to 100mm, why is that?

@lurch
Copy link

lurch commented Feb 2, 2024

The "200 mm" ribbon cable is very close to the L1 wavelength.

Luckily, our cables are available in a range of lengths! 😅
https://www.raspberrypi.com/products/camera-and-display-cable/
https://www.raspberrypi.com/products/camera-cable/

@SChancey
Copy link

SChancey commented Feb 2, 2024

@RolandasRazma Practically speaking, from @lurch's links above, going to 300 mm will lower the effective coupling frequency. You don't want 100 mm due to it being close to L1's half wavelength. Imagine a full sine cycle (full wave) and a 1/2 sine cycle (half wave). A simple antenna of appropriate length is "tuned" if it's length matches the effective full or half wave length. If the imposing EM wave, internal or external, matches the antenna length then coupling is increased. This is true for reception and transmission. Fortunately GNSS receiver modules typically employ input SAW filters (high Q bandpass filters) to reject out of band and unwanted wavelengths since they have extremely high input sensitivity (140 dB+). This means you don't have to de-tune the antenna very much to reduce interference. RF is part art, part "magic" and I'm no expert. A rule of thumb for cable / antenna length for an effective frequency is: Frequency (in MHz) = c (speed of light or propagation in meters / sec) / l (length in meters). At the speed of light, it takes ~ 1 nanosecond to travel 1 foot. Real cables don't support propagation at the speed of light and may have speeds in the ~ 0.8 - 0.9 c range. Optimized cables can have higher speeds. Further coupling reduction can be achieved via shielding and / or ferrite cores as mentioned in this thread. In the end, you have to test it. Hard to argue with reality.

@RolandasRazma
Copy link

@SChancey thanks for elaborating, make sense

@03vmate
Copy link

03vmate commented Mar 2, 2024

@naushir Any ideas when the HQ camera will have adjustable frequency as well? Using the camera jams all nearby phones' GPS as well, which is quite problematic.

@6by9
Copy link
Collaborator

6by9 commented Mar 3, 2024

@naushir Any ideas when the HQ camera will have adjustable frequency as well? Using the camera jams all nearby phones' GPS as well, which is quite problematic.

This is the problem with raising secondary issues under one issue in an issue tracker - the primary issue gets resolved and the issue closed. The secondary issue therefore gets lost.

I've raised raspberrypi/linux#6004 to track it.

@SChancey
Copy link

SChancey commented Mar 3, 2024

@03vmate Please consider the 300 or 500 mm camera cables as an option. Cable lengths are discussed above. The 200 mm Standard - Mini Camera Cable normally provided may couple interference in the L1 band. Raspberry Pi states the camera cables are shielded but they really only have a ground plane on one side. This is helpful for transmission lines. My Samsung S21 doesn't have GNSS interference problems with my Pi 5 using a 200 mm camera cable. Not all phones are designed equally of course. Good luck.

@03vmate
Copy link

03vmate commented Mar 3, 2024

@SChancey Used an FFC-HDMI adapter, so I can run fully shielded cable. Covered the camera and phi in copper tape as much as I could, but there was still significant interference. Adjusting the clock, even by just a few MHz would be nice to have, especially that a lot of other sensors support this.

@6by9
Copy link
Collaborator

6by9 commented Mar 4, 2024

@SChancey Used an FFC-HDMI adapter, so I can run fully shielded cable. Covered the camera and phi in copper tape as much as I could, but there was still significant interference. Adjusting the clock, even by just a few MHz would be nice to have, especially that a lot of other sensors support this.

It might be nice, but it isn't related to this issue which is regarding the v3 camera module.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests