DVB-C USB transport stream broken #82

Closed
r10r opened this Issue Aug 13, 2012 · 104 comments

Comments

Projects
None yet

r10r commented Aug 13, 2012

Hi,

I need some help debugging a USB transfer problem on my raspberry pi.

The problem is that the DVB Transport stream recorded from a USB DVB stick (cinergy htc stick)
attached to a usb hub which is attached to the raspberry pi is broken because packages are missing.
I plugged the USB DVB stick directly into the Pi's USB Port (with additional power) with the
same results. I use the latest media_build from linuxtv.org against the 3.1.9 raspberry pi kernel.
The same drivers from the media_build work perfectly with a 3.1.9 kernel on my thinkpad.

When turning on debugging in the DVB driver I get the following messages:

[  126.162705] tda18271: performing RF tracking filter calibration
[  128.056572] tda18271: RF tracking filter calibration complete
[  128.057855] em28xx #0/2-dvb: Using 5 buffers each with 64 x 940 bytes
[  129.052712] em28xx #0/2-dvb: ISO ERROR COUNT: 1.
[  129.052743] em28xx #0/2-dvb: URB packet 0, status -4001 [Unknown].
[  129.581352] em28xx #0/2-dvb: ISO ERROR COUNT: 1.
[  129.581384] em28xx #0/2-dvb: URB packet 0, status -4001 [Unknown].
[  129.872090] em28xx #0/2-dvb: ISO ERROR COUNT: 1.
[  129.872122] em28xx #0/2-dvb: URB packet 0, status -4001 [Unknown].
[  129.969224] em28xx #0/2-dvb: ISO ERROR COUNT: 1.
[  129.969258] em28xx #0/2-dvb: URB packet 44, status -4001 [Unknown].

My assumption is that the isochronous transfer is broken for the usb host driver.

The error code is defined in:

 drivers/usb/host/dwc_common_port/dwc_os.h:#define DWC_E_NO_STREAM_RES   4001

which is caused by a DWC_OTG_HC_XFER_FRAME_OVERRUN in the URB:

 dwc_otg/dwc_otg_hcd_intr.c:            frame_desc->status = -DWC_E_NO_STREAM_RES;

I'm a bit lost since I do not have any USB specific knowledge but I really want
to get the DVB Stick running. What can I do for further testing/debugging?
Thanks for your help!

Cheers,
Ruben

I guess this is closely related to the problem we are experiencing with the Kinect as documented here

r10r commented Aug 15, 2012

@Dexterp37 Do you have any solution for this problem? Or is there any work in progress?

Sadly.. no :( I noticed that latest kernel source has update USB driver so I'm trying to compile the kernel from source and see if it helps.

r10r commented Aug 15, 2012

Which kernel are you trying? The latest https://github.com/raspberrypi/linux.git from @popcornmix ? Please let me know if it works for you.

At the moment I'm trying to compile the one from @popcornmix . I'll also give 3.2.x from https://github.com/bootc/linux/tree/rpi-3.2.23 a try. I'm still fighting with some cross compilation/fork issues with Cygwin on Windows.

r10r commented Aug 16, 2012

I recommend you to use a debian VM for cross compilation. Setting up the toolchain there is really simple.
Last version I tried was 921a3d1 - but maybe the newest commit df9a562 fixes something.

r10r commented Aug 18, 2012

@Dexterp37 The error code has changed, but unfortunately USB isoc transfer is still broken with the latest raspberry pi kernel:

[   71.135721] drxk: frontend initialized.
[   71.195037] tda18271 0-0060: creating new instance
[   71.199269] TDA18271HD/C2 detected @ 0-0060
[   71.262233] drxk: status = 0x639260d9
[   71.262270] drxk: detected a drx-3926k, spin A3, xtal 20.250 MHz
[   75.581431] DRXK driver version 0.9.4300
[   75.927370] DVB: registering new adapter (em28xx #0)
[   75.927412] DVB: registering adapter 0 frontend 0 (DRXK DVB-C DVB-T)...
[   75.938859] em28xx #0: Successfully loaded em28xx-dvb
[   75.938895] Em28xx: Initialized (Em28xx dvb Extension) extension
[  122.687711] tda18271: performing RF tracking filter calibration
[  124.590611] tda18271: RF tracking filter calibration complete
[  124.591786] em28xx #0/2-dvb: Using 5 buffers each with 64 x 940 bytes
[  126.187359] em28xx #0/2-dvb: ISO ERROR COUNT: 1.
[  126.187393] em28xx #0/2-dvb: URB packet 11, status -63 [Buffer error (overrun)].
[  126.195557] em28xx #0/2-dvb: ISO ERROR COUNT: 1.
[  126.195581] em28xx #0/2-dvb: URB packet 0, status -63 [Buffer error (overrun)].

r10r commented Aug 18, 2012

@Dexterp37 You can try the patch from #29 (comment). With this patch I can at least use the TPLink wireless dongle and my USB keyboard together.

Collaborator

popcornmix commented Aug 20, 2012

Can you try latest firmware with:
dwc_otg.microframe_schedule=1
added to cmdline.txt

r10r commented Aug 20, 2012

Hi thanks for the info. I already tried 0872b20 with the microframe scheduler patch? I already tried the microframe scheduler patch before - USB is more stable, but isoc transfer is still broken.

I can confirm that the transfer is still broken. Tried this morning with latest kernel + microframe schedule and Microsoft Kinect.

Contributor

ghollingworth commented Aug 31, 2012

Can you do a lsusb -v to output the endpoint information, just be interested to see if it's highbandwidth isochronous endpoints that are the problem...

Likely issue is that it's just the same high latency problem we're seeing elsewhere (i.e. interrupts are disabled for long enough that we miss processing the isochronous requests until the source overrun's its buffers)

Gordon

@ghollingworth this is my lsusb -v for Microsoft Kinect. This was done by directly connecting to the pi, without an hub. If you need I can provide a lsusb -v listing with an hub too.

http://pastebin.com/wxY7NQCT

r10r commented Aug 31, 2012

@ghollingworth you can find the output from lsusb -v at https://gist.github.com/3549890. The output from the DVB-C stick is below the full output.

Are you working on this issue Gordon? Is this similar to the spinlock problem in the sdhci driver?

Contributor

ghollingworth commented Aug 31, 2012

Ah...

It seems that they are indeed using high bandwidth transfers... I can
believe that they're not going to work with the current driver...

I do have some ideas about how I can get this working one of which is to
push off the low latency work onto the multimedia processor (the thing most
people think is the GPU!) the other is to implement the low latency work in
the FIQ rather than in the IRQ.

Am going to have to first fix the split transaction issues first Then I
can have a look at this

Thanks

Gordon

On 31 August 2012 08:43, Ruben Jenster notifications@github.com wrote:

direct link is
https://gist.github.com/3549890#file_terratec%20dvb_c%20htc%20stick


Reply to this email directly or view it on GitHubhttps://github.com/raspberrypi/linux/issues/82#issuecomment-8184770.

r10r commented Sep 1, 2012

@ghollingworth Hey Gordon - sounds good.
Just drop me a message if you need someone for testing.

Good luck!
Ruben

Hi,

I was just wondering if anyone made any progress on this. I'm experiencing the same issue (USB buffer overruns) with a PCTv Nanostick T2 TV dongle. This results in a lot of lost MPEG packets when trying to record a TV stream.

Latest kernel & firmware?
If so, neither does smsc95xx.turbo_mode=N nor dwc_otg.speed=1 help separately or together in cmdline.txt ?

Contributor

ghollingworth commented Sep 27, 2012

Currently I don't have a device like that so cannot comment on any progress.

Is it a USB2.0 device?

Thanks

Gordon

On 27/09/2012, grizlak notifications@github.com wrote:

Hi,

I was just wondering if anyone made any progress on this. I'm experiencing
the same issue (USB buffer overruns) with a PCTv Nanostick T2 TV dongle.
This results in a lot of lost MPEG packets when trying to record a TV
stream.


Reply to this email directly or view it on GitHub:
#82 (comment)

Sent from my mobile device

r10r commented Sep 27, 2012

If possible I'll try out the current kernel this evening with the given options and give you feeback.

Regards,
Ruben

r10r commented Sep 27, 2012

@ghollingworth The terratec cinergy htc stick is a USB 2.0 device

The TV device I have is USB 2.0. I'm using Arch with kernel 3.2.27-8 and firmware 20120926.

  • smsc95xx.turbo_mode=N -- no improvement
  • dwc_otg.speed=1 -- no improvement, breaks USB altogether -- connecting a powered USB hub with a keyboard attached to my RPI started causing the OS to hang
  • smsc95xx.turbo_mode=N and dwc_otg.speed=1 -- same as with dwc_otg.speed=1 alone

Are there any newer kernels or firmware that I could try? If I can provide any debugging output that could help you, let me know.

By the way -- is there any reference page maybe with descriptions of the cmdline.txt parameters?

Contributor

ghollingworth commented Sep 27, 2012

What is the throughput you've expecting?

Gordon

On 27/09/2012, Ruben Jenster notifications@github.com wrote:

@ghollingworth The terratec cinergy htc stick is a USB 2.0 device


Reply to this email directly or view it on GitHub:
#82 (comment)

Sent from my mobile device

A typical DVB-T stream in my location is about 24Mbit/s. Ideally I'd like to be able to process DVB-T2 streams, whose bitrates can exceed 30MBit/s, but I'm not sure if it's not too much for the hardware in RPI. Anyway, getting the 24Mbit/s to work would be nice for starters.

BTW, I'm not trying to decode the streams -- just to record them, which I believe is a pure I/O excersise in which the CPU should not have a big impact (but I might be wrong here). I guess recording 24Mbit/s to an SD card isn't likely to work, but hopefully an external USB 2.0 hard drive could do it. But I need to sort out the lost packets problem in the first place.

r10r commented Sep 27, 2012

I think that should work if I'm not completely mistaken. Modern class 10 SDHC cards should make about 20 MByte/s. For me the DVB-C stream size is about 38Mbit/s which is about 4 MByte/s (38000/8/1000).
USB 2.0 can do a maximum of 60MByte/s (48000/8/1000). So this should not be the problem.

wiredbob commented Nov 7, 2012

Hi guys, great thread. Did anyone manage to identify a fix for isochronous transfers? I've got a PCTV Nanostick T2 and have the same issues with glitching of the transport stream as discussed above and would love to get it working.

Thanks.

Contributor

ghollingworth commented Nov 8, 2012

Does anyone have a link to the firmware for the PCTV Nanostick 73e I've got hold of one but don't have the firmware files...

Contributor

ghollingworth commented Nov 8, 2012

Thanks,

Found it... Now trying to work out all the random tools to actually
get it to output the dvb stream!

I'm sure I'll get there in the end but if anyone has a crib note I can
use to get hold of a single channel...

Gordon

On 08/11/2012, grizlak notifications@github.com wrote:

I believe this is what you need:
http://linuxtv.org/downloads/firmware/dvb-usb-dib0700-1.20.fw


Reply to this email directly or view it on GitHub:
#82 (comment)

Sent from my mobile device

You might try dvbstream for starters. This command will grab the whole transport stream (all channels broadcasted on a single frequency) and record 180 seconds of it to a file. You'll be able to play out the file later e.g. in VLC.

dvbstream 8192 -f -n 180 -o > file.mpg

The frequency value is based on where you're located... You can Google around for the list of frequencies available in your area, or scan the whole frequency range with a tool called w_scan.

The frequency comes after "-f", e.g. 650000

Contributor

ghollingworth commented Nov 8, 2012

OK, I ran a w_scan and it came up with the following channels...

T 498000000 8MHz 2/3 NONE QAM64 8k 1/32 NONE
T 522000000 8MHz 2/3 NONE QAM64 8k 1/32 NONE
T 690000000 8MHz 3/4 NONE QAM64 8k 1/32 NONE # Cambs & Beds
T 714000000 8MHz 3/4 NONE QAM64 8k 1/32 NONE # Cambs & Beds
T 722000000 8MHz 3/4 NONE QAM64 8k 1/32 NONE

I'm assuming then I need to do a dvbstream 8192 -f 722000000 -n 180 -o > file.mpg

It just gives me

"Not able to lock to the signal on the given frequency"

Might be that the signal isn't strong enough here (I'm just using the mini antenna supplied with the dongle so don't know how good the signal is going to be) What do you think? Do you think it should work correctly?

Thanks

Gordon

One quick fix you might want to try is input the frequency in kHz instead of Hz, so instead of 722000000 use 722000.
If this doesn't work, we'll try something different.

Contributor

ghollingworth commented Nov 8, 2012

Got it working using tzap and cat'ing dvr0 to a file... Don't get any
error messages yet though

Gordon

On 08/11/2012, grizlak notifications@github.com wrote:

One quick fix you might want to try is input the frequency in kHz instead of
Hz, so instead of 722000000 use 722000.
If this doesn't work, we'll try something different.


Reply to this email directly or view it on GitHub:
#82 (comment)

Sent from my mobile device

It probably won't give you any error messages at this point. You could try and open the resulting file in VLC and look at Tools -> Codec information -> Statistics to see how many corrupted/discontinued frames you get. Or even better, run it through a proper analyzer (http://www.tsreader.com/tsreader/index.html, Windows only) and see how many continuity errors it shows.

wiredbob commented Nov 8, 2012

Yep, you won't see any errors writing the file with tzap. Try playing back the ts with mplayer and you'll see the stats in the console -

e.g.

mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors
A:28462.2 V:28462.2 A-V: 0.008 ct: -0.280 173/173 10% 36% 0.7% 0 0
[mp2float @ 0x7ffaf07264c0]Header missing
[mpeg2video @ 0x7ffaf07264c0]00 motion_type at 8 29
[mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors
A:28462.5 V:28462.5 A-V: 0.033 ct: -0.263 179/179 10% 36% 0.7% 0 0
[mpeg2video @ 0x7ffaf07264c0]00 motion_type at 31 6
[mpeg2video @ 0x7ffaf07264c0]concealing 0 DC, 0 AC, 0 MV errors
A:28462.9 V:28462.9 A-V: 0.018 ct: -0.239 189/189 10% 36% 0.7% 0 0
[mpeg2video @ 0x7ffaf07264c0]00 motion_type at 12 18
[mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors
A:28463.0 V:28463.0 A-V: 0.017 ct: -0.234 192/192 10% 37% 0.7% 0 0
[mpeg2video @ 0x7ffaf07264c0]ac-tex damaged at 10 15
[mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors
A:28463.0 V:28463.0 A-V: 0.012 ct: -0.233 193/193 10% 37% 0.7% 0 0

Contributor

ghollingworth commented Nov 9, 2012

So I just captured a few seconds of video from the pi and simultaneously captured through my usb analyser and then wrote a nice little application to track the pids and output when the continuity count fails...

The only drops I find occurred in the device...

I've also noticed that the PCTV nanoStick 73E I've got seems to have some strange errors in its transmission. I get data length errors... Does anyone know if this has been seen before somewhere, couldn't find any reference to it on google...

Gordon

Contributor

ghollingworth commented Nov 11, 2012

I recorded about a minute of ITV1 using the 73e and had no errors in the stream.

Is there anyone who had trouble streaming from this device?

Gordon

On 08/11/2012, wiredbob notifications@github.com wrote:

Yep, you won't see any errors writing the file with tzap. Try playing back
the ts with mplayer and you'll see the stats in the console -

e.g.

mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors
A:28462.2 V:28462.2 A-V: 0.008 ct: -0.280 173/173 10% 36% 0.7% 0 0
[mp2float @ 0x7ffaf07264c0]Header missing
[mpeg2video @ 0x7ffaf07264c0]00 motion_type at 8 29
[mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors
A:28462.5 V:28462.5 A-V: 0.033 ct: -0.263 179/179 10% 36% 0.7% 0 0
[mpeg2video @ 0x7ffaf07264c0]00 motion_type at 31 6
[mpeg2video @ 0x7ffaf07264c0]concealing 0 DC, 0 AC, 0 MV errors
A:28462.9 V:28462.9 A-V: 0.018 ct: -0.239 189/189 10% 36% 0.7% 0 0
[mpeg2video @ 0x7ffaf07264c0]00 motion_type at 12 18
[mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors
A:28463.0 V:28463.0 A-V: 0.017 ct: -0.234 192/192 10% 37% 0.7% 0 0
[mpeg2video @ 0x7ffaf07264c0]ac-tex damaged at 10 15
[mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors
A:28463.0 V:28463.0 A-V: 0.012 ct: -0.233 193/193 10% 37% 0.7% 0 0


Reply to this email directly or view it on GitHub:
#82 (comment)

Sent from my mobile device

I've never used 73e. But I'm curious if you would get an error-free stream if you tried to record the whole multiplex (dvbstream with dummy PID 8192).

Contributor

ghollingworth commented Nov 12, 2012

I never got dvbstream to work, it just failed to lock no matter what I gave
it...

Gordon

On 12 November 2012 12:09, grizlak notifications@github.com wrote:

I've never used 73e. But I'm curious if you would get an error-free stream
if you tried to record the whole multiplex (dvbstream with dummy PID 8192).


Reply to this email directly or view it on GitHubhttps://github.com/raspberrypi/linux/issues/82#issuecomment-10285062.

I am having the same problem getting lock on a 290e. How did you take the ITV sample?

Contributor

ghollingworth commented Nov 12, 2012

I used x_scan to output a channels.conf file

You then need to copy this into somewhere like ~/.tzap/something

Then tune to the ITV1 channel (-r creates a streaming output) (you can use -S to shut it up
tzap -r "ITV1"

then while tzap is still running:

cat /dev/dvb/adapter0/dvr0 > file.mpg

Gordon

About dvbstream not being able to lock: I'm not sure, but 290e has two frontends (one for DVB-T/T2 and one for DVB-C). Maybe 73e is the same and dvbstream was choosing the wrong frontend by default?

The no-lock problem may be a power one (my 460e stick was not able to lock without being connected to a powered hub).
Anyway, DVB-S, DVB-S2 ou DVB-C is still not working properly here (lot of packet loss, count depends of stream bandwidth)

This happens with kernel 3.6.y too?

I did some tests last week with kernel 3.6.11 and PCTV Nanostick 290e. Same results as with 3.2.27: packets lost with DVB-T.

Contributor

ghollingworth commented Jan 7, 2013

What bitrate were you trying to download and what were you trying to do with it?

Were you just streaming it directly to a file?

Gordon

On 7 Jan 2013, at 07:40, grizlak notifications@github.com wrote:

I did some tests last week with kernel 3.6.11 and PCTV Nanostick 290e. Same results as with 3.2.27: packets lost with DVB-T.


Reply to this email directly or view it on GitHub.

It's the same if i'm streaming with VDR or TVHeadend or using XBMC via Live TV.
But some channels like BFM TV on Astra 19.2E aren't getting so much artefacts (but the bandwith is not high... so...)

Regards,

Contributor

ghollingworth commented Jan 7, 2013

When I was running a simple TV streaming (SD around 2 Mb) I was getting no drops in the data, that was in a place with a very good signal!

Be interested to see what bitrates work and which ones don't

Thanks

Gordon

On 7 Jan 2013, at 08:48, hadjedjvincent notifications@github.com wrote:

It's the same if i'm streaming with VDR or TVHeadend or using XBMC via Live TV.
But some channels like BFM TV on Astra 19.2E aren't getting so much artefacts (but the bandwith is not high... so...)

Regards,


Reply to this email directly or view it on GitHub.

I'm seeing the same dropped packets problem with a TechnoTrend TT-connect S2-3650 CI on a raspberry pi running the latest raspbmc.

I presume because I see in the syslog

'raspbmc kernel: dvb-usb: will pass the complete MPEG2 transport stream to the software demuxer'

The pi can't cope with all the data it is given. Is this likely to be fixed.

Signal is fine as I get no continuity errors using a windows laptop with same sat feed.

Thanks

AndyT

Hi all,

Is there any news about this issue ?

Regards,

ghollingworth was assigned Feb 9, 2013

@swarren swarren pushed a commit to swarren/linux-rpi that referenced this issue Feb 13, 2013

@nxpfrankli @davem330 nxpfrankli + davem330 net: fec: fix spin_lock dead lock
=========================================================
[ INFO: possible irq lock inversion dependency detected ]
3.8.0-rc5+ #82 Not tainted
---------------------------------------------------------
swapper/0/0 just changed the state of lock:
 (&(&fep->hw_lock)->rlock){..-...}, at: [<8034e2f8>] fec_enet_start_xmit+0x48/0x                      2cc
but this lock took another, SOFTIRQ-unsafe lock in the past:
(prepare_lock){+.+.+.}

and interrupts could create inverse lock ordering between them.
other info that might help us debug this:
Possible interrupt unsafe locking scenario:

CPU0				CPU1
----				----
lock(prepare_lock);
				local_irq_disable()
				lock(&(&fep->hw_lock)->rlock);
				lock(prepare_lock);
<Interrupt>
lock(&(&fep->hw_lock)->rlock);

*** DEADLOCK ***

Signed-off-by: Frank Li <Frank.Li@freescale.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
85bd179

Any updates on this? I can confirm having very similar errors with the em28xx and a PCTV 520e (Continuity counter error)...

No news at this time :-( Still not working...

Too bad that the RaspberryPi Team isn't telling us a final statement on
this.

2013/3/12 mkulicke notifications@github.com

Any updates on this? I can confirm having very similar errors with the
em28xx and a PCTV 520e (Continuity counter error)...


Reply to this email directly or view it on GitHubhttps://github.com/raspberrypi/linux/issues/82#issuecomment-14766743
.

Contributor

ghollingworth commented Mar 12, 2013

Current statement is the same as before, still working on USB 1.0 split transactions… Currently nearly at a place where I can release an alpha test release, but need a couple of obvious bugs fixing.

After that we may have time to look at Isochronous endpoints…

Gordon Hollingworth, Director of Engineering
RaspberryPi

On 12 Mar 2013, at 14:27, hadjedjvincent notifications@github.com wrote:

No news at this time :-( Still not working...

Too bad that the RaspberryPi Team isn't telling us a final statement on
this.

2013/3/12 mkulicke notifications@github.com

Any updates on this? I can confirm having very similar errors with the
em28xx and a PCTV 520e (Continuity counter error)...


Reply to this email directly or view it on GitHubhttps://github.com/raspberrypi/linux/issues/82#issuecomment-14766743
.


Reply to this email directly or view it on GitHub.

Hi Gordon,

It's nice that you give us some good news :-)
So it's not an hardware problem, and that is the most important.

Thank you for your feedback.

Regards,

2013/3/12 ghollingworth notifications@github.com

Current statement is the same as before, still working on USB 1.0 split
transactions… Currently nearly at a place where I can release an alpha test
release, but need a couple of obvious bugs fixing.

After that we may have time to look at Isochronous endpoints…

Gordon Hollingworth, Director of Engineering
RaspberryPi

On 12 Mar 2013, at 14:27, hadjedjvincent notifications@github.com
wrote:

No news at this time :-( Still not working...

Too bad that the RaspberryPi Team isn't telling us a final statement on
this.

2013/3/12 mkulicke notifications@github.com

Any updates on this? I can confirm having very similar errors with the
em28xx and a PCTV 520e (Continuity counter error)...


Reply to this email directly or view it on GitHub<
https://github.com/raspberrypi/linux/issues/82#issuecomment-14766743>
.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHubhttps://github.com/raspberrypi/linux/issues/82#issuecomment-14786662
.

Contributor

ghollingworth commented Mar 12, 2013

Well it depends upon what you call a hardware problem… Does a tree make a sound if nobody is around to hear it?

There is definitely a hardware problem, but I've written suitably clever software to work around it!

Gordon

Gordon Hollingworth, Director of Engineering

On 12 Mar 2013, at 17:38, hadjedjvincent notifications@github.com wrote:

Hi Gordon,

It's nice that you give us some good news :-)
So it's not an hardware problem, and that is the most important.

Thank you for your feedback.

Regards,

2013/3/12 ghollingworth notifications@github.com

Current statement is the same as before, still working on USB 1.0 split
transactions… Currently nearly at a place where I can release an alpha test
release, but need a couple of obvious bugs fixing.

After that we may have time to look at Isochronous endpoints…

Gordon Hollingworth, Director of Engineering
RaspberryPi

On 12 Mar 2013, at 14:27, hadjedjvincent notifications@github.com
wrote:

No news at this time :-( Still not working...

Too bad that the RaspberryPi Team isn't telling us a final statement on
this.

2013/3/12 mkulicke notifications@github.com

Any updates on this? I can confirm having very similar errors with the
em28xx and a PCTV 520e (Continuity counter error)...


Reply to this email directly or view it on GitHub<
https://github.com/raspberrypi/linux/issues/82#issuecomment-14766743>
.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHubhttps://github.com/raspberrypi/linux/issues/82#issuecomment-14786662
.


Reply to this email directly or view it on GitHub.

Hardware problem or not, it seems that you can fix it.
I don't know how, but anyway, the result is the most important.

So thanks you !

2013/3/12 ghollingworth notifications@github.com

Well it depends upon what you call a hardware problem… Does a tree make a
sound if nobody is around to hear it?

There is definitely a hardware problem, but I've written suitably clever
software to work around it!

Gordon

Gordon Hollingworth, Director of Engineering

On 12 Mar 2013, at 17:38, hadjedjvincent notifications@github.com
wrote:

Hi Gordon,

It's nice that you give us some good news :-)
So it's not an hardware problem, and that is the most important.

Thank you for your feedback.

Regards,

2013/3/12 ghollingworth notifications@github.com

Current statement is the same as before, still working on USB 1.0
split
transactions… Currently nearly at a place where I can release an alpha
test
release, but need a couple of obvious bugs fixing.

After that we may have time to look at Isochronous endpoints…

Gordon Hollingworth, Director of Engineering
RaspberryPi

On 12 Mar 2013, at 14:27, hadjedjvincent notifications@github.com
wrote:

No news at this time :-( Still not working...

Too bad that the RaspberryPi Team isn't telling us a final statement
on
this.

2013/3/12 mkulicke notifications@github.com

Any updates on this? I can confirm having very similar errors with
the
em28xx and a PCTV 520e (Continuity counter error)...


Reply to this email directly or view it on GitHub<
https://github.com/raspberrypi/linux/issues/82#issuecomment-14766743>
.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub<
https://github.com/raspberrypi/linux/issues/82#issuecomment-14786662>
.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHubhttps://github.com/raspberrypi/linux/issues/82#issuecomment-14797411
.

Hi Gordon,

Glad to hear you're making progress.

8< Does a tree make a sound if nobody is around to hear it?

I think you mean Does a falling tree make a sound if nobody is around to hear it?

Well if it doesn't it implies that the generation of the sound is instigated by:-

1 the tree - meaning the tree is aware of the presence of someone and that that someone is able to hear. This means not just detecting proximity but also detecting the ability to hear, i.e. not make a sound if a person is close but deaf.

2 the person(s) present - implying the person(s) can detect the tree falling without necessarily seeing the tree fall as they may have their back to the tree or be blind.

or

3 someone or something external - effectively proving the existence of an all seeing entity usually given the name 'God'.

Of course, unless the system is 100% bug free there would be cases where the sound is instigated incorrectly or not instigated when it should so there may be occasions when there would be:

a) a sound generated when there is indeed no-one to hear it

b) a tree falls with someone able to hear being present and yet makes no sound

c) the noise of a tree is generated but no tree actually falls

In case c) above there could be a noise generated with no-one to hear it so that generates the question

How can we prove the occurrence of the sound of a tree falling when no tree is actually falling and nobody is around to hear it?

Chasing USB bugs sounds simple in comparison.

Now I must go as I need to get in the fridge because I want to see what happens to the light when I close the door.

David

P.S. I hope this all means we are on the path to sort out #79

Hi there,

Is there any news about this issue, Gordon ?
I've seen a lot of changes into the dwc_otg code last days.

Is there any news about this issue ?

Best regards,
Vincent

Hi there,

I've tried today with the latest firmware & kernel (next branch).
It's working fine with most channels, excepted HD channels that are still having lot of artefacts.

It's near the perfection so ;)

Regards,
Vincent

Contributor

P33M commented May 25, 2013

The "tasklet patch" 79ec5aa fixed a lot of general isochronous behaviour; webcams and SDTV tv sticks now work quite well. With HD and/or uncompressed webcam output at high resolutions (implying very high isoc bitrates) there are still glitches. HD DVB sticks would be "nice" to have working - Pi HDPVR anyone? - but I think the fix for truly reliable isoc will require quite a lot more elbow grease.

Basically at the current state of affairs, if anything disables interrupts within the kernel for more than 125uS then an isoc frame doesn't get scheduled, leading to a "gap" in an otherwise guaranteed-bandwidth transport stream. If we get too many of these in a given time period then the actual vs promised bandwidth difference is likely to result in lost data.

The only way I can see this being less intrusive to other drivers (i.e. force them not to disable interrupts) is to include isoc scheduling in the FIQ. Which is going to be a bit difficult.

Maybe Gordon had some time to think about that since the last time he posted here.

Anyway, SD channels works, that's fine already.
But of course, fixing that bug will fix many others usb devices at the same time so...

Let's wait Gordon answer 👍

Regards,

Contributor

ghollingworth commented May 26, 2013

When you say HD isn't working, what are you doing with it? Are you just
writing the stream to a file? Or are you trying to decode the stream as
well?

I don't think the FIQ patch will make any further differences to this
other than by reducing the CPU overhead by a %Š

P33M's tasklet patch definitely seems to have fixed the problem with these
devices, as for HD, it should work, the bandwidth isn't particularly high
but it depends upon how loaded the processor isŠ

Gordon

I'm trying to stream it over the network. No transcoding.
I've checked the CPU usage, and when streaming an HD channels, it does not go up 60% so everything seems fine there.
I'm using tvheadend 3.4, latest git version from yesterday.
The channel was CANAL+ HD (Astra 19.2E), and was FTA at the moment i've tried.

Regards

zpon commented May 31, 2013

I tried out my nano 520e and got similar results. SD channels seems to able to stream over network okay, HD channels has artifacts or chopping or lagging. Recording HD channels has a little less artifacts, though I only experimented with short recordings (max 10 minutes).
I tested on a recently build image of openelec and the version of tvheadend supplied by openelecs add-on repository.

Let me know if I can done anything to help development.

KrfVan commented Jun 23, 2013

I'm also using a em28xx based dvb stick and I am experiencing the same problem:
em28xx #0/2-dvb: URB packet 11, status -63 [Buffer error (overrun)]
that appear about every 2-3 seconds. The error is caused by a frame overrun error of the dwc_otg usb driver.
The only way to solve this issue is to unbind the smsc95xx ethernet driver.
At that moment only em28xx (ISOC transfer) and mouse and keyboard (INTR transfer) are active on the USB bus. causing no problem at all, even with 100% cpu load.
I first expected the problem could be caused by the INTR endpoint of the the smsc95xx that is only used for reporting changes on the PHY. disabling this endpoint in the driver did however not solve the problem when the smsc95xx driver is active (binded). Even with no ethernet traffic.So it seems the regular polling of the bulk IN endpoint of the smsc95xx is causing the frame overrun of the perioded isoc transport of the em28xx?
I unbinded the smsc95xx and took a usb memory stick to test with another bulk transport device. And the DVB stick was getting the same frame overruns from the moment there was any activity of bulk transport on the usb bus.
My observations so far:
No problem with ISOC transport when there is only periodic transfer on the bus (INTR and ISOC)
Problem starts when there is non-periodic transfer (bulk), the load of bulk trafic does not seem to matter.

The em28xx drivers captures the complete dvb mux, meaning a stream of > 30Mbit/s, this is high but should not be a problem.
It seems sticks that use bulk transfer instead of isoc do not suffer from this problem. I guess the PCTV Nanostick 73e is using bulk transport?

Regarding the SD/HD channels. I've both SD and HD channels in the same mux, all captured together with the em28xx (they all go together over the USB bus and are demuxed on de raspberry). SD channels seem to be less sensitive to data packets that are lost. On HD channels you notice it directly in the image.

Contributor

P33M commented Jun 23, 2013

I profiled this previously using a rather crude approach to determine when transactions actually appear during a microframe.

It seems that the PHY core will sometimes do a periodic transaction later in the microframe than it "should" do if the requests were processed the way the code seems to do them (it does periodic channels first, queues them to the hardware, then uses any leftover bandwidth to do non-periodic channels). For reasons unknown, if you have a lot of non-periodic traffic on the bus then sometimes a periodic transaction will be deferred to later in the microframe.

This has an important consequence if the transaction happens to occur closer to the end of the microframe than the typical interrupt latency (20uS); namely the hardware will instead raise a frmovrun interrupt instead of transaction complete, despite the data being transferred successfully (for an IN). The driver then throws the data away and reports back with -63 in the iso_desc status field.

It is typical of cheaper DVB sticks to just stream the entire mux; that means they don't have to implement packet filtering hardware to selectively pick PID/VIDs out of the DVB stream. You will probably see more corruption in HD channels simply because they have the lion's share of the bandwidth - if you drop random packets, then chances are that it will be a HD one that drops and not an SD one.

This is easily reproducible with cheap webcams streaming at 640x480 with uncompressed formats - typical bandwidth used is of the order of the bandwidth used by DVB dongles.

KrfVan commented Jun 24, 2013

@P33M I agree with most of your explanation, except that a lot periodic traffic is needed to trigger this issue. I see the issue popping up with very little non-periodic activity.
I also altered the dwc_otg interrupt routines to handle a frmovrun as xfercomple, but it seems not all IN traffic is received correctly. I still get distortion in my image (although less then before).
the em28xx supports pid filter in hardware, but it's not implemented in the linux driver. Probably because of lack of publicly available datasheets.

KrfVan commented Aug 4, 2013

I have found a work-around to prevent this issue. It seems, when non-periodic transactions are queued to close to the end of a microframe, periodic transactions are sometimes pushed very late in the next microframe causing frame overrun. My trick is to prevent transactions being queued to a channel if we are close to the end of a microframe. I've currently implemented it on 1fe837e (commit before fiq_split patch) and have to disable fiq to get it working. This is what I add in dwc_otg_hcd.c (sorry I'm not familiar with git) in dwc_otg_hcd_select_transactions

@@ dwc_otg_transaction_type_e dwc_otg_hcd_select_transactions(dwc_otg_hcd_t * hcd)
        dwc_irqflags_t flags;
        dwc_spinlock_t *channel_lock = hcd->channel_lock;
        dwc_otg_transaction_type_e ret_val = DWC_OTG_TRANSACTION_NONE;
-
+       hfnum_data_t hfnum;
+       gintmsk_data_t intr_mask = {.d32 = 0 };
+       // Do not queue transitions of there is less then the selected time available in a uframe
+       hfnum.d32 =
+                DWC_READ_REG32(&hcd->core_if->host_if->host_global_regs->hfnum);
+       intr_mask.d32 = DWC_READ_REG32(&hcd->core_if->core_global_regs->gintmsk);
+       if ((hfnum.b.frrem < 3000) && intr_mask.b.sofintr) {return ret_val;}

It is working well with frrem < 3000 (I guess this is the middle of a microframe?). Probably a lot of bandwidth gets lost on the bus, but with this value I do not get any frame overuns and my image is clean. I can even stream it over ethernet without any issues.

Contributor

P33M commented Aug 4, 2013

That's odd. There's no sensible reason why the hardware should do that... But then this wouldn't be the first time the hardware has exhibited wierd behaviour.

I believe frrem runs off the PHY core clock which is either 60 or 48MHz. A count of 3000 is approximately 40% of uframe remaining with 60Mhz and 50% with 48MHz.

You are correct that this will clobber nonperiodic bandwidth - each completed non-periodic transaction will attempt to queue more transfers in a round-robin affair for all the endpoints with work to do. If we're losing a large chunk of a microframe then nonperiodic bandwidth, including ethernet, will suffer.

I shall have to test this and see if we can't include it as a big hacky module option - fix high-bandwidth isoc but hamstring everything else.

Contributor

Ferroin commented Aug 4, 2013

@P33M @KrfVan Is there any way it could be written to check the size of the transaction that is being queued and reject it if there isn't enough room left in the microframe?

KrfVan commented Aug 21, 2013

I've implemented my fix as a module parameter, the number of remaining clock cycles after which no transactions can be queued to the host can be controlled by "trans_backoff". If it is set to 0 (the default value), everything is as before.
Currently I'm using the following cmdline.txt parameters to get DVB reception and network streaming going well.

dwc_otg.fiq_fix_enable=0 dwc_otg.fiq_split_enable=0 dwc_otg.trans_backoff=3000

I also have nak_holdoff and microframe schedule disabled (not tested in enabled state yet)

Here is the patch for the latest 3.9.y branch: 56ec798

diff --git a/drivers/usb/host/dwc_otg/dwc_otg_driver.c b/drivers/usb/host/dwc_otg/dwc_otg_driver.c
index 4735f51..4057421 100644
--- a/drivers/usb/host/dwc_otg/dwc_otg_driver.c
+++ b/drivers/usb/host/dwc_otg/dwc_otg_driver.c
@@ -246,8 +246,8 @@ extern bool fiq_fix_enable;
 bool fiq_split_enable = true;
 //Global variable to switch the nak holdoff on or off
 bool nak_holdoff_enable = true;
-
-
+//Global variable to enable transaction uframe backoff
+int trans_backoff = 0;
 /**
  * This function shows the Driver Version.
  */
@@ -1384,6 +1384,9 @@ module_param(nak_holdoff_enable, bool, 0444);
 MODULE_PARM_DESC(nak_holdoff_enable, "Enable the NAK holdoff");
 module_param(fiq_split_enable, bool, 0444);
 MODULE_PARM_DESC(fiq_split_enable, "Enable the FIQ fix on split transactions");
+module_param(trans_backoff, int, 0444);
+MODULE_PARM_DESC(trans_backoff,
+        "Transaction uframe backoff in clock cycles");

 /** @page "Module Parameters"
  *
diff --git a/drivers/usb/host/dwc_otg/dwc_otg_hcd.c b/drivers/usb/host/dwc_otg/dwc_otg_hcd.c
index be1d25b..928afcf 100644
--- a/drivers/usb/host/dwc_otg/dwc_otg_hcd.c
+++ b/drivers/usb/host/dwc_otg/dwc_otg_hcd.c
@@ -48,6 +48,7 @@
 #include "dwc_otg_mphi_fix.h"

 extern bool microframe_schedule, nak_holdoff_enable;
+extern int trans_backoff;

 //#define DEBUG_HOST_CHANNELS
 #ifdef DEBUG_HOST_CHANNELS
@@ -1393,7 +1394,15 @@ dwc_otg_transaction_type_e dwc_otg_hcd_select_transactions(dwc_otg_hcd_t * hcd)
    dwc_irqflags_t flags;
    dwc_spinlock_t *channel_lock = hcd->channel_lock;
    dwc_otg_transaction_type_e ret_val = DWC_OTG_TRANSACTION_NONE;
-
+   hfnum_data_t hfnum;
+   gintmsk_data_t intr_mask = {.d32 = 0 };
+   if (trans_backoff) {
+       // Do not queue transitions of there is less then the selected time available in a uframe
+       hfnum.d32 =
+               DWC_READ_REG32(&hcd->core_if->host_if->host_global_regs->hfnum);
+       intr_mask.d32 = DWC_READ_REG32(&hcd->core_if->core_global_regs->gintmsk);
+       if ((hfnum.b.frrem < trans_backoff) && intr_mask.b.sofintr) {return ret_val;}
+       }
 #ifdef DEBUG_SOF
    DWC_DEBUGPL(DBG_HCD, "  Select Transactions\n");
 #endif
diff --git a/drivers/usb/host/dwc_otg/dwc_otg_hcd_intr.c b/drivers/usb/host/dwc_otg/dwc_otg_hcd_intr.c
index 19abea0..78172ea 100644
--- a/drivers/usb/host/dwc_otg/dwc_otg_hcd_intr.c
+++ b/drivers/usb/host/dwc_otg/dwc_otg_hcd_intr.c
@@ -742,8 +742,10 @@ int32_t dwc_otg_hcd_handle_sof_intr(dwc_otg_hcd_t * hcd)
    }

    /* Clear interrupt */
-   //gintsts.b.sofintr = 1;
-   //DWC_WRITE_REG32(&hcd->core_if->core_global_regs->gintsts, gintsts.d32);
+   if (!fiq_fix_enable) {
+       gintsts.b.sofintr = 1;
+       DWC_WRITE_REG32(&hcd->core_if->core_global_regs->gintsts, gintsts.d32);
+   }

    return 1;
 }

Hi,

Is anyone able to provide the kernel.img with your diff integrated so we can test too ?

Regards,

@swarren swarren pushed a commit to swarren/linux-rpi that referenced this issue Sep 10, 2013

Andrew Morton + Stephen Rothwell swap-make-swap-discard-async-checkpatch-fixes
WARNING: line over 80 characters
#81: FILE: include/linux/swap.h:243:
+	struct swap_cluster_info discard_cluster_head; /* list head of discard clusters */

WARNING: line over 80 characters
#82: FILE: include/linux/swap.h:244:
+	struct swap_cluster_info discard_cluster_tail; /* list tail of discard clusters */

ERROR: space prohibited before that '--' (ctx:WxO)
#255: FILE: mm/swapfile.c:449:
+				si->cluster_nr --;
 				               ^

total: 1 errors, 2 warnings, 310 lines checked

./patches/swap-make-swap-discard-async.patch has style problems, please review.

If any of these errors are false positives, please report
them to the maintainer, see CHECKPATCH in MAINTAINERS.

Please run checkpatch prior to sending patches

Cc: Shaohua Li <shli@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
163359d

@P33M P33M pushed a commit to P33M/linux that referenced this issue Oct 25, 2013

P33M dwc_otg: Implement trans_backoff paramater hack
This adds a configurable, disabled by default parameter that prevents
queueing of non-periodic packets if there are less than trans_backoff
PHY core clocks remaining in the microframe. This is noted to assist
isochronous performance with high bandwidths - see #82.

Caution: Do not set this value close to or above the number of PHY
core clocks in a microframe (6000). Doing so will prevent any non-
periodic transactions from happening.

Thanks to @KrfVan for the original experimentation work.
243570b
Contributor

P33M commented Oct 25, 2013

I've implemented @KrfVan's fix on my own repository https://github.com/P33M/linux. Can someone with a functional DVB device that is experiencing frame loss please build and test?

zpon commented Oct 28, 2013

I have a Quatro Stick nano 520e and will be able to test, but I will need help compiling the kernel and creating a functioning image. (I have build OpenElec before, but I guess it will not be possible to simply drop in the source from your repo?)

Contributor

P33M commented Oct 28, 2013

It's easiest to cross-compile from a Linux PC or similar.

http://elinux.org/RPi_Kernel_Compilation

kripton commented Oct 28, 2013

My experiences:
I never bothered to try the RPi as DVB receiver until today. I have one http://www.linuxtv.org/wiki/index.php/Technisat_SkyStar_USB_HD here and hooked it up to the Pi today. Currently I have 3.11.6+ running PLUS the "Implement trans_backoff paramater hack" commit on top.
On the RPi, I installed tvheadend and set it up. Now there's one HD (German "Das Erste HD") channel beeing streamed from the Pi via HTTP to my laptop where it is played live using mplayer. CPU usage on the Pi is a bit over 50% and mplayer doesn't report any errors/decoding glitches.
I didn't even bother playing around with the trans_backoff-parameter yet since it works just perfectly here. Maybe tomorrow there'll be time to test my Edirol UA-101 USB audio interface (no DVB but also a lot of isoc and didn't work previously with the Pi). However, I'm in the process of moving houses so testing might not have the highest priority.
P33M: What's the correct way to set the trans_backoff parameter when the USB driver is built into the kernel and not as module? "dwc_otg_module_params.trans_backoff=2000" ?
Anything special you want to be tested? Devices I can provide: Technisat SkyStar USB HD, Edirol UA-101, Twinhan VisionDTV/MagixBox (USB DVB-T)

zpon commented Oct 28, 2013

After spending some hours trying to get a working image created I realized that raspbmc already has this patch enabled (see http://svn.stmlabs.com/svn/raspbmc/patches/kernel-3.10.patch and http://svn.stmlabs.com/svn/raspbmc/testing/kernel/build_kernel-3.10.sh).
When streaming the video is still lagging, however, when recording the stream on the sd-card the video seems to quite good, however I need to do some more testing before being sure.
I was told that USB ports and the ethernet connection uses the same bus on the RPi, this might explain why I get better quality when recording the stream rather than streaming it via the ethernet connection.

Contributor

P33M commented Oct 28, 2013

To set the module parameter in my patch, add the commandline argument dwc_otg.trans_backoff=xxxx to /boot/cmdline.txt.

Most of the problem reports so far have been from high-bandwidth isochronous traffic. SD channels typically do not provide enough isoc bandwidth to start dropping packets, but HD channels do.

These are the key things that I would like tested:

  • What are the range of trans_backoff values (0 has no effect, 4000 is probably the useable maximum and 6000 breaks things) that result in better/worse video? Try to use player statistics and not just visual, some players may hide mpeg2 errors.
  • Is there any difference in video performance for equivalent values of trans_backoff between the patch in raspbmc and mine? The current one implemented in raspbmc will drop periodic transactions if they are queued when trans_backoff is in effect.

tvjon commented Oct 29, 2013

Kripton,

does your kernel have P33M's patch or Raspbmc's? If the former, can you post your kernel somewhere please? Sam Nazarko's patch is an improvement but it sounds like P33M's may finally fix the problem. I haven't moved to a DVB-S tuner for the Pi yet, as if its USB can't cope with terrestrial HD bitrate, there's no chance on DVB-S. However, it's now looking more promising (all my testing is MPEG4). I have a QBOX ii receiver but their support man says it's "incompatible" with the Pi. That leaves your Skystar for which drivers obviously exist, Skyview, & a couple of others, all of which are rather more recent than the Skystar, & lower power consumption.

Thanks.

zpon commented Oct 29, 2013

I'm not entirely sure what kind of statistics you are referring to, but here is the output from mplayer for two different recordings: http://pastebin.com/we86WDT2 - as you probably can tell the last one did not play well. Both recordings was made with the default raspbmc distribution and kernel.
If you need different statistics please let me know exactly how to gather them.

Update:
Additional tests: http://pastebin.com/5wZv77ev - the two first files was from the same tv channel as the one which had problems in the first pastbin link.

kripton commented Oct 29, 2013

tvjon,
I used the "offical" kernel from github, branch "rpi-3.11.y" (was at 99f40fe when I checked it out) and applied P33M's patch on top of it (git cherry-pick 243570b). The resulting kernel (alongside the modules) can be found at http://kripton.kripserver.net/linux/rpi_dvbs_kernel_and_modules_3.11.6.tar.xz The cmdline.txt is also there. However, it looks like the patch doesn't really work, it maybe depends on stuff only found in the 3.6.y branch. At least nothing breaks using "dwc_otg.trans_backoff=7000". Nothing is different relating to the value of the parameter.

As I told lately, with that kernel, I'm able to stream a DVB-S2 HD channel via HTTP (tvheadend) from the RPi to my laptop and mplayer there (running in the console) doesn't show any decoder errors. So it's not just visually okay. And since the SkyStar USB HD cannot do hardware TS-filtering, it's one complete DVB-S2 transponder pushed from the DVB device to the RPi (where it's filtered in software).
However when using my Logitech C920 webcam (I forgot that device in the list last time) and streaming using gstreamer (sender, rpi: "gst-launch-1.0 v4l2src ! video/x-h264, width=1024, height=576, framerate=30/1 ! h264parse ! matroskamux streamable=true ! tcpserversink host=192.168.189.53 port=5000" receiver, laptop: "gst-launch-1.0 -vvvv tcpclientsrc host=192.168.189.53 port=5000 ! matroskademux ! avdec_h264 ! xvimagesink sync=false") there are no decoding errors or glitches but the picture freezes from time to time (about two times in 3 minutes, lasting for 10-20 seconds each time). Might be another problem.

Since I won't have internet the next week: Happy bughunting!

tvjon commented Oct 30, 2013

kripton,

The argument I use for trans_backoff does make quite a difference with the Sam Nazarko posted files.

I'd like to use the same RPi for recording & playback, rather than several multiple Pi setups, so I haven't experimented with TVHeadend, etc. for a while.

This means using omxplayer for playback, & it usually does improve with each iteration.

I meant DVB-S2 in my previous post. DVB-S is ok anyway. I've concentrated on DVB-T2 so far though, to have at least a fighting chance of success :)
One problem with using DVB-T2 is that it's region dependent, so a Country not far away may have a different HD standard, so it's hard to draw conclusions from other posters' results.

Interestingly, my DVB-T2 tuner can do hardware TS filtering, but apparently the linux driver doesn't support that, so it's the whole multiplex.

It is encouraging that you're getting good DVB-S2 results. I already have a skystar usb box, but that's relatively old & won't do S2. I need to decide which to buy.

I've also seen freezing. Interestingly, no artefacts at that point, & AV resumes where it left off, as though I'd pressed the pause key :)

Thanks for the files. I've installed them, & RPi boots to the login prompt, but refuses to accept any keyboard input. SSH can't talk to it either. I'll try another installation later. If that fails, I'll dig out my serial to usb & see what I can find. I see your zimage is quite a bit smaller than the 3.10.x kernel I'm using.

No internet? For a week? You'll get withdrawal symptoms :)

Anything interesting, & I'll post back anyway. Thanks again for taking the time to post your files & your results.

Contributor

P33M commented Oct 30, 2013

I've got a DVB-T2 compatible dongle on order. Should be able to prove this myself before the week is out.

tvjon commented Oct 31, 2013

In fact P33M, I was about to offer to lend mine to PiTowers (or wherever) in case it helped for testing, so you beat me to it :) You won't be disappointed by the quality assuming you've cracked the USB problem. We've compared the quality with our Freesat HD STB, & the Freesat is a bit sharper, but I'm sure that's to do with DVB-S2's higher bit rate for the respective channels. The great thing about the DVB-T2 dongle compared to our STB, is that recordings are safe if anything goes wrong with the tuner. If our STB ever dies, so do our recordings, as they're "locked" (encrypted) to the actual STB itself!!
I'm reasonably encouraged now by recent progress from KrfVan, kripton & yourself, to source a decent, modern DVB-S2 usb box, thus having both terrestrial & satellite capability, handy in stormy weather :)

PCTV 290e DVB-T2 dongle is working great with raspbmc sept 2013.
NB that I am using as a headless backend with XBMC disabled - not tried with the GUI up too.

Contributor

P33M commented Nov 4, 2013

facepalm

I managed to get a DVB-T2 dongle that isn't supported by Linux and in fact has a nonstandard Windows 7 driver. The SD dongle that I bought does work however, and appears to stream the entire mux.

tvjon commented Nov 4, 2013

That's a shame. What's its make & model # please?

Yes, SD is a lot less taxing but not as nice to watch as its HD equivalent.

Contributor

P33M commented Nov 4, 2013

It is an August DVB-T210. Both the demodulator and frontend chips are apparently supported by Linux, it's just the bridge chip that appears to be unrecognised.

USB VID = 0x1F4D PID = 0xD220

Contributor

P33M commented Nov 6, 2013

@popcornmix that patch worked - now have a functional dongle.

With tvheadend and a streaming arrangement via the network, a single SD channel plays without error. If I then start streaming a second SD channel from the same multiplex, the second channel exhibits an order of magnitude more errors (by eye). I suspect that the isochronous stream is being packed with the first channel's PIDs being sent first, before the second filtered PIDs which means that the second channel is more likely to see lost packets.

Now I need to correlate this with error rates reported by the dwc driver and see if there is a link. THEN I can get the analyser on so that I know what I'm looking for in the 20Mbit/s deluge of packets in and out.

@swarren swarren pushed a commit to swarren/linux-rpi that referenced this issue Nov 22, 2013

Borislav Petkov + Ingo Molnar x86/boot: Further compress CPUs bootup message
Turn it into (for example):

[    0.073380] x86: Booting SMP configuration:
[    0.074005] .... node   #0, CPUs:          #1   #2   #3   #4   #5   #6   #7
[    0.603005] .... node   #1, CPUs:     #8   #9  #10  #11  #12  #13  #14  #15
[    1.200005] .... node   #2, CPUs:    #16  #17  #18  #19  #20  #21  #22  #23
[    1.796005] .... node   #3, CPUs:    #24  #25  #26  #27  #28  #29  #30  #31
[    2.393005] .... node   #4, CPUs:    #32  #33  #34  #35  #36  #37  #38  #39
[    2.996005] .... node   #5, CPUs:    #40  #41  #42  #43  #44  #45  #46  #47
[    3.600005] .... node   #6, CPUs:    #48  #49  #50  #51  #52  #53  #54  #55
[    4.202005] .... node   #7, CPUs:    #56  #57  #58  #59  #60  #61  #62  #63
[    4.811005] .... node   #8, CPUs:    #64  #65  #66  #67  #68  #69  #70  #71
[    5.421006] .... node   #9, CPUs:    #72  #73  #74  #75  #76  #77  #78  #79
[    6.032005] .... node  #10, CPUs:    #80  #81  #82  #83  #84  #85  #86  #87
[    6.648006] .... node  #11, CPUs:    #88  #89  #90  #91  #92  #93  #94  #95
[    7.262005] .... node  #12, CPUs:    #96  #97  #98  #99 #100 #101 #102 #103
[    7.865005] .... node  #13, CPUs:   #104 #105 #106 #107 #108 #109 #110 #111
[    8.466005] .... node  #14, CPUs:   #112 #113 #114 #115 #116 #117 #118 #119
[    9.073006] .... node  #15, CPUs:   #120 #121 #122 #123 #124 #125 #126 #127
[    9.679901] x86: Booted up 16 nodes, 128 CPUs

and drop useless elements.

Change num_digits() to hpa's division-avoiding, cell-phone-typed
version which he went at great lengths and pains to submit on a
Saturday evening.

Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: huawei.libin@huawei.com
Cc: wangyijing@huawei.com
Cc: fenghua.yu@intel.com
Cc: guohanjun@huawei.com
Cc: paul.gortmaker@windriver.com
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20130930095624.GB16383@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
a17bce4

is there a fix to get PCTV 521e also running? i think i need only a patched fw? right?

@justvanbloom i don't think that a there is a patched fw, that is a driver problem (i got 460e stick).
We need to wait @P33M @popcornmix @ghollingworth final patch in order to avoid all theses USB errors :-)

@swarren swarren pushed a commit to swarren/linux-rpi that referenced this issue Feb 12, 2014

@htejun @gregkh htejun + gregkh kernfs: make kernfs_deactivate() honor KERNFS_LOCKDEP flag
kernfs_deactivate() forgot to check whether KERNFS_LOCKDEP is set
before performing lockdep annotations and ends up feeding
uninitialized lockdep_map to lockdep triggering warning like the
following on USB stick hotunplug.

 usb 1-2: USB disconnect, device number 2
 INFO: trying to register non-static key.
 the code is fine but needs lockdep annotation.
 turning off the locking correctness validator.
 CPU: 1 PID: 62 Comm: khubd Not tainted 3.13.0-work+ #82
 Hardware name: empty empty/S3992, BIOS 080011  10/26/2007
  ffff880065ca7f60 ffff88013a4ffa08 ffffffff81cfb6bd 0000000000000002
  ffff88013a4ffac8 ffffffff810f8530 ffff88013a4fc710 0000000000000002
  ffff880100000000 ffffffff82a3db50 0000000000000001 ffff88013a4fc710
 Call Trace:
  [<ffffffff81cfb6bd>] dump_stack+0x4e/0x7a
  [<ffffffff810f8530>] __lock_acquire+0x1910/0x1e70
  [<ffffffff810f931a>] lock_acquire+0x9a/0x1d0
  [<ffffffff8127c75e>] kernfs_deactivate+0xee/0x130
  [<ffffffff8127d4c8>] kernfs_addrm_finish+0x38/0x60
  [<ffffffff8127d701>] kernfs_remove_by_name_ns+0x51/0xa0
  [<ffffffff8127b4f1>] remove_files.isra.1+0x41/0x80
  [<ffffffff8127b7e7>] sysfs_remove_group+0x47/0xa0
  [<ffffffff8127b873>] sysfs_remove_groups+0x33/0x50
  [<ffffffff8177d66d>] device_remove_attrs+0x4d/0x80
  [<ffffffff8177e25e>] device_del+0x12e/0x1d0
  [<ffffffff819722c2>] usb_disconnect+0x122/0x1a0
  [<ffffffff819749b5>] hub_thread+0x3c5/0x1290
  [<ffffffff810c6a6d>] kthread+0xed/0x110
  [<ffffffff81d0a56c>] ret_from_fork+0x7c/0xb0

Fix it by making kernfs_deactivate() perform lockdep annotations only
if KERNFS_LOCKDEP is set.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Fabio Estevam <festevam@gmail.com>
Reported-by: Alan Stern <stern@rowland.harvard.edu>
Reported-by: Jiri Kosina <jkosina@suse.cz>
Reported-by: Dave Jones <davej@redhat.com>
Tested-by: Fabio Estevam <fabio.estevam@freescale.com>
Tested-by: Jiri Kosina <jkosina@suse.cz>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
da9846a

@swarren swarren pushed a commit to swarren/linux-rpi that referenced this issue Feb 12, 2014

@htejun @gregkh htejun + gregkh kernfs: make kernfs_deactivate() honor KERNFS_LOCKDEP flag
kernfs_deactivate() forgot to check whether KERNFS_LOCKDEP is set
before performing lockdep annotations and ends up feeding
uninitialized lockdep_map to lockdep triggering warning like the
following on USB stick hotunplug.

 usb 1-2: USB disconnect, device number 2
 INFO: trying to register non-static key.
 the code is fine but needs lockdep annotation.
 turning off the locking correctness validator.
 CPU: 1 PID: 62 Comm: khubd Not tainted 3.13.0-work+ #82
 Hardware name: empty empty/S3992, BIOS 080011  10/26/2007
  ffff880065ca7f60 ffff88013a4ffa08 ffffffff81cfb6bd 0000000000000002
  ffff88013a4ffac8 ffffffff810f8530 ffff88013a4fc710 0000000000000002
  ffff880100000000 ffffffff82a3db50 0000000000000001 ffff88013a4fc710
 Call Trace:
  [<ffffffff81cfb6bd>] dump_stack+0x4e/0x7a
  [<ffffffff810f8530>] __lock_acquire+0x1910/0x1e70
  [<ffffffff810f931a>] lock_acquire+0x9a/0x1d0
  [<ffffffff8127c75e>] kernfs_deactivate+0xee/0x130
  [<ffffffff8127d4c8>] kernfs_addrm_finish+0x38/0x60
  [<ffffffff8127d701>] kernfs_remove_by_name_ns+0x51/0xa0
  [<ffffffff8127b4f1>] remove_files.isra.1+0x41/0x80
  [<ffffffff8127b7e7>] sysfs_remove_group+0x47/0xa0
  [<ffffffff8127b873>] sysfs_remove_groups+0x33/0x50
  [<ffffffff8177d66d>] device_remove_attrs+0x4d/0x80
  [<ffffffff8177e25e>] device_del+0x12e/0x1d0
  [<ffffffff819722c2>] usb_disconnect+0x122/0x1a0
  [<ffffffff819749b5>] hub_thread+0x3c5/0x1290
  [<ffffffff810c6a6d>] kthread+0xed/0x110
  [<ffffffff81d0a56c>] ret_from_fork+0x7c/0xb0

Fix it by making kernfs_deactivate() perform lockdep annotations only
if KERNFS_LOCKDEP is set.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Fabio Estevam <festevam@gmail.com>
Reported-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
a660793
Contributor

P33M commented Feb 17, 2014

As an informational point: I can confirm that queueing packets near to the end of a microframe causes odd behaviour. Either:

  1. The packet is queued at the EOF1 point
  2. The packet is queued right after SOF (due to interrupt latency) but with an incorrectly set oddfrm bit.

One or both of these cause one of two things to go wrong:

  1. The packet pops out in microframe n+2
  2. The packet pops out in microframe N+1, but the channel halt interrupt is not received until coincident with the SOF for microframe N+2.

The transaction backoff is a bit a of a blunderbuss to fix what is a rather fine-grained problem. The ideal solution is to use the FIQ to guarantee that packets are queued when they are meant to be queued.

KrfVan commented Feb 18, 2014

I agree the transaction backoff is not nice nor a good solution. It just skips transactions that would be queued to late and relies on the the buffer of the usb client devices being big enough to not loose data when it is not transferred every microframe. I've noticed however data loss every now and then.
Isoc transfers occurring every microframe get queued after a transfer in the current microframe is transferred to the host and the channel is released. This can be very late in the microframe and interrupt latency postpones the queueing even more. As P33M suggest, here the FIQ could help by having priority over other interrupts and reduce the latency.
A few months back I started looking at another solution without the FIQ, but due to lack of time my raspberry is collecting dust (real pity). By using two channels for high bandwidth isoc transfers that occur every microframe, one channel can be queued to the hardware for the next microframe at the SOF interrupt while the other channel starts transferring. Each channel then occurs every two microframes and does not have to wait for the release of the other channel to get queued to the hardware.

tvjon commented Feb 19, 2014

Thank you for the information P33M. It's encouraging to see it's still being worked on. One observation is that initially I unbound Ethernet before making a recording but later found that simply unplugging the Ethernet lead was sufficient to further reduce artefacts during reception.

I've been unable to use the 290e tuner with the last 3 kernel & firmware iterations as there's a

"gpiochip_find_base: cannot find free range"

error, resulting in failure to initialise its cxd2820r demodulator.

So far, in spite of limitations as you point out, KrfVan's modification has been the sole means of using the 290e for mostly artefact free reception.

I've now bought a 461e DVB-S2 tuner which I was hoping to test, as it has recently had Linux drivers written for it. However, although building in the "media build" Linux package without errors or warnings, causes kernel panic on attaching the tuner to RPi. That may not be the drivers' fault though, rather the particular libxxx-dev libraries' implementation.

KrfVan, your dual channel idea looks interesting. Let's hope you can squeeze some time in for your Pi!

Contributor

P33M commented Feb 19, 2014

@KrfVan the process you describe is in fact one of the mechanisms my FIQ rewrite uses with split transactions. A single host channel is reserved for the entirety of a transfer, and the FIQ just keeps poking it at the right time. Currently this only applies to split transactions but can be extended to general multi-transfer high-speed isochronous.

Interleaving 2 host channels in the current driver would still be vulnerable to interrupt latency greater than 125uS: consider the case where SoF is blocked for nearly a full microframe - the 2nd host channel doesn't get queued until the following microframe so the same symptom results.

If the FIQ did the transfer then the only critical points would be at the start/end of a full URB's worth of data (usually 32 or 64 microframes of transactions) - this would reduce the chance that randomly distributed interrupt latency will have any effect on the transaction.

KrfVan commented Feb 28, 2014

@P33M had a quick look to your FIQ code, really nice work. I noticed a lot is already in place to support the high-speed isochronous transfers, but currently only one microframe is handled. What are the missing pieces to handle a complete urb in the fiq?

Contributor

P33M commented Feb 28, 2014

It's a state machine, so it just stays in the same state until it runs out of iso_frame_desc elements.

It's also WIP and something that's in the "hacked together" state rather than beta.

Edit: if you're referring to one of my oblique comments about microframe intervals then yes, the case where interval=1 is the only one that's currently handled. I'd need to hook more stuff into the SOF handler to handle the case where the interval is >1, but such devices with the interval != 1 are rare.

Contributor

P33M commented Mar 5, 2014

Hm. I have Isoc IN support working in the FIQ...

As expected, the vast majority of transactions complete successfully according to the FIQ (with a failure rate of 1 in 10,000).

Using my terribad USB2.0 ebay special webcam, this pushes guvcview to near 30fps playback at 640x480 YUYV but with uvcvideo.nodrop=1 which still produces garbled frames. With nodrop=0 15-18fps is the norm. The CPU overhead of an isochronous stream is clearly less: there is idle CPU time whereas before there wasn't.

The problem now is feeding the FIQ fast enough. It seems that this webcam cannot tolerate a 1 microframe gap in transactions which seems to happen quite often in 32-microframe intervals (i.e. the transfer length of a URB). I wonder if I can do some sort of double-buffering and interleave transactions slightly...

Contributor

P33M commented Mar 6, 2014

The BRANCH=next firmware now has beta support for HS Isochronous acceleration performed by the FIQ. I can get about 25-27fps from a 640x480 webcam that uses a 192mbit/s (i.e. maxed out) stream size. The limitation appears to be CPU time - with the ARM at 900MHz it is at 99% usage. I've not yet profiled the FIQ to see how long it is taking but I assume that is is approx 1uS - similar to split transaction processing.

You will need to set dwc_otg.fiq_fsm_enable=1 and dwc_otg.fiq_fsm_mask=0x7 in /boot/cmdline.txt to enable the acceleration.

Annoyingly, the dvb-usb driver for both my SD and HD DVB-T dongles enables bulk mode which forces me to test using webcams. The irony is that isochronous may actually end up the better transport in terms of CPU loading when using the FIQ...

Contributor

P33M commented Mar 24, 2014

Closing: HS Isochronous IN now reliable in the FIQ.

P33M closed this Mar 24, 2014

@popcornmix popcornmix pushed a commit that referenced this issue Oct 1, 2014

Guido Martínez + Jiri Slaby drm/tilcdc: slave: fix dangling sysfs connector node
commit daa15b4 upstream.

Add a drm_sysfs_connector_remove call when we destroy the panel to make
sure the connector node in sysfs gets deleted.

This is required for proper unload and re-load of this driver as a
module. Without this, we would get a warning at re-load time like so:

   tda998x 0-0070: found TDA19988
   ------------[ cut here ]------------
   WARNING: CPU: 0 PID: 825 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x54/0x74()
   sysfs: cannot create duplicate filename '/class/drm/card0-HDMI-A-1'
   Modules linked in: [..]
   CPU: 0 PID: 825 Comm: modprobe Not tainted 3.15.0-rc4-00027-g9dcdef4 #82
   [<c0013bb8>] (unwind_backtrace) from [<c0011824>] (show_stack+0x10/0x14)
   [<c0011824>] (show_stack) from [<c0034e8c>] (warn_slowpath_common+0x68/0x88)
   [<c0034e8c>] (warn_slowpath_common) from [<c0034edc>] (warn_slowpath_fmt+0x30/0x40)
   [<c0034edc>] (warn_slowpath_fmt) from [<c01243f4>] (sysfs_warn_dup+0x54/0x74)
   [<c01243f4>] (sysfs_warn_dup) from [<c0124708>] (sysfs_do_create_link_sd.isra.2+0xb0/0xb8)
   [<c0124708>] (sysfs_do_create_link_sd.isra.2) from [<c02ae37c>] (device_add+0x338/0x520)
   [<c02ae37c>] (device_add) from [<c02ae6e8>] (device_create_groups_vargs+0xa0/0xc4)
   [<c02ae6e8>] (device_create_groups_vargs) from [<c02ae758>] (device_create+0x24/0x2c)
   [<c02ae758>] (device_create) from [<c029b4ec>] (drm_sysfs_connector_add+0x64/0x204)
   [<c029b4ec>] (drm_sysfs_connector_add) from [<bf0b1b40>] (slave_modeset_init+0x120/0x1bc [tilcdc])
   [<bf0b1b40>] (slave_modeset_init [tilcdc]) from [<bf0b2be8>] (tilcdc_load+0x214/0x4c0 [tilcdc])
   [<bf0b2be8>] (tilcdc_load [tilcdc]) from [<c029955c>] (drm_dev_register+0xa4/0x104)
      [..snip..]
   ---[ end trace 4df8d614936ebdee ]---
   [drm:drm_sysfs_connector_add] *ERROR* failed to register connector device: -17

Signed-off-by: Guido Martínez <guido@vanguardiasur.com.ar>
Tested-by: Darren Etheridge <detheridge@ti.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Jiri Slaby <jslaby@suse.cz>
d635b7b

@popcornmix popcornmix pushed a commit that referenced this issue Oct 8, 2014

Guido Martínez + Dave Airlie drm/tilcdc: slave: fix dangling sysfs connector node
Add a drm_sysfs_connector_remove call when we destroy the panel to make
sure the connector node in sysfs gets deleted.

This is required for proper unload and re-load of this driver as a
module. Without this, we would get a warning at re-load time like so:

   tda998x 0-0070: found TDA19988
   ------------[ cut here ]------------
   WARNING: CPU: 0 PID: 825 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x54/0x74()
   sysfs: cannot create duplicate filename '/class/drm/card0-HDMI-A-1'
   Modules linked in: [..]
   CPU: 0 PID: 825 Comm: modprobe Not tainted 3.15.0-rc4-00027-g9dcdef4 #82
   [<c0013bb8>] (unwind_backtrace) from [<c0011824>] (show_stack+0x10/0x14)
   [<c0011824>] (show_stack) from [<c0034e8c>] (warn_slowpath_common+0x68/0x88)
   [<c0034e8c>] (warn_slowpath_common) from [<c0034edc>] (warn_slowpath_fmt+0x30/0x40)
   [<c0034edc>] (warn_slowpath_fmt) from [<c01243f4>] (sysfs_warn_dup+0x54/0x74)
   [<c01243f4>] (sysfs_warn_dup) from [<c0124708>] (sysfs_do_create_link_sd.isra.2+0xb0/0xb8)
   [<c0124708>] (sysfs_do_create_link_sd.isra.2) from [<c02ae37c>] (device_add+0x338/0x520)
   [<c02ae37c>] (device_add) from [<c02ae6e8>] (device_create_groups_vargs+0xa0/0xc4)
   [<c02ae6e8>] (device_create_groups_vargs) from [<c02ae758>] (device_create+0x24/0x2c)
   [<c02ae758>] (device_create) from [<c029b4ec>] (drm_sysfs_connector_add+0x64/0x204)
   [<c029b4ec>] (drm_sysfs_connector_add) from [<bf0b1b40>] (slave_modeset_init+0x120/0x1bc [tilcdc])
   [<bf0b1b40>] (slave_modeset_init [tilcdc]) from [<bf0b2be8>] (tilcdc_load+0x214/0x4c0 [tilcdc])
   [<bf0b2be8>] (tilcdc_load [tilcdc]) from [<c029955c>] (drm_dev_register+0xa4/0x104)
      [..snip..]
   ---[ end trace 4df8d614936ebdee ]---
   [drm:drm_sysfs_connector_add] *ERROR* failed to register connector device: -17

Signed-off-by: Guido Martínez <guido@vanguardiasur.com.ar>
Tested-by: Darren Etheridge <detheridge@ti.com>
Cc: <stable@vger.kernel.org> #v3.9+
Signed-off-by: Dave Airlie <airlied@redhat.com>
daa15b4

@popcornmix popcornmix pushed a commit that referenced this issue Oct 8, 2014

@davem330 Eric Dumazet + davem330 ipv4: do not use this_cpu_ptr() in preemptible context
this_cpu_ptr() in preemptible context is generally bad

Sep 22 05:05:55 br kernel: [   94.608310] BUG: using smp_processor_id()
in
preemptible [00000000] code: ip/2261
Sep 22 05:05:55 br kernel: [   94.608316] caller is
tunnel_dst_set.isra.28+0x20/0x60 [ip_tunnel]
Sep 22 05:05:55 br kernel: [   94.608319] CPU: 3 PID: 2261 Comm: ip Not
tainted
3.17.0-rc5 #82

We can simply use raw_cpu_ptr(), as preemption is safe in these
contexts.

Should fix https://bugzilla.kernel.org/show_bug.cgi?id=84991

Signed-off-by: Eric Dumazet <edumazet@google.com>
Reported-by: Joe <joe9mail@gmail.com>
Fixes: 9a4aa9a ("ipv4: Use percpu Cache route in IP tunnels")
Acked-by: Tom Herbert <therbert@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
a35165c

@popcornmix popcornmix pushed a commit that referenced this issue Oct 12, 2014

@gregkh Guido Martínez + gregkh drm/tilcdc: slave: fix dangling sysfs connector node
commit daa15b4 upstream.

Add a drm_sysfs_connector_remove call when we destroy the panel to make
sure the connector node in sysfs gets deleted.

This is required for proper unload and re-load of this driver as a
module. Without this, we would get a warning at re-load time like so:

   tda998x 0-0070: found TDA19988
   ------------[ cut here ]------------
   WARNING: CPU: 0 PID: 825 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x54/0x74()
   sysfs: cannot create duplicate filename '/class/drm/card0-HDMI-A-1'
   Modules linked in: [..]
   CPU: 0 PID: 825 Comm: modprobe Not tainted 3.15.0-rc4-00027-g9dcdef4 #82
   [<c0013bb8>] (unwind_backtrace) from [<c0011824>] (show_stack+0x10/0x14)
   [<c0011824>] (show_stack) from [<c0034e8c>] (warn_slowpath_common+0x68/0x88)
   [<c0034e8c>] (warn_slowpath_common) from [<c0034edc>] (warn_slowpath_fmt+0x30/0x40)
   [<c0034edc>] (warn_slowpath_fmt) from [<c01243f4>] (sysfs_warn_dup+0x54/0x74)
   [<c01243f4>] (sysfs_warn_dup) from [<c0124708>] (sysfs_do_create_link_sd.isra.2+0xb0/0xb8)
   [<c0124708>] (sysfs_do_create_link_sd.isra.2) from [<c02ae37c>] (device_add+0x338/0x520)
   [<c02ae37c>] (device_add) from [<c02ae6e8>] (device_create_groups_vargs+0xa0/0xc4)
   [<c02ae6e8>] (device_create_groups_vargs) from [<c02ae758>] (device_create+0x24/0x2c)
   [<c02ae758>] (device_create) from [<c029b4ec>] (drm_sysfs_connector_add+0x64/0x204)
   [<c029b4ec>] (drm_sysfs_connector_add) from [<bf0b1b40>] (slave_modeset_init+0x120/0x1bc [tilcdc])
   [<bf0b1b40>] (slave_modeset_init [tilcdc]) from [<bf0b2be8>] (tilcdc_load+0x214/0x4c0 [tilcdc])
   [<bf0b2be8>] (tilcdc_load [tilcdc]) from [<c029955c>] (drm_dev_register+0xa4/0x104)
      [..snip..]
   ---[ end trace 4df8d614936ebdee ]---
   [drm:drm_sysfs_connector_add] *ERROR* failed to register connector device: -17

Signed-off-by: Guido Martínez <guido@vanguardiasur.com.ar>
Tested-by: Darren Etheridge <detheridge@ti.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
47b7b0c

@popcornmix popcornmix pushed a commit that referenced this issue Nov 7, 2014

@gregkh Guido Martínez + gregkh drm/tilcdc: slave: fix dangling sysfs connector node
commit daa15b4 upstream.

Add a drm_sysfs_connector_remove call when we destroy the panel to make
sure the connector node in sysfs gets deleted.

This is required for proper unload and re-load of this driver as a
module. Without this, we would get a warning at re-load time like so:

   tda998x 0-0070: found TDA19988
   ------------[ cut here ]------------
   WARNING: CPU: 0 PID: 825 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x54/0x74()
   sysfs: cannot create duplicate filename '/class/drm/card0-HDMI-A-1'
   Modules linked in: [..]
   CPU: 0 PID: 825 Comm: modprobe Not tainted 3.15.0-rc4-00027-g9dcdef4 #82
   [<c0013bb8>] (unwind_backtrace) from [<c0011824>] (show_stack+0x10/0x14)
   [<c0011824>] (show_stack) from [<c0034e8c>] (warn_slowpath_common+0x68/0x88)
   [<c0034e8c>] (warn_slowpath_common) from [<c0034edc>] (warn_slowpath_fmt+0x30/0x40)
   [<c0034edc>] (warn_slowpath_fmt) from [<c01243f4>] (sysfs_warn_dup+0x54/0x74)
   [<c01243f4>] (sysfs_warn_dup) from [<c0124708>] (sysfs_do_create_link_sd.isra.2+0xb0/0xb8)
   [<c0124708>] (sysfs_do_create_link_sd.isra.2) from [<c02ae37c>] (device_add+0x338/0x520)
   [<c02ae37c>] (device_add) from [<c02ae6e8>] (device_create_groups_vargs+0xa0/0xc4)
   [<c02ae6e8>] (device_create_groups_vargs) from [<c02ae758>] (device_create+0x24/0x2c)
   [<c02ae758>] (device_create) from [<c029b4ec>] (drm_sysfs_connector_add+0x64/0x204)
   [<c029b4ec>] (drm_sysfs_connector_add) from [<bf0b1b40>] (slave_modeset_init+0x120/0x1bc [tilcdc])
   [<bf0b1b40>] (slave_modeset_init [tilcdc]) from [<bf0b2be8>] (tilcdc_load+0x214/0x4c0 [tilcdc])
   [<bf0b2be8>] (tilcdc_load [tilcdc]) from [<c029955c>] (drm_dev_register+0xa4/0x104)
      [..snip..]
   ---[ end trace 4df8d614936ebdee ]---
   [drm:drm_sysfs_connector_add] *ERROR* failed to register connector device: -17

Signed-off-by: Guido Martínez <guido@vanguardiasur.com.ar>
Tested-by: Darren Etheridge <detheridge@ti.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
0f0fb98

@popcornmix popcornmix pushed a commit that referenced this issue Oct 25, 2015

@pshelar @davem330 pshelar + davem330 openvswitch: Fix ovs_vport_get_stats()
Not every device has dev->tstats set. So when OVS tries to calculate
vport stats it causes kernel panic. Following patch fixes it by
using standard API to get net-device stats.

---8<---
Unable to handle kernel paging request at virtual address 766b4008
Internal error: Oops: 96000005 [#1] PREEMPT SMP
Modules linked in: vport_vxlan vxlan ip6_udp_tunnel udp_tunnel tun bridge stp llc openvswitch ipv6
CPU: 7 PID: 1108 Comm: ovs-vswitchd Not tainted 4.3.0-rc3+ #82
PC is at ovs_vport_get_stats+0x150/0x1f8 [openvswitch]
<snip>
Call trace:
 [<ffffffbffc0859f8>] ovs_vport_get_stats+0x150/0x1f8 [openvswitch]
 [<ffffffbffc07cdb0>] ovs_vport_cmd_fill_info+0x140/0x1e0 [openvswitch]
 [<ffffffbffc07cf0c>] ovs_vport_cmd_dump+0xbc/0x138 [openvswitch]
 [<ffffffc00045a5ac>] netlink_dump+0xb8/0x258
 [<ffffffc00045ace0>] __netlink_dump_start+0x120/0x178
 [<ffffffc00045dd9c>] genl_family_rcv_msg+0x2d4/0x308
 [<ffffffc00045de58>] genl_rcv_msg+0x88/0xc4
 [<ffffffc00045cf24>] netlink_rcv_skb+0xd4/0x100
 [<ffffffc00045dab0>] genl_rcv+0x30/0x48
 [<ffffffc00045c830>] netlink_unicast+0x154/0x200
 [<ffffffc00045cc9c>] netlink_sendmsg+0x308/0x364
 [<ffffffc00041e10c>] sock_sendmsg+0x14/0x2c
 [<ffffffc000420d58>] SyS_sendto+0xbc/0xf0
Code: aa1603e1 f94037a4 aa1303e2 aa1703e0 (f9400465)

Reported-by: Tomasz Sawicki <tomasz.sawicki@objectiveintegration.uk>
Fixes: 8c87663 ("openvswitch: Remove vport stats.")
Signed-off-by: Pravin B Shelar <pshelar@nicira.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
83ffe99

@popcornmix popcornmix pushed a commit that referenced this issue Mar 21, 2017

@congwang @davem330 congwang + davem330 ipv6: reorder icmpv6_init() and ip6_mr_init()
Andrey reported the following kernel crash:

kasan: GPF could be caused by NULL-ptr deref or user memory access
general protection fault: 0000 [#1] SMP KASAN
Dumping ftrace buffer:
   (ftrace buffer empty)
Modules linked in:
CPU: 0 PID: 14446 Comm: syz-executor6 Not tainted 4.10.0+ #82
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
task: ffff88001f311700 task.stack: ffff88001f6e8000
RIP: 0010:ip6mr_sk_done+0x15a/0x3d0 net/ipv6/ip6mr.c:1618
RSP: 0018:ffff88001f6ef418 EFLAGS: 00010202
RAX: dffffc0000000000 RBX: 1ffff10003edde8c RCX: ffffc900043ee000
RDX: 0000000000000004 RSI: ffffffff83e3b3f8 RDI: 0000000000000020
RBP: ffff88001f6ef508 R08: fffffbfff0dcc5d8 R09: 0000000000000000
R10: ffffffff86e62ec0 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: ffff88001f6ef4e0 R15: ffff8800380a0040
FS:  00007f7a52cec700(0000) GS:ffff88003ec00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000061c500 CR3: 000000001f1ae000 CR4: 00000000000006f0
DR0: 0000000020000000 DR1: 0000000020000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
Call Trace:
 rawv6_close+0x4c/0x80 net/ipv6/raw.c:1217
 inet_release+0xed/0x1c0 net/ipv4/af_inet.c:425
 inet6_release+0x50/0x70 net/ipv6/af_inet6.c:432
 sock_release+0x8d/0x1e0 net/socket.c:597
 __sock_create+0x39d/0x880 net/socket.c:1226
 sock_create_kern+0x3f/0x50 net/socket.c:1243
 inet_ctl_sock_create+0xbb/0x280 net/ipv4/af_inet.c:1526
 icmpv6_sk_init+0x163/0x500 net/ipv6/icmp.c:954
 ops_init+0x10a/0x550 net/core/net_namespace.c:115
 setup_net+0x261/0x660 net/core/net_namespace.c:291
 copy_net_ns+0x27e/0x540 net/core/net_namespace.c:396
9pnet_virtio: no channels available for device ./file1
 create_new_namespaces+0x437/0x9b0 kernel/nsproxy.c:106
 unshare_nsproxy_namespaces+0xae/0x1e0 kernel/nsproxy.c:205
 SYSC_unshare kernel/fork.c:2281 [inline]
 SyS_unshare+0x64e/0x1000 kernel/fork.c:2231
 entry_SYSCALL_64_fastpath+0x1f/0xc2

This is because net->ipv6.mr6_tables is not initialized at that point,
ip6mr_rules_init() is not called yet, therefore on the error path when
we iterator the list, we trigger this oops. Fix this by reordering
ip6mr_rules_init() before icmpv6_sk_init().

Reported-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
15e6680
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment