V4L2 Improvements - videobuf2_contig allocator

[6by9]

(This is more a placeholder for a task that needs doing than a bug)

Currently the bcm2835-v4l2 driver uses the videobuf2-vmalloc buffer allocator, and relies on VCHI doing a copy of every frame from VC memory into ARM memory. This obviously has a hit on memory bandwidth.
The videobuf2-contig allocator should allow CMA allocated memory to be used by both VC and ARM, and would also work with the vc4 DRM/KMS driver. (Already demoed in anholt#12 although still with VCHI copying from VC to CMA memory)

Tasks:

• Change allocator
• Make use of VCSM Improvements - dmabuf support #1806 (or reimplement) to import the contiguous buffer into a VC MEM_HANDLE_T that can be used with MMAL.

The awkard bits are:

• currently very little memory is allocated to CMA. How can that be changed without impacting platforms unduly? Make the driver a device tree node and have an overlay load the driver and increase CMA (can it do that? May only be a kernel command line thing)
• does it throw up further caching issues?

[pelwell]

Overlays can add items to the kernel command line - see the vc4-kms-v3d overlay for an example. Note, however, that multiple cma= values may not have the desired effect.

[popcornmix]

With the vc4-kms-v3d driver we do run with cma=256M@256M so you might imagine in the future having a reasonable amount of cma by default is likely.

[6by9]

Thanks both.
With vc4-kms-v3d I think we have no real issue as there should be a CMA block big enough for most things.
With the standard VC side driver it's likely to be more interesting as CMA won't really be used other than for V4L2, so it's a waste.

It's one to have a discussion about come the end of the month - I'm unlikely to get any time to do much before then.

[Electron752]

Waiting for the discussion...

Doing some cleanup in this area sounds interesting and useful. I've also noticed a few posts and from using it myself that CMA is a bit broken in 4.10 at least as far as vc4-kms-v3d is concerned. Making it work with both the standard VC driver and VC4 sounds useful as well.

[anholt]

@6by9 well, switching to CMA instead of gpu_mem=128 like raspi-config does currently for camera means at least Linux gets to use those pages when the camera isn't active.

[6by9]

@anholt Yes, but for the case where the app uses MMAL directly (ie most Pi camera apps) we don't have the option of using CMA so have to have gpu_mem allocated.
In theory the vc_cma driver would let us do that (the GPU requested CMA memory as free mem hit a low water mark), but it was never that great, and I think using it would cause Phil to have kittens.
Ideally clients want to be using zero copy and VCSM allocated buffers (1), but it requires clients to set things up slightly differently, so we can't just drop the old mechanism.

FYI I start work at Pi Towers on Monday, hence my comment earlier about discussions likely to occur at the end of the month.
Unless Gordon has other things lined up for me these will be a couple of the early things I intend to be looking into. Hopefully given a modest run at it the VCSM change shouldn't take long - spare time just hasn't allowed me a clear enough run at it.

(1) Hmm, VCSM could be rejigged to use CMA by default as the allocator and map it into VC, as a new implementation of vc_sm_alloc(). I hadn't thought of that one, and it would allow gpu_mem to be set lower without any loss of functionality and a lower overhead on the client. Another one to discuss with the others.

[JamesH65]

ping @6by9

[6by9]

The firmware camera stack is deprecated, therefore limited point in pursuing this.

We do now have the frameworks to handle dmabufs and import them into the VPU memory space (they're used by the codec and ISP V4L2 drivers), but it has limited gain to switch bcm2835-v4l2.

[6by9]

Whoops, should have been a "not planned"