Added OS X Screen Capture device

[jbenden]

This pull request adds a new device, avcapture, that allows one to capture the primary display on OS X.

[michaelni]

this breaks build on linux
"gcc: error trying to exec 'cc1obj': execvp: No such file or directory"
also the patch should probably be submitted to the ffmpeg-devel mailing list for review.
you can also add yourself to the MAINTAINERs file in the patch if you like and i suggest to stash these into a single patch, they kind of seem to me all together just the initial support for OS X screen capture devices, it should make review easier. Or does this represent some public history of development and testing ? (considering it fails building on linux it doesnt seem so to me)

[jbenden]

The reason is that 'avcapture' doesn't disable itself on non-OS X platforms. I've briefly looked around, but I cannot figure out why it does that. Maybe someone with more experience can help...

[jbenden]

I figured it out and added the necessary change to configure.

[TimothyGu]

Please merge this line with

OBJS-$(CONFIG_AVFOUNDATION_INDEV)        += avfoundation.o

[TimothyGu]

Also, alphabetical order.

[TimothyGu]

Missing copyright. Also use AVDEVICE_AVFFRAMERECEIVER_H as multi-inclusion guard.

[TimothyGu]

Please add copyright.

[jbenden]

I have fixed the requested changes.

[TimothyGu]

No tabs are allowed in FFmpeg.

[TimothyGu]

This needs

Copyright (c) 2014 Thilo Borgmann <thilo.borgmann@mail.de>

too as you copied the contents from avfoundation.m.

[jbenden]

Ok, have a look again. Thanks.

[jbenden]

I've updated the code to merge appropriately with master branch.

[jbenden]

I've added a blog post about using this device. See it here: http://www.benden.us/journal/2014/avcapture-mac-os-x-screen-recording/

[davidherran]

Hello, I have been using the merge since it launched and it works great in Maveriks. Thank you very much for this sensational contribution. The only thing is that I could not set up the audio with OpenAL. Try to do the same to set it but I could not. I'm using SOX a python script with multhreding to capture audio and video simultaneously.

Thank you very much.

[michaelni]

Your branch contains commits that add conflict markers like "<<<<<<< HEAD", which are later removed in seperate commits, also it contains a seemingly random merge of ffmpeg master.
I cannot merge or cherry pick these patches, please clean this up somehow

[jbenden]

Ok, I will clean up the commit.

On a side note, should I add audio capture support to this module? It might be better than using sox. Thoughts?

[michaelni]

sure, if you like to add audio support.

[jbenden]

Ok, I cleaned up the merges. Can you merge it easily now?

[michaelni]

yes, but it seem/would breaks build on linux
gcc: error trying to exec 'cc1obj': execvp: No such file or directory
make: *** [libavdevice/avcapture.o] Error 1

[jbenden]

I fixed the Linux build problem. I am getting a linking error though. Not sure if it is due to code rot or if there is a problem. I'm researching this matter.

[jbenden]

It seems the updated configure script fixed the linking issues. I am about to test and commit audio support to this device.

[michaelni]

should lock/unlock_frames() not be static inline ?
iam asking as just inline was historically in C interpreted differently by different compilers. Not sure this issue applies here too

[jbenden]

I changed them to static inline.

I have a branch ready for testing that includes audio support. It is working locally, but has a few pops and clicks. I'm not sure of why it is doing this. Can anyone offer advise?

The branch is located here: https://github.com/jbenden/FFmpeg/tree/avcapture-audio

[jbenden]

As far as the sound branch goes, if you set the rate to 30; everything seems much better. So there is some sort of disconnect between the two streams rates (the video rate verses audio rate).

[davidherran]

YEA!!! OPENAL

[thiloborgmann]

This introduces a lot of code redundancy by introducing the avcapture device.

I think the avfoundation device should be extended to include the display inputs.

Also the audio issues can be solved for all inputs in one device.

[jbenden]

I now have a branch where I've merged avcapture with avfoundation. https://github.com/jbenden/FFmpeg/tree/avfoundation

It experiences the same troubles that plagued avcapture; namely that the frame rate must be fixed at 30 or the sound is sped up. Additionally, it does NOT seem as if any of the arguments passed to ffmpeg are honored - I had to hard code my device indexes to even test this out.

[thiloborgmann]

I'm currently adapting your changes but need some more days to do it.
Except the 30fps issue, I expect the issues to be solvable.

[thiloborgmann]

I've just posted a patch to ffmpeg-devel.
Thanks, Joe!
Remarks welcome.

[jbenden]

How come the command-line arguments are still not read? I still have to hard-code a video_device_index to use and list_devices doesn't work. Is it my copy or a general problem?

[jbenden]

I've updated to the latest code, still it doesn't read the command-line options for avfoundation correctly.

[jbenden]

In case people are curious, the FFmpeg mailing list post is here: https://ffmpeg.org/pipermail/ffmpeg-devel/2014-October/164470.html

[jbenden]

Thilo, you have an off by one calculation in listing supported devices. In the for loop for desktop screens, it should be i + index + 1.

[thiloborgmann]

I teste yesterday on a mini with no video devices so maybe this is true. Will test again later today.

I used: -i "#" and -i "Captire screen #" which are working for me.
-video_device_index should also work.

What command line does not work as expected for you?

[jbenden]

ffmpeg -f avfoundation -i "" -list_devices true

This does not list the available devices, like it should.

[thiloborgmann]

In that case -list_devices is interpreted as an output option. Move it bevore -i "".

[jbenden]

That did the trick! (Note I patched the n+1 problem in display enumeration)

→ ffmpeg -f avfoundation -list_devices true -i "" 10/25 @ 07:15
ffmpeg version N-67141-ge56425d Copyright (c) 2000-2014 the FFmpeg developers
built on Oct 24 2014 10:19:51 with Apple LLVM version 6.0 (clang-600.0.54) (based on LLVM 3.5svn)
configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-nonfree --enable-hardcoded-tables --enable-avresample --enable-vda --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libmp3lame --enable-libxvid
libavutil 54. 11.100 / 54. 11.100
libavcodec 56. 9.100 / 56. 9.100
libavformat 56. 10.100 / 56. 10.100
libavdevice 56. 1.100 / 56. 1.100
libavfilter 5. 2.100 / 5. 2.100
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
[AVFoundation input device @ 0x7f939b514280] AVFoundation video devices:
[AVFoundation input device @ 0x7f939b514280] [0] Built-in iSight
[AVFoundation input device @ 0x7f939b514280] [1] Capture screen 0
[AVFoundation input device @ 0x7f939b514280] AVFoundation audio devices:
[AVFoundation input device @ 0x7f939b514280] [0] Soundflower (64ch)
[AVFoundation input device @ 0x7f939b514280] [1] Built-in Microphone
[AVFoundation input device @ 0x7f939b514280] [2] Soundflower (2ch)
[AVFoundation input device @ 0x7f939b514280] [3] Built-in Input
: Input/output error

[thiloborgmann]

Your fix of the device listing works only if there are video devices available...

Fixed differently and updated the patch on ffmpeg-devel.

Thanks!

[michaelni]

applied thilos patch,
thanks to all!

[davidherran]

Hello, thank you very much for the help. I compiled the new version of the branch-avcapure audio. These are devices that shows me the system:

[AVFoundation input device @ 0x7faf53d0cfe0] AVFoundation video devices:
[AVFoundation input device @ 0x7faf53d0cfe0] [0] iSight
[AVFoundation input device @ 0x7faf53d0cfe0] AVFoundation audio devices:
[AVFoundation input device @ 0x7faf53d0cfe0] [0] Built-in Microphone
[AVFoundation @ 0x7faf53d0cfe0 input device] [1] Built-in Input

But I have doubts on how to execute the command. I need to capture video of your screen and audio from the microphone, but the camera is triggered and recording the screen but with many interruptions:

ffmpeg -i -f avcapture '0' -i -f avfoundation '0' /Users/user/5.mkv

as to differentiate avfoundation [0] Built-in Microphone and not [0] in iSight
avfoundation -f -i '0'

Use this version https://github.com/jbenden/FFmpeg/tree/avfoundation since the master does not recognize my microphone.

Thank you very much!

[thiloborgmann]

Interesting, on what hardware are you operating?

For the command, use
ffmpeg -f avfoundation -i "0:0" /Users/user/5.mkv

[davidherran]

Hi! Thank you very much for answering. I'm on a macbook pro with mavericks.
Run the command as you indicate and captured video from my isight camera is not from my screen.

I apologize if I did not understand. I need to capture video from my desk and microphone audio.

Thanks again

[thiloborgmann]

For screen capturing you should use current git HEAD.

If the microphone is an issue with git HEAD, then please report this issue in detail - best would be to open a new ticket on trac.ffmpeg.org if you think it is a bug.

[davidherran]

Ok... Muchas gracias!!!!!

[TimothyGu]

@davidherran I suspect you accidentally switched the -f and the -i:

 ffmpeg -i -f avcapture '0' -i -f avfoundation '0' /Users/user/5.mkv

The -f must be in front of -i.

P.S. ¿Hablas español?

[davidherran]

Yes, I am from Colombia, how are you? I take and I tell ... finished compiling FFMPEG master version, and success !!! recorded video and audio from my desk ... but I deleted audio frames ... audio cuts out and comes back when I play video. I'm on a Mac book pro with mavericks.

use this command: ffmpeg_master avfoundation -f -i '1: 0' /Users/user/1.mkv

Greetings from Colombia !!!!

Si, soy de Colombia, como estás? Aprovecho y te cuento... acabe de compilar la versión master de FFMPEG, y éxito!!! grabo video de mi escritorio y audio... pero me borra frames de audio... el audio se corta y regresa cuando reproduzco el video. Estoy en un Mac book pro con mavericks.

use este comando: ffmpeg_master -f avfoundation -i '1:0' /Users/user/1.mkv

Saludos de Colombia!!!!

[TimothyGu]

@davidherran

Hola David,

No hablo español mucho ni lo hablo bien porque es mi segundo año aprendiendo español en mi escuela. Vivo en los Estados Unidos, pero soy de China, y español es mi tercera idioma. Hay muchos mexicanos aquí en California y español es muy útil.

Voy a (give you) unas instrucciones para debugar el problema en inglés, pero pienso @thiloborgmann sabe más en el sujeto.

Saludos de Los Ángeles, y por favor corregir mi (writing)!

---

Hi David,

Spanish is my third language so I'm not really good at it. Please correct me in any way you could!

I suspect the order of 1:0 is wrong. Could you please test

ffmpeg_master -f avfoundation -i '0:0' /Users/user/1.mkv

and

ffmpeg_master -f avfoundation -i '0:1' /Users/user/1.mkv

and report back? Just so you know, the first number is the video device while the second is the audio device. You don't have a video device 1 so that might be causing the error.

If you still get some error, could you post the log of the failed command? It will help us debug the problem.

---

@thiloborgmann Two questions I have here:

1. The docs doesn't say you can pass input file names with colon, but the implementation clearly supports a notation with : as separators between video and audio devices. Would you mind adding some documentation on that?
2. Was there a reason you chose strtok() over av_strtok() which is NIH'd and Brings More Stability™?

[thiloborgmann]

1. will have a look as soon as I can find some time.
2. hm no, I think I copied that from another filter... going to change that when time comes, too.

[davidherran]

@TimothyGu

Hello and of course with pleasure ... my English is also very bad, so excuse me ...

Hola David,
No hablo español mucho ni lo hablo bien porque es mi segundo año aprendiendo español en mi escuela. Vivo en los Estados Unidos, pero soy de China, ( y español —> y "el" español ) (es mi tercera idioma — > es mi "tercer" idioma). Hay muchos mexicanos aquí en California (y español —> y "el" español) es muy útil.
Voy a (give you —> darle o darte) unas instrucciones para (debugar —> depurar o esta bien usar el termino en ingles debug) el problema en inglés, pero pienso @thiloborgmann sabe más en el (sujeto —> asunto o tema).
Saludos de Los Ángeles, y por favor corregir mi (writing —> Escritura)!

Use new version of FFMPEG master with the command ->

ffmpeg_master -f avfoundation -list_devices true -i ""

and this is the list of devices that I have:

[AVFoundation input device @ 0x7fe40b50d060] AVFoundation video devices:
[AVFoundation input device @ 0x7fe40b50d060] [0] iSight
[AVFoundation @ 0x7fe40b50d060 input device] [1] Capture screen 0
[AVFoundation input device @ 0x7fe40b50d060] AVFoundation audio devices:
[AVFoundation input device @ 0x7fe40b50d060] [0] Built-in Microphone
[AVFoundation @ 0x7fe40b50d060 input device] [1] Built-in Input

to execute the command

ffmpeg_master -f avfoundation -i '1:0' /Users/user/1.mkv

Capture great video from my desktop, but the audio has cut frames. that's the thing ...

[jbenden]

add -r 30 to the list of arguments. This helps my recordings.

[davidherran]

@jbenden Hello and tks...

please see the example in video http://youtu.be/d8kfFlh_yeg

Command:

ffmpeg_master -f avfoundation -i '1:0' -r 30 /Users/davidherran/1.mkv

[jbenden]

Well the sync is OK now, but there is popping clicks in the audio. I'm not sure why that's happening, maybe @thiloborgmann can shed some light on the situation.

[davidherran]

ok. thats is correct, this is the correct term or word "popping" tks