Ffmpeg 0.10.2 #810

Merged
merged 40 commits into from Apr 4, 2012

Conversation

Projects
None yet
Member

elupus commented Mar 23, 2012

Title says it all..

I see some minor regressions with semi corrupt h264 files. But have not tested very much yet.

Member

anssih commented Mar 23, 2012

Hmm, this seems to be dropping most of lib/ffmpeg/patches, how are you intending to handle divergence from upstream now? (i.e. reviewing / submitting / generally looking at / keeping track our patches, or applying them to an updated FFmpeg)

From a quick look this sequence of commits seems to also contain some unbuildable trees in between, making git-bisect a bit more painful... but I guess it might not be worth it to fix them now afterwards.

Also, not 100% sure if this drops too much backward compatibility, but I don't care myself right now as xbmc can't be built with old Mageia FFmpeg packages anyway for other reasons (if other distros/users complain, the needed compatibility can be added later).

Nice job converting us out of deprecated stuff :)

Owner

Memphiz commented Mar 23, 2012

mhh all horses on davilla now for getting this to build on ios :/ neither ffmpeg nor libav seems to care about linker issue for ios when compiling dylibs :(

Member

elupus commented Mar 23, 2012

I intend to re add the patches. Have it all in my ffmpeg fork.

Yea there are some broken steps in there, but hard to handle without a
single large commit.

Contributor

davilla commented Mar 24, 2012

One thought for iOS is to create static libs and link to them like we do for external ffmpeg.

Owner

Memphiz commented Mar 24, 2012

sounds like a plan :)

Member

anssih commented Mar 24, 2012

@elupus: OK, though a bit weird that you remove them first if you plan to readd them IMO. Also note that our tree missed a dozen or so patches in lib/ffmpeg/patches, and I just recently put them there, did you keep those applied? I see your commit doesn't remove them as it is based on an older tree.

On patches, it would also have been nice to have a list of patches that have been remove/added and the reasons (upstreamed, fixed in other way, dropped for some other reason)...

Member

elupus commented Mar 24, 2012

If you check my ffmpeg branch xbmc_0.10.2 on github the patch series is
ontop there.

I must admit I have lost track somewhat of why and when patches was
dropped. Was very long ago that I started this update.

Member

elupus commented Mar 24, 2012

Also it was actually fully build-able for the ffmpeg_0.9 series update (or perhaps it was 0.8) i had before. Not tested status on that in this series.

Member

anssih commented Mar 25, 2012

I really think we should double-check the patches in xbmc_0.10.2 against the patches currently in lib/ffmpeg/patches then to document the dropped patches. I guess it'd be preferable if you did it as the work is yours, but I can do it as well if you don't want to.

Instead of removing the patches in the first commit and adding them back later, IMO it would be better to just replace them in the same commit so that git can track the renames and patch history.

OTOH I wonder if there is some better way of keeping track of our patches on top of FFmpeg instead of plain patches in lib/ffmpeg/patches (which is tricky to keep in sync with the lib/ffmpeg itself to avoid losing anything), like some separate git repository that can be merged or rebased against new FFmpeg and the patches applied on-top (after all we use git when rebasing our patches anyway...) - but I don't know how would we keep that in sync then (and git submodules are probably not a good idea here). Does @theuni maybe have any ideas regarding this, or should we just continue as before?

Member

amejia1 commented Mar 25, 2012

What's wrong with using git submodules? We could have a clone of https://github.com/FFmpeg/FFmpeg with our changes and use the clone as the submodule.

Member

elupus commented Mar 25, 2012

Git submodules and our own ffmpeg branch makes sense to me. But I've just
about only heard bad things about git submodules so I'm not sure.

We would be rewriting history on that alot though since we always want to
rebase our commits in each update. May not be a huge problem.

I will try to restore the patch series. Will be a bit tricky thou since
order have changed.

Member

anssih commented Mar 25, 2012

@amejia1, I admit I'm not very familiar with git submodules, do they play well with git-bisect and branches in general? Also, the rebasing issue elupus notes would be a concern.

@elupus, no need to care about the order I think, just put the removal+addition in the same commit, git will find the renames then.

Member

amejia1 commented Mar 25, 2012

@anssih A git submodule is basically a git repository embedded in another git repository. You can run any git command on the submodule just like any git repository. A .gitmodules file is created and tracked in the superproject. This file stores the url of the submodule git repository and the path within the superproject where it should be cloned.

So in our case, adding ffmpeg as a submodule could help better track changes upstream. We clone ffmpeg upstream (or the github mirror) and place it within https://github.com/xbmc. We make our changes that we need (probably a good idea to use a branch other than master). Then within xbmc, we add this clone as a submodule.

$ git submodule add git://github.com/xbmc/ffmpeg.git lib/ffmpeg

Afterwards, run

$ git submodule init
$ git submodule update

And that's it. For anyone cloning xbmc anew, the command to clone xbmc and all submodules is

$ git clone --recursive git://github.com/xbmc/xbmc.git

Member

elupus commented Mar 25, 2012

First crude update with the patch series updated. I'll see if i can go throught it and split out the patches that are no longer needed. This is still WIP. but it's getting there.

Member

anssih commented Mar 26, 2012

@amejia1 As said, I'm more interested in how branches and bisect work there. i.e. if I pull someone else's xbmc branch or create my own, and git-checkout between them and other commits, the lib/ffmpeg directory better update along or it will be a mess. Similarly, when git-bisecting, lib/ffmpeg should follow xbmc checkout automatically or bisecting becomes nightmarish. I really doubt all that works well enough with submodules...

Member

theuni commented Mar 26, 2012

Submodules have many helpful uses, but they are a big pain in the ass. They're incredibly confusing at first, so it would mean confusion for everyone building XBMC.

If we move that way, we can turn all of our externals into submodules, and get rid of anything that's not ours. That would be really nice.

So.. imo it's all or nothing. If we flip the switch and deal with the pain, we may as well do it everywhere else we can.

Member

amejia1 commented Mar 26, 2012

@anssih I'm afraid submodules don't work like this. Essentially a submodule would be it's own self contained repository. Doing a git bisect won't effect the submodule and vice versa. Same with git commit and any other git command.

Member

elupus commented Mar 26, 2012

The submodules will point to a specific rev, and that rev number is part of
xbmc, so it should work fine.

Member

wsoltys commented Mar 26, 2012

compiles with the following additions: https://github.com/wsoltys/xbmc/commits/ffmpeg_0.10.2
still access violation when trying to play a video. Next try would be to reverse the gcc mingw update though it did work fine with the old ffmpeg.

Member

theuni commented Mar 26, 2012

@elupus the hash that the submodules point at won't be in xbmc, it'll be in the child repo. And it means the extra step of updating that repo any time the submodule is bumped. No big deal ofc, but something else to have to coach people through.

Member

wsoltys commented Mar 27, 2012

still fails playing a video when compiled with our old mingw env. Access violation when avcodec_alloc_frame.

Member

elupus commented Mar 27, 2012

I meant the hash number will be stored versioned in xbmc and update when
one bisect, but will refer to a revision of the child repo. Ie the chars
stored in xbmc doesn't refer to a revision of xbmc.

Member

wsoltys commented Mar 28, 2012

above rebased code doesn't compile on win32 and throws a lot of errors. I tried my old branch posted above with --disable_asm but the access violation is the same.

Member

elupus commented Mar 28, 2012

Sorry, should have checked it before pushing. I'll make sure it works on
Windows before pushing again.

Member

elupus commented Mar 29, 2012

Okey, now this builds and works fine on windows.

Would be nice if we could get some small tests and see what regressions we have.

Member

wsoltys commented Mar 30, 2012

Compiles fine but you might want to change postproc-51.dll to postproc-52.dll in dllpaths_win32.h.
Unfortunately this build seems to have problems with dxva. When I start the I'm Legend trailer the file menu still shows but no video no audio is shown. If I press X the last message in the log is "waiting for video thread to exit".
From the killa sample the sound is played but after pressing X the same happens.

Member

wsoltys commented Mar 30, 2012

Forgot to mentioned that I'm Legend and killa sample work fine when turning off DXVA acceleration.

Member

elupus commented Mar 30, 2012

Could you post a log of the failing DXVA?

Member

wsoltys commented Mar 30, 2012

First one starting I'm Legend, wait, press x, wait:
http://pastebin.com/P2hzBrAz

Second one playing kill sampla, wait, press x, wait:
http://pastebin.com/CgpFxT3B

Member

elupus commented Mar 30, 2012

Could you try and revert the following: http://pastebin.com/rg0imEj9

should be possibly by:
git apply --reverse --directory=lib/ffmpeg file.patch

Member

wsoltys commented Mar 30, 2012

I'm not at home any more and only have a small rig with me with almost no sample files. I probably can try something tomorrow since this Laptop has also an ati card.

Member

elupus commented Mar 30, 2012

didn't seem to help here. think i managed to revert the commit. It hangs on some weird critical section but can't get a backtrace. @CrystalP would be nice if you could help out here.

Member

elupus commented Mar 30, 2012

It works fine for mpeg2 it seems.

Member

elupus commented Mar 30, 2012

Okey.. multi threading causes the lockup. Something in how threading is handled with hwcodecs makes it break down.

Member

elupus commented Mar 31, 2012

Okey, so the DXVA2 and VAAPI issues are resolved. I suspect we should do a quick update again reasonably soon. Either to master, or 0.11 if that ends up being out.

elupus and others added some commits Mar 25, 2012

@elupus elupus Update ffmpeg to n0.10.2 (f139838d6473c7b5152178f602cb953a824c2ff9)
xbmc ffmpeg 05f8b5549c5e20cf9a417069838edd6841d7bd40
2836f95
@elupus elupus Drop neon patch that doesn't apply anymore 2d8334b
@elupus elupus Update patches c6a5330
@elupus elupus Drop now merged or redundant patches 32ab506
@elupus elupus Drop patch for win32 build
These scripts should not be in the ffmpeg build dir
3320259
@elupus elupus Drop patch for cdxa probe, it causes missdetections of certain streams
Sadly i can't remember an example
903284b
@elupus elupus replace depreciated av_open_input_stream/file with new avformat_open_…
…input
75eff0d
@elupus elupus replace depreciated ByteIOContext with AVIOContext 564d77c
@elupus elupus replace depreciated AVMetadata(Tag) with AVDictionary(Entry) fb3b3a2
@elupus elupus Don't listen to depreciated return value of av_read_frame 67c7a8e
@elupus elupus Don't use depreciated avcodec_thread_init 4ee00c7
@elupus elupus Replace depreciated av_find_stream_info with avformat_find_stream_info ab76b8b
@elupus elupus Replace more depreciated ffmpeg functions 1aed04a
@elupus elupus Drop support for ffmpeg version older than our built in version bddad01
@elupus elupus Replace deprecated is_streamed with new seekable flags 8d48767
@elupus elupus Remove old code depending on deprecated file_size
This may need some testing for growing files
29121de
@elupus elupus changed: replace old libavfilter integration with null output filter …
…with buffersink
1fccb60
@elupus elupus changed: replace old swscale pixel format conversion with libavfilter…
… automatic
f6610b1
@aballier @elupus aballier Drop url_set_interrupt_cb usage which was removed in libavformat 54 a…
…nd set the callback directly in the AVFormatContext, which was added in libavformat-53.15.0
0bf68bf
@aballier @elupus aballier Do not map url_fdopen as it is deprecated and gone in libavformat 54.…
… It is unused in xbmc code.
45a39f7
@aballier @elupus aballier Replace av_write_header by avformat_write_header.
The former is gone in libavformat 54.
e4396ee
@aballier @elupus aballier Remove av_set_parameters.
It is gone in libavformat 54 and avformat_write_header takes care of allocating data as needed.
1572062
@aballier @elupus aballier Replace av_metadata_(get/set2) by av_dict_(get/set).
The former is deprecated and gone in ffmpeg git.
2528bd0
@aballier @elupus aballier Merge av_close_input_{file,stream} into avformat_close_input.
The formers are deprecated and gone in libavformat 54.
Moreover, this allows removing some code because avformat_close_input takes care of freeing relevant structures.
d6669d7
@aballier @elupus aballier Replace init_put_byte by avio_alloc_context.
The former is deprecated as of ffmpeg 0.9 and is gone in libavformat 54.
9cf83f5
@aballier @elupus aballier Replace dump_format by av_dump_format. 3d4c4fe
@aballier @elupus aballier Replace url_* functions by their avio_* counterparts. c5774f9
@aballier @elupus aballier Remove av_alloc_put_byte and replace its occurences by avio_alloc_con…
…text.
05c142b
@aballier @elupus aballier Include libavfilter/avcodec.h for av_vsrc_buffer_add_frame. 135160b
@aballier @elupus aballier Remove assignment to stream_copy which is never read after, be it by …
…xbmc or ffmpeg.
75179cb
@aballier @elupus aballier Do not set AVFrame.age.
It is unused and does not exist anymore in libavcodec 54.
fd3506a
@aballier @elupus aballier Use av_opt_set instead of av_set_string3 and drop support for old API. 4caa264
@aballier @elupus aballier Replace deprecated av_get_bits_per_sample_fmt by av_get_bytes_per_sam…
…ple.

This, in addition, saves a couple of divisions.
1ee1787
@elupus elupus Convert to avcodec_decode_audio4 and drop avcodec_decode_audio3. 525834b
@Memphiz @elupus Memphiz [osx/ios/atv2] - build and link static libs for ffmpeg because of bui…
…ld issues on ios
3201695
@elupus @elupus elupus changed: Only allow slice based parallel decoding
Frame based parallel decoding causes HW decoding
to crash, and it's more sensitive to changes
in frame sizes during decoding.
a0fc63d
Contributor

CrystalP commented Apr 1, 2012

took me a while to build this... must use a brand new ffmpeg sub directory, the cleaning included in our scripts is not strong enough and there were a bunch of compilation errors.

I'm going through my samples, so far so good.

Contributor

CrystalP commented Apr 1, 2012

Something goes screwy and triggers an access violation in DVDAudioCodecPassthroughFFmpeg::Decode(), line 473 when transcoding for spdif is used. Comes from m_Codec->GetData() on line 438 setting decData to NULL. I haven't had the time to trace things further.

Member

wsoltys commented Apr 1, 2012

compiles fine for me (vs and mingw) without errors but did a "git clean untracked" before. My video samples work fine. Just got some problems with some wma samples but thats not that important :)

Member

wsoltys commented Apr 1, 2012

Just for the record one wma pro files crashes in DVDPlayerCodec::ReadPCM l288 because m_decoded is a bad pointer.
Logfile ist here:
http://xbmclogs.com/show.php?id=1133

Member

elupus commented Apr 1, 2012

Is it all transcoding that fails or just some specific case? If just some
cases, I don't think it should be blocking.

Member

elupus commented Apr 1, 2012

@wsoltys Could you get me that wma pro file?

Member

wsoltys commented Apr 1, 2012

the file is on teamftp in the wiso folder. I've uploaded another one (recit16bit.wma) which just fails to playback.

Member

elupus commented Apr 1, 2012

Think i have a fix for both issues already

Member

wsoltys commented Apr 1, 2012

no crash on the wma pro and overall nice work.

Contributor

CrystalP commented Apr 1, 2012

The audio issue is fixed.
Now with software decoding it's fairly easy to trigger an exception when skipping.
It happens in CDVDCodecUtils::CopyPicture() because the s pointer is to invalid memory for reading.

Another crash when playing a video, then stopping and starting another video quickly with the keyboard without waiting for the UI to follow the keypresses. Very often fails in CDVDPlayer::OpenFile(), looks like an object is destroyed and used there.

A minor cosmetic issue with vc1 and dxva hw accel to report the codec info (did not happen with Eden) due to a change in behavior.
With previous ffmpeg, in CDVDVideoCodecFFmpeg::Open(), m_pHardware was always null so a "-dxva2" was not added. That happened later through CDVDVideoCodecFFmpeg::SetHardware, called through GetFormat().

Now, with vc1/wmv3 for some reason, GetFormat() is called twice, once before and once after CDVDVideoCodecFFmpeg::Open()
so m_pHardware is valid, and the "-dxva2" gets added twice.

Quick patch here http://pastebin.com/Dv9ak3qZ , dunno if it interferes with other platforms but I don't think so.

The skipping crashes are annoying, but I wouldn't say blocking. Other than that I'd say we're in good shape. Well done!

I was having the issue with the CopyPicture() crash. I was able to fix it by commenting out lines 343 through 351 in DVDPlayerVideo.cpp which are located in the CDVDPlayerVideo::Process() function. I haven't had any trouble with playback or seeking after that and I don't know xbmc well enough to know what I may have broken or if this is the best solution, but I thought I would post it here in hopes that it might simplify someone's life for debugging. These were the lines I commented out:

// //Waiting timed out, output last picture
// if( picture.iFlags & DVP_FLAG_ALLOCATED )
// {
// //Remove interlaced flag before outputting
// //no need to output this as if it was interlaced
// picture.iFlags &= ~DVP_FLAG_INTERLACED;
// picture.iFlags |= DVP_FLAG_NOSKIP;
// OutputPicture(&picture, pts);
// pts+= frametime;
// }

  continue;
Member

elupus commented Apr 2, 2012

Thanx for the pointer @burriedu2, could you test if above patch resolves the issue?

Contributor

CrystalP commented Apr 3, 2012

codec name and the skipping are fine now. It's ready as far as I am concerned.

The fixes wouldn't hurt in Eden, right? We have weird reports of "crashing" when skipping that no dev was able to repro, I'm wondering if it could be the same thing. Unfortunately, the crashes only happen to users when debug logging is turned off for some strange reason...

Member

elupus commented Apr 3, 2012

Yea the fixes wont hurt Eden.

It looks like that patch works. Thank you!

@ghost

ghost commented Apr 3, 2012

elupus, is there a way to confirm that multithreading is being used? There are reports on the forum that commit a0fc63d breaks multithreaded decoding. Without it, many users cannot decode HD hi10 content. Granted, they could be mistaken, but I cannot find anything in the codec info window that tells how many threads are being used.

I have discovered a bug with this branch where I open the folder containing a particular file and xbmc crashes. It only happens with one particular file and everything else seems to work great. I don't think it should be a blocking issue or anything, but I figured I mention it. I have a xbmc debug log and a stack trace and I can upload the file somewhere if it would help.

stack trace:
#0 0x742b9270 in ?? ()
#1 0x0859597f in CDVDDemuxFFmpeg::AddStream (this=0xb1633a78, iId=100)
at DVDDemuxFFmpeg.cpp:1087
#2 0x0859727c in CDVDDemuxFFmpeg::Open (this=0xb1633a78, pInput=0xb1606ec0)
at DVDDemuxFFmpeg.cpp:493
#3 0x0859d5b1 in CDVDFactoryDemuxer::CreateDemuxer (pInputStream=0xb1606ec0)
at DVDFactoryDemuxer.cpp:66
#4 0x08544b27 in CDVDFileInfo::ExtractThumb (strPath=..., strTarget=...,
pStreamDetails=0xb5be0d8) at DVDFileInfo.cpp:104
#5 0x08b3a18b in CThumbExtractor::DoWork (this=0xb162cd00)
at ThumbLoader.cpp:142
#6 0x082ef3c0 in CJobWorker::Process (this=0xb36db120) at JobManager.cpp:65
#7 0x08a720a2 in CThread::staticThread (data=0xb36db120) at Thread.cpp:177
#8 0x00b68d31 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9 0x0259f46e in clone () from /lib/i386-linux-gnu/libc.so.6

Member

elupus commented Apr 4, 2012

Ok, it apperently has more than 100 streams. (rather strange to start with, but we should handle it gracefully)

@elupus elupus merged commit 3a43c68 into xbmc:master Apr 4, 2012

This is bugged, if the source material is S16 already m_pBuffer1/2 will never have the data copied into them from m_pFrame1 producing no output.

Member

elupus replied Apr 12, 2012

Hmm.. src is normally S16, and we do have audio with this. So i'm not following.

Member

elupus replied Apr 12, 2012

And m_pBuffer1 is not used anymore.

Member

gnif replied Apr 12, 2012

sorry, please ignore, I overlooked the change in GetData

Unknown referenced this pull request Apr 19, 2012

Closed

enable hardware decoding for DVDs #399

jmanley commented on a0fc63d May 3, 2012

I tried a recompile with the m_pCodecContext->thread_type = FF_THREAD_SLICE line commented out, in order to re-enable multithreaded decoding. Software decoding with multithreading seems to work great, except that the aspect ratio is incorrect if I enable "sync refresh rate to fps". If I leave it at default (60Hz), it's correct. Is this what you mean by sensitivity to changes in frame size? Why does the refresh rate affect the aspect ratio?

Member

elupus replied May 3, 2012

What do you think of this, instead :
/* Only allow slice threading, since frame threading is more sensitive to changes in frame sizes, and it causes crashes during HW accell */
bool disable_mt_frame=true;

ifdef HAS_DX

disable_mt_frame=g_guiSettings.GetBool("videoplayer.usedxva2");

endif

if (disable_mt_frame) m_pCodecContext->thread_type = FF_THREAD_SLICE;
else m_pCodecContext->thread_type = FF_THREAD_FRAME;

I use my own build with this for more than a month, without noticing any issue (of course, i'm not using DVXA...).

Owner

elupus replied May 31, 2012

From what I read it is even easier:

if (!m_bSoftware)
m_pCodecContext->thread_type = FF_THREAD_SLICE;

So, keep the previous behaviour, when Software is used for decoding.

@HolgerW1 HolgerW1 pushed a commit to HolgerW1/xbmc that referenced this pull request Sep 26, 2014

@midgetspy midgetspy Merge pull request #810 from CalBoy/development
Fix scene exceptions URL
fcee809

@HolgerW1 HolgerW1 pushed a commit to HolgerW1/xbmc that referenced this pull request Sep 26, 2014

Patrick Vos Merge branch 'development'
Add require/ignore words per show. Pull #799
Some notable examples below of using ignore/require words,
also the global ignore words in the webui under search settings.
Ignore webrip or 'xvid' while still getting web-dl.
Require a specific's groups rip, like QRUS or ignore one like mSD.
Grab HD rips but require it to have DD5 audio.
Require VFR (variable frame rate) for the show Top Gear.
Require Uncut or XL versions of a show (since TVDB does not allow 'duplicate' shows).
Do note that the ignore word logic comes before required words. So be careful on what you ignore.
The words are not case sensitive, and if you use multiple words then a release will be rejected if ANY match.

Fix bug on massEdit removing values not displayed (air-by-date, language, release ignore/require words).

Remove superfluous logging message when editing show.

Do not set a show to air-by-date if it contains Documentary -- fixes Top Gear

Update scene_exception url to point new github domain. Pull #810

Change PostProcessing checks and rules. Pull #803
Ignore subfolders that start with . instead of a specific ignore list.
Pass (inherit) nzbName to subfolder
Add postProcessor checks: status, quality/filesize, already processed.
Change use quality from snatch history instead of status quality
Delete empty folder (scripts and automatic modes, not manual).
Change search for airdate in database instead of thetvdb.
Includes PR-620-jayme-github Process files by size but reversed (biggest first).
Stop postprocessing when failing to create destination folder.

Update autoProcessTV to work in python 2 and 3 (note that sickbeard itself only works in 2.5+ and not 3). Pull #802

API and NZBGet cleanup. Pull #804
Updated NZBGet to allow setting username (requires NZBGet v11+).
Updated NZBGet to have real priority for (requires NZBGet v9+) instead of just adding to the top of the queue.
Change NZBGet host field to an url so that people can specify the protocol (http/https).

SB-API: PEP8 fixes, use unicode not string when returning show name for show.refresh / show.pause / show.update.
APIBuilder: Limit show list to 100 shows to ensure performance
(since the show list is used multiple times for different linked list)

Lower default RSS search freq. from 60 min to 40 mins.

Upgrade Jquery UI from 1.10.3 to 1.10.4. Pull #793

Quick pass over our .js to cleanup whitespace, add missing semicolons, url redirect fixes. Pull #795

Handle malformed tvshow.nfo files when using the Add Existing Show page and resolved
the loading indicator to not finish.

Fix Exception on adding the initial rootdir.

Add routines to check or clean config input from the GUI.

Change usenet-crawler to https

Use with openwhen opening/writing files so it will automatically close the file,
even if there is an exception. Applied whitespace PEP8 fixes to the files touched,
also remove obsolete functions using open.

Fix AttributeError exception for metadata if stripping genre/actor name when None. Pull #801

Torrent provider HDbits now supports backlog and proper searches.

Metadata: Mede8er added.
f64b94f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment