Ticket #6614 (closed Patches: Obsolete)

Opened 5 years ago

Last modified 19 months ago

Seeking in mpegts files stops playback

Reported by: dteirney Owned by:
Priority: 4 - Normal Milestone:
Component: DVD-Video menu navigation and playback Version: GIT
Severity: Normal Keywords: seek mpegts
Cc: elupus, theuni, davilla, Blocked By:
Blocking: Platform: All
Revision: 20403

Description

We are slowly building a collection of files that we can't seek in. All of them are mpegts files and as soon as we try to seek either forward or backwards the video closes.

Debug log for 3 files that aren't working can be found at  http://pastebin.com/f598ed316

Attachments

1001_20090219212800.mpg Download (9.4 MB) - added by dteirney 5 years ago.
1001_20090312212800.mpg Download (9.4 MB) - added by dteirney 5 years ago.
1001_20090402212800.mpg Download (9.4 MB) - added by dteirney 5 years ago.
mpegts-discontinuity-seeking.patch Download (4.3 KB) - added by dteirney 5 years ago.

Change History

comment:1 Changed 5 years ago by spiff

sample please

comment:2 Changed 5 years ago by dteirney

What's the best way to create a sample? The files that cause problems are over 6Gb in size (Digital HD DVB-T recordings from MythTV).

comment:3 Changed 5 years ago by dteirney

I'm pretty sure it relates to the "start" / "duration" time. For all of the recordings that don't work the current duration is completely poked in the OSD (shows as 100% complete even though it's not). There's an example screenshot at  http://trac.xbmc.org/ticket/6010

I've raised this as a separate ticket, because there are some files that still have the incorrect duration and end time displayed but seeking still seems to work. As soon as the OSD displays the duration as being 100% complete (even though it's not) seeking is busted.

comment:4 Changed 5 years ago by spiff

e.g.

mencoder -ovc copy -oac copy -endpos 10 -o remuxed.ts -of lavf input.ts

might do it (grabs the first 10 seconds). if the resulting file does not have the problem, just copying a chunk from the beginning should work for .ts files as well.

i suspect broken timestamps as well but without a sample we cannot reproduce and hence it would all be guesswork...

comment:5 Changed 5 years ago by dteirney

I couldn't copy the sound because mencoder doesn't appear to be able to deal with the LATM encapsulated AAC audio codec (I have another patch in XBMC to deal with this).

Curiously all 3 files barfed with a floating point exception when trying to encode using mencoder.

xbmc@davmyth:/var/multimedia/videos/Playback Samples$ mencoder -endpos 9.9Mb -ovc copy -nosound /var/lib/mythtv/recordings/1001_20090402212800.mpg -of lavf -o 1001_20090402212800.mpg MEncoder 2:1.0~rc2-0ubuntu17+medibuntu1 (C) 2000-2007 MPlayer Team CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 6000+ (Family: 15, Model: 67, Stepping: 3) CPUflags: Type: 15 MMX: 1 MMX2: 1 3DNow: 1 3DNow2: 1 SSE: 1 SSE2: 1 Compiled with runtime CPU detection. success: format: 0 data: 0x0 - 0x3a7fc14 TS file format detected. VIDEO H264(pid=250) NO AUDIO! NO SUBS (yet)! PROGRAM N. 1 FPS seems to be: 50.000000 [V] filefmt:29 fourcc:0x10000005 size:0x0 fps:50.00 ftime:=0.0200 MUXER_LAVF * REMEMBER: MEncoder's libavformat muxing is presently broken and can generate INCORRECT files in the presence of B frames. Moreover, due to bugs MPlayer will play these INCORRECT files as if nothing were wrong! * OK, exit videocodec: framecopy (0x0 24bpp fourcc=10000005) VIDEO CODEC ID: 0 Writing header... [mpeg @ 0x8721b48]dimensions not set Floating point exception

xbmc@davmyth:/var/multimedia/videos/Playback Samples$ mencoder -endpos 9.9Mb -ovc copy -nosound /var/lib/mythtv/recordings/1001_20090219212800.mpg -of lavf -o 1001_20090219212800.mpgMEncoder 2:1.0~rc2-0ubuntu17+medibuntu1 (C) 2000-2007 MPlayer Team CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 6000+ (Family: 15, Model: 67, Stepping: 3) CPUflags: Type: 15 MMX: 1 MMX2: 1 3DNow: 1 3DNow2: 1 SSE: 1 SSE2: 1 Compiled with runtime CPU detection. success: format: 0 data: 0x0 - 0xe1d11210 TS file format detected. VIDEO H264(pid=250) NO AUDIO! NO SUBS (yet)! PROGRAM N. 1 FPS seems to be: 50.000000 [V] filefmt:29 fourcc:0x10000005 size:0x0 fps:50.00 ftime:=0.0200 MUXER_LAVF * REMEMBER: MEncoder's libavformat muxing is presently broken and can generate INCORRECT files in the presence of B frames. Moreover, due to bugs MPlayer will play these INCORRECT files as if nothing were wrong! * OK, exit videocodec: framecopy (0x0 24bpp fourcc=10000005) VIDEO CODEC ID: 0 Writing header... [mpeg @ 0x8721b48]dimensions not set Floating point exception

xbmc@davmyth:/var/multimedia/videos/Playback Samples$ mencoder -endpos 9.9Mb -ovc copy -nosound /var/lib/mythtv/recordings/1001_20090312212800.mpg -of lavf -o 1001_20090312212800.mpgMEncoder 2:1.0~rc2-0ubuntu17+medibuntu1 (C) 2000-2007 MPlayer Team CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 6000+ (Family: 15, Model: 67, Stepping: 3) CPUflags: Type: 15 MMX: 1 MMX2: 1 3DNow: 1 3DNow2: 1 SSE: 1 SSE2: 1 Compiled with runtime CPU detection. success: format: 0 data: 0x0 - 0xdc11427c TS file format detected. VIDEO H264(pid=250) NO AUDIO! NO SUBS (yet)! PROGRAM N. 1 FPS seems to be: 50.000000 [V] filefmt:29 fourcc:0x10000005 size:0x0 fps:50.00 ftime:=0.0200 MUXER_LAVF * REMEMBER: MEncoder's libavformat muxing is presently broken and can generate INCORRECT files in the presence of B frames. Moreover, due to bugs MPlayer will play these INCORRECT files as if nothing were wrong! * OK, exit videocodec: framecopy (0x0 24bpp fourcc=10000005) VIDEO CODEC ID: 0 Writing header... [mpeg @ 0x8721b48]dimensions not set Floating point exception

dd did the job though.

xbmc@davmyth:/var/multimedia/videos/Playback Samples$ dd if=/var/lib/mythtv/recordings/1001_20090219212800.mpg of=1001_20090219212800.mpg bs=1000 count=9900 9900+0 records in 9900+0 records out 9900000 bytes (9.9 MB) copied, 0.213566 s, 46.4 MB/s

The files are only 9.9Mb long due to the upload restrictions, which is all of 10 or so seconds of video. The duration is clearly "not quite right" during playback. I've adjusted the seek time to be 2 seconds in advanced settings and it seems to be working in these short files - damn. I'll upload the files none-the-less because the duration is clearly bizzare and might provide some clues.

With a seek time of 2 seconds the original 6Gb file still barfs regardless of whether played through  myth:// or directly off disk. I noticed that when I played back the large 6Gb recording off disk the duration showed as 1:02:00 (or something like that, which looked correct, and then quickly changed to the mangled variant of 11:56:18.

Changed 5 years ago by dteirney

Changed 5 years ago by dteirney

Changed 5 years ago by dteirney

comment:6 Changed 5 years ago by dteirney

Debug log of playback with skip being pressed using short 2 second skip time at  http://pastebin.com/f6806b6e6

comment:7 Changed 5 years ago by dteirney

Spiff, can you think of any other ways that I might be able to get a developer a sample file that fails? I've poked around in the code for quite a while with some additional debugging and haven't uncovered anything obvious. Really needs someone who knows much more about the demuxer / timestamp handling (at a guess).

WAF factor declines each time we run into a recorded program that stops when we go to skip commercials (meaning we need to play from the beginning again and try our best not to touch the remote).

comment:8 Changed 5 years ago by Gamester17

  • Cc elupus added
  • Version changed from 9.04 "Babylon" to SVN

comment:9 Changed 5 years ago by spiff

if bandwith concerns aren't a problem, pop by irc ..

comment:10 Changed 5 years ago by dteirney

I've done some more testing to try and narrow the problem down further. I kept increasing the file size until it stopped working.

When the file size is somewhere between 1,800,000 bytes and 1,900,000 bytes it goes from working to not working. That size is curiously close to 2.0Gb, so I'm wondering whether there is some issue with using plain integers for the seeking?

The current 1.9Gb file would take eons to upload with my ADSL 2+ connection (upload only 512Mbit/s) so I'm not sure how I can get this file to someone - or is the fact that the issue is related to file size somehow enough to investigate further?

Since this isn't happening for all our files there must be some issue when the file size is large and there is a start time set within the mpegts stream.

I'm running on a 32 bit install of Mythbuntu.

David

comment:11 Changed 5 years ago by dteirney

Dammit. I missed some zeros off the end of those numbers. Either way it was somewhere between 1.8 and 1.9Gb that it started to fail. I've had a look around the code that displays the current playback time and can't for the life of me see how I'm getting 11:46:23 displaying as soon as the mpegts file starts. Based on what I can see it should be using m_clock.GetClock() as the display time, which as I understand it is 0 based from the start of playback.

comment:11 Changed 5 years ago by dteirney

Hmmm, so I thought that m_clock.GetClock() was supposed to return the time since the start of the video, i.e. assume = 0 when playback starts. That is not happening for some mpegts files which is the cause of a lot of my problems.

Have I misunderstood how m_clock.GetClock() is supposed to behave??? (I don't think so as this is how a lot of code assumes it to be).

Debug log with output of m_State.time (directly derived from m_clock.GetClock()) in UpdatePlayState at  http://pastebin.com/m1b06328c

comment:12 Changed 5 years ago by spiff

elupus mind commenting on that one?

comment:13 Changed 5 years ago by d4rk

Issue appears to be fixed in current ffmpeg (18911). Can you test by copying over mpegts*.* from ffmpeg/libavformat/ (from a clean ffmpeg checkout) into xbmc/cores/dvdplayer/Codecs/ffmpeg/libavformat and rebuilding.

comment:14 Changed 5 years ago by dteirney

I've applied the new ffmpeg mpegts*.* files to the linuxport branch. The clock related issue seems to be better as the playback time seems to be correct for some of the files I've tested. However, there are still some issues with seeking (some files aren't playing at all). I'll test more later. My suspicion is that some code doesn't expect the physical seek time to be different to the requested seek time.

Thanks for the help getting progress this far though!

comment:15 Changed 5 years ago by dteirney

The clock related issues seem to be working for all mpegts files now. The videos that weren't playing was because there was a commercial break from time 0, and that automatic seek somehow aborted playback (the same as if pressing skip with the remote would have done).

So, one issue resolved, but still have a number of files where seeking will nuke the playback. And now that commercial skip is working since the time markers match, I need to turn of EDL if we want to watch a show that is broken. Complete PITA.

I've put debug code in a bunch of other places, including where the stream length is guessed based on the current PTS, size and pb->pos. That output occurs every now and then, but doesn't seem like it's related to any of the problems.

One of the files that plays back but stops when attempting to skip continually has the duration on the OSD changing. It seems to be roughly 5 seconds longer than the playback time. There's no logging from the XBMC code that tries to guess the duration if FFMPEG returns a dodgy length, so that must be happening in the FFMPEG code directly.

comment:16 Changed 5 years ago by elupus

Mpeg-ts will try to guess duration based on timestamps in the stream. If the stream doesn't have contineous timestamps, things will go heywire.

Is it possible to seek in these files using ffplay? If so, there could be a point in reverting our patch on mpegts seeking. (then again if you copied new ones from mplayerhq.hu already, that is already done).

And yea, clock.GetTime() should be zero based normally. But it isn't really any requirement.

comment:17 Changed 5 years ago by dteirney

I've just finished a test using the SVN version of ffmpeg and the 1.9Gb test file that stops as soon as XBMC tries to skip.

Seeking doesn't work using ffplay either. An error is spat out "filename: error while seeking". Nothing seems to work from then on. Each time I press one of the arrow keys the same error message is shown.

comment:18 Changed 5 years ago by dteirney

I've had a go at creating a ticket for this issue with the ffmpeg community.  https://roundup.ffmpeg.org/roundup/ffmpeg/issue1128

comment:19 Changed 5 years ago by dteirney

  • Cc theuni, davilla, added

Attached is a patch that I have been using to better support playback of mpegts files that have a timestamp discontinuity within it:

Patch is a rollup of the patches applied to related tickets for ffmpeg:  https://roundup.ffmpeg.org/roundup/ffmpeg/issue1128  https://roundup.ffmpeg.org/roundup/ffmpeg/issue452

 https://roundup.ffmpeg.org/roundup/ffmpeg/file370/seek-wrap.diff  https://roundup.ffmpeg.org/roundup/ffmpeg/file254/ffmpeg-wrapping-revert2.diff

Note that all I have done is apply the patches, I haven't reviewed to see if they can be improved or are sane etc. I just know they seem to be working, i.e. playback continues past a discontinuity.

Not sure how this relates to some of the mpegts patches that XBMC already has applied.

Changed 5 years ago by dteirney

comment:20 Changed 4 years ago by malloc

  • Type changed from Bugs to Patches
  • Milestone changed from Future / Pending to 10.05

@dtierney, does your attached patch fix this bug?

comment:21 Changed 4 years ago by dteirney

@malloc, the attached patch allows seeking to work without playback stopping. However, after some magic point during playback seeking forwards will actually go backwards to the same point. I assume this magic point is where the discontinuity is.

The patches above are what I found on the ffmpeg bug tracker and were intended for a vanilla mpegts.c and utils.c file. As I understand it, the XBMC version of mpegts.c has had some reasonably significant changes - I think to do with seeking.

If someone who knows about the existing mpegts.c seeking related patches could incorporate the patches attached that would be great.

comment:22 Changed 4 years ago by malloc

@elupus, would you mind taking a look at these patches?

comment:23 Changed 4 years ago by elupus

since it doesn't seem to solve the issue completely, i don't see the point.

comment:24 Changed 4 years ago by malloc

  • Blocking 7429 added

comment:26 Changed 4 years ago by dteirney

The patch actually only affects utils.c. I have tried the new seeking API for mpegts, and this still needs the patch in utils.c for seeking within files with timestamp discontinuities to work.

Additionally, as noted in #7429, for files where ffmpeg failed to determine the duration, seeking doesn't stop playback or go back to a single place, instead it just doesn't appear to do anything.

@elupus, I'm not sure of the technical capabilities of the chap who created the patches. It may become obvious what is needed to do to fix the issue completely after a look. Of course it might also not be possible to fix. I've looked and didn't see anything obviously wrong (other than the formating) but clearly there is still room for improvement.

comment:27 Changed 4 years ago by dteirney

Another forum post with users with this problem:  http://forum.xbmc.org/showthread.php?t=70099

Is there any XBMC developer that knows about the ffmpeg seeking routines that can have a look at this issue based on the semi-working patch, or is it a terminal problem? Once the PVR branch goes into trunk, anyone playing back DVB-T recordings is going to run into this issue at some point or another and it's very frustrating.

The latest update in  https://roundup.ffmpeg.org/roundup/ffmpeg/issue1128 from Michael Niedermayer (head ffmpeg guru) has the following suggestion to resolve. I don't know how to start implementing that though...

On Fri, Feb 05, 2010 at 11:45:09PM +0000, David Teirney wrote:

What is the appropriate way to get the seeking fixed? The revision that was mentioned only fixed ffplay, and that change was to use byte based seeking rather than time based seeking. What can be done so time based seeking behaves as expected?

linear search from a position of known timestamp

[...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

comment:28 Changed 4 years ago by malloc

  • Blocking 7429 removed

(In #7429) I have no reason to believe that this is related to #6614.

comment:29 Changed 4 years ago by elupus

  • Milestone changed from "Dharma" to Future / Pending

This isn't going to get fixed for Dharma

comment:30 Changed 3 years ago by sho

@dteirney, has there been any change in this area?

comment:31 Changed 3 years ago by dteirney

I haven't been following the ffmpeg CVS log for a while so not sure if there have been any improvements directly within ffmpeg. I will be updating our HTPC in the next few weeks (still haven't migrated it from the subversion tree to git). Will test some files without the patch applied and see if it still stops playback.

comment:32 Changed 3 years ago by elupus

We do linear search from a known timestamp at the moment. But i suppose just switching to byte-based seeks could be done. The problem is that we can't do it easily unless we know the full time of the file.

comment:33 Changed 2 years ago by fiveisalive

Also reported in the tsp's cmyth branch here:  https://github.com/tsp/xbmc/issues/1

comment:34 Changed 19 months ago by Martijn

  • Status changed from new to closed
  • Resolution set to Obsolete

Closing as it is for an outdated version and created long time ago

comment:35 Changed 19 months ago by Martijn

  • Milestone Future / Pending deleted
Note: See TracTickets for help on using tickets.