Interesting, I wonder what this is / why you'd want it. In particular, when you have the DTS but not the PTS.
The recent gstreamer 1.22 release [2] had what I read as the opposite—calculate a plausible DTS from the order and PTS. They did a nice job of explaining why it's useful. AFAICT, this approach is the only viable way to get B frames to work properly from a received RTP stream.
> H.264/H.265 timestamp correction elements ... Muxers are often picky and need proper PTS/DTS timestamps set on the input buffers, but that can be a problem if the encoded input media stream comes from a source that doesn't provide proper signalling of DTS, such as is often the case for RTP, RTSP and WebRTC streams or Matroska container files. Theoretically parsers should be able to fix this up, but it would probably require fairly invasive changes in the parsers, so two new elements h264timestamper and h265timestamper bridge the gap in the meantime and can reconstruct missing PTS/DTS.
Looks like the ffmpeg thing is dts2pts_bsf.c. [3] I haven't really read the implementation, but I was hoping the comment at the top would illuminate things, but "Derive PTS by reordering DTS from supported streams" isn't enough for me.
Thanks. Looks like the input in question was an .avi file. That's a container format I don't know anything about (and don't particularly care to). I suspect then it's not relevant for anything more modern.
A surprising number of mp4 files are missing the ctts atom that contains pts since it’s sort-of optional outside the Apple stack (like ffmpeg generated such files from raw h264, which is probably the actual motivation for finally fixing this.) This should allow you to not generate or fix such files.
It’s sort-of optional for most playback stacks because they leave frame reordering to individual decoders as a codec-specific implementation detail, but Apple’s stack actually cares about frame-accurate random access so it relies on the codec-independent container timestamps.
The inaccurate seeking you get without container pts is okay for playback but it falls apart with editing or stuff like av1an.
Thanks, that makes more sense! Raw (Annex B) h264 in particular isn't exactly an ideal media format but I've definitely gone through it while debugging stuff and can see how you'd need this plugin to get from it to a correctly muxed file with B frames.
I have a bit of a love-hate relationship with ffmpeg. Love because it does things nothing else does (e.g. has a main-profile software H.264 decoder) and is so comprehensive, hate because code quality varies (lots of long-unfixed bugs/quirks in trac) and documentation/API/CLI experience is poor. E.g., I wish the documentation for this new bsf explained what you just said.
You're right, this comment should just be added to the file header and maybe the docs about H.264 processing in FFmpeg; otherwise this comment will get lost in a couple hours and the knowledge about all this will only be known by the people that made the change and not many more...
I really wonder why it costs people so much to write the whys of everything technical they do in at least a source code comment, but it is indeed an everyday's uphill battle.
> - DTS to PTS reorder bsf
Interesting, I wonder what this is / why you'd want it. In particular, when you have the DTS but not the PTS.
The recent gstreamer 1.22 release [2] had what I read as the opposite—calculate a plausible DTS from the order and PTS. They did a nice job of explaining why it's useful. AFAICT, this approach is the only viable way to get B frames to work properly from a received RTP stream.
> H.264/H.265 timestamp correction elements ... Muxers are often picky and need proper PTS/DTS timestamps set on the input buffers, but that can be a problem if the encoded input media stream comes from a source that doesn't provide proper signalling of DTS, such as is often the case for RTP, RTSP and WebRTC streams or Matroska container files. Theoretically parsers should be able to fix this up, but it would probably require fairly invasive changes in the parsers, so two new elements h264timestamper and h265timestamper bridge the gap in the meantime and can reconstruct missing PTS/DTS.
Looks like the ffmpeg thing is dts2pts_bsf.c. [3] I haven't really read the implementation, but I was hoping the comment at the top would illuminate things, but "Derive PTS by reordering DTS from supported streams" isn't enough for me.
[1] https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/rel...
[2] https://gstreamer.freedesktop.org/releases/1.22/
[3] https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/rel...