[x264-devel] Reconfiguring x264 aspect ratio in the middle of stream

[Jarmo Torvinen]

Is it possible to change the aspect ratio (x264_param_t.vui.i_sar_width &
height) after the encoder
has been initialized with the x264_encoder_open()-call?

There is the x264_encoder_reconfig()-call but while inspecting the source
it seems that the vui*-parameters are
not copied in that function call.

Further down, the aspect_ratio_idc seems to be written at the
x264_sps_write() call in encoder
x264_encoder_encode():

/* Write SPS and PPS */
if( i_nal_type == NAL_SLICE_IDR && h->param.b_repeat_headers )
{
...

/* generate sequence parameters */
x264_nal_start( h, NAL_SPS, NAL_PRIORITY_HIGHEST );
x264_sps_write( &h->out.bs, h->sps );
x264_nal_end( h );


x264_param_default() sets the b_repeat_headers to 1, so the SPS is written
again by default for each IDR.


The whole point of this exercise would be to be able to reconfigure the
aspect ratio of x264 in the middle of the stream.
This happens for example in DVB-streams and when transcoding the DVB MPEG2
to 264, it would be nice
to automatically to preseve the correct aspect ratio. Any pointers to
correct direction would be appreciated,
I'm ready to contribute patches.


Jarmo Torvinen

[Gabriel Bouvigne]

Post by Jarmo Torvinen
Is it possible to change the aspect ratio (x264_param_t.vui.i_sar_width &
height) after the encoder
has been initialized with the x264_encoder_open()-call?
There is the x264_encoder_reconfig()-call but while inspecting the source
it seems that the vui*-parameters are
not copied in that function call.

[...]

Post by Jarmo Torvinen
I'm ready to contribute patches.

That would be a handy features, as there are many ATSC/DVB cases where
this would be usefull. Would you like to try implementing this?

--
Gabriel

[Jarmo Torvinen]

Post by Gabriel Bouvigne

Post by Jarmo Torvinen
Is it possible to change the aspect ratio

(x264_param_t.vui.i_sar_width &

Post by Gabriel Bouvigne

Post by Jarmo Torvinen
height) after the encoder
has been initialized with the x264_encoder_open()-call?
There is the x264_encoder_reconfig()-call but while inspecting the source
it seems that the vui*-parameters are
not copied in that function call.

[...]

Post by Jarmo Torvinen
I'm ready to contribute patches.

That would be a handy features, as there are many ATSC/DVB cases where
this would be usefull. Would you like to try implementing this?

Sure, actually I have already made a proof-of-concept style modification
which seems to work. I'm using
to to change dynamically the aspect ratio of the video while transcoding
from a DVB ts stream to a ts containing
h264 video content.

The POC contains modifications to
-x264 encoder/encoder.c,
-ffmpeg/libavcodec/libx264.c,
-ffmpeg/libavcodec/h264.c
-ffmpeg/ffmpeg.c

The modification to the x264 encoder part is really small, actually only
couple of lines:

// add these lines to x264_encoder_reconfig()

+ h->sps->vui.i_sar_width = h->param.vui.i_sar_width;
+ h->sps->vui.i_sar_height = h->param.vui.i_sar_height;


I see some problems with this approach:

1) it is too time consuming to call the x264_encoder_reconfig() just for
the purpose
of changing the aspect ratio, I'd rather have separate API function which
just would do that.
But on the other hand, is there enough justification to add another api
call just for that purpose?

2) is it correct way to directly to modify the h->sps.* parameters, maybe
it would be better to call
x264_sps_init() instead?

3) according to initial tests, the aspect ratio changes, but the timing is
not always correct. This is because the
aspect ratio info is sent in the SPS, so the timing depends on the timing
of the scene cut. To have correct timing,
there should be some way to force a scenecut or sending a SPS through the
API when the aspect ratio changes.

Any thoughts?


Jarmo

[Gabriel Bouvigne]

Post by Jarmo Torvinen
1) it is too time consuming to call the x264_encoder_reconfig() just for
the purpose
of changing the aspect ratio, I'd rather have separate API function which
just would do that.
But on the other hand, is there enough justification to add another api
call just for that purpose?

I don't think this would justify another API call

Post by Jarmo Torvinen
3) according to initial tests, the aspect ratio changes, but the timing is
not always correct. This is because the
aspect ratio info is sent in the SPS, so the timing depends on the timing
of the scene cut. To have correct timing,
there should be some way to force a scenecut or sending a SPS through the
API when the aspect ratio changes.

Yes, ideally you should have a way to force a new IDR.
However, I've seen many real DVB cases where the aspect ratio change was
slightly delayed from its theoretical placement. This is often not a big
issue, as TV broadcaster are often using short gops (about 15 to 30 frames).

--
Gabriel

[Loren Merritt]

Post by Jarmo Torvinen
1) it is too time consuming to call the x264_encoder_reconfig() just for
the purpose of changing the aspect ratio, I'd rather have separate API
function which just would do that. But on the other hand, is there
enough justification to add another api call just for that purpose?

x264_encoder_reconfig takes 7k cycles (2 microseconds), and you call it at
most once per GOP. I don't see the problem.

Post by Jarmo Torvinen
3) according to initial tests, the aspect ratio changes, but the timing
is not always correct. This is because the aspect ratio info is sent in
the SPS, so the timing depends on the timing of the scene cut. To have
correct timing, there should be some way to force a scenecut or sending
a SPS through the API when the aspect ratio changes.

x264_picture_t->i_type = X264_TYPE_IDR

What isn't guaranteed is that reconfig changes take place on the same
frame they're input with, as opposed to whatever frame is encoded next.

--Loren Merritt

[Jarmo Torvinen]

Post by Loren Merritt

Post by Jarmo Torvinen
1) it is too time consuming to call the x264_encoder_reconfig() just for
the purpose of changing the aspect ratio, I'd rather have separate API
function which just would do that. But on the other hand, is there
enough justification to add another api call just for that purpose?

x264_encoder_reconfig takes 7k cycles (2 microseconds), and you call it at
most once per GOP. I don't see the problem.

Ok.

Post by Loren Merritt

Post by Jarmo Torvinen
3) according to initial tests, the aspect ratio changes, but the timing
is not always correct. This is because the aspect ratio info is sent in
the SPS, so the timing depends on the timing of the scene cut. To have
correct timing, there should be some way to force a scenecut or sending
a SPS through the API when the aspect ratio changes.

x264_picture_t->i_type = X264_TYPE_IDR
What isn't guaranteed is that reconfig changes take place on the same
frame they're input with, as opposed to whatever frame is encoded next.

Thanks for the tip. I tried to set that parameter but unfortunately it
seems that the
parameter is ignored in a call to x264_encoder_encode().
Maybe this comment in x264.h (struct x264_picture_t) is valid?


...

/* In: force picture type (if not auto) XXX: ignored for now
* Out: type of the picture encoded */
int i_type;

...

Jarmo

[Steven Walters]

Post by Jarmo Torvinen
Thanks for the tip. I tried to set that parameter but unfortunately it
seems that the
parameter is ignored in a call to x264_encoder_encode().
Maybe this comment in x264.h (struct x264_picture_t) is valid?
...
/* In: force picture type (if not auto) XXX: ignored for now
* Out: type of the picture encoded */
int i_type;
...
Jarmo

It's been a bit since I've seen anything on the list about this,
and I'm somewhat interested in the feature...

as for the problem you were facing above, it could be fixed now by r1076

do you have any current differences against the repository to see how
things are progressing?

[Jarmo Torvinen]

Steven Walters <kemuri9 at gmail.com>
Sent by: x264-devel-bounces at videolan.org

Post by Jarmo Torvinen
Thanks for the tip. I tried to set that parameter but unfortunately it
seems that the
parameter is ignored in a call to x264_encoder_encode().
Maybe this comment in x264.h (struct x264_picture_t) is valid?
...
/* In: force picture type (if not auto) XXX: ignored for now
* Out: type of the picture encoded */
int i_type;
...
Jarmo

It's been a bit since I've seen anything on the list about this,
and I'm somewhat interested in the feature...
as for the problem you were facing above, it could be fixed now by r1076
do you have any current differences against the repository to see how
things are progressing?

Hi,

Great news. I tested with the latest git trunk and the IDR frame force
seems to now work
as it should. The timings seem to be perfect.

Please find attached the simple patch for allowing to dynamically adjust
the aspect ratio.

In order to test this feature for example with ffmpeg (encoding) and
mplayer (decoding), you need to modify
both of them as they don't yet support setting the correct parameters.
--
Jarmo



-------------- next part --------------
A non-text attachment was scrubbed...
Name: dynamic_aspect_ratio.patch.diff
Type: application/octet-stream
Size: 644 bytes
Desc: not available
Url : http://mailman.videolan.org/pipermail/x264-devel/attachments/20090119/c7d96788/attachment.obj