Android/frameworks_base
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge "media: clarify the use of profiles for MediaCodec encoders" into udc-dev

Browse Source
This commit is contained in:
Lajos Molnar
2023-06-06 19:14:52 +00:00
committed by Android (Google) Code Review
parent 7999f726b9 a9b03a6bc2
commit 5335811d6d
2 changed files with 49 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

45
media/java/android/media/MediaCodec.java
View File

@@ -359,6 +359,51 @@ import java.util.concurrent.locks.ReentrantLock;
codec to use a previously created {@linkplain #createPersistentInputSurface persistent input
surface} by calling {@link #setInputSurface}.
<h4 id=EncoderProfiles><a name="EncoderProfiles"></a>Encoder Profiles</h4>
<p>
When using an encoder, it is recommended to set the desired codec {@link MediaFormat#KEY_PROFILE
profile} during {@link #configure configure()}. (This is only meaningful for
{@link MediaFormat#KEY_MIME media formats} for which profiles are defined.)
<p>
If a profile is not specified during {@code configure}, the encoder will choose a profile for the
session based on the available information. We will call this value the <i>default profile</i>.
The selection of the default profile is device specific and may not be deterministic
(could be ad hoc or even experimental). The encoder may choose a default profile that is not
suitable for the intended encoding session, which may result in the encoder ultimately rejecting
the session.
<p>
The encoder may reject the encoding session if the configured (or default if unspecified) profile
does not support the codec input (mainly the {@link MediaFormat#KEY_COLOR_FORMAT color format} for
video/image codecs, or the {@link MediaFormat#KEY_PCM_ENCODING sample encoding} and the {@link
MediaFormat#KEY_CHANNEL_COUNT number of channels} for audio codecs, but also possibly
{@link MediaFormat#KEY_WIDTH width}, {@link MediaFormat#KEY_HEIGHT height},
{@link MediaFormat#KEY_FRAME_RATE frame rate}, {@link MediaFormat#KEY_BIT_RATE bitrate} or
{@link MediaFormat#KEY_SAMPLE_RATE sample rate}.)
Alternatively, the encoder may choose to (but is not required to) convert the input to support the
selected (or default) profile - or adjust the chosen profile based on the presumed or detected
input format - to ensure a successful encoding session. <b>Note</b>: Converting the input to match
an incompatible profile will in most cases result in decreased codec performance.
<p>
To ensure backward compatibility, the following guarantees are provided by Android:
<ul>
<li>The default video encoder profile always supports 8-bit YUV 4:2:0 color format ({@link
CodecCapabilities#COLOR_FormatYUV420Flexible COLOR_FormatYUV420Flexible} and equivalent
{@link CodecCapabilities#colorFormats supported formats}) for both Surface and ByteBuffer modes.
<li>The default video encoder profile always supports the default 8-bit RGBA color format in
Surface mode even if no such formats are enumerated in the {@link CodecCapabilities#colorFormats
supported formats}.
</ul>
<p class=note>
<b>Note</b>: the accepted profile can be queried through the {@link #getOutputFormat output
format} of the encoder after {@code configure} to allow applications to set up their
codec input to a format supported by the encoder profile.
<p>
<b>Implication:</b>
<ul>
<li>Applications that want to encode 4:2:2, 4:4:4, 10+ bit or HDR video input <b>MUST</b> configure
a suitable profile for encoders.
</ul>
<h4 id=CSD><a name="CSD"></a>Codec-specific Data</h4>
<p>
Some formats, notably AAC audio and MPEG4, H.264 and H.265 video formats require the actual data

5
media/java/android/media/MediaFormat.java
View File

@@ -1136,11 +1136,14 @@ public final class MediaFormat {
* may fail if other parameters are not compatible with the desired
* profile or if the desired profile is not supported, but it may also
* fail silently (where the encoder ends up using a different, compatible profile.)
* <p>
* It is recommended that the profile is set for all encoders. For more information, see
* the <i>Encoder Profiles</i> section of the {@link MediaCodec} API reference.
* <p class="note">
* <strong>Note:</strong> Codecs are free to use all the available
* coding tools at the specified profile, but may ultimately choose to not do so.
* <p class="note">
* <strong>Note:</strong> When configuring video encoders, profile must be
* <strong>Note:</strong> When configuring video encoders, profile (if set) must be
* set together with {@link #KEY_LEVEL level}.
*
* @see MediaCodecInfo.CodecCapabilities#profileLevels