Merge "media: clarify the use of profiles for MediaCodec encoders" into udc-dev am: 5335811d6d

Original change: https://googleplex-android-review.googlesource.com/c/platform/frameworks/base/+/23575693

Change-Id: Iffe3424da196ea08c0506937f31ee0caa53f505d
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>

media/java/android/media/MediaCodec.java

@@ -359,6 +359,51 @@ import java.util.concurrent.locks.ReentrantLock;
  codec to use a previously created {@linkplain #createPersistentInputSurface persistent input
  surface} by calling {@link #setInputSurface}.
 
+ <h4 id=EncoderProfiles><a name="EncoderProfiles"></a>Encoder Profiles</h4>
+ <p>
+ When using an encoder, it is recommended to set the desired codec {@link MediaFormat#KEY_PROFILE
+ profile} during {@link #configure configure()}. (This is only meaningful for
+ {@link MediaFormat#KEY_MIME media formats} for which profiles are defined.)
+ <p>
+ If a profile is not specified during {@code configure}, the encoder will choose a profile for the
+ session based on the available information. We will call this value the <i>default profile</i>.
+ The selection of the default profile is device specific and may not be deterministic
+ (could be ad hoc or even experimental). The encoder may choose a default profile that is not
+ suitable for the intended encoding session, which may result in the encoder ultimately rejecting
+ the session.
+ <p>
+ The encoder may reject the encoding session if the configured (or default if unspecified) profile
+ does not support the codec input (mainly the {@link MediaFormat#KEY_COLOR_FORMAT color format} for
+ video/image codecs, or the {@link MediaFormat#KEY_PCM_ENCODING sample encoding} and the {@link
+ MediaFormat#KEY_CHANNEL_COUNT number of channels} for audio codecs, but also possibly
+ {@link MediaFormat#KEY_WIDTH width}, {@link MediaFormat#KEY_HEIGHT height},
+ {@link MediaFormat#KEY_FRAME_RATE frame rate}, {@link MediaFormat#KEY_BIT_RATE bitrate} or
+ {@link MediaFormat#KEY_SAMPLE_RATE sample rate}.)
+ Alternatively, the encoder may choose to (but is not required to) convert the input to support the
+ selected (or default) profile - or adjust the chosen profile based on the presumed or detected
+ input format - to ensure a successful encoding session. <b>Note</b>: Converting the input to match
+ an incompatible profile will in most cases result in decreased codec performance.
+ <p>
+ To ensure backward compatibility, the following guarantees are provided by Android:
+ <ul>
+ <li>The default video encoder profile always supports 8-bit YUV 4:2:0 color format ({@link
+ CodecCapabilities#COLOR_FormatYUV420Flexible COLOR_FormatYUV420Flexible} and equivalent
+ {@link CodecCapabilities#colorFormats supported formats}) for both Surface and ByteBuffer modes.
+ <li>The default video encoder profile always supports the default 8-bit RGBA color format in
+ Surface mode even if no such formats are enumerated in the {@link CodecCapabilities#colorFormats
+ supported formats}.
+ </ul>
+ <p class=note>
+ <b>Note</b>: the accepted profile can be queried through the {@link #getOutputFormat output
+ format} of the encoder after {@code configure} to allow applications to set up their
+ codec input to a format supported by the encoder profile.
+ <p>
+ <b>Implication:</b>
+ <ul>
+ <li>Applications that want to encode 4:2:2, 4:4:4, 10+ bit or HDR video input <b>MUST</b> configure
+ a suitable profile for encoders.
+ </ul>
+
  <h4 id=CSD><a name="CSD"></a>Codec-specific Data</h4>
  <p>
  Some formats, notably AAC audio and MPEG4, H.264 and H.265 video formats require the actual data

media/java/android/media/MediaFormat.java

@@ -1136,11 +1136,14 @@ public final class MediaFormat {
      * may fail if other parameters are not compatible with the desired
      * profile or if the desired profile is not supported, but it may also
      * fail silently (where the encoder ends up using a different, compatible profile.)
+     * <p>
+     * It is recommended that the profile is set for all encoders. For more information, see
+     * the <i>Encoder Profiles</i> section of the {@link MediaCodec} API reference.
      * <p class="note">
      * <strong>Note:</strong> Codecs are free to use all the available
      * coding tools at the specified profile, but may ultimately choose to not do so.
      * <p class="note">
-     * <strong>Note:</strong> When configuring video encoders, profile must be
+     * <strong>Note:</strong> When configuring video encoders, profile (if set) must be
      * set together with {@link #KEY_LEVEL level}.
      *
      * @see MediaCodecInfo.CodecCapabilities#profileLevels