Merge \"media: document output image crop and layout for MediaCodec\" into nyc-dev

am: 939484b8ab

Change-Id: Iee0b8c9d07ac7847248e18f23fc26c5d8939f5bb

media/java/android/media/MediaCodec.java

@@ -125,6 +125,77 @@ import java.util.Map;
  All video codecs support flexible YUV 4:2:0 buffers since {@link
  android.os.Build.VERSION_CODES#LOLLIPOP_MR1}.
 
+ <h4>Accessing Raw Video ByteBuffers on Older Devices</h4>
+ <p>
+ Prior to {@link android.os.Build.VERSION_CODES#LOLLIPOP} and {@link Image} support, you need to
+ use the {@link MediaFormat#KEY_STRIDE} and {@link MediaFormat#KEY_SLICE_HEIGHT} output format
+ values to understand the layout of the raw output buffers.
+ <p class=note>
+ Note that on some devices the slice-height is advertised as 0. This could mean either that the
+ slice-height is the same as the frame height, or that the slice-height is the frame height
+ aligned to some value (usually a power of 2). Unfortunately, there is no way to tell the actual
+ slice height in this case. Furthermore, the vertical stride of the {@code U} plane in planar
+ formats is also not specified or defined, though usually it is half of the slice height.
+ <p>
+ The {@link MediaFormat#KEY_WIDTH} and {@link MediaFormat#KEY_HEIGHT} keys specify the size of the
+ video frames; however, for most encondings the video (picture) only occupies a portion of the
+ video frame. This is represented by the 'crop rectangle'.
+ <p>
+ You need to use the following keys to get the crop rectangle of raw output images from the
+ {@linkplain #getOutputFormat output format}. If these keys are not present, the video occupies the
+ entire video frame.The crop rectangle is understood in the context of the output frame
+ <em>before</em> applying any {@linkplain MediaFormat#KEY_ROTATION rotation}.
+ <table style="width: 0%">
+  <thead>
+   <tr>
+    <th>Format Key</th>
+    <th>Type</th>
+    <th>Description</th>
+   </tr>
+  </thead>
+  <tbody>
+   <tr>
+    <td>{@code "crop-left"}</td>
+    <td>Integer</td>
+    <td>The left-coordinate (x) of the crop rectangle</td>
+   </tr><tr>
+    <td>{@code "crop-top"}</td>
+    <td>Integer</td>
+    <td>The top-coordinate (y) of the crop rectangle</td>
+   </tr><tr>
+    <td>{@code "crop-right"}</td>
+    <td>Integer</td>
+    <td>The right-coordinate (x) <strong>MINUS 1</strong> of the crop rectangle</td>
+   </tr><tr>
+    <td>{@code "crop-bottom"}</td>
+    <td>Integer</td>
+    <td>The bottom-coordinate (y) <strong>MINUS 1</strong> of the crop rectangle</td>
+   </tr><tr>
+    <td colspan=3>
+     The right and bottom coordinates can be understood as the coordinates of the right-most
+     valid column/bottom-most valid row of the cropped output image.
+    </td>
+   </tr>
+  </tbody>
+ </table>
+ <p>
+ The size of the video frame (before rotation) can be calculated as such:
+ <pre class=prettyprint>
+ MediaFormat format = decoder.getOutputFormat(&hellip;);
+ int width = format.getInteger(MediaFormat.KEY_WIDTH);
+ if (format.containsKey("crop-left") && format.containsKey("crop-right")) {
+     width = format.getInteger("crop-right") + 1 - format.getInteger("crop-left");
+ }
+ int height = format.getInteger(MediaFormat.KEY_HEIGHT);
+ if (format.containsKey("crop-top") && format.containsKey("crop-bottom")) {
+     height = format.getInteger("crop-bottom") + 1 - format.getInteger("crop-top");
+ }
+ </pre>
+ <p class=note>
+ Also note that the meaning of {@link BufferInfo#offset BufferInfo.offset} was not consistent across
+ devices. On some devices the offset pointed to the top-left pixel of the crop rectangle, while on
+ most devices it pointed to the top-left pixel of the entire frame.
+
  <h3>States</h3>
  <p>
  During its life a codec conceptually exists in one of three states: Stopped, Executing or

media/java/android/media/MediaFormat.java

@@ -295,17 +295,19 @@ public final class MediaFormat {
      * Stride (or row increment) is the difference between the index of a pixel
      * and that of the pixel directly underneath. For YUV 420 formats, the
      * stride corresponds to the Y plane; the stride of the U and V planes can
-     * be calculated based on the color format.
+     * be calculated based on the color format, though it is generally undefined
+     * and depends on the device and release.
      * The associated value is an integer, representing number of bytes.
      */
     public static final String KEY_STRIDE = "stride";
 
     /**
      * A key describing the plane height of a multi-planar (YUV) video bytebuffer layout.
-     * Slice height (or plane height) is the number of rows that must be skipped to get
+     * Slice height (or plane height/vertical stride) is the number of rows that must be skipped
-     * from the top of the Y plane to the top of the U plane in the bytebuffer. In essence
+     * to get from the top of the Y plane to the top of the U plane in the bytebuffer. In essence
      * the offset of the U plane is sliceHeight * stride. The height of the U/V planes
-     * can be calculated based on the color format.
+     * can be calculated based on the color format, though it is generally undefined
+     * and depends on the device and release.
      * The associated value is an integer, representing number of rows.
      */
     public static final String KEY_SLICE_HEIGHT = "slice-height";