Playing videos and music is a popular activity on Android devices. The Android framework + provides {@link android.media.MediaPlayer} as a quick solution for playing media with minimal + code, and the {@link android.media.MediaCodec} and {@link android.media.MediaExtractor} classes + are provided for building custom media players. The open source project, ExoPlayer, is a + solution between these two options, providing a pre-built player that you can extend.
+ +ExoPlayer supports features not currently provided by + {@link android.media.MediaPlayer}, including Dynamic adaptive streaming + over HTTP (DASH), SmoothStreaming, and persistent caching. ExoPlayer can be extended + to handle additional media formats, and because you include it as part of your app code, + you can update it along with your app.
+ +This guide describes how to use ExoPlayer for playing Android supported media formats, as well as + DASH and SmoothStreaming playback. This guide also discusses ExoPlayer events, messages, DRM + support and guidelines for customizing the player.
+ ++ Note: ExoPlayer is an open source project that is not part of the Android + framework and is distributed separately from the Android SDK. The project contains a library and + a demo app that shows both simple and more advanced use of ExoPlayer:
+ +ExoPlayer is a media player built on top of the {@link android.media.MediaExtractor} and + {@link android.media.MediaCodec} APIs released in Android 4.1 (API level 16). At the core of this + library is the {@code ExoPlayer} class. This class maintains the player’s global state, but makes few + assumptions about the nature of the media being played, such as how the media data is obtained, + how it is buffered or its format. You inject this functionality through ExoPlayer’s {@code + prepare()} method in the form of {@code TrackRenderer} objects.
+ +ExoPlayer provides default {@code TrackRenderer} implementations for audio and + video, which make use of the {@link android.media.MediaCodec} and {@link android.media.AudioTrack} + classes in the Android framework. Both renderers require a {@code SampleSource} object, from which + they obtain individual media samples for playback. Figure 1 shows the high level object model for + an ExoPlayer implementation configured to play audio and video using these components.
+ +
++ Figure 1. High level object model for an ExoPlayer configured to play audio + and video using {@code TrackRenderer} objects +
+ + +A {@code TrackRenderer} processes a component of media for playback, such as + video, audio or text. The ExoPlayer class invokes methods on its {@code TrackRenderer} instances from a + single playback thread, and by doing so causes each media component to be rendered as the global + playback position is advanced. The ExoPlayer library provides {@code MediaCodecVideoTrackRenderer} as + the default implementations rendering video and {@code MediaCodecAudioTrackRenderer} for audio. + Both implementations make use of {@link android.media.MediaCodec} to decode individual media + samples. They can handle all audio and video formats supported by a given Android device + (see Supported Media + Formats for details). The ExoPlayer library also provides an implementation for rendering + text called {@code TextTrackRenderer}. +
+ +The code example below outlines the main steps required to instantiate an ExoPlayer to play video + and audio using the standard {@code TrackRenderer} implementations.
+ ++// 1. Instantiate the player. +player = ExoPlayer.Factory.newInstance(RENDERER_COUNT); +// 2. Construct renderers. +MediaCodecVideoTrackRenderer videoRenderer = … +MediaCodecAudioTrackRenderer audioRenderer = ... +// 3. Inject the renderers through prepare. +player.prepare(videoRenderer, audioRenderer); +// 4. Pass the surface to the video renderer. +player.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, + surface); +// 5. Start playback. +player.setPlayWhenReady(true); +... +player.release(); // Don’t forget to release when done! ++ +
For a complete example, see the {@code SimplePlayerActivity} in the ExoPlayer demo app, which + correctly manages an ExoPlayer instance with respect to both the {@link android.app.Activity} and + {@link android.view.Surface} lifecycles.
+ + +A standard {@code TrackRenderer} implementation requires a {@code SampleSource} to + be provided in its constructor. A {@code SampleSource} object provides format information and + media samples to be rendered. The ExoPlayer library provides {@code FrameworkSampleSource} and + {@code ChunkSampleSource}. The {@code FrameworkSampleSource} class uses {@link + android.media.MediaExtractor} to request, buffer and extract the media samples. The {@code + ChunkSampleSource} class provides adaptive playback using DASH or SmoothStreaming, and + implements networking, buffering and media extraction within the ExoPlayer library.
+ + ++ In order to render media formats supported by the Android framework, the {@code + FrameworkSampleSource} class uses {@link android.media.MediaExtractor} for networking, + buffering and sample extraction functionality. By doing so, it supports any media container format + supported by the version of Android where it is running. For more information about media formats + supported by Android, see Supported + Media Formats. +
+ +The diagram in Figure 2 shows the object model for an ExoPlayer implementation using + {@code FrameworkSampleSource}.
+ +
++ Figure 2. Object model for an implementation of ExoPlayer that renders + media formats supported by Android using {@code FrameworkSampleSource} +
+ +The following code example outlines how the video and audio renderers are constructed to + load the video from a specified URI.
+ ++FrameworkSampleSource sampleSource = new FrameworkSampleSource( + activity, uri, null, 2); +MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer( + sampleSource, null, true, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT, 0, + mainHandler, playerActivity, 50); +MediaCodecAudioTrackRenderer audioRenderer = new MediaCodecAudioTrackRenderer( + sampleSource, null, true); ++ +
The ExoPlayer demo app provides a complete implementation of this code in + {@code DefaultRendererBuilder}. The {@code SimplePlaybackActivity} class uses it to play one + of the videos available in the demo app. Note that in the example, video and audio + are muxed, meaning they are streamed together from a single URI. The {@code FrameworkSampleSource} + instance provides video samples to the {@code videoRenderer} object and audio samples to the + {@code audioRenderer} object as they are extracted from the media container format. It is also + possible to play demuxed media, where video and audio are streamed separately from different URIs. + This functionality can be achieved by having two {@code FrameworkSampleSource} instances instead + of one.
+ + +ExoPlayer supports adaptive streaming, which allows the quality of the + media data to be adjusted during playback based on the network conditions. DASH + and SmoothStreaming are examples of adaptive streaming technologies. Both these approaches + load media in small chunks (typically 2 to 10 seconds in duration). Whenever a chunk of media + is requested, the client selects from a number of possible formats. For example, a client may + select a high quality format if network conditions are good, or a low quality format if network + conditions are bad. In both techniques, video and audio are streamed separately.
+ +ExoPlayer supports adaptive playback through use of the {@code ChunkSampleSource} class, + which loads chunks of media data from which individual samples can be extracted. Each {@code + ChunkSampleSource} requires a {@code ChunkSource} object to be injected through its constructor, + which is responsible for providing media chunks from which to load and read samples. The {@code + DashMp4ChunkSource} and {@code SmoothStreamingChunkSource} classes provide DASH and SmoothStreaming + playback using the FMP4 container format. The {@code DashWebMChunkSource} class uses the WebM + container format to provide DASH playback.
+ +All of the standard {@code ChunkSource} implementations require a {@code FormatEvaluator} and + a {@code DataSource} to be injected through their constructors. The {@code FormatEvaluator} + objects select from the available formats before each chunk is loaded. The {@code DataSource} + objects are responsible for actually loading the data. Finally, the {@code ChunkSampleSources} + require a {@code LoadControl} object that controls the chunk buffering policy.
+ +The object model of an ExoPlayer configured for a DASH adaptive playback is shown in the + diagram below. This example uses an {@code HttpDataSource} object to stream the media over the + network. The video quality is varied at runtime using the adaptive implementation of {@code + FormatEvaluator}, while audio is played at a fixed quality level.
+ +
++ Figure 3. Object model for a DASH adaptive playback using ExoPlayer +
+ +The following code example outlines how the video and audio renderers are constructed.
+ ++Handler mainHandler = playerActivity.getMainHandler(); +LoadControl loadControl = new DefaultLoadControl( + new BufferPool(BUFFER_SEGMENT_SIZE)); +BandwidthMeter bandwidthMeter = new BandwidthMeter(); + +// Build the video renderer. +DataSource videoDataSource = new HttpDataSource(userAgent, + HttpDataSource.REJECT_PAYWALL_TYPES, bandwidthMeter); +ChunkSource videoChunkSource = new DashMp4ChunkSource(videoDataSource, + new AdaptiveEvaluator(bandwidthMeter), videoRepresentations); +ChunkSampleSource videoSampleSource = new ChunkSampleSource(videoChunkSource, + loadControl, VIDEO_BUFFER_SEGMENTS * BUFFER_SEGMENT_SIZE, true); +MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer( + videoSampleSource, null, true, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT, + 0, mainHandler, playerActivity, 50); + +// Build the audio renderer. +DataSource audioDataSource = new HttpDataSource(userAgent, + HttpDataSource.REJECT_PAYWALL_TYPES, bandwidthMeter); +ChunkSource audioChunkSource = new DashMp4ChunkSource(audioDataSource, + new FormatEvaluator.FixedEvaluator(), audioRepresentation); +SampleSource audioSampleSource = new ChunkSampleSource(audioChunkSource, + loadControl, AUDIO_BUFFER_SEGMENTS * BUFFER_SEGMENT_SIZE, true); +MediaCodecAudioTrackRenderer audioRenderer = new MediaCodecAudioTrackRenderer( + audioSampleSource, null, true); ++ +
In this code, {@code videoRepresentations} and {@code audioRepresentation} are {@code + Representation} objects, each of which describes one of the available media streams. In the DASH + model, these streams are parsed from a media presentation description (MPD) file. The ExoPlayer + library provides a {@code MediaPresentationDescriptionParser} class to obtain {@code + Representation} objects from MPD files.
+ ++ Note: Building Representation objects from MPD files is not required. You can + build Representation objects from other data sources if necessary. +
+ +The ExoPlayer demo app provides complete implementation of this code in + {@code DashVodRendererBuilder}. The {@code SimplePlaybackActivity} class uses this builder to + construct renderers for playing DASH sample videos in the demo app. It asynchronously fetches a + specified MPD file in order to construct the required {@code Representation} objects. For an + equivalent SmoothStreaming example, see the {@code SmoothStreamingRendererBuilder} class in the + demo app.
+ + +For DASH and SmoothStreaming playback, consider both static format selection at the + start of playback and dynamic format selection during playback. Static format selection should be + used to filter out formats that should not be used throughout the playback, for example formats + with resolutions higher than the maximum supported by the playback device. Dynamic selection varies + the selected format during playback, typically to adapt video quality in response to changes in + network conditions.
+ +When preparing a player, you should consider filtering out some of the available formats if + they are not useable for playback. Static format selection allows you to filter out + formats that cannot be used on a particular device or are not compatible with your player. + For audio playback, this often means picking a single format to play and discarding the others.
+ +For video playback, filtering formats can be more complicated. Apps should first + eliminate any streams that whose resolution is too high to be played by the device. For H.264, + which is normally used for DASH and SmoothStreaming playback, ExoPlayer’s {@code MediaCodecUtil} + class provides a {@code maxH264DecodableFrameSize()} method that can be used to determine what + resolution streams the device is able to handle, as shown in the following code example:
+ +
+int maxDecodableFrameSize = MediaCodecUtil.maxH264DecodableFrameSize();
+Format format = representation.format;
+if (format.width * format.height <= maxDecodableFrameSize) {
+ // The device can play this stream.
+ videoRepresentations.add(representation);
+} else {
+ // The device isn't capable of playing this stream.
+}
+
+
+This approach is used to filter {@code Representations} in the {@code DashVodRendererBuilder} + class of the ExoPlayer demo app, and similarly to filter track indices in {@code + SmoothStreamingRendererBuilder}.
+ +In addition to eliminating unsupported formats, it should be noted that the ability to + seamlessly switch between H.264 streams of different resolution is an optional decoder feature + available in Android 4.3 (API level 16) and higher, and so is not supported by all devices. The + availability of an adaptive H.264 decoder can be queried using {@code MediaCodecUtil}, as shown in + the following code example:
+ ++boolean isAdaptive = MediaCodecUtil.getDecoderInfo(MimeTypes.VIDEO_H264).adaptive; ++ +
The {@code MediaCodecVideoTrackRenderer} class is still able to handle resolution changes on + devices that do not have adaptive decoders, however the switch is not seamless. Typically, the + switch creates a small discontinuity in visual output lasting around 50-100ms. For devices that + do not provide an adaptive decoder, app developers may choose to adapt between formats at + a single fixed resolution so as to avoid discontinuities. The ExoPlayer demo app + implementation does not pick a fixed resolution.
+ + +During playback, you can use a {@code FormatEvaluator} to dynamically select from the + available video formats. The ExoPlayer library provides a {@code FormatEvaluator.Adaptive} + implementation for dynamically selecting between video formats based on the current network + conditions.
+ +This class provides a simple, general purpose reference implementation, however you are + encouraged to write your own {@code FormatEvaluator} implementation to best suit your particular + needs.
+ + +During playback, your app can listen for events generated by the ExoPlayer that indicate the + overall state of the player. These events are useful as triggers for updating the app user + interface such as playback controls. Many ExoPlayer components also report their own component + specific low level events, which can be useful for performance monitoring.
+ + +ExoPlayer allows instances of {@code ExoPlayer.Listener} to be added and removed using its + {@code addListener()} and {@code removeListener()} methods. Registered listeners are notified of + changes in playback state, as well as when errors occur that cause playback to fail. For more + information about the valid playback states and the possible transitions between them, see the + ExoPlayer source code.
+ +Developers who implement custom playback controls should register a listener and use it to + update their controls as the player’s state changes. An app should also show an + appropriate error to the user if playback fails.
+ +In addition to high level listeners, many of the individual components provided by the + ExoPlayer library allow their own event listeners. For example, {@code + MediaCodecVideoTrackRenderer} has constructors that take a {@code + MediaCodecVideoTrackRenderer.EventListener}. In the ExoPlayer demo app, {@code SimplePlayerActivity} + acts as a listener so that it can adjust the dimensions of the target surface to have the correct + height and width ratio for the video being played:
+ +
+@Override
+public void onVideoSizeChanged(int width, int height) {
+ surfaceView.setVideoWidthHeightRatio(height == 0 ? 1 : (float) width / height);
+}
+
+
+The {@code RendererBuilder} classes in the ExoPlayer demo app inject the activity as the + listener, for example in the {@code DashVodRendererBuilder} class:
+ ++MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer( + videoSampleSource, null, true, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT, + 0, mainHandler, playerActivity, 50); ++ +
Note that you must pass a {@link android.os.Handler} object to the renderer, which determines + the thread on which the listener’s methods are invoked. In most cases, you should use a + {@link android.os.Handler} associated with the app’s main thread, as is the case in this example. +
+ +Listening to individual components can be useful for adjusting UI based on player events, as + in the example above. Listening to component events can also be helpful for logging performance + metrics. For example, {@code MediaCodecVideoTrackRenderer} notifies its listener of dropped video + frames. A developer may wish to log such metrics to track playback performance in their + app.
+ +Many components also notify their listeners when errors occur. Such errors may or may not + cause playback to fail. If an error does not cause playback to fail, it may still result in + degraded performance, and so you may wish to log all errors in order to track playback + performance. Note that an ExoPlayer instance always notifies its high level listeners of errors that + cause playback to fail, in addition to the listener of the individual component from which the error + originated. Hence, you should display error messages to users only from high level listeners. + Within individual component listeners, you should use error notifications only for informational + purposes.
+ + +Some ExoPlayer components allow changes in configuration during playback. By convention, you make + these changes by passing asynchronous messages through the ExoPlayer to the component. + This approach ensures both thread safety and that the configuration change is + executed in order with any other operations being performed on the player.
+ +The most common use of messaging is passing a target surface to + {@code MediaCodecVideoTrackRenderer}:
+ ++player.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, + surface); ++ +
Note that if the surface needs to be cleared because + {@link android.view.SurfaceHolder.Callback#surfaceDestroyed + SurfaceHolder.Callback.surfaceDestroyed()} has been invoked, then you must send this + message using the blocking variant of {@code sendMessage()}:
++ +
+player.blockingSendMessage(videoRenderer, + MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, null); ++ +
You must use a blocking message because the contract of {@link + android.view.SurfaceHolder.Callback#surfaceDestroyed surfaceDestroyed()} requires that the + app does not attempt to access the surface after the method returns. The {@code + SimplePlayerActivity} class in the demo app demonstrates how the surface should be set and + cleared.
+ + +One of the main benefits of ExoPlayer over {@link android.media.MediaPlayer} is the ability to + customize and extend the player to better suit the developer’s use case. The ExoPlayer library + is designed specifically with this in mind, defining a number of abstract base classes and + interfaces that make it possible for app developers to easily replace the default implementations + provided by the library. Here are some use cases for building custom components:
+ +If a custom component needs to report events back to the app, we recommend that you + do so using the same model as existing ExoPlayer components, where an event listener is passed + together with a {@link android.os.Handler} to the constructor of the component.
+ +We recommended that custom components use the same model as existing ExoPlayer components to + allow reconfiguration by the app during playback, as described in + Sending messages to components. + To do this, you should implement a {@code ExoPlayerComponent} and receive + configuration changes in its {@code handleMessage()} method. Your app should pass + configuration changes by calling ExoPlayer’s {@code sendMessage()} and {@code + blockingSendMessage()} methods.
+ + +On Android 4.3 (API level 18) and higher, ExoPlayer supports Digital Rights Managment (DRM) + protected playback. In order to play DRM protected content with ExoPlayer, your app must + inject a {@code DrmSessionManager} into the {@code MediaCodecVideoTrackRenderer} and {@code + MediaCodecAudioTrackRenderer} constructors. A {@code DrmSessionManager} object is responsible for + providing the {@code MediaCrypto} object required for decryption, as well as ensuring that the + required decryption keys are available to the underlying DRM module being used.
+ +The ExoPlayer library provides a default implementation of {@code DrmSessionManager}, called + {@code StreamingDrmSessionManager}, which uses {@link android.media.MediaDrm}. The session + manager supports any DRM scheme for which a modular DRM component exists on the device. All + Android devices are required to support Widevine modular DRM (with L3 security, although many + devices also support L1). Some devices may support additional schemes such as PlayReady.
+ +The {@code StreamingDrmSessionManager} class requires a {@code MediaDrmCallback} to be + injected into its constructor, which is responsible for actually making provisioning and key + requests. You should implement this interface to make network requests to your license + server and obtain the required keys. The {@code WidevineTestMediaDrmCallback} class in the + ExoPlayer demo app sends requests to a Widevine test server.
diff --git a/docs/html/images/exoplayer/adaptive-streaming.png b/docs/html/images/exoplayer/adaptive-streaming.png new file mode 100644 index 0000000000000..9fc650c9eead9 Binary files /dev/null and b/docs/html/images/exoplayer/adaptive-streaming.png differ diff --git a/docs/html/images/exoplayer/frameworksamplesource.png b/docs/html/images/exoplayer/frameworksamplesource.png new file mode 100644 index 0000000000000..fd1e314ad6641 Binary files /dev/null and b/docs/html/images/exoplayer/frameworksamplesource.png differ diff --git a/docs/html/images/exoplayer/object-model.png b/docs/html/images/exoplayer/object-model.png new file mode 100644 index 0000000000000..e0d6e55a944fb Binary files /dev/null and b/docs/html/images/exoplayer/object-model.png differ