Files
frameworks_base/services/java/com/android/server/wm/Session.java
Svetoslav Ganov 1cf70bbf96 Screen magnification - feature - framework.
This change is the initial check in of the screen magnification
feature. This feature enables magnification of the screen via
global gestures (assuming it has been enabled from settings)
to allow a low vision user to efficiently use an Android device.

Interaction model:

1. Triple tap toggles permanent screen magnification which is magnifying
   the area around the location of the triple tap. One can think of the
   location of the triple tap as the center of the magnified viewport.
   For example, a triple tap when not magnified would magnify the screen
   and leave it in a magnified state. A triple tapping when magnified would
   clear magnification and leave the screen in a not magnified state.

2. Triple tap and hold would magnify the screen if not magnified and enable
   viewport dragging mode until the finger goes up. One can think of this
   mode as a way to move the magnified viewport since the area around the
   moving finger will be magnified to fit the screen. For example, if the
   screen was not magnified and the user triple taps and holds the screen
   would magnify and the viewport will follow the user's finger. When the
   finger goes up the screen will clear zoom out. If the same user interaction
   is performed when the screen is magnified, the viewport movement will
   be the same but when the finger goes up the screen will stay magnified.
   In other words, the initial magnified state is sticky.

3. Pinching with any number of additional fingers when viewport dragging
   is enabled, i.e. the user triple tapped and holds, would adjust the
   magnification scale which will become the current default magnification
   scale. The next time the user magnifies the same magnification scale
   would be used.

4. When in a permanent magnified state the user can use two or more fingers
   to pan the viewport. Note that in this mode the content is panned as
   opposed to the viewport dragging mode in which the viewport is moved.

5. When in a permanent magnified state the user can use three or more
   fingers to change the magnification scale which will become the current
   default magnification scale. The next time the user magnifies the same
   magnification scale would be used.

6. The magnification scale will be persisted in settings and in the cloud.

Note: Since two fingers are used to pan the content in a permanently magnified
   state no other two finger gestures in touch exploration or applications
   will work unless the uses zooms out to normal state where all gestures
   works as expected. This is an intentional tradeoff to allow efficient
   panning since in a permanently magnified state this would be the dominant
   action to be performed.

Design:

1. The window manager exposes APIs for setting accessibility transformation
   which is a scale and offsets for X and Y axis. The window manager queries
   the window policy for which windows will not be magnified. For example,
   the IME windows and the navigation bar are not magnified including windows
   that are attached to them.

2. The accessibility features such a screen magnification and touch
   exploration are now impemented as a sequence of transformations on the
   event stream. The accessibility manager service may request each
   of these features or both. The behavior of the features is not changed
   based on the fact that another one is enabled.

3. The screen magnifier keeps a viewport of the content that is magnified
   which is surrounded by a glow in a magnified state. Interactions outside
   of the viewport are delegated directly to the application without
   interpretation. For example, a triple tap on the letter 'a' of the IME
   would type three letters instead of toggling magnified state. The viewport
   is updated on screen rotation and on window transitions. For example,
   when the IME pops up the viewport shrinks.

4. The glow around the viewport is implemented as a special type of window
   that does not take input focus, cannot be touched, is laid out in the
   screen coordiates with width and height matching these of the screen.
   When the magnified region changes the root view of the window draws the
   hightlight but the size of the window does not change - unless a rotation
   happens. All changes in the viewport size or showing or hiding it are
   animated.

5. The viewport is encapsulated in a class that knows how to show,
   hide, and resize the viewport - potentially animating that.
   This class uses the new animation framework for animations.

6. The magnification is handled by a magnification controller that
   keeps track of the current trnasformation to be applied to the screen
   content and the desired such. If these two are not the same it is
   responsibility of the magnification controller to reconcile them by
   potentially animating the transition from one to the other.

7. A dipslay content observer wathces for winodw transitions, screen
   rotations, and when a rectange on the screen has been reqeusted. This
   class is responsible for handling interesting state changes such
   as changing the viewport bounds on IME pop up or screen rotation,
   panning the content to make a requested rectangle visible on the
   screen, etc.

8. To implement viewport updates the window manger was updated with APIs
   to watch for window transitions and when a rectangle has been requested
   on the screen. These APIs are protected by a signature level permission.
   Also a parcelable and poolable window info class has been added with
   APIs for getting the window info given the window token. This enables
   getting some useful information about a window. There APIs are also
   signature protected.

bug:6795382

Change-Id: Iec93da8bf6376beebbd4f5167ab7723dc7d9bd00
2012-09-06 18:56:17 -07:00

484 lines
19 KiB
Java

/*
* Copyright (C) 2011 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.android.server.wm;
import com.android.internal.view.IInputContext;
import com.android.internal.view.IInputMethodClient;
import com.android.internal.view.IInputMethodManager;
import com.android.server.wm.WindowManagerService.H;
import android.content.ClipData;
import android.content.Context;
import android.content.res.Configuration;
import android.graphics.Rect;
import android.graphics.Region;
import android.os.Binder;
import android.os.Bundle;
import android.os.IBinder;
import android.os.Parcel;
import android.os.RemoteException;
import android.os.ServiceManager;
import android.util.Slog;
import android.view.Display;
import android.view.IWindow;
import android.view.IWindowSession;
import android.view.InputChannel;
import android.view.Surface;
import android.view.SurfaceSession;
import android.view.WindowManager;
import java.io.PrintWriter;
/**
* This class represents an active client session. There is generally one
* Session object per process that is interacting with the window manager.
*/
final class Session extends IWindowSession.Stub
implements IBinder.DeathRecipient {
final WindowManagerService mService;
final IInputMethodClient mClient;
final IInputContext mInputContext;
final int mUid;
final int mPid;
final String mStringName;
SurfaceSession mSurfaceSession;
int mNumWindow = 0;
boolean mClientDead = false;
public Session(WindowManagerService service, IInputMethodClient client,
IInputContext inputContext) {
mService = service;
mClient = client;
mInputContext = inputContext;
mUid = Binder.getCallingUid();
mPid = Binder.getCallingPid();
StringBuilder sb = new StringBuilder();
sb.append("Session{");
sb.append(Integer.toHexString(System.identityHashCode(this)));
sb.append(" uid ");
sb.append(mUid);
sb.append("}");
mStringName = sb.toString();
synchronized (mService.mWindowMap) {
if (mService.mInputMethodManager == null && mService.mHaveInputMethods) {
IBinder b = ServiceManager.getService(
Context.INPUT_METHOD_SERVICE);
mService.mInputMethodManager = IInputMethodManager.Stub.asInterface(b);
}
}
long ident = Binder.clearCallingIdentity();
try {
// Note: it is safe to call in to the input method manager
// here because we are not holding our lock.
if (mService.mInputMethodManager != null) {
mService.mInputMethodManager.addClient(client, inputContext,
mUid, mPid);
} else {
client.setUsingInputMethod(false);
}
client.asBinder().linkToDeath(this, 0);
} catch (RemoteException e) {
// The caller has died, so we can just forget about this.
try {
if (mService.mInputMethodManager != null) {
mService.mInputMethodManager.removeClient(client);
}
} catch (RemoteException ee) {
}
} finally {
Binder.restoreCallingIdentity(ident);
}
}
@Override
public boolean onTransact(int code, Parcel data, Parcel reply, int flags)
throws RemoteException {
try {
return super.onTransact(code, data, reply, flags);
} catch (RuntimeException e) {
// Log all 'real' exceptions thrown to the caller
if (!(e instanceof SecurityException)) {
Slog.e(WindowManagerService.TAG, "Window Session Crash", e);
}
throw e;
}
}
public void binderDied() {
// Note: it is safe to call in to the input method manager
// here because we are not holding our lock.
try {
if (mService.mInputMethodManager != null) {
mService.mInputMethodManager.removeClient(mClient);
}
} catch (RemoteException e) {
}
synchronized(mService.mWindowMap) {
mClient.asBinder().unlinkToDeath(this, 0);
mClientDead = true;
killSessionLocked();
}
}
@Override
public int add(IWindow window, int seq, WindowManager.LayoutParams attrs,
int viewVisibility, Rect outContentInsets, InputChannel outInputChannel) {
return addToDisplay(window, seq, attrs, viewVisibility, Display.DEFAULT_DISPLAY,
outContentInsets, outInputChannel);
}
@Override
public int addToDisplay(IWindow window, int seq, WindowManager.LayoutParams attrs,
int viewVisibility, int displayId, Rect outContentInsets,
InputChannel outInputChannel) {
return mService.addWindow(this, window, seq, attrs, viewVisibility, displayId,
outContentInsets, outInputChannel);
}
@Override
public int addWithoutInputChannel(IWindow window, int seq, WindowManager.LayoutParams attrs,
int viewVisibility, Rect outContentInsets) {
return addToDisplayWithoutInputChannel(window, seq, attrs, viewVisibility,
Display.DEFAULT_DISPLAY, outContentInsets);
}
@Override
public int addToDisplayWithoutInputChannel(IWindow window, int seq, WindowManager.LayoutParams attrs,
int viewVisibility, int displayId, Rect outContentInsets) {
return mService.addWindow(this, window, seq, attrs, viewVisibility, displayId,
outContentInsets, null);
}
public void remove(IWindow window) {
mService.removeWindow(this, window);
}
public int relayout(IWindow window, int seq, WindowManager.LayoutParams attrs,
int requestedWidth, int requestedHeight, int viewFlags,
int flags, Rect outFrame, Rect outContentInsets,
Rect outVisibleInsets, Configuration outConfig, Surface outSurface) {
if (false) Slog.d(WindowManagerService.TAG, ">>>>>> ENTERED relayout from "
+ Binder.getCallingPid());
int res = mService.relayoutWindow(this, window, seq, attrs,
requestedWidth, requestedHeight, viewFlags, flags,
outFrame, outContentInsets, outVisibleInsets,
outConfig, outSurface);
if (false) Slog.d(WindowManagerService.TAG, "<<<<<< EXITING relayout to "
+ Binder.getCallingPid());
return res;
}
public void performDeferredDestroy(IWindow window) {
mService.performDeferredDestroyWindow(this, window);
}
public boolean outOfMemory(IWindow window) {
return mService.outOfMemoryWindow(this, window);
}
public void setTransparentRegion(IWindow window, Region region) {
mService.setTransparentRegionWindow(this, window, region);
}
public void setInsets(IWindow window, int touchableInsets,
Rect contentInsets, Rect visibleInsets, Region touchableArea) {
mService.setInsetsWindow(this, window, touchableInsets, contentInsets,
visibleInsets, touchableArea);
}
public void getDisplayFrame(IWindow window, Rect outDisplayFrame) {
mService.getWindowDisplayFrame(this, window, outDisplayFrame);
}
public void finishDrawing(IWindow window) {
if (WindowManagerService.localLOGV) Slog.v(
WindowManagerService.TAG, "IWindow finishDrawing called for " + window);
mService.finishDrawingWindow(this, window);
}
public void setInTouchMode(boolean mode) {
synchronized(mService.mWindowMap) {
mService.mInTouchMode = mode;
}
}
public boolean getInTouchMode() {
synchronized(mService.mWindowMap) {
return mService.mInTouchMode;
}
}
public boolean performHapticFeedback(IWindow window, int effectId,
boolean always) {
synchronized(mService.mWindowMap) {
long ident = Binder.clearCallingIdentity();
try {
return mService.mPolicy.performHapticFeedbackLw(
mService.windowForClientLocked(this, window, true),
effectId, always);
} finally {
Binder.restoreCallingIdentity(ident);
}
}
}
/* Drag/drop */
public IBinder prepareDrag(IWindow window, int flags,
int width, int height, Surface outSurface) {
return mService.prepareDragSurface(window, mSurfaceSession, flags,
width, height, outSurface);
}
public boolean performDrag(IWindow window, IBinder dragToken,
float touchX, float touchY, float thumbCenterX, float thumbCenterY,
ClipData data) {
if (WindowManagerService.DEBUG_DRAG) {
Slog.d(WindowManagerService.TAG, "perform drag: win=" + window + " data=" + data);
}
synchronized (mService.mWindowMap) {
if (mService.mDragState == null) {
Slog.w(WindowManagerService.TAG, "No drag prepared");
throw new IllegalStateException("performDrag() without prepareDrag()");
}
if (dragToken != mService.mDragState.mToken) {
Slog.w(WindowManagerService.TAG, "Performing mismatched drag");
throw new IllegalStateException("performDrag() does not match prepareDrag()");
}
WindowState callingWin = mService.windowForClientLocked(null, window, false);
if (callingWin == null) {
Slog.w(WindowManagerService.TAG, "Bad requesting window " + window);
return false; // !!! TODO: throw here?
}
// !!! TODO: if input is not still focused on the initiating window, fail
// the drag initiation (e.g. an alarm window popped up just as the application
// called performDrag()
mService.mH.removeMessages(H.DRAG_START_TIMEOUT, window.asBinder());
// !!! TODO: extract the current touch (x, y) in screen coordinates. That
// will let us eliminate the (touchX,touchY) parameters from the API.
// !!! FIXME: put all this heavy stuff onto the mH looper, as well as
// the actual drag event dispatch stuff in the dragstate
mService.mDragState.register(callingWin.mDisplayContent.getDisplay());
mService.mInputMonitor.updateInputWindowsLw(true /*force*/);
if (!mService.mInputManager.transferTouchFocus(callingWin.mInputChannel,
mService.mDragState.mServerChannel)) {
Slog.e(WindowManagerService.TAG, "Unable to transfer touch focus");
mService.mDragState.unregister();
mService.mDragState = null;
mService.mInputMonitor.updateInputWindowsLw(true /*force*/);
return false;
}
mService.mDragState.mData = data;
mService.mDragState.mCurrentX = touchX;
mService.mDragState.mCurrentY = touchY;
mService.mDragState.broadcastDragStartedLw(touchX, touchY);
// remember the thumb offsets for later
mService.mDragState.mThumbOffsetX = thumbCenterX;
mService.mDragState.mThumbOffsetY = thumbCenterY;
// Make the surface visible at the proper location
final Surface surface = mService.mDragState.mSurface;
if (WindowManagerService.SHOW_LIGHT_TRANSACTIONS) Slog.i(
WindowManagerService.TAG, ">>> OPEN TRANSACTION performDrag");
Surface.openTransaction();
try {
surface.setPosition(touchX - thumbCenterX,
touchY - thumbCenterY);
surface.setAlpha(.7071f);
surface.setLayer(mService.mDragState.getDragLayerLw());
surface.show();
} finally {
Surface.closeTransaction();
if (WindowManagerService.SHOW_LIGHT_TRANSACTIONS) Slog.i(
WindowManagerService.TAG, "<<< CLOSE TRANSACTION performDrag");
}
}
return true; // success!
}
public void reportDropResult(IWindow window, boolean consumed) {
IBinder token = window.asBinder();
if (WindowManagerService.DEBUG_DRAG) {
Slog.d(WindowManagerService.TAG, "Drop result=" + consumed + " reported by " + token);
}
synchronized (mService.mWindowMap) {
long ident = Binder.clearCallingIdentity();
try {
if (mService.mDragState == null) {
// Most likely the drop recipient ANRed and we ended the drag
// out from under it. Log the issue and move on.
Slog.w(WindowManagerService.TAG, "Drop result given but no drag in progress");
return;
}
if (mService.mDragState.mToken != token) {
// We're in a drag, but the wrong window has responded.
Slog.w(WindowManagerService.TAG, "Invalid drop-result claim by " + window);
throw new IllegalStateException("reportDropResult() by non-recipient");
}
// The right window has responded, even if it's no longer around,
// so be sure to halt the timeout even if the later WindowState
// lookup fails.
mService.mH.removeMessages(H.DRAG_END_TIMEOUT, window.asBinder());
WindowState callingWin = mService.windowForClientLocked(null, window, false);
if (callingWin == null) {
Slog.w(WindowManagerService.TAG, "Bad result-reporting window " + window);
return; // !!! TODO: throw here?
}
mService.mDragState.mDragResult = consumed;
mService.mDragState.endDragLw();
} finally {
Binder.restoreCallingIdentity(ident);
}
}
}
public void dragRecipientEntered(IWindow window) {
if (WindowManagerService.DEBUG_DRAG) {
Slog.d(WindowManagerService.TAG, "Drag into new candidate view @ " + window.asBinder());
}
}
public void dragRecipientExited(IWindow window) {
if (WindowManagerService.DEBUG_DRAG) {
Slog.d(WindowManagerService.TAG, "Drag from old candidate view @ " + window.asBinder());
}
}
public void setWallpaperPosition(IBinder window, float x, float y, float xStep, float yStep) {
synchronized(mService.mWindowMap) {
long ident = Binder.clearCallingIdentity();
try {
mService.setWindowWallpaperPositionLocked(
mService.windowForClientLocked(this, window, true),
x, y, xStep, yStep);
} finally {
Binder.restoreCallingIdentity(ident);
}
}
}
public void wallpaperOffsetsComplete(IBinder window) {
mService.wallpaperOffsetsComplete(window);
}
public Bundle sendWallpaperCommand(IBinder window, String action, int x, int y,
int z, Bundle extras, boolean sync) {
synchronized(mService.mWindowMap) {
long ident = Binder.clearCallingIdentity();
try {
return mService.sendWindowWallpaperCommandLocked(
mService.windowForClientLocked(this, window, true),
action, x, y, z, extras, sync);
} finally {
Binder.restoreCallingIdentity(ident);
}
}
}
public void wallpaperCommandComplete(IBinder window, Bundle result) {
mService.wallpaperCommandComplete(window, result);
}
public void setUniverseTransform(IBinder window, float alpha, float offx, float offy,
float dsdx, float dtdx, float dsdy, float dtdy) {
synchronized(mService.mWindowMap) {
long ident = Binder.clearCallingIdentity();
try {
mService.setUniverseTransformLocked(
mService.windowForClientLocked(this, window, true),
alpha, offx, offy, dsdx, dtdx, dsdy, dtdy);
} finally {
Binder.restoreCallingIdentity(ident);
}
}
}
public void onRectangleOnScreenRequested(IBinder token, Rect rectangle, boolean immediate) {
synchronized(mService.mWindowMap) {
final long identity = Binder.clearCallingIdentity();
try {
mService.onRectangleOnScreenRequested(token, rectangle, immediate);
} finally {
Binder.restoreCallingIdentity(identity);
}
}
}
void windowAddedLocked() {
if (mSurfaceSession == null) {
if (WindowManagerService.localLOGV) Slog.v(
WindowManagerService.TAG, "First window added to " + this + ", creating SurfaceSession");
mSurfaceSession = new SurfaceSession();
if (WindowManagerService.SHOW_TRANSACTIONS) Slog.i(
WindowManagerService.TAG, " NEW SURFACE SESSION " + mSurfaceSession);
mService.mSessions.add(this);
}
mNumWindow++;
}
void windowRemovedLocked() {
mNumWindow--;
killSessionLocked();
}
void killSessionLocked() {
if (mNumWindow <= 0 && mClientDead) {
mService.mSessions.remove(this);
if (mSurfaceSession != null) {
if (WindowManagerService.localLOGV) Slog.v(
WindowManagerService.TAG, "Last window removed from " + this
+ ", destroying " + mSurfaceSession);
if (WindowManagerService.SHOW_TRANSACTIONS) Slog.i(
WindowManagerService.TAG, " KILL SURFACE SESSION " + mSurfaceSession);
try {
mSurfaceSession.kill();
} catch (Exception e) {
Slog.w(WindowManagerService.TAG, "Exception thrown when killing surface session "
+ mSurfaceSession + " in session " + this
+ ": " + e.toString());
}
mSurfaceSession = null;
}
}
}
void dump(PrintWriter pw, String prefix) {
pw.print(prefix); pw.print("mNumWindow="); pw.print(mNumWindow);
pw.print(" mClientDead="); pw.print(mClientDead);
pw.print(" mSurfaceSession="); pw.println(mSurfaceSession);
}
@Override
public String toString() {
return mStringName;
}
}