From 1d229b08905faf62b3f4de5bab29f700163264a6 Mon Sep 17 00:00:00 2001 From: Derek Sollenberger Date: Fri, 25 Feb 2022 21:04:10 +0000 Subject: [PATCH] Add class level JavaDocs for RuntimeShader. Bug: 213869522 Test: N/A Change-Id: I66e52575023839f70fa05048e4c08a4c6fc5faeb --- .../java/android/graphics/RuntimeShader.java | 220 +++++++++++++++++- 1 file changed, 218 insertions(+), 2 deletions(-) diff --git a/graphics/java/android/graphics/RuntimeShader.java b/graphics/java/android/graphics/RuntimeShader.java index 2ff888b06dd89..6abe34b1d6758 100644 --- a/graphics/java/android/graphics/RuntimeShader.java +++ b/graphics/java/android/graphics/RuntimeShader.java @@ -19,12 +19,228 @@ package android.graphics; import android.annotation.ColorInt; import android.annotation.ColorLong; import android.annotation.NonNull; +import android.view.Window; import libcore.util.NativeAllocationRegistry; /** - * Shader that calculates per-pixel color via a user defined Android Graphics Shading Language - * (AGSL) function. + *

A {@link RuntimeShader} calculates a per-pixel color based on the output of a user defined + * Android Graphics Shading Language (AGSL) function.

+ * + *

Android Graphics Shading Language

+ *

The AGSL syntax is very similar to OpenGL ES Shading Language, but there are some important + * differences that are highlighted here. Most of these differences are summed up in one basic fact: + * With GPU shading languages, you are programming a stage of the GPU pipeline. With AGSL, you + * are programming a stage of the {@link Canvas} or {@link RenderNode} drawing pipeline.

+ * + *

In particular, a GLSL fragment shader controls the entire behavior of the GPU between the + * rasterizer and the blending hardware. That shader does all of the work to compute a color, and + * the color it generates is exactly what is fed to the blending stage of the pipeline.

+ * + *

In contrast, AGSL functions exist as part of a larger pipeline. When you issue a + * {@link Canvas} drawing operation, Android (generally) assembles a single GPU fragment shader to + * do all of the required work. This shader typically includes several pieces. For example, it might + * include:

+ * + * + *

A {@link RuntimeShader}, like other {@link Shader} types, effectively contributes a function + * to the GPU’s fragment shader.

+ * + *

AGSL Shader Execution

+ *

Just like a GLSL shader, an AGSL shader begins execution in a main function. Unlike GLSL, the + * function receives as an input parameter the position of the pixel within the {@link Canvas} or + * {@link RenderNode} coordinate space (similar to gl_fragCoord) and returns the color to be shaded + * as a vec4 (similar to out vec4 color or gl_FragColor in GLSL).

+ * + * 
+ * vec4 main(vec2 canvas_coordinates);
+ *
+ * + *

AGSL and GLSL use different coordinate spaces by default. In GLSL, the fragment coordinate + * (fragCoord) is relative to the lower left. AGSL matches the screen coordinate system of the + * Android {@link Canvas} which has its origin as the upper left corner. This means that the + * coordinates provided as a parameter in the main function are local to the canvas with the + * exception of any {@link Shader#getLocalMatrix(Matrix)} transformations applied to this shader. + * Additionally, if the shader is invoked by another using {@link #setInputShader(String, Shader)}, + * then that parent shader may modify the input coordinates arbitrarily.

+ * + *

AGSL and Color Spaces

+ *

Android Graphics and by extension {@link RuntimeShader} are color managed. The working + * {@link ColorSpace} for an AGSL shader is defined to be the color space of the destination, which + * in most cases is determined by {@link Window#setColorMode(int)}.

+ * + *

When authoring an AGSL shader, you won’t know what the working color space is. For many + * effects, this is fine because by default color inputs are automatically converted into the + * working color space. For certain effects, it may be important to do some math in a fixed, known + * color space. A common example is lighting – to get physically accurate lighting, math should be + * done in a linear color space. To help with this, AGSL provides two intrinsic functions that + * convert colors between the working color space and the + * {@link ColorSpace.Named#LINEAR_EXTENDED_SRGB} color space: + * + * 

+ * vec3 toLinearSrgb(vec3 color);
+ * vec3 fromLinearSrgb(vec3 color);
+ * + *

AGSL and Premultiplied Alpha

+ *

When dealing with transparent colors, there are two (common) possible representations: + * straight (unassociated) alpha and premultiplied (associated) alpha. In ASGL the color returned + * by the main function is expected to be premultiplied. AGSL’s use of premultiplied alpha + * implies: + *

+ * + * + * + *

Uniforms

+ *

AGSL, like GLSL, exposes the concept of uniforms. An AGSL uniform is defined as a read-only, + * global variable that is accessible by the AGSL code and is initialized by a number of setter + * methods on {@link RuntimeShader}. AGSL exposes two primitive uniform data types (float, int) and + * two specialized types (colors, shaders) that are outlined below.

+ * + *

Primitive Uniforms

+ *

There are two primitive uniform types supported by AGSL, float and int. For these types and + * uniforms representing a grouping of these types, like arrays and matrices, there are + * corresponding {@link RuntimeShader} methods to initialize them. + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + * + *
Java Type AGSL Type Method
Floatsfloat{@link RuntimeShader#setFloatUniform(String, float)}
vec2{@link RuntimeShader#setFloatUniform(String, float, float)}
vec3{@link RuntimeShader#setFloatUniform(String, float, float, float)}
vec4{@link RuntimeShader#setFloatUniform(String, float, float, float, float)}
Integersint{@link RuntimeShader#setIntUniform(String, int)}
ivec2{@link RuntimeShader#setIntUniform(String, int, int)}
ivec3{@link RuntimeShader#setIntUniform(String, int, int, int)}
ivec4{@link RuntimeShader#setIntUniform(String, int, int, int, int)}
Matrices and Arraysmat2, mat3, and mat4, and float[]{@link RuntimeShader#setFloatUniform(String, float[])}
int[]{@link RuntimeShader#setIntUniform(String, int[])}
+ * + * For example, a simple AGSL shader making use of a float uniform to modulate the transparency + * of the output color would look like:

+ * + * 
+ * uniform float alpha;
+ * vec4 main(vec2 canvas_coordinates) {
+ *     vec3 red = vec3(1.0, 0.0, 0.0);
+ *     return vec4(red * alpha, alpha);
+ * }
+ * + *

After creating a {@link RuntimeShader} with that program the uniform can then be initialized + * and updated per frame by calling {@link RuntimeShader#setFloatUniform(String, float)} with the + * value of alpha. The value of a primitive uniform defaults to 0 if it is declared in the AGSL + * shader but not initialized.

+ * + *

Color Uniforms

+ *

AGSL doesn't know if uniform variables contain colors, it won't automatically convert them to + * the working colorspace of the shader at runtime. However, you can label your vec4 uniform with + * the "layout(color)" qualifier which lets Android know that the uniform will be used as a color. + * Doing so allows AGSL to transform the uniform value to the working color space. In AGSL, declare + * the uniform like this: + * + * 

+ * layout(color) uniform vec4 inputColorA;
+ * layout(color) uniform vec4 inputColorB;
+ * vec4 main(vec2 canvas_coordinates) {
+ *     // blend the two colors together and return the resulting color
+ *     return mix(inputColorA, inputColorB, 0.5);
+ * }
+ * + *

After creating a {@link RuntimeShader} with that program the uniforms can + * then be initialized and updated per frame by calling + * {@link RuntimeShader#setColorUniform(String, int)}, + * {@link RuntimeShader#setColorUniform(String, long)}, or + * {@link RuntimeShader#setColorUniform(String, Color)} with the desired colors. The value of a + * color uniform is undefined if it is declared in the AGSL shader but not initialized.

+ * + *

Shader Uniforms

+ * In GLSL, a fragment shader can sample a texture. For AGSL instead of sampling textures you can + * sample from any {@link Shader}, which includes but is not limited to {@link BitmapShader}. To + * make it clear that you are operating on an {@link Shader} object there is no "sample" + * method. Instead, the shader uniform has an "eval()" method. This distinction enables AGSL shaders + * to sample from existing bitmap and gradient shaders as well as other {@link RuntimeShader} + * objects. In AGSL, declare the uniform like this: + * + * 
+ * uniform shader myShader;
+ * vec4 main(vec2 canvas_coordinates) {
+ *     // swap the red and blue color channels when sampling from myShader
+ *     return myShader.sample(canvas_coordinates).bgra;
+ * }
+ * + *

After creating a {@link RuntimeShader} with that program the shader uniform can + * then be initialized and updated per frame by calling + * {@link RuntimeShader#setInputShader(String, Shader)} with the desired shader. The value of a + * shader uniform is undefined if it is declared in the AGSL shader but not initialized.

+ * + *

Although most {@link BitmapShader}s contain colors that should be color managed, some contain + * data that isn’t actually colors. This includes bitmaps storing normals, material properties + * (e.g. roughness), heightmaps, or any other purely mathematical data that happens to be stored in + * a bitmap. When using these kinds of shaders in AGSL, you probably want to initialize them with + * {@link #setInputBuffer(String, BitmapShader)}. Shaders initialized this way work much like + * a regular {@link BitmapShader} (including filtering and tiling), with a few major differences: + *

+ * + *

In addition, when sampling from a {@link BitmapShader} be aware that the shader does not use + * normalized coordinates (like a texture in GLSL). It uses (0, 0) in the upper-left corner, and + * (width, height) in the bottom-right corner. Normally, this is exactly what you want. If you’re + * evaluating the shader with coordinates based on the ones passed to your AGSL program, the scale + * is correct. However, if you want to adjust those coordinates (to do some kind of re-mapping of + * the bitmap), remember that the coordinates are local to the canvas.

+ * */ public class RuntimeShader extends Shader {