AnyPortrait > Manual > Writing Command Buffer

Writing Command Buffer


Rendering is done automatically by the Unity engine, but you can also extend its functionality by writing scripts.
One of the many ways to do this is to use "Command Buffer".
In a nutshell, Command Buffer is "scheduling a request to render the target mesh to specified conditions".
You can check the detailed explanation in the official Unity manual.
- Extending the Built-in Render Pipeline with CommandBuffers
- Scheduling and executing rendering commands in the Scriptable Render Pipeline
- CommandBuffer API
- CameraEvent API

If you use Unity's "CommandBuffer" class, AnyPortrait's character can be rendered through the command buffer, but it is difficult to implement because there is a lot of data that is difficult for users to access.
The "apCustomCommandBuffer" class added in AnyPortrait v1.4.0 is the version for AnyPortrait of the "CommandBuffer" class.
You can easily implement various techniques using this class.

This page covers how to write scripts using apCustomCommandBuffer and implement some fun techniques.
A detailed description of the functions of the apCustomCommandBuffer class can be found in the Related Page.

Draw a character to another camera with the Command Buffer

Before we get started, let's briefly understand how the command buffer is passed.
When a script creates a command buffer and passes it to Unity's rendering pipeline, additional rendering is executed according to the request of the reserved command buffer when the rendering pipeline is running.
The required elements that need to be passed here are "Mesh to be drawn" and "Camera to draw".
With this in mind, let's implement it.

Let's make the simplest example using the command buffer.
Prepares the environment for "the main camera (Camera 1)" to render the character's original meshes.
And let's make "the character (Cloned Meshes)" visible by the command buffer in "another camera (Camera 2)".
Let's remember "Original Meshes" and "Camera 2" as the necessary elements to create the command buffer.

We've set up the Unity scene to fit your description.
(1) You can see that there are 2 cameras and 1 AnyPortrait character.
(2) This is the AnyPortrait character (apPortrait).
(3) This is the main camera for drawing characters. It plays the same role as “Camera 1” in the previous diagram.
(4) This is an overlay camera that renders additionally when the main camera finishes rendering. It acts the same as "Camera 2" in the previous diagram.

These are the settings for each camera.
Camera 1 is set to render first as the main camera.
Camera 2 is rendered later than Camera 1 by the "Depth" value, and the character is not rendered by the "Culling Mask".

So if you run the game, the character will only be drawn on the main camera.

Now let's write a simple script that creates and updates the command buffer.
The following script does a very simple job of making the "input apPortrait character" draw on the "input camera".

Let's take a look at the key statements of the script one by one.

These are member objects.
(1) Make apPortrait and Camera variables, which are essential elements of the command buffer, so that can be connected from the Unity scene.
(2) Create the command buffer variable as apCustomCommandBuffer type.

This code work for that creates a command buffer and registers it with the render pipeline.
(1) In the example, apPortrait may not be initialized because initialization is written in the Start function, so we wrote initialization code. It is not required code.
(2) When creating a command buffer instance, input the Camera and apPortrait as arguments, and additionally input the command buffer's name.
(3) Register the command buffer in the rendering pipeline using the "AddToCamera" function.
At this time, it sets when the request of the command buffer will be processed during the rendering process.
If the project uses "Scriptable Render Pipeline (SRP)" environment, you should use "AddToCameraSRP" instead.

If any of the command buffer elements "character", "camera" and "this script that manages the command buffer" do not exist in the scene, the above code must be called to remove the command buffer from the rendering pipeline.

If any elements of the command buffer are changed, the contents of the command buffer must be rewritten.
Since AnyPortrait updates the mesh every frame, you need to rewrite the contents of the command buffer in the Update or LateUpdate function.
(1) To write the contents of the command buffer, the contents of the existing buffer must be cleared.
(2) Input the rendering View-Projection Matrix into the command buffer. This function is not supported in versions earlier than Unity 2019.
(3) Input which meshes to render in the command buffer in apPortrait. "DrawAllMeshes" asks to render all currently visible meshes if possible.

Let's put the script we wrote into our Unity scene.
(1) Create a new GameObject.
(2) Register the created script as a component, and input "apPortrait character" and "Overlay camera (Camera 2)" respectively.

If you run the game again, you should see the character being drawn on the second camera by the command buffer now.

Even small changes to the code that writes the contents of the command buffer can make a huge difference to the rendering result.
As an experiment, let's add a line of code like this:

In Overlay Camera (Camera 2), just before the character is rendered by the command buffer, the rendering target, that is, the screen color, was initialized to magenta, so you can see the above result.

Rendering the character in Grayscale

The use of command buffers is primarily for creating special rendering effects.
To create a rendering effect, a separate material must be available in the command buffer.
This time, let's implement the command buffer to render the character in grayscale.

This time, we need to write a Custom Shader that renders the character in a solid color and the Command Buffer Script described above.
First, write the following Custom Shader.

Modify or create a new command buffer script as described above.
This time we need to take the external material and write it to be used for rendering.

(1) Create a Material asset using the completed Custom Shader.

(1) Select the GameObject created earlier.
(2) Assign Material asset using the custom shader to the added variable of the script.

This is the finished result.
When rendering by the command buffer, you can see that the rendering is replaced by a different material.
By applying this method, you will be able to use various rendering techniques.

Rendering the character with the Outline

By utilizing the above explanations a little more, you can make a practical example.
A typical technique is "Outline".
In this example, you can implement it easily by using a simple custom shader that draws an outline using UV.
Additionally, when creating a command buffer, you need to change the rendering point.
Finally, unlike the previous example, this technique will make the command buffer run against the Main Camera.

Write a custom shader that draws the outline.
There are various ways to draw the outline, but here we will simply use the method of expanding the color using the Alpha value from the surrounding UV as a reference.
Create a new shader asset and write it like this:

Next, write the command buffer script.
It can be written almost identically to the code in the previous example, but the rendering point needs to be changed.
The custom shader we wrote has code that extends the character image a bit more and draws it in a solid color, so it needs to be "drawn before the character" for it to act as an outline.

(1) Create a Material asset with the outline shader applied.
(2) Select the Material asset and set the Thickness and Color of the outline.

(1) Select the GameObject that controls the command buffer.
(2) For the outline, you need to set the Main Camera that renders the character as the target, not the overlay camera.
(3) Assign the Material asset that draws the outline.

When you run the game, you can see that the outline is drawn with a reasonable quality.

Rendering the character which has two or more images

Through the previous explanation, we learned the process of rendering using a command buffer and an alternative material.
Next, in the case of that there is a character who has two or more images, let's render it assigning external materials.
If you want to render by applying a different "Secondary Texture" depending on the image, you need to modify the materials you need to prepare and the initialization code of the command buffer.

We prepared a character that uses the two textures on the left.
Let's make a material that is multiplied by the color of the secondary texture on the right corresponding to each image and render it through the command buffer.

Let's create a custom shader this time as well.
The "_MainTex" property is managed by AnyPortrait and cannot be used, so you need to create a separate texture property.

Before writing the script to control the command buffer, let's create a material.
(1) Create Materials according to the number of images.
(2) Assign each Secondary Texture to the "_SecondaryTex" attribute.

Now let's write a script to render by assigning two or more materials to the command buffer.

(1) This is a Unity scene featuring a character using two images.
(2) The Unity scene is the same as in the previous description. Select the GameObject with the script added.
(3) Assign the name of the image used by the apPortrait character and the material of the custom shader that matches it, in order.

If you run the game, you will see the above result with the appropriate alternate material corresponding to the image on the mesh applied.

Rendering the Render Texture with a Transparent background

This issue has been requested by several users.
When rendering a character to Render Texture rather than the screen, if the background of the render texture is made transparent, the mesh disappears.
For more information on how to render a character with a render texture, see the Related Page for detailed instructions.
This page describes the script that controls the "Render Target", introducing how to solve the problem of rendering with a render texture with a command buffer.

This is a diagram of how to render a character by saving the rendered result in a render texture and then overlaying it on another mesh.
Render textures in which characters are rendered are often used because they can be used in various ways in the game.
The command buffer is not used in this step yet.

We configured the scene according to the above idea.
It consists of two layers (UI, Default), and AnyPortrait character, 2 Cameras, and a Quad Mesh with render texture are placed in the scene.
(1) Character and the First Camera that renders it. It is not rendered on the Main Camera.
(2) Quad Mesh ("RT Quad") and Main Camera with render texture applied.

(1) And this is the “Render Texture” asset which is the key of this technique.
(2) This Render Texture is registered as "Target Texture" of the First Camera that renders the character.

The character is set to a non-Default layer (in this example, set to UI) so that it doesn't render on the Main Camera.

(1) This is the Quad Mesh rendered by the Main Camera.
(2) The texture of this mesh's material is the "Render Texture" asset used by the First Camera.
Because the material of this Quad Mesh and the First Camera share the same Render Texture, the result of rendering with the First Camera will appear in the Quad Mesh.

When you run the game, you can see the rendering result of the First Camera on the Quad Mesh, that is, the character is displayed normally.

The problem arises when the background color of the render texture is not opaque.
(1) Select the First Camera to render the character.
(2) Select the Background color property. (Clear Flags must be Solid Color.)
(3) Reduce the Alpha value of the background color.
(4) You can see that the entire render texture including the character as well as the background becomes increasingly transparent.

This is the idea how to solve this problem.
Complete the render texture of the opaque background, and separately create "Mask Texture" using the Command Buffer.
The Mask can be used to separate the "Background" and "Character" areas.
Combine this Mask Texture with the original Render Texture to render the image with only the background transparent.

Note that.
For clarity, the original render texture generated by the camera is called "Color Render Texture", and the render texture that acts as a mask generated by the command buffer is called "Mask Render Texture".

A lot of work is required to implement this technique.
(1) Write a Custom Shader for mask rendering that stores "the Alpha value in Grayscale".
(2) Create an additional "Mask Render Texture".
(3) Write a "Command Buffer Script" to render the mask render texture.
(4) Create a Shader and Material to be applied to the Quad Mesh that merges the two Render Textures.

First, let's write a custom shader to use in the command buffer, that is, a custom shader that stores the Alpha value as a Grayscale.

Next, write a script to control the Command Buffer.
Most of the code in the previous example is the same, except that it specifies an arbitrary "Render Texture" as "Render Target".

Finally, create a Shader for the material to be applied to the Quad Mesh.
Combining "Color Render Texture" and "Mask Render Texture" serves to make only the background transparent.

(1) Create "Mask Render Texture" and "Material" to be used in the command buffer. Apply a custom shader to the material that stores the Alpha value as a color.
(2) Create a GameObject to control the command buffer.
(3) Add Script controlling the Command Buffer and assign "apPortrait character" and "First camera", and also assign "Material" and "Mask" Render texture" were created in Step (1).

Since the transparency of the entire image is calculated from the Mask Render Texture, the background of the "Color Render Texture" should be "Opaque" as it is.
(1) Select the First Camera.
(2) Set a background color (3) as it is opaque.
In this case, you can get a cleaner result by setting the color a little darker than the edges of the image.

(1) Then, create a Material to be applied to the Quad Mesh and apply a custom shader that combines the two render textures.
(2) Assign "Color Render Texture" to First Property (_MainTex).
(3) Assign "Mask Render Texture" to Second Property (_MaskTex).

(1) Select Quad Mesh.
(2) Apply the "Material that combines two render textures" created earlier to this mesh.

This is the finished result.
Get great results with only the characters rendered clearly on a transparent background.

You can see that "Color Render Texture" and "Mask Render Texture" are merged to solve the rendering problem.
The command buffer is effectively used to create additional render textures, etc. to solve the rendering problem as above.