Today's Tech Blog is courtesy of Hannes “Ariak” Schmieding, and it covers an interesting alternative to using "depth = -y" for depth sorting in 2.5D games. This tech blog comes with an example .yyz, available here (provided by Hannes Schmieding under the MIT License), and we highly recommend getting it and playing around with it before reading any further. Try moving around with the 3D camera to get an idea for how the technique we'll be explaining works. Note that setting up a 3D camera is not part of this tutorial, although it is included in the example project for illustration purposes.
This last year I have been slowly but steadily chipping away creating systems for my orthographic 2.5D RPG:
Since releasing the first screenshots, the question I received the most is how I handled the z axis and depth sorting. The introduction of the layer system with GMS2 promoted the use of alternatives to the omnipresent depth = -y and its derivatives. Discussions about depth sorting were a major topic, and I contributed by comparing different solutions, including one that I myself created (for those interested the post can be found here). In this tech blog I would like to walk you through that solution, explaining how it works for depth sorting...
Let's begin with a list of goals that the solution had to achieve:
- Take advantage of the newly-introduced layer system (simultaneously giving more performance in-game and being easier to organise in the room editor)
- Minimise taxing CPU-based depth sorting, such as depth order lists or depth=-y
- Deal with multiple layers of height and narrow canyons to uphold 2.5D perspective illusion
- Effortlessly handle a vast number of sprites and objects
- Facilitate silhouette system
- (Nice to have) Does not require special camera/view adaptations
Perhaps the easiest way to illustrate the final solution I came up with is depicting what I like to call the "narrow canyon" problem. Using a 2.5D perspective, when a character walks into a narrow canyon, we would like its feet to disappear behind the occluding wall and its head to still be visible over the wall behind him. However, with techniques like depth = -y the sprite is drawn flat (all pixels on the same depth / z). Consequently, modifying the characters depth to be above or below the tile layers will not do the trick. You could get around this by using two layers at different depths for the tiles, however, this quickly becomes a problem when you deal with multiple levels of height – as my game does.
WHAT IS Z-TILTING?
As can be seen in the above .gif, the idea behind z-tilting is to tilt the sprites in 3D space instead of having them draw flat at a single depth. This technique enables us to make use of the GPU’s occlusion culling - testing which parts of a sprite are currently occluded by previously-drawn sprites and which parts aren’t - thus getting rid of cpu-based depth sorting. The technique is derived from billboarding, which was developed for early 3D games such as Mario Kart 64, where it was often used to make far away objects such as trees appear three dimensional with minimal performance impact. From a distance, the illusion was maintained well enough, even when the camera rotated. Even though rotating the camera in our 2.5D orthographic perspective quickly dispels the illusion, we can still make use of this system to solve the narrow canyon problem.
A BUMPY PATH
In the early stages, to achieve this effect I tinkered around with changing the perspective. This was a huge hassle and the results where moderate at best.
My next attempt was to use vertex buffers. They allow access to the z axis, which meant that tilting the sprites was just editing the z coordinate of the upper half of the sprite. However, a major problem with vertex-buffers is that they represent purely static geometry. The animation was out of the question without extensive buffer modification or shader trickery, all of which defeated the purpose of easily tying in with GameMaker's default renderer.
The final solution works by making use of a single shader, which allows for effortless animation using the draw functions we’re all used to. Here is how it works from a technical perspective:
A HIGH LEVEL RUNDOWN
Using layer scripts, the z buffer and shader corner-ids are enabled and the shader is set after the background layer has cleared the application surface. The shader tilts sprites by modifying the z coordinate of the upper vertices. These can be identified by the corner ids. As the alpha channel is problematic in 3D, it is piggybacked to encode the sprite height instead, providing information on how many pixels to raise the z-coordinate. This trick allows the shader to function based on the default GM vertex format. Drawing is locked to opaque (non-opaque pixels are discarded).
PREPARATION AND GENERAL NOTES
Before getting stuck into the meat of the procedure, first we need to do some setting up and be aware of a few potential issues:
- Ensure that texture pages do not have automatic cropping enabled. Go to Tools > Texture Groups and uncheck "automatically crop".
- Sprites that we wish to tilt should have their origin on the bottom.
- All sprites that share a depth should ideally be on the same layer.
- The depth between layers is important. In my example, tile layers are one tile width apart.
- Depth values of layers are always integers and layers cannot share the same depth. This is not a limitation of the technique, but simply how GM handles layer depth.
In the following sections I will outline the functions and features that are required and how they come together.
THE Z BUFFER
If you’ve ever used 3D in GameMaker and are already familiar with the z buffer, then you can skip this section.
The z buffer allows for z-testing, which is how the GPU tests which pixels are visible and which are already occluded by something that was previously drawn. To enable z-testing we call the function gpu_set_ztestenable(true). In order to record what we draw on the z buffer we will also need to call gpu_set_zwriteenable(true). If z-writing is disabled we can draw sprites and have them test for occlusion, yet not flag the positions as occupied, which is useful for e.g. lighting and shadows. Once these functions are called they become effective immediately and affect all further drawing until disabled.
It is worth noting that the z buffer ignores alpha values. What this means is that even if a tree is drawn at 50% alpha whilst z-writing is on, when we later draw anything behind the tree, the occluded parts will not be visible at all. The same goes for sprites that have pixels with zero alpha. This is almost any sprite! It is especially noticeable on the trees in the .gif below. Alpha testing solves this issue. Since we are using a custom shader, the functions gpu_set_alphatestenable and gpu_set_alphatestrefvalue will not work, as they affect GameMaker's built-in default shader only, so we will need to modify the fragment shader and use a discard operation. Here is a visual where alpha testing is toggled on and off:
Z-fighting is another common problem. It occurs when two sprites occupy exactly the same screen space. It may at first appear to only be an issue when using a 3D camera, but, as seen in the lower left corner of the below image, z-fighting can also lead to undesirable visual glitches in 2.5D perspective, such as the red line bleeding through the bush:
The shader I will use features an optional line that tweaks z-fighting by making use of the sprite height and offsetting ever so slightly on the y-axis.
PIGGYBACKING THE ALPHA CHANNEL
As we’ve just discussed, the alpha channel can be tricky. For the most part we want all our drawing to be opaque as long as z-writing is on. This works very well for pixel art games, and by locking all drawing to opaque we don't need the alpha channel for transparency and can use it instead to encode another vital piece of information - the height of the given sprite we are drawing. This information is required to determine by how much to lift the upper vertices of the sprite on the z axis, thus ensures a consistent 45° tilt for all sprites. If the angle of tilted sprites varied we would see visual glitches. As a shader converts the colours from [0-255] integers to [0-1] floats the shader needs to multiply by 255 to reconvert to the height value in pixels. This has a side-effect of limiting how large a sprite can be - 255 pixels - before its tilt will no longer be accurate and we start seeing inconsistencies. I don't consider this to be much of a problem, but if you do use higher-fidelity graphics a solution could be to have alpha encoded in multiple pixels of the sprite height instead of one.
With the shader, an alpha blend of zero will not be tilted and still draws fully opaque.
SHADER BASICS: VERTEX FORMAT
In order to understand why the alpha trick enables us to use the draw functions, we are all used to, a quick glance at the different vertex formats is required. The gist of it is that we can use the existing default format which GM uses to encode all the information we need. In GameMaker all sprite (including tiles, background, objects, etc.) related drawing is based on 2 triangles (called a quad), each consisting of 3 points or vertices. The vertex format defines what information each vertex has, and the default vertex format contains the following information:
vec3 in_Position (x,y,z) [roomspace] // x,y,depth of the corners of each vertex.vec4 in_Colour (r,g,b,a) [0-1] // image_blend and image_alpha, converted from [0-255]vec2 in_TextureCoord (u,v) [0,1] // the position of the vertex on the texture page
These attributes are visible to the vertex shader, where our modification will take place. The in_Position defines where to draw, the in_TextureCoord defines what sprite to use for the texture, and in_Colour lets us apply additional colour blending. For our purposes, it is this last property that we can use to encode additional information by "stealing" the alpha channel to encode sprite height. Furthermore, a tiny bit of information is stolen from the red and blue channel, but more on that in the next section...
CORNER ID
When the function shader_enable_corner_id is active, the least significant bit of the red and blue channel are "stolen" from in_Colour to encode information that allows us to identify which of the 6 vertices that make up a sprite we are currently dealing with. This does not change the sprite's original colour values, it merely limits the in_Colour value used for blending (a far more detailed explanation can be found here on the GameMaker Community forum). For our purposes we can identify an upper vertex with the following code snippet in the vertex shader:
float top = 1.0 - mod( in_Colour.b * 255.0, 2.0 ); // upper vertex? [0:1]
We work with a float since this allows for easy multiplication - you’ll see this being used later on. Note! Currently, tile-blending is not supported. For now, the in_Colour component (image blend and alpha) of tiles are locked at pure white (RGBA: 1,1,1,1) and are not affected by the modifications to the colour channels via shader_enable_corner_id. A blue value of 1 resolves to a lower vertex with the formula provided above. As a result tiles will always be flat.
Now that we have gone over how the z buffer works, which quirks it involves, and how we can make use of the default vertex formats in_Colour attribute to encode additional information, we can move on towards applying what we have covered. To this point, layer script are a handy new tool.
LAYER SCRIPTS
The newly-introduced layer scripts with GMS2 provide us with a more accessible means of organising our render pipeline. We will use them to set and unset the shader and other GPU properties, thus defining the (depth) region of our rendering that uses the z-tilting shader.
Layer scripts only need to be set once and will then run every frame, provided the layer is visible. It’s important to keep in mind that layer scripts are executed multiple times per frame, namely once for every draw event (pre-draw, draw, post-draw, etc.). Of these events, only one is currently of interest for us: the default draw event. This is where all elements that reside on layers render. The event number for the standard draw event is zero. To avoid multiple executions the layer script will begin with if event_number != 0 exit;
The following code is run after the background layer has rendered using layer_script_end and enables the z-tilting shader.
if event_number!=0 exit;
gpu_set_ztestenable(true);
gpu_set_zwriteenable(true);
shader_set(shd_zsort);
shader_enable_corner_id(true);
This code reverts the settings and is run after all the stuff we want to depth sort is drawn.
if event_number!=0 exit;
shader_reset();
shader_enable_corner_id(false);
gpu_set_ztestenable(false);
gpu_set_zwriteenable(false);`
Finally, it is time to present the centerpiece of this tutorial: the z-tilting shader. I have commented the parts that deviate from a default pass-through shader.
Vertex Shader: Z-Tilting
attribute vec3 in_Position;
attribute vec4 in_Colour;
attribute vec2 in_TextureCoord;
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
void main()
{
vec4 col = in_Colour;
vec4 object_space_pos = vec4( in_Position, 1.0);
float top = 1.0 - mod( col.b * 255.0, 2.0); // identify upper vertex
object_space_pos.z -= 255.0 * col.a * top; //tilt using alpha
object_space_pos.y += col.a / 10.0; //tweak zfighting
gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;
v_vColour = vec4(col.rgb, 1.0); // lock alpha blend to fully opaque
v_vTexcoord = in_TextureCoord;
}
Fragment Shader: Discard Non-Opaque
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
void main()
{
vec4 sprite_col = texture2D( gm_BaseTexture, v_vTexcoord );
if (sprite_col.a < 1.0) { discard; } // discard non opaque sprite pixels
gl_FragColor = sprite_col * v_vColour;
}
FINAL REMARKS
This is not the perfect solution... far from it in fact! A consequence of piggybacking the alpha channel is that for the most part lightning and other alpha-based effects are more difficult to pull off. Furthermore, effects like sword slashes should not be tilted. If they are, then striking south will often make the tip of the sword disappear into the ground, or if swinging north, poke through tiles such as bridges on higher layers. This causes major headaches for artists!
On the plus side, it forgoes CPU-based depth sorting and upholds the 2.5D illusion nicely, whilst being able to utilise the drawing functions we're all used to and effortlessly handle a vast number of sprites. If you want your 2.5D game to feature a plethora of grass blades (ideally using vertex buffers) and have each of them correctly depth-sort vs all the other sprites in your game, z-tilting handles this smoothly.
Additionally, silhouettes tie in nicely to this system by flipping the z-buffer logic from checking for non-occluded areas to drawing only where they're occluded. Since silhouettes are such a sought-after feature, I've gone ahead and included a little bonus in the attached demo file.
ABOUT THE AUTHOR
Two years ago Hannes “Ariak” Schmieding picked up GameMaker on a whim. Since then he has released absolutely no games at all, as he thinks tinkering with tech and systems is much more fun. He keeps telling himself he will make a game, but his projects are eternally stuck in the prototyping phase. Which is why we find ourselves here with this tech blog! You can follow his descent into madness on Twitter
