Dithering > clipping

Recently I made the camera zoom in while the player performs certain actions. This looked good in most cases, except when there are tall objects between the player and the camera. It’s looks very strange and immersion breaking when the game camera clips through objects.

So I wrote a dithering transparency function in my standard shader that adjusts the dithering amount based on the distance from the camera. It uses bayer matrix pattern and pixel discarding, so it’s not true transparency (which would be expensive). It looks so much better than unnatural clipping!

I think this is a standard technique for 3rd person games. When camera gets too close to objects, or when there’s an object between the camera and the player, objects become dithered or transparent. I noticed it while playing Sekiro recently, Super Mario Odyssey does a similar thing, and Uncharted 4 does it too. Thank you to Ming-Lun Chou for sharing this technique - it didn’t take me long to figure this out because I remembered reading about it on his postmortem.

Vertex Shader fun

I really enjoy writing shaders that breathe life into the game world, especially vertex shaders. A vertex shader’s main function is to transform a vertex’s 3d position into a 2d position on the screen for rendering and coloring, but you can manipulate that position in interesting ways to do fun things.

For cloth, hair, vegetation, or anything that needs to be affected by wind, vertex shader is the perfect tool. And the code is super small - for my waving flag, the vertex shader is just 3 lines.

//x is length, y is depth, z is Y-height
//horizontal wave - more wavy towards the end
v.vertex.y += _WavingAmount * pow(v.vertex.x, 2) * pow(v.vertex.z, 3) * cnoise(float2(_Time.z * 0.7, v.vertex.z * 2)) * 1 * sin(v.vertex.x * 4 + _Time.z * 2);
//height wave
v.vertex.z += pow(v.vertex.x * 0.4, 3) * 0.6 * cnoise(float2(_Time.z * 0.7, v.vertex.z * 2)) + (pow(v.vertex.x,2) * -0.15);
//more horizontal wave
v.vertex.y += pow((1 - abs(v.vertex.x + 1)), 0.5) * cnoise(float2(_Time.z * 0.5, v.vertex.z)) * _WavingAmount;

Essentially it’s a bunch of sine and noise functions stacked on top of each other to create a natural waving pattern. And a cool thing is that all this can be multiplied by a global float value. Meaning, one number can control the waving amount of ALL objects that are programmed to wave in the wind. Also all these things run on the GPU, freeing precious CPU resource for important things like AI and physics.

The main reason why I find shader development so fun is that everything can be parameterized and previewed instantly. It’s super easy to expose the waving amount and the sine wave octave values to the material, and turn them into a slider. Then you can go out into the scene and play with those values until you get one that looks just right.

Vertex shaders aren’t good for every kind of animation though.. For my main character’s pony tail, I use the built in cloth component in Unity (which runs on CPU), because it needs collision support. It’s possible to write a vertex shader that reacts to colliders, but it requires some set up and steps that I haven’t investigated yet. On the other hand, the ghost mode long hair runs on the GPU, because it doesn’t collide with anything.

One interesting issue I was running into with vertex shaders is that it doesn’t change depth and normals textures that are used for some post processing effects (ex: ambient occlusion), even when you add ‘addshadow’ tag in surface shaders… Turns out, there’s a built-in hidden shader in Unity that takes care of depth normals. I had to modify that shader and add custom render types with the same vertex displacement functions to make sure that the vertices used for depth normals were waving in the same way as the vertices in the normal shader.

Anyway, graphics programming is fun and super useful. Everyone should learn it!

Dialogue System

Got the dialogue system working in the game! I’ve been meaning to implement this for a while, but I kept getting sidetracked by other features… also it felt more daunting to make than other things for me. This is the first time I’m working with a node-based dialogue system with choices and multiple characters.

I actually built a prototype of a node-based dialogue tree system a few months back, just for fun, but it was very basic and it would’ve taken some time to polish it up. I’m kinda new to the whole custom editor thing, though I’ve been writing more and more editor code to speed things up…

For this game I thought it would be better to purchase an asset that’s widely used by other Unity users. I really should rely on the asset store more to save time… Anyway, the one I picked is the Dialogue System for Unity by Pixel Crushers.

This asset is great. Lots of features, compatibility with other assets, etc. But the UI templates are kind of finicky to use and not what I need for this project. I’m not knocking the asset at all, it just didn’t allow me to display text in ways I want it to because I have specific needs. So basically I’m only using the node dialogue editor UI from this asset. I wrote my own code for parsing and fetching text from its conversation system, as well as a different UI system that let’s me do what I want it to do.

One reason I hate using Unity assets for anything related to code is that I have to read other people’s code when I’m using it or modifying it. Working with poorly documented code written by others is THE WORST. I think every programmer will agree with me there. Luckily, Pixel Crushers has the best documentation for a Unity Asset Store product I’ve seen. It wasn’t too hard to figure out how to fetch and parse text from the dialogue database file that is generated by this plug-in.

Also, it’s very easy to export a .csv file to upload to Google Drive, so I can collaborate with others on localization. Importing a .csv file and turning it into a dialogue database is super easy too. When I was working on Maruta Escape, I put off localization code till the last minute and it was SUPER PAINFUL to figure out how to do all that AFTER all of the English text had been put into the game. Yeah yeah, I wasn’t being smart. This plug-in makes localization much more streamlined, so I’m happy that I went with this asset.

I still have some work to do to polish up the dialogue system in this game, but I think the hardest parts are done. Thanks Pixel Crushers!

Faking volumetric lights

Today, I decided I was going to add some volumetric lights into the game. I’ve used really clever solutions like Aura before, but I wanted something more lightweight that’ll run on lower end PCs and maybe nintendo switch… so I decided to fake it.

Start with an unlit cone -> Use the vertex position (z-axis) to set the alpha along the height of the cone -> Make the edges fade out nicely using the dot product of world normal and view direction.

It sounds so simple when I write it out like that, but it took me some time to figure out. Feels good to fake volumetric lights with just a cone and a relatively small shader. It has pretty much zero impact on performance.

Ragdoll bugs

Sometimes you try to fix bugs and then your bug fix creates even more serious bugs…

So there was this weird issue with corpses where sometimes they’d rotate around 90 degrees on the y-axis for no reason. When enemy soldiers die, I disable their animator and navigation agent, so there’s nothing that should be modifying their rotation.

First thing I thought of was that the ragdoll effect was causing problems, so I wrote a coroutine that puts constraints on the rigidbody joints and makes them kinematic (probably not necessarily to do both) 3 seconds after the body touches the ground.

And this created even worse problems. Some corpses (maybe 1 in 5) would twitch around like in the GIF above, and others (approx 1 in 10) would fold in on themselves like in the GIF below.

This was completely unacceptable. The original bug was much less distracting and preferable to these problems… And since these issues were happening seemingly randomly, I had no idea what to do. After spending hours playing with different values in the physics settings, rigidbody settings, and character joint component settings, I was ready to give up.

And then I remembered… I was setting the neck joint angle manually in the enemies to make them look forward (overriding the animation). I thought maybe that was the cause of the twitching at least. So I deleted the coroutine that locks the corpses, and made the neck joint function only run while the enemy is alive. And this fixed everything, even the original bug of enemy corpses turning randomly while dead.

Lesson learned - don’t modify transforms directly when ragdoll is active. I thought I wasn’t doing that, but I was. Glad it all got resolved though. It’s like a curse has been lifted. No more possessed corpses in this game. …Oh wait, isn’t this game about possessing NPCs? Maybe these corpses SHOULD be doing weird things. Hmm…

Water progress

I’ve been working on some realistic water for the game. Previous I created a simple stylized water with scrolling textures at different scales, and it looked fine when the game looked more flat (due to orthographic camera). But with the change to perspective camera and things appearing more 3D, the old water wasn’t cutting it. So I decided to make a new water shader.

I used Amplify shader editor for the first time for this water shader. I’m really loving it. It’s very intuitive to use, and there are very useful built-in nodes that would've taken more time for me to figure out if I were writing the water shader from scratch.

This water shader is massive and has a lot of features - depth transparency, refraction based on normals, foam around objects, trail behind moving things. And splashes using particle system emitter. I learned a ton making this water for the past few days.

Also I learned how to use command buffers, which is super cool as well. Before this, I relied on render textures and multiple cameras to get the job done. I'd create a new layer, make a child camera that only sees that layer, set a new render texture to that camera, and set that render texture as a global texture accessible from shaders. This worked well and I set those render texture to render at half res, so it wasn't bad for performance. But I was starting to run out of layers, because Unity only gives you 32 layers. I have a lot of layers for different physics and rendering masks I need (I probably need to look at all of my layers and remove some unnecessary ones).

Then I came across command buffers and it seemed like an easier way to do what I was doing before, with the added flexibility of being able to choose specific objects instead of relying on tags or layers. So now my water trail particle systems are put into an array, and I render everything in that array into the water trail command buffer, turn that into a global texture, and use that to draw trails on to the water plane. Command buffer opens a whole new world of possibilities for selective effects I can apply to objects without using more layer masks and cameras, so I'm super excited. I think it should be much more efficient than using multiple cameras too - I think it uses the main camera for rendering these buffers, so it should be rendering only once? Command buffer experts, please correct me if I'm wrong.

A lot of learning on graphics programming in the last few days. I'm going to use Amplify shader editor more and more, because it's so convenient and easier to visualize results. And I'm going to use command buffers more as well. I should actually use command buffers for everything and get rid of my children cameras. I have cameras for rendering characters only - for highlighting them when they're behind obstacles - and rendering enemy sightcones only - for rendering them on top of grass and water. For both of these things I probably could've used shader render order to render them on top, but I had good reasons for not doing that (both technical and stylistic). Now I can use command buffers for all those and get rid of those annoying children cameras. Woo hoo!

UPDATE: 2019_07_26

So the water was looking pretty good, but it was missing something big… I didn’t react to the characters that much. There’s the bubble trail as player and NPCs move around on water, but where are the water ripples?

This effect is achieved with a particle system that fires donut shapes when something collides with the water surface. These donut shapes are rendered on a separate buffer and then added to the normal map of the water.

I would like to try directional waves with vertex displacement and tessellation later, but for now this water is already looking more interactive than before. It’s fun to run around and create ripples everywhere.

Visuals and lighting

Originally I wanted to make the game look more like a painting, so I’ve been using an orthographic camera… but then I realized - wait a minute, I’m making all these 3D assets, and my main art inspiration is those miniature towns at Korean museums… so why am I trying to flatten the visuals?

So I switched the camera to perspective mode and added some edge blur (I need to add more) to make it look more miniature. I’m liking this change a lot. Keeping it! Also I like seeing different elements pop out more. I need to tone down the ground texture normals though. It’s a bit too high in the above gif.

Also I’ve been playing with the lighting more. I want most of the game to take place during night time, because that’s when ghosts come out to play, but also I don’t want the game to look dark. So I’m using a blue directional light and really soaking the scene in it. Like super bright moon light? I’m still fiddling with it, but it’s looking pretty good I think.

Funky effects

I’m taking some time away from crafting gameplay and assets to work on some funky visual effects. I don’t know if I’ll use any of this, but sometimes it’s good to shift focus and experiment. The first gif is using some refraction using particle system, jiggling UVs, and optical feedback. Using some shader code and render texture. Most of it is basically UV jiggling. Feed some noise to the UV and jiggle it around, you get water like refraction. It’s neat.

And this one, I might actually use this one, I like it a lot, it’s a compute shader particle system. I think there are about a million particles spawned in that gif above. The starting color of each particle is the color of the pixel at the location of that particle. And over time it becomes more faint. This makes it looks like the character herself is disintegrating, which is neat. Compute shader is so powerful for these kinds of things. Each particle is moving independently and using noise equations to move around behind the character. Compute shader code is a pain to set up, but it’s very cool to be able to spawn millions of particles without using much CPU resource at all. I can spawn about 5 million particles before the frame rate drops below 60fps, on my GTX 1070 GPU. This looks cool and all, but would it be distracting for gameplay? i’m not sure. I’d have to test it.

Friendly fire!

Possessing enemy soldier and shooting his rifle at another enemy soldier is now working! This is pretty fun. Maybe a bit too easy, but I’ll balance that later. Also, I added a shrine maiden that can see the player ghost and alert soldiers.

For both possession and adding another character, I was afraid that I’d have to write a lot of new code or rewrite old code, but I figured out ways to a lot of the current code, so I was able to do these quicker than anticipated. For controlling enemy characters, basically I turn off the inputs from the AI director, and then feed it player input. This allowed me to basically make no changes to how the enemy AI works right now while giving me all of the control over the enemy. Similar thing for the shrine maiden behaviour, I turn off some methods while running some new methods, but most of the AI is similar to the current soldier AI. Adding more characters should be a breeze since my code is more flexible now.

Possession BUG

So I’m working on ghost possession mechanic right now. Turn into a ghost, go into a soldier’s body, control him for a while and shoot other enemies while controlling the soldier. Sounds fun, right? Well, it’s my fault for not writing the enemy code from the beginning to take player input - I’ve been running into some programming challenges. Fun challenges though. I’m close to making everything work.

Also, in order to make the ghost body disappear I’m scaling the ghost body using scripts and…. well you can see in the GIF. I created a weird bug where player’s body gets struck and infinitely increases in size while upside down. Probably because the scaling doesn’t stop at zero and goes into negative values. What a fun looking bug though. Almost lovecraftian.

Knocking over lights

Fun little detail here. I added physics to the pit fire objects, so you can knock them over. Although, now that the pit fire is interactive, maybe I need to make everything else interactive too. Hmm. Maybe I’ll take out intractability to save some work…

Ghost Dash

Got the ghost dash ability working! It uses up the ghost meter, so you can’t use it too much, but it’s pretty useful. Zipping around as a ghost like I see on Korean horror movies is so fun. I need to make the ghost leave a lot of dust or fog as she dashes around.

Blood on enemy's uniform

Enemies now bleed on to their uniform as they die! It’s pretty hard to see, because of the top down camera, but I like this detail. A little more realistic. This was done with some shader code. Blood pattern is always there on the uniform material, just invisible. And there’s a number that controls the size of the blood. As the enemy dies, that number increases on the enemy death script. Right now, that blood only appears on the back of the uniform. I need to change this so that the blood appears exactly on where the enemy gets stabbed. I think I have to convert the direction of the player and enemy’s facing direction to figure out where he gets stabbed, and then apply that to the shader somehow. Something to figure out. Small detail but sometimes player will stab the enemy from behind, so I think I have to figure it out.

Suicide ghost

Here’s another fun little thing that probably won’t go into the game. Committing suicide! Actually this might be a cool mechanic for a game with resurrection. Kill yourself to sacrifice one of your lives, but become invisible for a time and sneak into places…

I scripted the slashing motion in c# script with IK on player’s arms. It doesn’t look as refined as other animations which are using traditional humanoid animations, but with some blends and lerps I could make the motions smoother.

Also, I separated the ghost from the player body for now. I’m not sure if I’m keeping this change though, since I like the look of player seamlessly transitioning from ghost mode to human mode. It would be pretty cool if every time you turns into a ghost, you had to kill yourself and leave a body… and when you turn back into a human you just get a new body. So your old body would still be back there… or maybe you just possess another body? That would be cool too. Like a ghost simulator game. Hop from one body to another. Is this a sneaking game anymore? Maybe this is a ghost simulator now.

Ghost mode

Alright here it is, a key feature of the game finally working. GHOST MODE. Player character can temporarily turn into a ghost and become invisible. Took some time to figure out the mechanics for that and visual effects for the ghost mode. Floating, long hair, seethrough clothing, all that stuff. I don’t think I’ll keep the resurrection though. I like games where you die a lot and you have to go back all the way to the beginning.

The hair uses a geometry shader with some physics simulation running on GPU. It’s very efficient for the number of hair strands it renders. I hope I won’t run into compatibility issues with consoles though. Shader languages and compatibility issues are scary… So far running nicely on PC and iOS, so maybe it’ll be fine on other platforms too. I don’t have plans to make this game for mobile, but I just wanted to test the performance. Solid 30fps on my old iPhone 6s, pretty good.

Also, interactive grass makes ghost mode cooler. Basically I make the invisible grass circle (with direction info embedded into RGB) larger and spinning while player is in ghost mode. This makes the grass under ghost’s feet wave around wildly. I like it.

Interactive grass

Characters can move grass now. This was fun to figure out. Basically I have an invisible circle with x,y,z direction embedded into RGB colors under every moving character, and I can use this to bend the grass in the right direction near characters. It’s like 6 lines of code added to Unity’s built-in grass shader. The grass shader already has code for moving grass blade vertices - for wind - so I just had to add the render texture for grass camera (which only sees those hidden circles with character direction) and parse the colors into direction and add that to the grass vertex displacement function. It sounds pretty complicated when written out, but it’s pretty simple actually.

I have to tweak some settings to make them look more convincing, but at least the functionality is there. Also, right now the terrain is rendering each grass blade separately, but I really should make a new grass texture with multiple grass blades to save some GPU power. Might not see each grass blade moving independently with many blades per texture, but grass blades aren’t the most important thing in this game.

How did Metal Gear Solid 3 do individually moving grass blades? I remember being impressed when I played that game for the first time. And all that stuff was running on PS2 hardware. How?! Man, my accomplishments in this game pale in comparison to those retro games and their black magic. MGS3 also had those flowers moving independently during the boss battle with The Boss. How did they do all that stuff? How does ThatGameCompany’s Flower render all those grass blades and flowers on PS3 hardware? Maybe there’s a better way to render and move these grass blades… Hmm.

Painting blood

I’m painting with blood! I wanted to try this for a while, finally figured out a way to do it. Using a fairly small terrain texture (1024 x 1024) combined with some shader magic (mostly noise and edge blur), I can paint blood on the ground without wasting too much CPU resources, and without it looking like a pixel drawing. It looks super blocky without the shader magic.

I’m not sure if I’ll keep the corpse dragging function in the game though, since I’m running into a lot of issues with the ragdoll going crazy while soldier’s body is being dragged. I could make the corpse’s pose static while it’s being dragged, I guess. Or just figure out how to use the ragdoll better. I’m not sure yet.

But also, I think I’m probably going to have a lot of enemies on each level with patrol routes covering every inch of the level (it sounds impossible to play but I have an idea I want to try), so dragging and hiding corpses might be useless. But even if blood puddle gets painted in unique patterns when each enemy dies, this will be cool. Painting with blood is fun.

Bomb

I’m still brainstorming types of abilities and weapons the player will have, but here’s a fun one. A bomb! Basically it’s a big sphere collider, and when it touches an enemy the the enemy turns into a ragdoll and flies away. And there’s a particle effect for fire and explosion. it looks pretty goofy and fun. Maybe I’ll just call this game ‘bomber girl’ and make it all about throwing bombs and enemies.

Multi enemies and corpse detection

Enemy AI is now functional! I made an AI director system that sets the patrol pattern for multiple enemy AIs. Plus, they react to corpses as well. It’s working pretty well now. When I make more assets, I can drop these guys on the level and actually start testing gameplay now. Exciting! I need to add more functions to the AI director, like letting the soldier call a nearby soldier for help. And maybe alert phases where every soldier goes into alert for a while after one of them sees the player.

Homing bullet

I’ve been working on enemy AI and shooting. I’m trying different ideas for the bullet and enemy rifle.

  • Actual bullet with physics, run sideways to dodge

  • Actual bullet with some homing, so you see the bullet trail but it’s super hard to dodge it

  • Just a line from rifle to the player and instant death

I think the 3rd choice is probably what I’m going to go with, because it fits the rest of the gameplay ideas I have, but I’m trying the other ideas too.

Homing bullets are pretty funny. I tried slowing down the bullet flying speed, and it looks like a red snake flying in the air and biting the player. Maybe I should keep this and make this game just ridiculous.

I also added a ‘hearing range’ collider to the enemy. If you sprint close to the enemy, he’ll turn towards the player and shoot. I’m experimenting with the right turning speed, so player can stop running just before the enemy sees the player and have fun outsmarting the enemy AI.

Another homing bullet with some terrain bug thrown in there too. Homing bullets are so fun. Not for this game but maybe for another future game, I’ll try these weird slow homing bullets. Could be neat.