More developments soon to come with project Morphscape, but for now I just wanted to add the recut version of the preview cinematic with music, written by drummer and composer, Eric Reeves! You can find more of his work at www.ericreevesjazz.com.
This past week, I assembled a slideshow for my final senior thesis defense presentation at Rochester Institute of Technology. If you haven't been following my blog, it is a pretty thorough overview and documentation of the work I have done and the workflows I've developed over the course of the past two semesters working on this independent project.
Although my thesis project is "officially" complete and I've demonstrated what I set out to prove initially at the beginning of the year, I will still be refining the project as well as continuing my research and development on the theme, processes, and visual style I have been pursuing-- so stay tuned!
After putting together a visual showcase for last week, I've decided to switch gears for a bit to focus on the interactive component of this project, which is my intended end product. All of the dream events featured in the cinematic are distance based, and so they happen automatically as the player moves forward. This is works well for a movie, but for a game, it is a bit linear and passive. I've been revisiting the work I did several months back to create a volumetric painting system using Unreal Engine 4's Canvas Render Target 2D. Additionally, I've started to design a very simple character and its movement controls.
CHARACTER + 3D CURSOR
My initial idea was that the character would be a flying character set up in third-person. The camera would aim to look at a 3D cursor that is at the projected 3D location of your current mouse position. To control the character, you would use W A S and D. When the character would move forward, it would move forward in the direction of the camera's aim, towards the 3D cursor.
After testing this out, it felt pretty hard to control. I've always found flying controls in games to be the trickiest to get the hang of -- and they are definitely difficult to make intuitive as well. Additionally, I couldn't work out a camera arrangement that prevented the character from blocking a good chunk of the environment.
I decided instead to make the character act as a 3D cursor, which feels much more effective already. This also is much clearer and easier to control the painting mechanism, where the 3D cursor/character acts as the paintbrush. Now, the orb character flies to the projected location of your mouse in the world, while the player camera remains stable. The player camera is still controllable with WASD and will rotate to look at the cursor, but it is grounded.
The 3D cursor is set up mostly in the Player Controller class. To enable the mouse events, I've checked the Mouse Interface features in the Player Controller settings.
The base feature of the 3D cursor is that the mouse's 2D location needs to be projected into 3D space. I've used a node called Convert Mouse Location to World Space. This node alone doesn't put the location in the correct spot though. You can multiply the Direction vector from this node by a user-specified distance, and then add that to the Mouse Location. This essentially puts your cursor at the specified distance from the initial mouse location by moving it out along the direction vector. (See below.)
For testing purposes, I had attached a sphere to the cursor location to be able to easily visualize where it was in 3D space. As an additional control, I mapped the Mouse Wheel input to be able to increment and decrement the Cursor Distance variable, within a certain range so it is capped at a min and max distance.
Upon testing this, especially with the painting mechanism I am implementing (since the 3D cursor serves as the brush for painting), I realized it would help a lot to be able to have the cursor "stick" to nearby surfaces, essentially detecting anything in its path within a certain radius. Without this function, you would really have to guess where invisible, paintable objects are by just clicking around aimlessly in world space.
On every tick, I've made this the first thing that the Player Controller class checks. This makes it so, if there is an object hit under the mouse, it takes precedence over the default free-floating cursor. The node Get Hit Result Under Cursor by Channel checks for and returns any object hits under the mouse cursor.
The most basic way to implement this is to then simply set the 3D cursor location to that exact resulting hit location. However, getting a hit under the mouse doesn't account for how far that hit away is from the 3D cursor location in world space. This makes it difficult to break your 3D cursor away from objects that your cursor may unintentionally be on top that are far away, due to perspective. My solution is to get the world location of the character and find its distance to the hit location. If the distance is greater than the user-specified threshold, then the cursor location defaults to the free-floating 3D cursor.
Additional things I'd like to implement here are some more visual indicators - perhaps when your cursor sticks to an object that is invisible, I could add a decal to the surface of that object.
There isn't much more that goes on in the Character class, but it does need to be set to the cursor location. You can grab the Cursor Location variable from the Player Controller class by getting the Player Controller and casting it to your specific Player Controller class.
I've used a VInterp To node to interpolate from the current location of the character to the current cursor location. This makes for much smoother movement, and adds the effect of the cursor following/lagging behind where you are mousing to. My character also has ribbon particles trailing behind it, so this allows the particles to trail for a bit longer.
Here is a very rough first pass of my idea for a look for the character that I have been testing with. I imagined it as a light orb because I feel like this is reminiscent of neurons, synapses, and electrical brain activity. I also wanted its look to allude to the dominant painting theme that defines both the visual style and interactive function in this dreamscape world.
The Character Blueprint contains a Particle System that moves along with the location of the character meshes. This Particle System uses the Ribbon Type Data to emit as a ribbon trail. It is emitted from the sphere character mesh by using the Bone/Socket Location module in Cascade to hook it up to the joint of the character skeletal mesh.
This example of the character moving on the left shows a (slightly choppy due to the screenshot capturing process) example of how the particles follow behind.
Ribbon particles apply materials such that the leftmost part of the texture is towards the start of the ribbon trail and the rightmost side of the texture is towards the end of the trail. So, I've used a painted brush stroke texture as the opacity map, but I've blended it with a linear gradient that becomes more black towards the right side of the opacity mask. This makes it so the particle trail dwindles towards the end of the trail. (See right.)
VOLUMETRIC PAINT SYSTEM
My project idea started with the idea of painting in world space to reveal different worlds/objects. This is a concept I want to carry through to my final product. Early on in my progress, I had done some testing with Unreal Engine 4's Canvas Render Target 2D Blueprints. Due to a couple of issues I encountered with them, this testing fell by the wayside for a while -- but after resolving these
This is my progress so far in implementing this feature. As of now, it is working pretty well to my liking, though there is a lot of potential to explore here, as this is just a test with a simple sphere.
Here are some features I've added this week:
- Left mouse button applies paint.
- When applied, paint stays permanent.
- Objects can have paint only partially applied/be revealed in parts.
- Paint can be cleared completely (mostly for testing purposes at the moment.)
- Brush size is adjustable.
In the near future, I'd also like to add the ability to erase paint in parts. Decals on the surface of the object where you are about to paint would also serve as a nice visual indicator.
In a few releases of Unreal, the Canvas Render Target 2D was causing crashes, but this has thankfully now been resolved, which allowed me to continue working with them.
I also realized that the Create Canvas Render Target 2D node cannot be created within a Construction Script. Doing so resulted in an error that prevented me from saving the map, due to graphs being linked to objects in an external map. Instead, create it on Event Begin Play and the problem is solved.
This week I've made a cinematic to summarize and showcase my work on my senior thesis project so far.
At the beginning of this project, I set out to create a real-time dreamscape environment as a means of achieving two major goals in my work. The first, to build an environment and assets equipped with the parameters to enable visual transformations before your eyes; and second, to develop a custom painted aesthetic.
This aesthetic I have developed relates the spirit of impressionist painting conceptually to the transience of form in dreams -- how objects are ever shifting, becoming other things entirely as your brain assigns them new identities. With loose, expressive paint strokes breaking up the silhouettes of models, my goal has been to suggest the vague impressions of our experiences that appear in our dreams. What exists is just a malleable impression at the whim of your subconscious mind. Impressionist painting can be considered a study in the way we perceive. I find that this relates to the way our brain "sees" and records visual information to memory. We recall that condensed perception in our dreams and memories.
From a technical standpoint, working in 3D graphics as opposed to painting requires much more overhead to overcome the default look of the tools and be able to demonstrate a personal style. This is an area that has been very important to me in my work, to avoid letting the medium I use dictate my vision for a project.
One of the main issues I've addressed has been the hard-edged borders of models and the seams between intersecting models. This is an area where 3D can really reveal itself as 3D, as opposed to a painting, where strokes are able to make form fluid and seamless. The hard-edged 3D model contradicts the idea of malleable form, because it exists as a very concrete, defined form. To me, this breaks the dream-like illusion, as our dreams aren't always so clear.
While rendering progresses in terms of achieving greater and greater photo realism, making a material interact with light as a painter might depict light still stands as a challenge. In my materials, I wanted to regain some of the expressive, happy accidents that can give a painting so much life and character. Using a technique based in meshes and materials as opposed to a post process gives this look a more dimensional quality that I wanted for this project, rather than looking like an additional layer on top of everything.
Another goal of mine over the course of this year has been working with the idea of "smart assets." The development of systems like Unreal Engine 4's Blueprints have made the gap between art and programming smaller than ever. Every asset I have created for this project is thoroughly parameterized, with parameters communicating between materials and Blueprints. Building assets to be "smart" allows for more functional assets but also more flexible assets in a production pipeline to accommodate rapid visual iterations on the same asset. My semi-procedural spline tree system built in Blueprints is a testament to the flexibility that the overhead of setting up a smart asset yields.
Looking ahead, I'd like to realize my vision of making this environment explorable and interactive. The trees will sprout up from nothing, and the rock formations that line the shore will melt away into the ocean. My vision is that this will be triggered by user input rather than simply distance from an object. A character and music will help to tie everything together into a cohesive, experiential dream playground.
Over my spring break, I went on a week long trip to the Southwest with the College of Imaging Arts and Sciences honors group at Rochester Institute of Technology. Seeing so many unique rock formations was enlightening and a great source of appropriately timed inspiration and reference. This past week I've been dipping my toes back into my project with some newfound perspective.
Here are several things I've worked on this week, or will be working on in the near future:
- Populating the environment. Although minimalism can sometimes feel dream-like, the map I've been working with has primarily been a test scene. I'd like to give this shoreline area a sense of space and add more rocks to accompany the main rock arch I've been testing with.
- Pushing the painted visual style to be even more loose and expressive. I've started to do that this week just a bit with the rock, but I'll be continuing to work on this for the ground, the ocean waves, and the boat.
Recently I rediscovered my love for JMW Turner's paintings. His ability to effectively capture and communicate a scene or a moment with such expressive brush strokes and colors is something that's really inspiring to me, and my vision for this piece. I admire the way solid objects seem to dematerialize into atmosphere -- that fluidity of matter is something I'd like to achieve in my work as well. With all of his seaside themed paintings, it's a wonder I haven't been looking to these paintings all along! But it's never too late for some fresh inspiration.
- Melting rocks. My vision for the rocks on the beach was that they represent hardened, fossilized memories. Your memories from your conscious brain are what dictate the things your subconscious dreams of. As the player, you can melt them to access these memories and fuel your dreams, creating more of the environment around you. I hope to have the rocks melt and form the ocean that the boat materializes onto. I've been thinking of this as a mechanism that will tie the interactive environment together into a cohesive, self-contained dream playground.
Here's a test I did with flow maps for the wax melting. This setup involves a normal and height map's UVs panning by a 2D flow map I've painted. It is revealed by a linear gradient mask being pushed down. The mask is blended with the flow mapped height map and a noise to break up the linear quality.
What works pretty simply on a cylinder is a little bit more difficult on a complex model. So I've been spending some time trying to work out a method to get the wax flowing along the surface angle of the rock arch model seamlessly. I've painted a world space flow map using Mari's Vector Brush tool, and I'm using that as a flow map to modify the projected UVs of my flow texture.
- Materialization effects. I want to create the feeling that the environment is forming organically as you continue to walk, or dream. Here is an example of the material effect I've been working on where the rock arch is materializing as you walk forward.
- A sky. Here's something I've neglected so far! A sky can be quite expressive and contribute a lot to the mood of an environment. I've started to set one up, though this still has a ways to go. My idea is to use flow maps to create clouds that move in a streaky, swirling pattern. As you near the boat, I'd like this sky to darken and become even more tumultuous.
- Refinements. This week I tweaked the boat's buoyancy and added a rock to it so it isn't a straight up and down bob. This is done by simply modifying the rotation of the meshes within the boat Blueprint on a sine wave over time.
- Procedural spline tree system, animated to enable growing trees. I hadn't worked with it in a while, but last semester I spent quite a bit of my time developing a procedural spline tree system, which is based around Unreal Engine 4's spline and spline mesh actors and uses a recursive method in a Blueprint for generating branches. The idea that motivated me to develop this was to enable trees to grow around you dynamically.
This week I've started to figure out my method for animating the components of the spline tree. My idea is to have an auxiliary spline that serves as a motion path curve that the spline points of each component can follow. Their movement, or distance along this motion path spline would be determined by a Timeline in the level blueprint, so that its movement can ease and not be completely linear.
This works well for the trunk so far. The issue with the primary branches and all of the following tiers will be maintaining the relative position of the child branch on its parent branch as they both animate. Additionally, a custom set motion path for every single branch would be a ridiculous amount of setup and user control, so I hope to automate the motion paths for the recursive, procedurally generated branches.
Here are some lower priority, but still important tasks I've been considering:
A character. I've been imagining this environment being explored in third-person, perhaps by a flying light orb character with a ribbon-like light trail.
Music. An ambient soundscape would set a nice, quiet mood for this environment.
Here's my latest iteration of the boat deterioration sequence I've been working on refining, with most elements added -- including the sand rippling over the rock, the shoreline being swallowed up by the sand, and the painted style implemented into the assets. Although it may not look much different than the video I recorded last week, I've made a lot of tweaks here.
Sand rippling over rocks - material blending based on height
Here's a close up of the material blending of the sand rippling over top of the rocks based on their respective height maps. This was something I focused in on this week. It was a bit of a challenge because I wanted to add displacement from the sand ripples, but only add it in areas where there weren't large rocks - to make it look as if the sand were moving over top of the rocks rather than displacing them with it. This gives the sense that the ground is still solid underneath the sand.
Procedural boat rusting
This past week I've improved my technique for the boat's shader-based rusting and deterioration sequence. My initial approach was to hand paint mask keyframes that designated where the rust formed, and blend between them. This worked out fine, and accomplished what I needed it to for the most part. If you notice in my previous work in progress videos of the boat rusting, most things are working, but when the boat starts to deteriorate away, the transition gets a bit choppy and it disappears in pretty large chunks. This is my own error, due to me jumping too much between these keyframes -- kind of like a choppy animation with not enough in-betweens. I was going to go in and repaint some of these keyframes again, when I realized this was about the fourth time I've repainted these keyframes and it can be a pretty painstaking process. I finally decided to pursue a procedural method that involved blending different masks in the material.
Each method has definitely presented its share of pros and cons, but ultimately this new method of blending together different masks in the shader has proven more versatile. The keyframed mask animation offers a lot of customized artistic control in deciding where the boat will rust, but this procedural method offers enough variety in its parameters to provide for a similar amount of control to achieve the desired effect. This method takes a bit more trial and error and tooling around with parameters to get the look I want out of it, but its ability to be tweaked based on in-engine results wins out.
There are three components to the sequence that happen at different times and are controlled by slightly modified masks, which in the end are used as masks for the blending between three different materials. The default boat hull has a painted metal material applied. When the boat begins to rust, the paint chips away revealing the rusted metal underneath. I call this deep red rust material "rust flakes." The secondary effect of this rusting is the "rust drips" material layer, where you see streaks of rust leak down the boat from these areas where rusted metal is exposed. Finally, the boat deteriorates, the metal being eroded through in layers. This deterioration mask is fed into the opacity mask for the material. The deterioration mask deteriorates in areas where the structure of the boat is weakest - starting with the areas that rusted first. The structure is the strongest following the skeleton/framework of the hull - so holes appear in areas where the skeleton is not first. This is why the skeleton of the boat is one of the last parts of the hull that remains.
Essentially, the entire transition is controlled by a linear gradient mask from black to white, projected onto the boat. The boat starts out masked by black, and the gradient is moved down until the boat is white. So the transition variable that controls the progression of the sequence is controlling the downward offset of the projected linear gradient in world space. The reason that the main mask is a linear gradient sliding down is because I do want this deterioration to generally happen from top to bottom. I feel like downward is a believable pattern for rusting, and deteriorating from top to bottom makes for a nice reveal of the skeleton underneath. Lastly, it helps to make sure that when parts of the boat start deteriorating away, there aren't any "floating" and unsupported pieces left in the air.
The linear gradient mask is just the base, though. This mask needs to be broken up for a little bit more serendipity in the rusting pattern. To do this, I warp the projected UVs of the linear gradient by multiplying them by a general perlin noise pattern - just to get some broad wavy variation instead of a straight horizontal line moving downwards. Warping this enough can allow for rust to emerge in areas lower down on the boat but still follow an overall trend of top to bottom rusting. The rust drips use the same warped noise as the areas where the rust flakes appear, but the noise is offset downwards a bit more, to give the directional appearance of rust dripping.
Getting into even more specific, small scale variation to break up this warped linear gradient mask, I overlay another noise into the grayscale values of the gradient. I generate a curvature map from the normal of the boat hull using Substance Designer. This highlights just the peaks of the normal map - the areas with ribbing. I use this mask to give these areas of the boat rusting priority. In real life, the most exposed features and areas that water could collect would rust first, so this adds a bit to the believability of where the rust starts to form first.
On the deterioration mask, however, I use a mask I hand painted that designates the underlying skeleton structure of the boat. The skeleton of the boat would be the most durable areas of the hull, and probably deteriorate away last. So I overlay this mask to make sure that it rusts away last.
Finally, I blend in the most specific detail - the edge pattern of each mask. Rust flakes follow a "flakey" cell-like pattern, rust drips follow a directional, streaky noise pattern, and the deterioration mask is a blend of both just to get some ragged edge variation.
I relate the deterioration mask back to the "mesh shell" technique I use to achieve a painted look by layering the mesh with the same material applied on top of itself, save for being pushed outwards several times using world position offset. These layered meshes help to achieve a look of thickness to the thin-walled base mesh of the boat hull. The layers help also with adding randomness to the deteriorated edge- the noises used to break up the deterioration mask are offset slightly on each layer, so each layer of the boat is rusted away in a slightly different pattern, but generally follows the same broad pattern. The broken up edges per layer, paired with dithering in the opacity mask, also contribute to a slightly softer, painted edge look.
Rocks and painted style
In addition to re-engineering the boat's rusting transition, I've been working through all of my materials to standardize the overall painted look I want to be consistent across assets. This is characterized in the material mostly by the stepped normals and dithered pattern that breaks up smooth gradients. Here are some screenshots showcasing the rocks with this look applied. Sphere reflection captures contribute a lot to the rim lit look that relates to the dramatic lighting of many paintings I enjoy.
I've been working with the sand material, which includes a blending of sand, rocks, small pebbles that blend in around the larger rocks, and water. With all of these components coming together in one material, I've had to do quite a bit of tidying up and figuring out the best organization for blending these all together. The rocks and water can be blended in by painting different Vertex Colors on the mesh. I break up the Vertex Color mask by blending in the height maps of each material. This ground material is also set up with displacement. I'm enjoying the way that these different components can be combined (for example, I can have the sand with rocks blending in and water blending over top.)
I've also been hooking up all the parameters to change throughout the boat deterioration sequence. This includes the boat's animated mask revealing rust on all of its parts, the buoyancy height displacing the boat changing, the wave height and speed of the water changing, and the shoreline mask clamping in, overtaking the water. All of these parameters are based on your distance to the boat, which is mapped 0-1. Each parameter has a float curve on a Timeline in the Level Blueprint, which maps its overall progression from 0-1 throughout the sequence. (For example, the wave height may grow as you come closer to the boat, but then decrease again as the shoreline swallows the ocean.) Each parameter has the true, user-defined min and max value, and the 0-1 from the timeline is remapped to that range. Then, the corresponding Material Parameter is set.
I'll be updating this soon with a newer video that shows the water waves growing in height and the sand surging towards the shoreline, but here is the boat sequence as of now!
A little bit on some developments I've made in terms of the concept that has been motivating my design. My idea is that, metaphorically speaking, your everyday experiences and memories are hardened and fossilized into rocks by your conscious mind. While you're dreaming, your subconscious brain accesses all of your memories. You can melt the rocks, which reveals these memories to your subconscious - you bring light to them and they become fluid again. I'd like the environment to be fairly empty at first, and as you melt rocks, accessing your memories, the environment will materialize - your subconscious brain is forming new dream memories from your collective experiences. Although your subconscious mind often accesses your permanent, fossilized memories while you dream, dreams themselves are very impermanent and hard to remember after waking. This is why many of my visuals deal with the ephemeral, like the boat deteriorating away and the constant flux of the environment. Sand also relates to this idea, as it is composed of a seemingly infinite amount of eroded rocks. It is the dust of time, constantly blowing around and never settling permanently.
I've been working on putting together a test map to see some of my pieces coming together. It helps to get a sense of the space, mood, and color I've been imagining.
I'm starting to get a believable look to the water, though I haven't yet worked out how to incorporate the painted look. I feel like there is a lot of opportunity for some nice foam paint particles off of the peaks of waves.
To control the movement of the wave, I implemented the Gertsner wave into my material. Here is some interesting information on the Gertsner wave. This tutorial did an even more wonderful job breaking down the math formula for the Gertsner wave and its translation into the Unreal material editor. I was a bit unsatisfied with the resulting mathematically calculated normal map, so I added my own custom normal, panning across the waves as they are displaced.
For this sequence, these waves will start building as you come closer to the boat and the shoreline. The sand will ripple forward as it swallows up the increasingly chaotic sea. Meanwhile, rust is growing all over the surface of the boat as it deteriorates away to its skeleton left in the sand. I've also hooked up the boat bobbing up and down at a controllable height and speed. Although both viable solutions, this is done through controlling the World Position Offset of the boat materials, rather than by a translation of the mesh within a Blueprint.
Materializing objects using the visual particle paint style as a functional mechanism
I had a thought that the particle paint that makes up the visual style of objects could also serve as a functional component to materialize objects in front of you. Imagine as you walk forward there are loose, brush stroke particles floating in the air, gathered vaguely in the form of an object's silhouette. As you come closer, these particles could suck in to the final shape of the object and the full mesh could dither/fade into existence.
Since the mesh particles are emitted on the surface of the base mesh, they follow the surface normal of that mesh. So, when the particles have velocity, they can move along that surface normal, inward or outwards. Here's a quick test I did to suck the particles inward towards the shape of the rock arch.
I've been debating the amount to which I want the player to be able to control their surroundings. If I make this a bit more active in gameplay, rather than just having things happen as you move around, it could make for a really interesting interactive experience. I have been thinking along the lines of an environment shifting and transforming around you as you explore in a dream-like way, but it might be interesting to experiment with the idea of your control in a dream, especially when you become conscious you are dreaming.
I've successfully made a seamless blend between the rock arch and the sandy ground. To do this, I UV mapped the bottom three rows of faces on the rock arch to UV set 2. The UVs for these three rows stretch to fit the 0-1 space. I use a linear gradient applied to UV set 2 that masks the material blend between the sand and the rock material. I use the gradient also to dither out the opacity of the rock arch mesh at the very bottom where it meets the sand. This completely gets rid of any hard line between the two separate meshes. To stylize this transition, I could go further and break up the linear gradient mask with a rougher, brush stroke texture.
Rocks in the sand
I wanted to create tiling materials for larger rocks and pebbles to vertex paint on the sandy landscape to add a bit of texture to the endless sandscape. These rocks would be blended with the base sand using their height maps. Seeing some shores with rockier areas got me thinking that this landscape would look more visually interesting and tie the big rock arches into the ground with some rocks peeking out of the sand.
Using Substance Designer to create procedural alphas for sculpting in ZBrush
These are possibly some of the most successful normal/height maps I've created. After attempting to sculpt many a tiling texture tirelessly, I've finally figured out strategies for making the process a little less painstaking but still customized rather than automated.
When sculpting in Zbrush, using different alphas on your brushes is very helpful for achieving forms quickly. I've personally never enjoyed using default alphas. Painting custom alphas in Photoshop can prove to be a difficult task. Even though you may think you're painting smooth gradients of grayscale, it's a bit hard to control the values that signify different heights, so it can get a bit messy and it's not very versatile to modify. Grabbing height maps from photos can sometimes work, but my problem with this method is that, since photos have lighting information baked in, they can't ever really provide the true relief of an object. Additionally, photos almost come with too much busy information and too many artifacts, and they only really work for a photo real style. Grabbing alphas from something you've sculpted in Zbrush is an awesome tool, but I have had trouble working this way because I can get overly carried away in specific detail while an alpha is supposed to serve as a generalized, reusable brush. Having overly distinguishable features might nullify the reusability of that alpha.
Substance Designer, which I use frequently for creating tiling texture maps, is a great tool for the job of making alphas for Zbrush. Here are a few of the alphas I created to sculpt these rocks.
I particularly enjoy this method because it's a great blend of hand-crafted and procedural, which is something I strive for in my technique.
To create these, I take a Paraboloid shape and I warp it (using Mosaic Grayscale) a bit by a Perlin Noise Zoom (which allows me to control the size of a blurry, generalized noise that works well for larger warping deformations.) This makes the circle gradient a bit more uneven in shape.
Next, I use Tile Generator (grayscale) with a Paraboloid as the shape to tile. By using Max rather than Add Sub for the blending method, and setting the scale to 4+, these paraboloid shapes "collide" with each other and only take the max values where they collide, creating sort of a bubble shape and hard crease in between them. I also dial up Luminance Variation to get some value variation. I invert the resulting tile generated noise so that it is brighter in areas where there are the hard creases formed by the maxed out transition point between the paraboloids. Now, when I multiply this noise over top of the original Paraboloid shape I've warped, it will create some sharp, rock-like peaks. Auto leveling is also an important step to make sure your values are full 0-1 for the height map.
This chain can be repeated to create different rocks by varying the random seed and other parameters. By using the Transformation 2D and setting the tiling parameter to be Absolute and changing the dropdown from H and V tiling to No Tiling, you can place and scale the resulting rock shape anywhere on your texture canvas. This can be blended with other rocks with the Max (Lighten) method as well.
Continuing my tests with the particle paint method I've been developing has been really exciting! I love experimenting with my tools to enable new aesthetic possibilities.
In most of these tests, I've used the light vector from a directional light to create a gradient of color from warm to cool within the material for the particles. The goal is to emulate the warm/cool contrast of light and shadows that many paintings utilize so well. I wanted to do this in a more artistic way than simply coloring lights, which would provide a smooth falloff across the surface of the object.
When I'm painting, I like to break down form into planar facets to simplify how a light will hit that surface. This thought process I use for painting objects is simulated well by the particle paint method because these
brush strokes exist in 3D space on planes that follow the surface of the object. But essentially, these planes are generalizing the form of the object, just like many painters, myself included, do while painting. This stepped, rather than smooth, textural gradient has a lot of character to it. The overlapping of the particle paint planes really emulates the layering of real paint strokes. When translated into a three-dimensional world, this overlapping has a pretty interesting quality of abstraction.
This is the setup for my custom light Blueprint that passes its directional vector to a Material Parameter Collection. This parameter can be used in any material. This Blueprint contains a Directional Light. The rotation of the Directional Light is set by the rotation of the Blueprint. By getting the Rotation XVector, we can compare the alignment of the light's directional vector with the vertex normal of any surface within a material.
This is part of the chain for the setup of the material for my particles. I take the Dot product of the light vector and the vertex normal vector to compare the degree to which they're facing each other. As the normal of the particle's plane becomes more and more in line with the light vector, the light is hitting that plane more directly.
The Dot product will put out values of 1 where they are facing each other and values of 0 where they are perpendicular. This makes a gradient falloff on the surface from white to black until the "terminator."
The one caveat is that, the Dot product needs to be clamped from 0 to 1 so it doesn't yield the negative results (if the surface normal is aligned with the light vector, but in the negative direction.) You can see this happening in this image, where the opposite side of the sphere is white as well.
At the end, I power the clamped result to control the sharpness of the falloff of the gradient. This can be tweaked to personal preference.
After that, I've fed the 0-1 gradient into the alpha of a 3ColorBlend node. This allows me to map 3 colors to the low, mid, and high ranges of this gradient. Here I have them as simple constant RGB colors, but anything can be plugged into the 3ColorBlend. I could even blend in a different BaseColor texture at different points on this light falloff. Perhaps I want to use texture map that has some speckly detail in the higher ranges where the light hits most, and a rougher, blockier looking texture map in the shadows. These are some things I'll be experimenting with soon to see how I can achieve some artistic abstractions of material definition that respond dynamically to light.
This is a material graph I set up to randomize the brush stroke type (the opacity mask used for the material) per particle. I wanted to randomize parameters for particles within their material without having to make a different Dynamic Material Instance assigned to each particle. So, I decided to use the Particle Color attribute, determined by Initial Color in Cascade.
A side note: to set particles to be a random color at birth, make sure that in Cascade you use a Distribution Vector Uniform for Start Color to set a Min and a Max color (I used 0,0,0 as min and 1,1,1 as max.) Also, if you're using this in the same way I am, to drive material parameters for static particles, delete any Color Over Life modules. They override the random birth color of the particle.
The comments I've made on this material graph best explain this graph, but basically I've set up the randomly determined values in the Red and Green channels of the Particle Color to correlate to a Row # and a Column # on a 4x4 texture grid for brush stroke masks. The UVs of the texture grid are tiled .25 to crop to the size of one grid square on a 4x4 grid, and the U and V are shifted independently to the resulting Row and Column #. From there, this resulting mask can be plugged into anything, but I've used it as my opacity mask to determine the per particle stroke "texture."
If you look carefully, these three screenshots show the randomized brush strokes being used.
Something else I've been working with is using World Position Offset to rotate the planes along the "edge" of the mesh (dependent on viewing angle) to face the player's camera. Although the edge I'm getting from pushing the particle planes out from the mesh using radial velocity (the Sphere module in Cascade) isn't terrible, it can sometimes reveal the thin, non-dimensionality of planes perpendicular to your viewing angle.
It looks like I could pass the Camera aim vector from the player Blueprint to a material parameter and compare that to the vertex normal of the planes in the same way I have the compared the light vector to the vertex normal with a Dot product. The more perpendicular the planes, the more black the resulting value from the Dot product can be. And so, I can use that as a mask for a RotateAboutAxis operation that plugs into World Position Offset.
I threw on a normal map to see what kind of results that might yield with this technique. The bit of lighting information and relief it gives the paint strokes is nice, though I like the look of it a bit subtler.
Here's an early preview. I tried applying the particle paint to a mesh that wasn't a sphere, and the results weren't bad, but there are so many ways I have yet to explore to implement this thoroughly with different types of meshes and more complex forms. Even so, the results are definitely interesting. I'm enjoying the textural differentiation it's adding to the rock by simplifying the information in the areas that the particles overlap.
I'm also developing my workflow for systematizing all of this. I'm establishing a central Blueprint that contains my main Particle Paint particle system. I'm parameterizing the Cascade Particle System and connecting those parameters to user-defined Blueprint variables, so that the particle system is easily customized per instance.
Today I was inspired by this article that outlines an approach to achieve a non-photorealistic, painterly style. The idea is that particles are placed on the surface of a mesh with brush stroke shaders applied. There's not really any reason this can't be translated into Unreal, so I figured I'd give it a shot.
My original idea months ago was similar to this method, except I was scripting the placement of plane meshes onto the surface of a mesh using Blueprints. The problem was, this was only feasible to calculate with simple shapes like cylinders.
Particles in Unreal's Cascade have a module called Skeletal Mesh/Surface Location (read Unreal's documentation here) that allow particles to be emitted from the surface of a mesh -- though as far as I know, this is only works with Skeletal Meshes.
I've made a quick illustration demonstrating the two methods I've been working with to achieve this painted style, and each of their features/disadvantages that I'm aware of.
The first method is the one I've set up with all of my assets so far. It has been pretty faithful but there are some drawbacks. This idea was loosely based on this video I saw a while back, which essentially displaces the mesh several times to produce an imprecise, painted silhouette.
Here are some of the results I've gotten working with this today. Right now the material is just using one brush stroke opacity mask and the color is assigned based on the vertex normal, just to see it working with a bit of color variety. It's interesting to see it working and I'm looking forward to investigating some settings more thoroughly to see what kind of results I might be able to get on different sorts of meshes.
This method is also interesting because there are a lot of parameters I can play with on an individual stroke basis. Here's a sheet of brush stroke types that I can use as masks for each particle plane. In this way, I can easily vary the brush stroke type by sliding the UVs around this sheet. This sheet would be shared by all assets and different assets might use different stroke types. (Leaves might use something more loose while rocks might use something blockier and more chiseled.)
I've begun to test this method on more complicated meshes, and there are some issues - for example, some areas of the mesh I might not want brush strokes to generate on. More strokes are going to generate where the mesh is most dense. I might be able to control this density by making another version of the base mesh that will be bound to the same position in a Blueprint, but hidden. Additionally, this mesh would be a Skeletal Mesh to allow the particles to emit from the surface location, while the base mesh could remain a Static Mesh.
I would say that the main disadvantage of the "Mesh Shell Layer" method I've been using is that there's quite a bit of set up involved for the Blueprints for each asset. It relies heavily on communication between the asset's Blueprint and the material parameters per dynamic material instance assigned to each mesh shell instance. Creating instances of these mesh shell layers, assigning a new dynamic material instance per layer, and modifying the material parameters per layer becomes a hefty operation. I've got this consolidated into a function, but the issue is that this function can't easily be shared between each asset's Blueprint. A solution might be to put this function into a Blueprint Function Library that can be used across multiple Blueprints without having to copy and paste the whole graph, but since the function deals with adding Static Mesh components to the world dynamically, it can't be done in a Function Library function.
The mesh shell layer method might also prove a bit more intensive than the particle method. This could be improved by making the mesh shells even lower res versions of the base mesh. I recently had difficulty with this on the rock asset. The rock's base material was tessellated to allow for displacement with the wax. Each mesh shell layer would normally be assigned an instance of this base material. But there is no material override to turn off tessellation, so an entirely new material without tessellation would have to be made. Even though a lot of duplicate information can be consolidated into a Material Function, syncing up these two materials proves to be a pretty tedious process.
Overall though, both methods have their advantages. The mesh shell layer approach works well on the boat because each layer adds a painterly thickness to the thin-walled boat mesh. Each layer can have varying edge texture, be multiplied by a different color, and the holes eating through the structure of the boat can be clamped differently.
On the other hand, the particle method would work better for something with a more organic silhouette, like tree leaves. The ability of particles to move may also come in handy for some effects as well. Thinking back on the boat rusting sequence, it would be a great in addition to the mesh shells to create the illusion of parts of the boat crumbling away as it deteriorates. I like that by working on the level of individual brush strokes, it gives a bit more visual variety and is more akin to the process of actual painting.
A bit more development on the look of the wax and the rock materials and the blending/emergence of the wax into the cracks of the rock. The wax and rock are set up as two different Material Functions, masked by vertex color. This vertex color mask is modulated by the two height maps of each material to break up the vertex-like blending. In the future, the mask won't likely be determined by vertex color, since Skeletal Meshes can't make use of vertex color information. Nonetheless, it's great for testing the look and how the materials are working together at the moment.
A quality that I wanted to capture and I think is starting to work here is that the subsurface color bleeding should be less in areas where the wax drip is thicker and more in thinner areas. The best subsurface effect comes from being lit from the inside of the mesh. I plan to include several point lights on the inside of the mesh that are components within the rock's Blueprint. This should make it easy to manage them from one place.
The rock should be looking a bit rockier and rougher than last week. I've redone some of my normal maps that were looking a little ill-defined and made tweaks to my custom painted normal map node (fewer posterized steps for less smooth gradients, and more intense warping to simulate splotchy brush strokes.) Still to come are the painted mesh layers.
Here's a super rough layout/blockout of a shore area just to get an idea of how everything is going to fit together into a space, with the candle rock mountain as an overhanging cliff. The candle rock will be a beacon, with the wick lit, that the player will have to climb up to. You can see that the boat sequence will be just off of the shore, and won't appear until you've gone up to the top of the mountain. After you come back down from the mountain, if you look back up at the top, the wick will have been replaced with a lighthouse.
I'll be populating the shore and landscape with wax rock assets. Additionally, my next steps with this are to refine the candle rock model and landscape detail, as well as fix scale and proportions that aren't working.
Revisiting the rust material and the formation of rust on the boat hull, I realized it wasn't very believable. I observed a lot of rust on metallic surfaces and realized that my rust was lacking a differentiation between the areas of rust flaking and the areas of rust dripping. The rust flaking areas often appear to be a deeper red orange, and the areas of dripping rust are lighter orange and fade off softly.
The boat's skeleton is slightly improved from before, a little bit less fantasy. I tried to make them appear more fragile, deformed, and decaying. One of the main problems with the previous skeleton iteration was that it didn't follow the form of the hull very well, and it looked more like a skeleton inside of a boat than a skeleton underlying the structure of the boat.
I added the wheelhouse structure and applied the same rusting effect to it by painting keyframes for its rusting and deterioration pattern.
This is an early preview. The materials aren't well defined on the wheelhouse yet and I plan to add some further model-specific detail maps like AO, curvature, and normal.
I've been starting to work with the environmental effects for this sequence. My idea is that, as you approach the boat, the sand will start to ripple in front of you in waves that surge forward and "swallow" the water that the boat is floating on. The first component of this is to displace the sand, using variations of a sine wave. I have the sand rippling at the moment, but I'll be working on making it more directed and less random.
Details on the process so far:
I've made a lot of progress in terms of organizing my graphs. The boat is one Blueprint, BP_Boat, that initializes all of the mesh layers that make up the boat (for the painted effect) and assigns dynamic material instances to each mesh layer. This can be simply dragged into the editor and should automatically rust away based on distance, as well as connect with other relevant materials in the sequence.
BP_Boat sets material parameters in all of these dynamic material instances. Different components of the transition are determined by different curves. The wheelhouse deteriorates at a faster rate than the hull, and so I've set up different curves to control their rate (i.e. 60% through the transition, the wheelhouse has completely deteriorated.) The water drying up may be another curve, or the increasing size of the waves/sand ripples throughout the transition.
One of the main material parameters that BP_Boat sets is the Rust Transition, a 0-1 float value determined by the player's distance from the boat. This essentially flips through the hand-painted mask keyframes to animate the boat rusting and deteriorating. Mask keyframes are scalar and so they are stored in each of the RGBA channels to conserve texture space within the shader. I've painted the mask keyframes so that black on the mask is solid, 0 - .5 is where rust begins to form, and .5 - 1 is deterioration, contributing to the opacity mask for the material.
The rust drips are formed by shifting the UVs for the mask downwards and lerping it over itself incrementally several times. This gives a somewhat soft blend directionally downwards. Using this mask, I blend in a lighter rust material. In the standard mask, I blend in a deeper, flakier rust material. The masks, which originally are soft gradients, are blended with different noises - the flakes blend in a cell pattern at the edges of the rusted areas, where the drips use a more directional vertical noise. These are all blended in the UE4 shader so tiling can be adjusted, etc.
Another example of material parameter that BP_Boat sets is a mask based on which mesh layer the material instance corresponds to. This allows for effects like a color multiplier per mesh layer, or a UV offset per mesh layer. Connecting the various materials back to a centralized Blueprint has proven very helpful in directing all of the parameters for this sequence.
Some things left to do:
-Refine the wheelhouse materials, add glass to windows
-Replace placeholder sand and water materials
-Refine sand and water wave displacement
-Natural displacement of boat along with sand/water waves
-Sand particles blowing
I'm looking to finish the majority of this in the next couple of weeks, but some of these tasks might take a backburner for a bit as I start venturing into my idea for my next sequence.
Wax rock beach:
Over the past few weeks I've devised a much clearer plan and scope for this project that I feel like encompasses a lot of the visuals and moods I wanted to create. Essentially, the project is one transforming dreamscape environment. The primary environment is a beach with large, arching rock formations. The color palette will be desaturated and cool initially. If you approach the rocks, they will start to melt into wax wherever you aim, and the melting will spread throughout the rock from there. The color palette will shift to warmer as the wax melts into an amber-like color. The hero element in this environment will be a large rock formation that looks like a candle. My rough idea is that the candle wax rocks will all melt down into a pool of water, which is where you'll find the boat.
I'm interested in incorporating morph targets to actually melt these rock meshes down. The great news is that Canvas Render Target 2D has been fixed and is no longer crashing in UE 4.7, so I'll be continuing to work on my world canvas mask painting to material blend to the wax material wherever the player aims. The wax material will be much more translucent and use flow maps and displace areas achieve the appearance of wax dripping.
A broad summary of the tasks for this sequence:
-Create rock mesh pieces
-Create hero candle rock formation
-Create materials for rock and wax
-Implement morph targets for candle rock meshes
-Implement canvas render target 2D masking for material blending
-Transition into boat sequence
After approaching the boat, leaving only bones, vines will grow up the bones and grass will start to grow and overtake the sand. Trees will sprout up alongside the path you walk, and increasingly grow larger, with branches looming over you.
I've already started to develop a system within my tree Blueprint to animate the primary branches and the trunks. By developing this, I'll hopefully be able to "grow" the trees around the player.
Some known stuff I'll have to do:
-Hopefully add curved tangents on branches
-Add leaves to trees
-Implement tree animating system
-Add vine growth to bone material OR add a spline-based vine
I made a couple of videos to recap my progress on my thesis project this semester.
Next steps for the ship deterioration:
-Tweak keyframes for the hull deterioration (some parts of the boat appear to be suspended in the air towards the end of the transition.)
-Add interest to the rusting/deterioration of the structure (a crumbling effect?)
-Better visually integrate the bones with boat hull. Currently they don't look enough like they are a part of the structure.
-Add wheelhouse structure to the deck of the boat.
-Work on sand and water material.
-Refine rust material and how it forms on the surface.
-Add interest to the water drying up/sand overtaking the water. As you walk closer to the boat, the sand will ripple from where you are walking and displace in wave-like formations.
-Additional secondary visual elements (seaweed? anchor?)
-Skydome with clouds, transitioning to overcast, as well as lighting.
It's still quite a ways from being done but it also has come a long way and I've learned a lot in the process.
Next steps for the tree system:
-Achieve a more angular look to the branches. An issue here currently is that I'm not able to set the tangents of splines through Blueprints. I am able to set the tangents of the spline mesh, but the child branches are placed on the parent branch by finding a location along the spline. If the tangents of the spline cannot be set to match the tangents of the spline mesh, then the branches are placed slightly off wherever the spline mesh deforms away from the path too far. I'm currently looking into other strategies to get the look I'd like.
-Add leaves. The leaves are an area I'm looking to achieve a really impressionist look.
-Improve the meshes I'm using. They work decently but they're sort of stand-ins at the moment. I'd like to refine the look of them.
I had posted a rock arch sculpt I was working on several weeks ago. I've started over with that here and I think I'm starting to better grasp the look I'm going for- a hybrid of melting wax and rock.
A happy accident in developing a look for my clouds. This might be an interesting look to explore.
Some progress on the boat hull rusting away sequence. I hand-painted mask keyframes that denote where rust forms and where deterioration starts to happen. Black on the mask is solid, black to 50% gray is rust forming, and 50% gray to white is where deterioration occurs. There are 8 keyframes in total. The result is getting there, but I still would like to do a bit of tweaking. Some of the keyframes towards the end jump a bit too much from one step to the next.
This week I focused on developing this shader-based rusting/deterioration transition (23+ hours.) I had attempted a few different methods for making this happen. I originally was working with two separate masks to make this transition happen - one large detail control map (the overall, top to bottom rusting of the boat, holes, etc.) and one small detail control map (the ribbing of the boat.) This method I was first going with allowed for a lot of versatility in the timing of the transition and the blending, since it was all blended together in the shader, but finding a way to blend them together well in the material turned out to be probably too much overhead without necessity to get the visual result I needed. Painting all of these details in one mask and separating them by value seemed to allow for a more natural, organic blending across the whole boat.
These last trees are still broken in many ways, but I've been working a lot on the Blueprint to consolidate into functions that are able to perform the repeatable processes of placing child branches on parent branches randomly. For example, now you can just up the number of tiers of branches and see the branches increasingly split off from each other, getting smaller and smaller. I also realized that, before, my child branches weren't being rotated relative to the parent branch, so this has that fixed, which is a major improvement. Something seems to have messed up in my branch aware placement, because some of the branches are looking pretty crowded. This is something I need to look into.
The system is much more organized than it used to be, but these trees lack a lot that I'm still looking to implement. Some branches still aren't being placed on the spline correctly (if a branch bends too far from its spline.) I've also been trying to decide how to modify the tangents of the randomly placed branches so they aren't straight sticks like in the first image.
I think the most exciting part of working on procedurally constructed assets is being surprised by the types of results that your system yields. Although you build the system with intention, there's a ton of room for error and unexpected combinations that can inspire you to strive for new visual goals, especially while testing. Neither of these trees look like how I planned, but the forms they created are inspiring alternatives to explore.
Bringing the boat into the same painted style I was able to achieve last week on the tree was a major development this week. Now that the look development for the boat is mostly figured out, I've been working on the masks to make the boat rust away as you come closer to it. I'm looking to finish the rusting away of the hull by the end of this week. (18 hours)
My tree system underwent some much needed cleanup and restructuring, but still has a long way to go. It's more downhill than uphill from here though. (16 hours)
The results of my grass tests were surprising. I was playing around with this on a whim. I set up cross section layers in a Blueprint that controls how many layers there are and the overall height of the grass to pack those layers into. Each of these layers has the grass material applied. The Blueprint is able to feed information into the material. For example, the material can have a different effect on the top layer than the bottom layer by taking in the current layer from the Blueprint. The opacity mask for each layer is actually the same painted mask, but clamped slightly differently at the top so the top thins just a bit. This grass also moves very slightly in a sort of dreamy way by just offsetting the UVs by a noise that shifts over time. (2-3 hours)
I've been playing around with materials to keep working at achieving the painted style I want for this project. Although I enjoy the look of hand-painted textures, my goal is to take this style beyond the typical flatly painted look by making it a convincing part of a 3D world with dynamic lighting. Particularly, I've been trying to create custom painted looking normal maps to accomplish this.
In my attempt to sculpt normal maps, I still hadn't been getting the look I wanted. Normal maps seemed to kill any painted illusion and overwhelmed my hand painted textures. Also, the chunkier style of my sculpts conflicted a bit with the more textural quality of my paintings. While revising the material for my tree bark, I made a bit of a breakthrough in my method.
These images are of the full tree with a modified, painted looking normal map. These also have the additional opacity mapped shell of the mesh over top, adding the textural edges on the silhouette.
You can see the effect a bit clearer in the image on the right. Two instances of my sculpted, tiling bark normal map texture are each warped slightly differently by noise patterns, and blended over top of each other by a dithered mask. This resulting warped normal map is blended back over top the original normal very slightly to only get a slight warped and dithered effect. It adds a slight pixel-level textural effect to the normal map.
Then, I step the values of the normal map so that the transitions between different depths and angles are sharper and less gradiented, sort of like a posterization or a cel shading effect, but on the normal map. This helps in getting a more faceted look that I just couldn't achieve in my original sculpt.
It was upsetting to see at first that the effect of this normal map wasn't really coming through once my original painted diffuse color was applied, even with a really direct point light shining on the surface.
Something simpler works a bit better to show the normal map. Although I could work to get a bit more slight hue variations in this map, it's a good base. With the painted quality integrated into the normal map, it looks so much more well integrated into a 3D world.
I spent the majority of this week working with materials to find this painted style (18 hours.) I also completely revised and reorganized my tree Blueprint (13 hours) and some things still need to be done to unbreak some things I broke in the process. I spent a bit of time with post process materials to soften the edges of objects (5 hours.) Lastly, I spent a lot of time trying to refine the look of my boat by painting model-specific detail maps, only to find that so far, I still like my iteration from last week better (12 hours.) Also, the development in my process with the normal maps makes the work I was doing on the look of the boat a bit unusable now, so I'll be working next week on applying this method to the normal maps I'll be creating for the boat.
My goal for this week has been to push the look of my boat further through defining its material and to work on the material blending between the painted boat material and the rusted boat material.
The opacity masked look has been working decently to get a sense of a painted edge quality, but something I have been lacking that I really admire about paintings is the mix between hard edges and lost edges. The brush strokes on an edge aren't always all completely hard and opaque. The problem is UE4 masked materials don't allow for grayscale alpha values, they clip black and white- it's either 100% or 70% opacity. Translucent materials can provide this, but they're more intensive.
I discovered that in the UE4 content examples, they demonstrate a technique used to fake translucence with masked materials by dithering. Dithering is a graphics technique used to simulate value using the density of a bunch of tiny dots when a color palette is limited. The DitherTemporalAA node in the material editor provides that kind of effect.
This same method seems to work to my advantage to give my opacity masks a softer look. Even though the little dots might technically be considered an artifact, it unexpectedly works to my advantage to give my opacity masks a softer, posterized/quantized look that aids in the painted style.
These images show a sphere with the dithered opacity masked by fresnel so it only shows up on the view angle dependent edges of the sphere. Here I've also tried multiplying that dithering by different brush stroke opacity masks to yield slightly different edge qualities. I had tried this same technique before with translucent materials to soften the edges by decreasing their opacity with fresnel as a mask, but the transition was very soft and it tended to look like the object was fading away.
Thinking forward, I would love to create a universal Material Parameter that can be modified by Blueprints across all materials to modify their edge quality, globally or locally. As things become more intense, the brush strokes at the edges might become less calm and more hard-edged and haphazard. I'd also like to do some modification to this edge effect based on how much light is hitting the object. I find in paintings, areas that fall into shadow tend to have lost edges more so than ones in the light. This kind of subtlety might really add depth to the visuals. Another parameter this effect could be dependent on would be distance or even focal point. Paintings use strongly defined edges to emphasize a focal point. Usually foreground elements might also be sharper, and background elements might have more lost edges if they are not the focus.
I've been trying to find a good way to get the model specific detail I need to define the structure of the boat with more than just tiling textures. A method of sculpting or modeling a high poly and baking it down for additional normal detail doesn't really work well here. Normal maps I have tried to create so far all seem to break the loose painted look I'm going for. I hope to continue investigating non-traditional ways I can apply normal maps to bend light and enhance the painted look rather than diminish it.
This map functions as sort of a hand painted AO mask for the boat. It's not entirely complete as is-- I've found recently that instead of spending too long getting carried away painting for hours, I'll just test a little bit in one area to bring in and preview if I'm getting the look I need from it before I continue. You can see here I've only added some of the studs at the front of the hull. On top of this, I can overlay a hand painted curvature map mask to get some edge detail in specific areas.
Both of these images above show the paint blending to the rust, which follows a similar but more clamped version of the opacity mask for the deteriorating parts of the boat (left.) I think the transition between the rust and paint could be taken much further by having even an additional mask that follows this one but helps to define the edge/transitional border between these two materials. Right now, to break up the blending with more than just an opacity blend, the mask on the left is multiplied in the material by a black and white hand painted brush stroke map for an additional textural quality to these edges.
You can see in the image on the left, the boat looks more thin-walled than the one on the right. This is where the duplicated mesh shells with different material instances applied work as cross sections to really help along the illusion of multiple layers of rust and textural depth.
Some more hand painted maps from this week - the first a rust-like mask I'm using to blend between the paint and rust, and the second a varied paint color for the painted boat material (which still needs more work.)
I've found tremendous use in my work so far compiling assets within a Blueprint that can be dragged into the editor with all of the components it needs to be a complete asset. For example, for my boat, I have a few duplicate shells of the original mesh. This allows me to keep them all hooked up to one master asset I can drag around.
Another benefit is that any global Material Parameter Collection values can be exposed to the Blueprint, so that I can modify material effects by clicking on the boat blueprint and changing values in the details panel. This is how I plan to manage and test the different sequential stages of deterioration that I've allowed for in the material. The transition will happen by blending/lerping the hand-painted mask that designates areas of deterioration to the next hand-painted mask with increased area of deterioration (the next stage or "keyframe" of the transition.) So to control the transition, I'll divide up 0 to 1 into the stages of the sequence, and at the end have one controllable value to slide back and forth between 0 and 1 to make the transition happen. This makes it easy to hook up to anything - distance, triggers, timelines or matinee.
Progress on the rusted boat in combination with painted style. Still quite a ways to go in terms of blending between the not rusted boat and getting the right amount of material definition with the painted style.
Hand-painted tiling rusted metal texture (left). On top of this, I've started to overlay hand-painted model specific details.
Some thumbnails illustrating the key points in the sequence. The boat rusts over as you approach it. The sand closes in on the ocean in waves, leaving the boat in a puddle. The sky becomes increasingly darker and overcast. Finally the boat has deteriorated to skeleton bones sticking out of the sand.
Concept art for the top of the mountain, the candle that melts into amber, and the amber that solidifies into rock.
PROCEDURAL SPLINE TREES
• Splines for primary branches can be dragged out.
• Tangents of splines can be rotated and scaled.
• User can add up to 10 primary branches that split form trunk.
• Secondary branches are generated randomly along primary branches with awareness for other branches.
• Implemented random streams so you can modify a random seed to change the tree rather than have it change every time the Blueprint recompiles.
• As the tree branches out to further tiers with smaller branches, more small branches will be included in one piece. Experiment with what these pieces are.
• Continue tweaking parameters on randomly placed branches to get a more natural result.
• Figure out what the loose style should be to generalize detail as it becomes more busy (tiny branches and leaves.)
• Modeled the boat to prepare for the boat deterioration dream sequence - this is an understatement. The modeling was the easiest part once I figured out how a boat actually works, which has been the majority of the problem solving. They have a very specific and subtle form and curvature that was a bit hard to capture initially until studying a variety of different boats. The shape of the hull is complete, everything else is in progress (railing is a stand in for scale.)
This came a long way from what I started on last week, which looked like a toy boat. I can't stress enough how important it was that I got the form of this right before proceeding.
Very early tests just to demonstrate the concept of layered meshes to get a painted look with expressive detail (i.e. a rusted boat showing through to an underlying skeleton.)
• Mesh is added to a Blueprint as a component, duplicated multiple times (in this picture, 3x)
• Each duplicate has a different material instance applied of the main material.
• The main material is a hand painted B+W mask with a smooth falloff from black to white. This is to get enough of a gradient in the gray values to multiply in a tiling hand painted B+W brush stroke mask, that adds the textural quality to these masked edges.
• Rather than hand painting the mask for the boat in the exact way I want it, I compile the mask in the shader to allow for different levels of clamping on each material instance. This is what allows for the dynamic transitioning of the deterioration of this boat.
• Parameters for UV offset and tiling multiplier to ensure the brush stroke pattern is not repeated on each "layer" of the material instance.
• Each mesh is "pushed" in on within the shader on the vertex normals of the mesh. Each layer/material instance is pushed in an increasing amount. This makes it so the mesh does not have to have these layers baked in, so every time I want to modify the distance between each layer, I don't have to tweak the model in Maya and re-export.
ROCKS AND WAX
Concept wax sculpts to use in the creation of tiling textures. Not sure I like the result of these - they might be too sculpted and detailed for the painted style I am going for.
I had a thought that I could use Substance Designer FX maps to generate procedural tiling brush stroke maps that follow a hand painted flow map for their directionality - a similar approach to this .
This might be somewhere I start to explore, especially to gain better control with the B+W brush stroke maps. Hand painting is great for some things, but hand painting tiling textures can often be a painstaking process, only to find you don't like the result after hours of painting. It is also hard to get a very evenly distributed level of detail when hand painting tiling textures. Naturally you hand craft some areas more than others, and these tend to be the areas that stick out when the texture tiles a couple of times.