Cat creator

The kitty uncanny valley is less detectable to humans, making creating virtual cats slightly less freaky than virtual human characters. Although it may disturb cats?

Recently I've been working on a personal project -- a cat character creator that allows the user to customize their own virtual cat! This is the first step in a larger cat simulator project I'd like to do using UE4. I am a big fan of cats in real life, so it's a joy to experiment with. 

So far I've sculpted a default cat head to test the different features I'm looking forward to creating. 

 

 

 

I've created cat eyes enabled with pupil dilation/contraction. This is done using a flow map to warp the UVs of the pupil to either expand or contract.

The goal is to be able to have a virtual toy and hide it behind something, and see the cat AI respond immediately by having its eyes dilate to full roundness with an urgent interest.

 

The fur in the images is done using a shell method, which (so far) has proved to be the most effective for both performance and with consideration to future needs for rigging and animation. The shell meshes are the same as the original base mesh of the cat's face, pushed out along its surface normals using World Position Offset in its shader. These shells are stacked on themselves to create the appearance of depth. The alpha mask/opacity mask used in the material on each shell is just a cross section of all the strands of fur (which essentially looks like a bunch of dots.) 

A progression of cat fur lengths, from Sphinx to Persian (though without some weight and directionality to the fur at the longest length, it appears as if the Persian's fur has exploded.)

Fur length is determined by a 0-1 black and white texture that I've painted on the model to match cat fur length patterns. This is used as an alpha/mask to modulate the World Position Offset for the shells. In areas where it is closer to 0, the fur length will be shorter (use the min offset for the shells.) In areas where it is closer to 1, the fur length will be longer (use the max offset for the shells.)

Some tests I'll be doing soon:

  • Fur directionality - The fur is currently following a linear path out along the surface normal of the mesh. If I were to create a flow map to nudge the fur strands in a certain direction progressively from the bottom to the top shell, it might provide the look of directionality. There's also the possibility I could create a flow map in real time by using Canvas Render Target 2Ds. This could allow the user to pet the cat and brush its fur in different directions.

 

  • Fur color - I could provide preset patterns for fur based on breed. These patterns could be procedurally customizable to an extent, to provide for some of the variety you'd see between different cats of the same breed. I think though, it would be more awesome to be able to custom paint your cat, to allow endless possibilities. I'd like to experiment with a feature using Canvas Render Target 2Ds to allow the user to take a brush and paint patterns and colors on their cat. 

 

  • Customizable eye patterns and colors - There are an invariable amount of different eye patterns. Using a Substance in engine could provide for a lot of variety, or alternatively different presets with customizability by exposed material parameters, similar to the fur.

 

  • Wet nose and rough tongue - Because what would a cat be without a cold, cold, wet nose and a sandpaper tongue?
     
  • More progress soon to come! Feel free to ask any questions or share any ideas you might have with me.

Spline tree builder updates

Through my work on project Morphscape, I had developed a Blueprint based system for semi-procedural trees based on spline and spline mesh components in Unreal Engine 4. Looking forward, I'm revisiting and restructuring my old work to release the system on the-Unreal Marketplace. 

In the work of many 3D game artists, I see sort of a road block in artistry when it comes to designing original trees, foliage, and other natural elements. It can be difficult to create these assets believably, with the right balance of consistency and variation to populate expansive natural environments. As a result of this, many turn to software like Speedtree, PaintFX, or download from pre-made kits.

While these are certainly not bad options, I am designing this system with the idea of maximizing the amount of artistic control the user will have in designing their tree, while still automating the laborious process of modeling branches on branches. 

This lets you still develop your own meshes and materials for the trunk and branches, but enables you to deform them using spline meshes, which can be both manually placed or automatically, procedurally generated based on user-defined parameters. 

Additionally, having this as a Blueprint system makes it so that you can have an infinite variety of trees that are customizable in-engine. The advantage being that you don't have to export a bunch of different static tree assets, like birch_tree_tall, maple_tree_short and rely on good placement to minimize noticeable repetition. Using this kind of tool has the added benefit of flexibility, since you can adapt your tree assets to your level design later in the pipeline.

So, you can simply drag the Spline Tree Builder Blueprint asset into the viewport. Every instance of that Blueprint asset you pull into the scene can be a different tree, but they are all built on the same customizable system. Just drag out splines and adjust parameters to rapidly iterate different variations. 

I had built a previous version of this system, but I am looking towards improving it in terms of user-friendliness, optimization, and adding new options like leaves and self-collision detection on branches. Look forward to more progress updates and the release on the Unreal Marketplace!

 

Southwest rock material studies

Last week I posted about the project I did revolving around my trip to the Pacific Northwest region back in January. In late March on my spring break, I had the awesome opportunity of traveling west yet again with the College of Imaging Arts and Sciences' honors program at Rochester Institute of Technology -- this time to see various national parks in the Southwest.

Travel has been one of the most worthwhile resources in terms of expanding my visual library. There is no reference quite like being able to stand in a space and get a sense of the atmosphere, light, and scale of an environment. Becoming more knowledgeable, in any realm, and experiencing new places can only help in generating new visual possibilities and informing my art and conceptualization.

Being particularly interested in texture and material creation, I found myself captivated by unique patterns of erosion, imagining the forces that shaped these rock formations, and the visual evidence. On the trip, I spent a lot of time studying and sketching to try to dissect the systems of the rock formations I saw -- what are the formal qualities that distinguish this type of rock, and to what extent can there be deviation from that pattern without losing the essence of that rock type? 

A thought that is always inspiring to me is the strong visual and conceptual link between mathematical patterns and natural patterns. For this reason, throughout the trip and in the weeks following, I decided to do some material studies of several of the rock types I saw on the trip using Substance Designer to create procedural textures for my materials. The method I use to create procedural textures is similar to how I draw: I define the major shapes and patterns of the rock, and I decide which parameters about the rock can be modified to keep it within the same rock system.

Here are a few examples of the tiling materials I created, loaded into Unreal Engine.

Here is a common layered shale rock I noticed in many places throughout the trip.

This rock material exemplifies the characteristics I saw in a lot of the rock structures I saw at the Arches national park. 

This one was based on sort of a specific moment for me on the trip. We visited Horseshoe Bend at sunset, and I loved the way that the low angle sunlight was emphasizing the staggered layers of the shale on the ground. 

It might not seem like the most extravagant of rocks, but I always found it important to stop and look at the ground.

This rock is from Glen Canyon/Lake Powell. With its distinctly different colors and variety in texture, it was one of my favorite rocks on the trip.

Based on the rock walls of Slot Canyon. I loved the way that the surface of this rock looks and feels so smooth, yet still has so many small imperfections. I enjoyed seeing the sharp, hard-edged contours in the rock that seemed to echo the wave forms and wind that may have shaped the rock. 

Bryce canyon certainly felt like it was on another planet. These bulbous rock formations called hoodoos are known for sometimes being shaped like people. 

Seattle underwater

Over winter break in January, I took a trip with the College of Imaging Arts and Sciences honors program at RIT to visit the Pacific Northwest area -- Seattle, Portland, and Vancouver B.C. 

On the trip, we took a tour of Seattle Underground. I was fascinated by the history of the disaster that caused the former ground-level of Seattle to be built on top of. Perhaps influenced by the seaside themes around me and our aquarium visits throughout the trip, I began to imagine a future disaster scenario in which present day Seattle exists flooded, below sea level.

As an independent project, I digitally painted illustrations for hypothetical postcards advertising the future disaster tourist attraction, Seattle Underwater. I've illustrated the scenarios in a somewhat comical, over-the-top style.

PIKE PLACE FISH MARKET

Seattle's Pike Place Fish Market is known for its fish mongers, making an attraction of throwing fish to entertain its customers. Now under the sea, its claim of "fresh fish" is in question.

SINKING SHIP PARKING GARAGE

This one is a play on the actual "Sinking Ship" parking garage we visited in Pioneer Square in Seattle. You can see here, post-disaster, that it has become a literal sinking ship!

THE SPACE NEEDLE

This one is an interior view of the Space Needle, sunken and facing destruction by a giant octopus. In real life, the Chihuly Sculpture Garden is situated directly underneath the space needle. I couldn't help but see this Chihuly glass sculpture as a bunch of Dr. Seuss-like creatures that seemed to be making noise. They also visually parallel the suction cups on the octopus's tentacle!

STARBUCKS

Seattle is home to the original Starbucks. The iconic two-tailed mermaid of their logo is thrilled that they have finally decided to open up shop under water!

Morphscape project recap

This past week, I assembled a slideshow for my final senior thesis defense presentation at Rochester Institute of Technology. If you haven't been following my blog, it is a pretty thorough overview and documentation of the work I have done and the workflows I've developed over the course of the past two semesters working on this independent project.

Although my thesis project is "officially" complete and I've demonstrated what I set out to prove initially at the beginning of the year, I will still be refining the project as well as continuing my research and development on the theme, processes, and visual style I have been pursuing-- so stay tuned! 

MORPHSCAPE: Interactivity

After putting together a visual showcase for last week, I've decided to switch gears for a bit to focus on the interactive component of this project, which is my intended end product. All of the dream events featured in the cinematic are distance based, and so they happen automatically as the player moves forward. This is works well for a movie, but for a game, it is a bit linear and passive. I've been revisiting the work I did several months back to create a volumetric painting system using Unreal Engine 4's Canvas Render Target 2D. Additionally, I've started to design a very simple character and its movement controls.

CHARACTER + 3D CURSOR

My initial idea was that the character would be a flying character set up in third-person. The camera would aim to look at a 3D cursor that is at the projected 3D location of your current mouse position. To control the character, you would use W A S and D. When the character would move forward, it would move forward in the direction of the camera's aim, towards the 3D cursor.

After testing this out, it felt pretty hard to control. I've always found flying controls in games to be the trickiest to get the hang of -- and they are definitely difficult to make intuitive as well. Additionally, I couldn't work out a camera arrangement that prevented the character from blocking a good chunk of the environment.

I decided instead to make the character act as a 3D cursor, which feels much more effective already. This also is much clearer and easier to control the painting mechanism, where the 3D cursor/character acts as the paintbrush. Now, the orb character flies to the projected location of your mouse in the world, while the player camera remains stable. The player camera is still controllable with WASD and will rotate to look at the cursor, but it is grounded.

SETUP

The 3D cursor is set up mostly in the Player Controller class. To enable the mouse events, I've checked the Mouse Interface features in the Player Controller settings.

 

The base feature of the 3D cursor is that the mouse's 2D location needs to be projected into 3D space. I've used a node called Convert Mouse Location to World Space. This node alone doesn't put the location in the correct spot though. You can multiply the Direction vector from this node by a user-specified distance, and then add that to the Mouse Location. This essentially puts your cursor at the specified distance from the initial mouse location by moving it out along the direction vector. (See below.)

For testing purposes, I had attached a sphere to the cursor location to be able to easily visualize where it was in 3D space. As an additional control, I mapped the Mouse Wheel input to be able to increment and decrement the Cursor Distance variable, within a certain range so it is capped at a min and max distance.

Upon testing this, especially with the painting mechanism I am implementing (since the 3D cursor serves as the brush for painting), I realized it would help a lot to be able to have the cursor "stick" to nearby surfaces, essentially detecting anything in its path within a certain radius. Without this function, you would really have to guess where invisible, paintable objects are by just clicking around aimlessly in world space.

On every tick, I've made this the first thing that the Player Controller class checks. This makes it so, if there is an object hit under the mouse, it takes precedence over the default free-floating cursor. The node Get Hit Result Under Cursor by Channel checks for and returns any object hits under the mouse cursor. 

cursor_hit.png

The most basic way to implement this is to then simply set the 3D cursor location to that exact resulting hit location. However, getting a hit under the mouse doesn't account for how far that hit away is from the 3D cursor location in world space. This makes it difficult to break your 3D cursor away from objects that your cursor may unintentionally be on top that are far away, due to perspective. My solution is to get the world location of the character and find its distance to the hit location. If the distance is greater than the user-specified threshold, then the cursor location defaults to the free-floating 3D cursor.

Additional things I'd like to implement here are some more visual indicators - perhaps when your cursor sticks to an object that is invisible, I could add a decal to the surface of that object.

There isn't much more that goes on in the Character class, but it does need to be set to the cursor location. You can grab the Cursor Location variable from the Player Controller class by getting the Player Controller and casting it to your specific Player Controller class.

I've used a VInterp To node to interpolate from the current location of the character to the current cursor location. This makes for much smoother movement, and adds the effect of the cursor following/lagging behind where you are mousing to. My character also has ribbon particles trailing behind it, so this allows the particles to trail for a bit longer.

DESIGN

Here is a very rough first pass of my idea for a look for the character that I have been testing with. I imagined it as a light orb because I feel like this is reminiscent of neurons, synapses, and electrical brain activity. I also wanted its look to allude to the dominant painting theme that defines both the visual style and interactive function in this dreamscape world.

The Character Blueprint contains a Particle System that moves along with the location of the character meshes. This Particle System uses the Ribbon Type Data to emit as a ribbon trail. It is emitted from the sphere character mesh by using the Bone/Socket Location module in Cascade to hook it up to the joint of the character skeletal mesh.

This example of the character moving on the left shows a (slightly choppy due to the screenshot capturing process) example of how the particles follow behind. 

Ribbon particles apply materials such that the leftmost part of the texture is towards the start of the ribbon trail and the rightmost side of the texture is towards the end of the trail. So, I've used a painted brush stroke texture as the opacity map, but I've blended it with a linear gradient that becomes more black towards the right side of the opacity mask. This makes it so the particle trail dwindles towards the end of the trail. (See right.)

 

VOLUMETRIC PAINT SYSTEM

My project idea started with the idea of painting in world space to reveal different worlds/objects. This is a concept I want to carry through to my final product. Early on in my progress, I had done some testing with Unreal Engine 4's Canvas Render Target 2D Blueprints. Due to a couple of issues I encountered with them, this testing fell by the wayside for a while -- but after resolving these

This is my progress so far in implementing this feature. As of now, it is working pretty well to my liking, though there is a lot of potential to explore here, as this is just a test with a simple sphere. 

 

 

 

Here are some features I've added this week:

  • Left mouse button applies paint.
  • When applied, paint stays permanent.
  • Objects can have paint only partially applied/be revealed in parts.
  • Paint can be cleared completely (mostly for testing purposes at the moment.)
  • Brush size is adjustable.

In the near future, I'd also like to add the ability to erase paint in parts. Decals on the surface of the object where you are about to paint would also serve as a nice visual indicator.

In a few releases of Unreal, the Canvas Render Target 2D was causing crashes, but this has thankfully now been resolved, which allowed me to continue working with them.

I also realized that the Create Canvas Render Target 2D node cannot be created within a Construction Script. Doing so resulted in an error that prevented me from saving the map, due to graphs being linked to objects in an external map. Instead, create it on Event Begin Play and the problem is solved.

MORPHSCAPE: Preview cinematic and thesis summary

This week I've made a cinematic to summarize and showcase my work on my senior thesis project so far.

At the beginning of this project, I set out to create a real-time dreamscape environment as a means of achieving two major goals in my work. The first, to build an environment and assets equipped with the parameters to enable visual transformations before your eyes; and second, to develop a custom painted aesthetic.

This aesthetic I have developed relates the spirit of impressionist painting conceptually to the transience of form in dreams -- how objects are ever shifting, becoming other things entirely as your brain assigns them new identities. With loose, expressive paint strokes breaking up the silhouettes of models, my goal has been to suggest the vague impressions of our experiences that appear in our dreams. What exists is just a malleable impression at the whim of your subconscious mind. Impressionist painting can be considered a study in the way we perceive. I find that this relates to the way our brain "sees" and records visual information to memory. We recall that condensed perception in our dreams and memories.

From a technical standpoint, working in 3D graphics as opposed to painting requires much more overhead to overcome the default look of the tools and be able to demonstrate a personal style. This is an area that has been very important to me in my work, to avoid letting the medium I use dictate my vision for a project.

One of the main issues I've addressed has been the hard-edged borders of models and the seams between intersecting models. This is an area where 3D can really reveal itself as 3D, as opposed to a painting, where strokes are able to make form fluid and seamless. The hard-edged 3D model contradicts the idea of malleable form, because it exists as a very concrete, defined form. To me, this breaks the dream-like illusion, as our dreams aren't always so clear.

While rendering progresses in terms of achieving greater and greater photo realism, making a material interact with light as a painter might depict light still stands as a challenge. In my materials, I wanted to regain some of the expressive, happy accidents that can give a painting so much life and character. Using a technique based in meshes and materials as opposed to a post process gives this look a more dimensional quality that I wanted for this project, rather than looking like an additional layer on top of everything.

Another goal of mine over the course of this year has been working with the idea of "smart assets." The development of systems like Unreal Engine 4's Blueprints have made the gap between art and programming smaller than ever. Every asset I have created for this project is thoroughly parameterized, with parameters communicating between materials and Blueprints. Building assets to be "smart" allows for more functional assets but also more flexible assets in a production pipeline to accommodate rapid visual iterations on the same asset. My semi-procedural spline tree system built in Blueprints is a testament to the flexibility that the overhead of setting up a smart asset yields.

Looking ahead, I'd like to realize my vision of making this environment explorable and interactive. The trees will sprout up from nothing, and the rock formations that line the shore will melt away into the ocean. My vision is that this will be triggered by user input rather than simply distance from an object. A character and music will help to tie everything together into a cohesive, experiential dream playground.

MORPHSCAPE: WEEK 9

Over my spring break, I went on a week long trip to the Southwest with the College of Imaging Arts and Sciences honors group at Rochester Institute of Technology. Seeing so many unique rock formations was enlightening and a great source of appropriately timed inspiration and reference. This past week I've been dipping my toes back into my project with some newfound perspective.

Here are several things I've worked on this week, or will be working on in the near future:

  • Populating the environment. Although minimalism can sometimes feel dream-like, the map I've been working with has primarily been a test scene. I'd like to give this shoreline area a sense of space and add more rocks to accompany the main rock arch I've been testing with.
  • Pushing the painted visual style to be even more loose and expressive. I've started to do that this week just a bit with the rock, but I'll be continuing to work on this for the ground, the ocean waves, and the boat.

Recently I rediscovered my love for JMW Turner's paintings. His ability to effectively capture and communicate a scene or a moment with such expressive brush strokes and colors is something that's really inspiring to me, and my vision for this piece. I admire the way solid objects seem to dematerialize into atmosphere -- that fluidity of matter is something I'd like to achieve in my work as well. With all of his seaside themed paintings, it's a wonder I haven't been looking to these paintings all along! But it's never too late for some fresh inspiration.

  • Melting rocks. My vision for the rocks on the beach was that they represent hardened, fossilized memories. Your memories from your conscious brain are what dictate the things your subconscious dreams of. As the player, you can melt them to access these memories and fuel your dreams, creating more of the environment around you. I hope to have the rocks melt and form the ocean that the boat materializes onto. I've been thinking of this as a mechanism that will tie the interactive environment together into a cohesive, self-contained dream playground.

Here's a test I did with flow maps for the wax melting.  This setup involves a normal and height map's UVs panning by a 2D flow map I've painted. It is revealed by a linear gradient mask being pushed down. The mask is blended with the flow mapped height map and a noise to break up the linear quality.

What works pretty simply on a cylinder is a little bit more difficult on a complex model. So I've been spending some time trying to work out a method to get the wax flowing along the surface angle of the rock arch model seamlessly. I've painted a world space flow map using Mari's Vector Brush tool, and I'm using that as a flow map to modify the projected UVs of my flow texture.

 

  • Materialization effects. I want to create the feeling that the environment is forming organically as you continue to walk, or dream. Here is an example of the material effect I've been working on where the rock arch is materializing as you walk forward.

 

  • A sky. Here's something I've neglected so far! A sky can be quite expressive and contribute a lot to the mood of an environment. I've started to set one up, though this still has a ways to go. My idea is to use flow maps to create clouds that move in a streaky, swirling pattern. As you near the boat, I'd like this sky to darken and become even more tumultuous. 
  • Refinements. This week I tweaked the boat's buoyancy and added a rock to it so it isn't a straight up and down bob. This is done by simply modifying the rotation of the meshes within the boat Blueprint on a sine wave over time.

 

 

  • Procedural spline tree system, animated to enable growing trees. I hadn't worked with it in a while, but last semester I spent quite a bit of my time developing a procedural spline tree system, which is based around Unreal Engine 4's spline and spline mesh actors and uses a recursive method in a Blueprint for generating branches. The idea that motivated me to develop this was to enable trees to grow around you dynamically.

This week I've started to figure out my method for animating the components of the spline tree. My idea is to have an auxiliary spline that serves as a motion path curve that the spline points of each component can follow. Their movement, or distance along this motion path spline would be determined by a Timeline in the level blueprint, so that its movement can ease and not be completely linear.

This works well for the trunk so far. The issue with the primary branches and all of the following tiers will be maintaining the relative position of the child branch on its parent branch as they both animate. Additionally, a custom set motion path for every single branch would be a ridiculous amount of setup and user control, so I hope to automate the motion paths for the recursive, procedurally generated branches.

Here are some lower priority, but still important tasks I've been considering:

A character. I've been imagining this environment being explored in third-person, perhaps by a flying light orb character with a ribbon-like light trail.

Music. An ambient soundscape would set a nice, quiet mood for this environment.

 

 

 

 

MORPHSCAPE: Sand material blending & hooking up the dream sequence

I've been working with the sand material, which includes a blending of sand, rocks, small pebbles that blend in around the larger rocks, and water. With all of these components coming together in one material, I've had to do quite a bit of tidying up and figuring out the best organization for blending these all together. The rocks and water can be blended in by painting different Vertex Colors on the mesh. I break up the Vertex Color mask by blending in the height maps of each material. This ground material is also set up with displacement. I'm enjoying the way that these different components can be combined (for example, I can have the sand with rocks blending in and water blending over top.)

I've also been hooking up all the parameters to change throughout the boat deterioration sequence. This includes the boat's animated mask revealing rust on all of its parts, the buoyancy height displacing the boat changing, the wave height and speed of the water changing, and the shoreline mask clamping in, overtaking the water. All of these parameters are based on your distance to the boat, which is mapped 0-1. Each parameter has a float curve on a Timeline in the Level Blueprint, which maps its overall progression from 0-1 throughout the sequence. (For example, the wave height may grow as you come closer to the boat, but then decrease again as the shoreline swallows the ocean.) Each parameter has the true, user-defined min and max value, and the 0-1 from the timeline is remapped to that range. Then, the corresponding Material Parameter is set.

I'll be updating this soon with a newer video that shows the water waves growing in height and the sand surging towards the shoreline, but here is the boat sequence as of now!

CONCEPT

A little bit on some developments I've made in terms of the concept that has been motivating my design. My idea is that, metaphorically speaking, your everyday experiences and memories are hardened and fossilized into rocks by your conscious mind. While you're dreaming, your subconscious brain accesses all of your memories. You can melt the rocks, which reveals these memories to your subconscious - you bring light to them and they become fluid again. I'd like the environment to be fairly empty at first, and as you melt rocks, accessing your memories, the environment will materialize - your subconscious brain is forming new dream memories from your collective experiences. Although your subconscious mind often accesses your permanent, fossilized memories while you dream, dreams themselves are very impermanent and hard to remember after waking. This is why many of my visuals deal with the ephemeral, like the boat deteriorating away and the constant flux of the environment. Sand also relates to this idea, as it is composed of a seemingly infinite amount of eroded rocks. It is the dust of time, constantly blowing around and never settling permanently.

MORPHSCAPE: Particle paint

Today I was inspired by this article that outlines an approach to achieve a non-photorealistic, painterly style. The idea is that particles are placed on the surface of a mesh with brush stroke shaders applied. There's not really any reason this can't be translated into Unreal, so I figured I'd give it a shot.

My original idea months ago was similar to this method, except I was scripting the placement of plane meshes onto the surface of a mesh using Blueprints. The problem was, this was only feasible to calculate with simple shapes like cylinders.

Particles in Unreal's Cascade have a module called Skeletal Mesh/Surface Location (read Unreal's documentation here) that allow particles to be emitted from the surface of a mesh -- though as far as I know, this is only works with Skeletal Meshes.

I've made a quick illustration demonstrating the two methods I've been working with to achieve this painted style, and each of their features/disadvantages that I'm aware of.

The first method is the one I've set up with all of my assets so far. It has been pretty faithful but there are some drawbacks. This idea was loosely based on this video I saw a while back, which essentially displaces the mesh several times to produce an imprecise, painted silhouette.

Here are some of the results I've gotten working with this today. Right now the material is just using one brush stroke opacity mask and the color is assigned based on the vertex normal, just to see it working with a bit of color variety. It's interesting to see it working and I'm looking forward to investigating some settings more thoroughly to see what kind of results I might be able to get on different sorts of meshes.

This method is also interesting because there are a lot of parameters I can play with on an individual stroke basis. Here's a sheet of brush stroke types that I can use as masks for each particle plane. In this way, I can easily vary the brush stroke type by sliding the UVs around this sheet. This sheet would be shared by all assets and different assets might use different stroke types. (Leaves might use something more loose while rocks might use something blockier and more chiseled.)

I've begun to test this method on more complicated meshes, and there are some issues - for example, some areas of the mesh I might not want brush strokes to generate on. More strokes are going to generate where the mesh is most dense. I might be able to control this density by making another version of the base mesh that will be bound to the same position in a Blueprint, but hidden. Additionally, this mesh would be a Skeletal Mesh to allow the particles to emit from the surface location, while the base mesh could remain a Static Mesh.

I would say that the main disadvantage of the "Mesh Shell Layer" method I've been using is that there's quite a bit of set up involved for the Blueprints for each asset. It relies heavily on communication between the asset's Blueprint and the material parameters per dynamic material instance assigned to each mesh shell instance. Creating instances of these mesh shell layers, assigning a new dynamic material instance per layer, and modifying the material parameters per layer becomes a hefty operation. I've got this consolidated into a function, but the issue is that this function can't easily be shared between each asset's Blueprint. A solution might be to put this function into a Blueprint Function Library that can be used across multiple Blueprints without having to copy and paste the whole graph, but since the function deals with adding Static Mesh components to the world dynamically, it can't be done in a Function Library function.

The mesh shell layer method might also prove a bit more intensive than the particle method. This could be improved by making the mesh shells even lower res versions of the base mesh. I recently had difficulty with this on the rock asset. The rock's base material was tessellated to allow for displacement with the wax. Each mesh shell layer would normally be assigned an instance of this base material. But there is no material override to turn off tessellation, so an entirely new material without tessellation would have to be made. Even though a lot of duplicate information can be consolidated into a Material Function, syncing up these two materials proves to be a pretty tedious process.

Overall though, both methods have their advantages. The mesh shell layer approach works well on the boat because each layer adds a painterly thickness to the thin-walled boat mesh. Each layer can have varying edge texture, be multiplied by a different color, and the holes eating through the structure of the boat can be clamped differently.

On the other hand, the particle method would work better for something with a more organic silhouette, like tree leaves.
The ability of particles to move may also come in handy for some effects as well. Thinking back on the boat rusting sequence, it would be a great in addition to the mesh shells to create the illusion of parts of the boat crumbling away as it deteriorates. I like that by working on the level of individual brush strokes, it gives a bit more visual variety and is more akin to the process of actual painting.

MORPHSCAPE: Boat sequence updates and looking forward

Revisiting the rust material and the formation of rust on the boat hull, I realized it wasn't very believable. I observed a lot of rust on metallic surfaces and realized that my rust was lacking a differentiation between the areas of rust flaking and the areas of rust dripping. The rust flaking areas often appear to be a deeper red orange, and the areas of dripping rust are lighter orange and fade off softly.

An updated screenshot of the rusting boat hull.

The boat's skeleton is slightly improved from before, a little bit less fantasy. I tried to make them appear more fragile, deformed, and decaying. One of the main problems with the previous skeleton iteration was that it didn't follow the form of the hull very well, and it looked more like a skeleton inside of a boat than a skeleton underlying the structure of the boat.   

I added the wheelhouse structure and applied the same rusting effect to it by painting keyframes for its rusting and deterioration pattern.

This is an early preview. The materials aren't well defined on the wheelhouse yet and I plan to add some further model-specific detail maps like AO, curvature, and normal.

 

I've been starting to work with the environmental effects for this sequence. My idea is that, as you approach the boat, the sand will start to ripple in front of you in waves that surge forward and "swallow" the water that the boat is floating on. The first component of this is to displace the sand, using variations of a sine wave. I have the sand rippling at the moment, but I'll be working on making it more directed and less random.

Details on the process so far:

I've made a lot of progress in terms of organizing my graphs. The boat is one Blueprint, BP_Boat, that initializes all of the mesh layers that make up the boat (for the painted effect) and assigns dynamic material instances to each mesh layer. This can be simply dragged into the editor and should automatically rust away based on distance, as well as connect with other relevant materials in the sequence.

BP_Boat sets material parameters in all of these dynamic material instances. Different components of the transition are determined by different curves. The wheelhouse deteriorates at a faster rate than the hull, and so I've set up different curves to control their rate (i.e. 60% through the transition, the wheelhouse has completely deteriorated.) The water drying up may be another curve, or the increasing size of the waves/sand ripples throughout the transition.

One of the main material parameters that BP_Boat sets is the Rust Transition, a 0-1 float value determined by the player's distance from the boat. This essentially flips through the hand-painted mask keyframes to animate the boat rusting and deteriorating. Mask keyframes are scalar and so they are stored in each of the RGBA channels to conserve texture space within the shader. I've painted the mask keyframes so that black on the mask is solid, 0 - .5 is where rust begins to form, and .5 - 1 is deterioration, contributing to the opacity mask for the material.

The rust drips are formed by shifting the UVs for the mask downwards and lerping it over itself incrementally several times. This gives a somewhat soft blend directionally downwards. Using this mask, I blend in a lighter rust material. In the standard mask, I blend in a deeper, flakier rust material. The masks, which originally are soft gradients, are blended with different noises - the flakes blend in a cell pattern at the edges of the rusted areas, where the drips use a more directional vertical noise. These are all blended in the UE4 shader so tiling can be adjusted, etc.

Another example of material parameter that BP_Boat sets is a mask based on which mesh layer the material instance corresponds to. This allows for effects like a color multiplier per mesh layer, or a UV offset per mesh layer. Connecting the various materials back to a centralized Blueprint has proven very helpful in directing all of the parameters for this sequence.

Some things left to do:

-Refine the wheelhouse materials, add glass to windows
-Replace placeholder sand and water materials
-Refine sand and water wave displacement
-Natural displacement of boat along with sand/water waves
-Sand particles blowing
-Skydome

I'm looking to finish the majority of this in the next couple of weeks, but some of these tasks might take a backburner for a bit as I start venturing into my idea for my next sequence.

Wax rock beach:

Over the past few weeks I've devised a much clearer plan and scope for this project that I feel like encompasses a lot of the visuals and moods I wanted to create. Essentially, the project is one transforming dreamscape environment. The primary environment is a beach with large, arching rock formations. The color palette will be desaturated and cool initially. If you approach the rocks, they will start to melt into wax wherever you aim, and the melting will spread throughout the rock from there. The color palette will shift to warmer as the wax melts into an amber-like color. The hero element in this environment will be a large rock formation that looks like a candle. My rough idea is that the candle wax rocks will all melt down into a pool of water, which is where you'll find the boat.

I'm interested in incorporating morph targets to actually melt these rock meshes down. The great news is that Canvas Render Target 2D has been fixed and is no longer crashing in UE 4.7, so I'll be continuing to work on my world canvas mask painting to material blend to the wax material wherever the player aims. The wax material will be much more translucent and use flow maps and displace areas achieve the appearance of wax dripping.

A broad summary of the tasks for this sequence:

-Create rock mesh pieces
-Create hero candle rock formation
-Create materials for rock and wax
-Implement morph targets for candle rock meshes
-Implement canvas render target 2D masking for material blending
-Transition into boat sequence

Forest:

After approaching the boat, leaving only bones, vines will grow up the bones and grass will start to grow and overtake the sand. Trees will sprout up alongside the path you walk, and increasingly grow larger, with branches looming over you.

I've already started to develop a system within my tree Blueprint to animate the primary branches and the trunks. By developing this, I'll hopefully be able to "grow" the trees around the player.

Some known stuff I'll have to do:

-Hopefully add curved tangents on branches
-Add leaves to trees
-Implement tree animating system
-Add vine growth to bone material OR add a spline-based vine