I've been playing around with materials to keep working at achieving the painted style I want for this project. Although I enjoy the look of hand-painted textures, my goal is to take this style beyond the typical flatly painted look by making it a convincing part of a 3D world with dynamic lighting. Particularly, I've been trying to create custom painted looking normal maps to accomplish this.
In my attempt to sculpt normal maps, I still hadn't been getting the look I wanted. Normal maps seemed to kill any painted illusion and overwhelmed my hand painted textures. Also, the chunkier style of my sculpts conflicted a bit with the more textural quality of my paintings. While revising the material for my tree bark, I made a bit of a breakthrough in my method.
These images are of the full tree with a modified, painted looking normal map. These also have the additional opacity mapped shell of the mesh over top, adding the textural edges on the silhouette.
You can see the effect a bit clearer in the image on the right. Two instances of my sculpted, tiling bark normal map texture are each warped slightly differently by noise patterns, and blended over top of each other by a dithered mask. This resulting warped normal map is blended back over top the original normal very slightly to only get a slight warped and dithered effect. It adds a slight pixel-level textural effect to the normal map.
Then, I step the values of the normal map so that the transitions between different depths and angles are sharper and less gradiented, sort of like a posterization or a cel shading effect, but on the normal map. This helps in getting a more faceted look that I just couldn't achieve in my original sculpt.
It was upsetting to see at first that the effect of this normal map wasn't really coming through once my original painted diffuse color was applied, even with a really direct point light shining on the surface.
Something simpler works a bit better to show the normal map. Although I could work to get a bit more slight hue variations in this map, it's a good base. With the painted quality integrated into the normal map, it looks so much more well integrated into a 3D world.
I spent the majority of this week working with materials to find this painted style (18 hours.) I also completely revised and reorganized my tree Blueprint (13 hours) and some things still need to be done to unbreak some things I broke in the process. I spent a bit of time with post process materials to soften the edges of objects (5 hours.) Lastly, I spent a lot of time trying to refine the look of my boat by painting model-specific detail maps, only to find that so far, I still like my iteration from last week better (12 hours.) Also, the development in my process with the normal maps makes the work I was doing on the look of the boat a bit unusable now, so I'll be working next week on applying this method to the normal maps I'll be creating for the boat.