Baking information into vertex color

This past year at GDC 2017, there were a few talks on the use of vertex shaders to provide movement. While not a new stunning revelation or paradigm shift, it does mark the beginning of an interesting shift of thinking – that vertex information can be used to store data that’s more than just information for blending textures or tinting the mesh.

But painting that information be fairly boring or error prone since it’s hard to visualize. Here’s a sample Python script that can help paint vertex position in worldSpace / local space from Maya

import pymel.core as pm

obj = pm.ls(sl=True)[0]
vtxPosition = []

for x in obj.vtx:
    worldPos = pm.xform(x, q=True, t=True, ws=True)
    pm.select(x)
    pm.polyColorPerVertex (rgb = (worldPos[0], worldPos[1], worldPos[2]))

So fairly simple – iterate through each vertex in a meshShape, grab it’s position in worldSpace (or omit the ws flag to do it in object space).

We can observe that the vertex information is correctly painted by looking at the Component editor in and selecting a few vertices and simply turning on vertex colors in the viewport. Neat!

You may notice that the color values go into negatives (as expected) and if you work with a large mesh that has vertices beyond translation values of (1,1,1) that the mesh starts getting painted weirdly, but the data is still correct. However, your game engine may look at vertex color information as RGB8 and you’ll need to remap the values to fit.  The following adjustment will now remap based on the bounding box of the mesh  to (0,1) with a fairly basic adjustment to the script.

def remapValue(originalValue, oldMin, oldMax, newMin, newMax):

    oldRange = (oldMax - oldMin)
    newRange = (newMax - newMin)
    newValue = (((originalValue - oldMin) * newRange) / oldRange) + newMin
    return newValue


import pymel.core as pm

obj = pm.ls(sl=True)[0]
boundingBox = pm.polyEvaluate(b=True, ae=True)

vtxPosition = []

for x in obj.vtx[:]:
    originalPos = pm.xform(x, q=True, t=True)
    newPos = [
        remapValue(originalPos[0], boundingBox[0][0], boundingBox[0][1], 0, 1), 
        remapValue(originalPos[1], boundingBox[1][0], boundingBox[1][1], 0, 1), 
        remapValue(originalPos[2], boundingBox[2][0], boundingBox[2][1], 0, 1)]
    pm.select(x)    
    pm.polyColorPerVertex (rgb = (newPos[0], newPos[1], newPos[2]))

This results in a paler looking mesh, but now with better values to work with. One use for this could include painting gradients on tree trunks to anchor them with a black value at the base as a vertex shader is played for animation.

Python and images

During mid-development of Batman and Walking Dead Season 3,  an artist requested the ability to strip out alpha from textures without having to necessarily open it up in Photoshop. As it turned out, she noticed that Substance Painter had been exporting images with an alpha channel, regardless of whether or not it was using it. While this has been fixed in more recent releases of Painter, I ended up using part of the script to basically get a better understanding of how assets were being generated by the art department. Keeping track of it helped get ‘easy wins’ when it came down to optimizations, as having an alpha channel in a texture that wasn’t being used just took up more memory.

There are a number of image manipulation / operation modules available for Python – but I found PIL ( https://pypi.python.org/pypi/PIL ) to be fairly straight forward to use when analyzing targas – the texture format of choice at Telltale. Below is a snippet of how easy it is to find textures with an alpha channel and strip them out. At Telltale I ended up passing the file-list to a QListWidget and connected a QPushButton to a function to re-save.

for tga in fileList[:]:
			
			print ("Processing {0}").format(tga)
		 	TGA = Image.open(tga)
		 	if TGA.mode == 'RGBA':
		 		print ("Found Alpha channel")
		 		try:
		 			TGA = TGA.convert('RGB')
		 			TGA.save (tga)
		 			print ("Successfully saved without alpha")
		 		except IOError as e:
		 			print ("Couldn't save - please make sure the file is checked out")
		 		finally: 
		 			pass

Looking at this old code I noticed I unnecessarily use fileList[:]  instead of just fileList . I’m not sure why I used to do this, but it’s definitely not something I do anymore.

A full list of image modes can be found on the PIL docs here: http://effbot.org/imagingbook/concepts.htm#mode , but as a tech artist in games you’ll probably run into L, RGB and RGBA the most.

First posts are always the hardest

Many moons ago, I did some contract work for the now non-existent Maxis team over in Emeryville, California (not to be confused with the Maxis team in Redwood Shores that works on the Sims).

We had an issue with a lot of copy and pasted assets in Maya – as it happens that Maya absolutely loves to append ‘pasted__’ into the namespace of transforms, mesh shapes and almost every node that ends up being pasted into the Maya scene. While Maya has the ability to search and replace names on transforms and mesh shapes in the Maya scene, it did not and still does not have the ability to clean up the hypershade nodes – which can lead to fairly unreadable hypershade, as at some point you’ll have scenes with material names similar to ‘pasted__pasted__pasted__blinn3’.

One of the artists made a script that removed ‘pasted__’ from the material names, but unfortunately only removed the first instance. While I never ended up looking at the script at the time, I imagine he simply wrote something to check the first x characters and remove if it matched. This lead artists into having to run a few times to clean up the hypershade.

When I eventually worked at Telltale, I noticed the art team had similar issues, except it was much worse as the production times for each episode of the game were very short, and a lot of copy and pasting was used – either through the standard Ctrl+C/Ctrl+V or through a custom import script.

I ended up writing the below script to catch all instances of a defined set of characters via regex – as I did not want artists to have to click on buttons multiple times.

# Author : Farhan Noor 7/13/2015
# Hypershade Cleanup
import pymel.core as pm
import re


def renameHypershadeNodes(hypRemove, hypReplace):
    
    hypSelection = pm.ls(type='shadingEngine', mat=True)
    print hypSelection
    for shadeNode in hypSelection:
        if re.search(hypRemove, str(shadeNode.name())):
            print (("Renaming %s to %s") % (
                shadeNode.name(), shadeNode.name().replace(hypRemove, hypReplace)))
            shadeNode.rename(shadeNode.name().replace(hypRemove, hypReplace))

renameHypershadeNodes("pasted__", "")

Note that we did not care about the file texture nodes – only the materials themselves as the names were baked into the mesh export, so this doesn’t really look at any other nodes. You can easily adjust the file texture nodes by adjusting to look for type=’ftn’.