This past year at GDC 2017, there were a few talks on the use of vertex shaders to provide movement. While not a new stunning revelation or paradigm shift, it does mark the beginning of an interesting shift of thinking – that vertex information can be used to store data that’s more than just information for blending textures or tinting the mesh.

But painting that information be fairly boring or error prone since it’s hard to visualize. Here’s a sample Python script that can help paint vertex position in worldSpace / local space from Maya

import pymel.core as pm

obj = pm.ls(sl=True)[0]
vtxPosition = []

for x in obj.vtx:
    worldPos = pm.xform(x, q=True, t=True, ws=True)
    pm.select(x)
    pm.polyColorPerVertex (rgb = (worldPos[0], worldPos[1], worldPos[2]))

So fairly simple – iterate through each vertex in a meshShape, grab it’s position in worldSpace (or omit the ws flag to do it in object space).

We can observe that the vertex information is correctly painted by looking at the Component editor in and selecting a few vertices and simply turning on vertex colors in the viewport. Neat!

You may notice that the color values go into negatives (as expected) and if you work with a large mesh that has vertices beyond translation values of (1,1,1) that the mesh starts getting painted weirdly, but the data is still correct. However, your game engine may look at vertex color information as RGB8 and you’ll need to remap the values to fit.  The following adjustment will now remap based on the bounding box of the mesh  to (0,1) with a fairly basic adjustment to the script.

def remapValue(originalValue, oldMin, oldMax, newMin, newMax):

    oldRange = (oldMax - oldMin)
    newRange = (newMax - newMin)
    newValue = (((originalValue - oldMin) * newRange) / oldRange) + newMin
    return newValue


import pymel.core as pm

obj = pm.ls(sl=True)[0]
boundingBox = pm.polyEvaluate(b=True, ae=True)

vtxPosition = []

for x in obj.vtx[:]:
    originalPos = pm.xform(x, q=True, t=True)
    newPos = [
        remapValue(originalPos[0], boundingBox[0][0], boundingBox[0][1], 0, 1), 
        remapValue(originalPos[1], boundingBox[1][0], boundingBox[1][1], 0, 1), 
        remapValue(originalPos[2], boundingBox[2][0], boundingBox[2][1], 0, 1)]
    pm.select(x)    
    pm.polyColorPerVertex (rgb = (newPos[0], newPos[1], newPos[2]))

This results in a paler looking mesh, but now with better values to work with. One use for this could include painting gradients on tree trunks to anchor them with a black value at the base as a vertex shader is played for animation.