Hawkins Heroes 3D Asset Optimization Pipeline

The following is a partial reproduction of the documentation provided to our client summarizing the asset pipeline techniques we developed for translating their VFX assets from Stranger Things to game-ready assets in Unity.

The high-level VFX assets of the Demobat creature were an ideal candidate for demonstrating an optimization process including retopology and rigging for in-game animation.

The assets provided were of a very high level of detail (LOD)—great for visual effects but too resource-heavy to use in-game when duplicated many times in the “Demobat Assault” phase of gameplay. Benchmark testing on the iPad Air (5th Generation / M1 SoC) proved that we could display the model at full detail, but only in isolation. Placing it in an environment of our target quality resulted in unacceptable performance with more than one instance on screen at a time.

We were provided with four models of the Demobat, one with a lower LOD (~150,000 triangles), and three high-LOD models with variations in the edges of their wings. All files were in native Autodesk Maya format, with conversions to Alembic for interchange with other software packages.

This file was chosen as a starting point for optimization because it had the most complete geometry and useful information for later baking into optimized textures. The lower-LOD model was still too high-poly to use in the quantities we wanted, and was missing the detailed normals information that this one had.

All models did include texture information, tiled into 78 UDIMs. These were discarded, as it would be impossible to match new topology to the existing UV maps and optimization would require a reduction to a single UV tile.

A first attempt at optimization was simply applying a decimation modifier on the imported geometry with 2 iterations of un-subdivision to preserve quad edge flow. This did reduce the polygon count to a useful 22,000 triangles, but more complex areas did not deform well during animation tests.

This decimated mesh was used to project newly drawn topology onto for the next step.

To achieve a reasonable polygon count (~20,000 triangles) with topological flow that is easily deformable during animation, a new mesh was roughly drawn on top of the original mesh and smoothed with a Catmull-Clark subdivision modifier.

Finer details present in the original high-LOD model were preserved by projecting the normals of that mesh onto the simplified model. This process reduces smaller pieces of geometry into texture data that is used to influence lighting in the game engine, but does not contribute to the memory footprint or animation deformation of the mesh itself.

The smoothly retopologized mesh was then attached to an animation rig with bones for the major wing and body segments, omitting some of the finer details like mouth and claws that were not included in the simplified mesh. This allowed us to create multiple animation tracks, including a wing-flapping flight loop, attack movements, and a pose for the bat to transition to when killed by the player.

Adopt USD Workflows

The Virtual Production projects made with Unreal Engine contain a good number of assets, and could be useful in the specific case that a game is developed with the same engine; mesh, material and environment data is all transferable to a new project or level. These were of limited use to us, though, as our project was built in Unity.

The Universal Scene Description file format is intended to increase compatibility across multiple software packages by storing information in a neutral format while still allowing for interactive use. Unreal Engine supports the use of USD directly as an alternative to its internal formats. This would instantly increase the flexibility of assets used in virtual production. Similarly, using the format in other VFX would increase interoperability between assets used for offline rendering and those used in real-time effects.

Preserve Surfacing Art

Surface design elements of VFX assets, including texture sets and shaders used by rendering engines, were not available for most of the assets we encountered. For those assets that did include texture sets, they were formatted in a way that was not useful in the game engine.

Including the working art assets for surface design—for example, an Adobe Substance project—would streamline the process of adapting high-resolution, complex textures and shaders for use in-game.