June 21, 2023

Process

• Blender 3D → render looping animation as image sequence

• Auto1111 → used the rendered image sequence as a base layer to apply image-to-image diffusion models using natural language prompts (and negative prompts), and traversing seed values in the vast latent space

Remarks:

Generative AI used hand-in-hand with 3D animation tools feels like strapping an alien jetpack to my creative toolkit. I’ve found it inspiring to be able to draw upon not only my knowledge of design and art history at a high level, but also on a more specific level with technical terminology and more specific process-oriented vocabulary. Negative prompting is such an interesting capability as well. I find it quite silly/lovely that you actually will get a better output by using such simple negative prompts as “bad” or “amateur.”

Interdimensional photography… There is some level of control provided through a system of various sliders, knobs, check boxes, scripts, etc., and one develops an intuition for the ranges of these variables over time, along with how they respond as an interconnected symphony of feedback layers. The latent space is also quite non-uniform [based on my traversals thus far], where there will be some highly dynamic hot spots that are very sensitive to small changes in variable values (or ranges of values), and even small variations can yield wildly different outputs. At the same time, these small areas of dense interest are often spread out between vast areas of somewhat featureless space, where certain variable ranges will barely impact the output. It’s punctuated equilibrium all the way down.

While the tools can operate unpredictably at times, it’s undeniably impressive how fast it can output high-quality imagery with endless variations. This mind-blowing speed of creation and variation amplifies and integrates with existing skillsets, where creators can expand on their source material with swift realization of complex visions, which is magnified by one’s ability to use descriptive language and reference prior art to create something new. With this synergy, the journey from concept to creation is not just accelerated but also enriched, allowing for rapid experimentation and iteration, thus elevating the potential for innovation in visual storytelling.

While the AI-generated animation shown here lacks temporal visual smoothness, each frame of the video is its own beautiful image— all uniquely different in structure, but connected in style. Below is a selection of just 36 of the 600 images I rendered out in just a couple hours. (click to enlarge)