Paint a blocked-out scene with your image model

An assistant can't paint, but it can lay a scene out exactly: where everything is, how big, what moves and what reacts. An image model can paint, but can't hold a layout or a timeline. So your assistant blocks the scene out with plain shapes and animates it (a greybox), your image model paints over a picture of it, and Polymation cuts the painting back onto the same objects. Every keyframe and interaction now drives painted art.

You bring
An idea of the scene and the look, and your own image model (ChatGPT, Gemini or any that takes an attached picture). For 3D, also an image-to-3D generator (Tripo, Meshy, Hunyuan3D, TRELLIS).
You get
2D: a Rive file whose animation and interactions drive the painting. 3D: a Spline scene with your generated models in place.
Needs
Nothing beyond the install.

Ask your assistant

  • Make a cosy watercolour storybook village at night: a lit cottage, an oak tree swaying, a fox blinking. Block it out first, then give me the brief for my image model.
  • Here's the painting (painted.png). Put it on the greybox.
  • Make a guide video of the greybox's fly animation for my video model. Here's the painted clip: put it back on the scene.
  • Circle the 3D greybox so my 3D generator gets several views of each thing.
  • Build my game's main menu as a greybox with working buttons, then have it painted as clean flat vector UI in pastels.
  • A small 3D town square in Spline: block it out and tell me what to generate for each model.

In 2D

  1. Block it out. An artboard the size it will play at; a full-size rectangle for the setting ("sky", "kitchen"); then one shape or group per thing, named for what it is ("oak_tree", "cottage", "fox"), each in roughly its own colour. A piece that moves on its own (a swinging door, a waving arm, a blinking eye) gets its own name. Text stays text: it is left out of the painting and stays live, sharp and bindable.
  2. Animate and wire it as the finished piece should be. The painted result keeps all of it.
  3. The brief. greybox_brief renders every object's exact outline in Rive's own runtime and writes the guide picture and the prompt, in the style you describe. It also returns the guide with every object named over it, to check nothing merged.
  4. You paint it. Attach the guide in your image model, use the prompt as written, and save the result as painted.png. Only if the brief asks: a short extra picture for something partly hidden behind a thing that moves.
  5. Put the painting on. greybox_apply_art fits the painting to the guide (models shift and rescale a little), cuts each object out from its known outline while keeping what the painting added at its edges, fills in what nearer things hid, and puts each object's art on its own node. It returns the painting beside the result at rest.
  6. See it move with preview_riv_gif, then save it, or share a page with export_web_player.

Paint it as a video

Video-to-video models (Runway Aleph, Luma Modify Video, Wan VACE) repaint a whole clip while keeping its motion. Polymation gives them the greybox playing, then puts the painted clip back.

  1. greybox_brief with video renders the greybox's animation (or a run of its state machine) as guide.mp4, with its first frame as guide-first.png and a prompt that asks to keep the motion, camera, timing and layout. It knows which object shows at every pixel of every frame. Five to ten seconds suits most models.
  2. You paint it in your video model, the painted still as the style reference if you made one.
  3. greybox_apply_art with clip fits the clip to the guide once, then cuts each object from every frame:
    • Each thing that moves gets its painted frames as a flipbook on its own node. They are carried back to where it rests, so its own keys still move it, and its state machine and listeners keep working.
    • Things that stay still get one clean picture (or the still painting, if you give it).
    • Text stays live.
    • Ask for flipbookPath to also get the whole clip as a plain flipbook for a cutscene.

In 3D

  1. Block it out in 3D with generate_spline_scene: one shape or group per thing, named for what it is. Things that repeat are numbered (house_1, house_2) and share one model. A greybox from Blender works too, exported as .glb.
  2. greybox_brief writes the guide from the camera you choose, and a prompt per kind of thing.
  3. You paint the guide, then one picture of each kind of thing in the same style.
  4. greybox_apply_art writes each kind's picture on a transparent background. You turn each one into a textured .glb in your image-to-3D generator.
  5. Views from every side: with video, the guide clip circles the scene (or plays its own motion). After your video model paints it, greybox_apply_art with clip writes up to four pictures of each kind of thing from different angles. These are for generators that take several views (Meshy Multi-Image to 3D, Tripo multiview, Rodin, Hunyuan3D-2mv, TRELLIS multi-image).
  6. greybox_place_models places each model where the greybox had that thing: sized to its box, centred and standing on its floor, keeping its name, motion and interactions. Flat things like the ground take the painting's colours.

Good to know

  • Names reach the prompt, so a clear name gets a better painting. Two neighbours in the same grey can merge; give each its own colour.
  • One painting covers the whole scene. To change only the motion later, edit the greybox's animation: the layout at rest is the same, so the same painting fits. Moving or resizing things at rest needs a new painting.
  • A painted picture can't follow a key on a shape's colour, size or outline. Keep that piece as a shape, or animate its position, turn, scale and opacity instead. The notes say when this happens.
  • If one object comes out with a low match, the model moved or reshaped it: ask the model again with the same guide to keep every thing exactly where the shapes are.

Related