Beyond Vibe Coding: 3D Apps You Can Control

You're prompting your way to a 3D app on Vercel. This is everything you need to direct the build.

01What you'll need

Two things:

• A computer set up for coding locally. If you're on a Mac, here's a quick walkthrough.
• An AI coding agent. Claude Code, Cursor, or Codex — whichever you like. Installed and able to edit files in a project.

The stack tonight: Vite + React for the app, React Three Fiber (R3F) for 3D, and Vercel to ship it.

02The four nouns of a 3D app

Vibe coding works when you name specifics. "Make me a 3D thing" gives you a generic scene. Name these four pieces in your prompt and the AI gives you exactly what you meant.

Scene — the container

Everything 3D lives inside one. In R3F that's a <Canvas>. Say in your prompt: "inside a <Canvas> filling the viewport…"

Camera — what the viewer sees

You get a default perspective camera unless you say otherwise. Say: "a camera at [3, 3, 5] looking at the origin…"

Mesh — every object = shape + look

Each object is a geometry (the shape) plus a material (the look). Say: "a sphere with a standard material in hot pink…"

Lights — without one, silhouettes

A standard material with no light renders black. Always ask for at least two. Say: "an ambient light and a point light at [10, 10, 10]…"

03What your AI will write

Your AI will produce code like this — for every prompt. You don't write it, you read it. When something looks wrong on screen, point at the part that doesn't match what you asked for and tell the AI to fix that.

import { Canvas } from '@react-three/fiber'
import { OrbitControls } from '@react-three/drei'

export default function App() {
return (
  <Canvas style={{ height: '100vh' }}>
    <ambientLight />
    <pointLight position={[10, 10, 10]} />
    <mesh>
      <boxGeometry />
      <meshStandardMaterial color="hotpink" />
    </mesh>
    <OrbitControls />
  </Canvas>
)
}

Once you can read that, most fixes are one sentence to your agent:

• Screen is black? Missing a light. "Add an ambientLight and a pointLight."
• Can't see anything? The camera's inside the mesh. "Move the camera to [3, 3, 5]."
• Can't rotate the view? No OrbitControls. "Import OrbitControls from drei and add it."
• Want it to animate? No useFrame. "Use useFrame to spin the mesh on its Y axis."

04Set up. Seed. Iterate.

Run the first two yourself. Then open your AI agent and drop in the seed prompt. From there, it's all prompting.

1. Scaffold a Vite + React + TS app. Pick a name and create the project:

$npm create vite@latest my-3d-app -- --template react-ts

Then move into it and install the dependencies:

$cd my-3d-app && npm install

2. Add R3F, drei, and the Three types.

$npm install three @types/three @react-three/fiber @react-three/drei

Then start the dev server and keep it running:

$npm run dev   # keep this running

3. Open the project in your AI editor and drop in this seed prompt. It gives you a sphere on screen you can iterate from.

Once you have a sphere on screen, every next prompt is just "add X" / "change Y."

Ship it to the web

When it's working locally, push it live so you can share the link.

1. Create a new GitHub repo. Go to github.com/new, name it, skip the README/license/.gitignore checkboxes, and create.

2. Initialize git, commit, and push. The Vite starter isn't a git repo yet, so initialize one first, then copy the repo URL GitHub showed you and run:

$git init$git add .$git commit -m "Initial 3D scene"$git remote add origin https://github.com/YOUR-USERNAME/YOUR-REPO.git$git push -u origin main

3. Deploy on Vercel. Either import the GitHub repo at vercel.com/new, or ship straight from the terminal — the defaults work for Vite:

$npm i -g vercel$vercel   # follow the prompts; defaults work for Vite

Vercel builds and hands you a public URL — share it.

05Two worked examples

Both of these are reachable from the same seed prompt and the same four nouns — they just pull in different directions.

• Spotify Roller Disco — the reveal. You skate a '70s roller rink from your #10 artist up to your #1. Each level-up re-skins the room: sprite, lighting, vinyl pickups. It ends on a shareable scorecard. Brand experience as content — the app is the marketing.
• Piano Hero — hardware as input. Notes fall down a Guitar-Hero-style highway. Input comes from a 25-key MIDI keyboard plugged into the laptop via WebMIDI — no keyboard, no mouse, your fingers play the game. The control scheme is the idea.

Yours can pull in a third direction entirely.

06Pick one constraint

These aren't ideas — they're constraints. Five people picking the same one should build five different things. Pick one and prompt your way there.

• Input — no keyboard, no mouse. Build something controlled only by mic, camera, gyro, MIDI, or gamepad. The control scheme is the idea.
• Output — ends on a single shareable image. Whatever you build, the climax has to compress into one screenshot worth posting.
• Output — the first 3 seconds carry the whole thing. If someone closes the tab at 3 seconds, they should still feel something.
• Aesthetic — one geometry, one material. The whole world is copies of a single shape. Cubes only, spheres only, planes only. Light and color do the rest.
• Aesthetic — black and white only. Color is forbidden. Pure value, shadow, contrast.
• Domain — replace something boring at work. Your standup, your inbox, your tracker, a Slack ritual. The 3D version has to be how you'd actually do it.
• Domain — a "Wrapped" for something that's never had one. Pick any number from your life — sleep, screen time, reading, commits, texts — and end-of-year it.
• Wild — sound shapes the world. The user's voice or any audio input drives geometry, motion, or color. No pointer input at all.

Have fun building — Jeff.