Modes and sound sets

A mode is one switch that changes several things about the robot at once: his eyes, his behaviour preset, his personality, the noises he makes, and whether he chatters when left alone. Switching mode is a single choice in the dashboard, and everything follows from it.

What a mode is made of

Each mode is a folder holding one file:

{
  "name": "cat",
  "description": "Behaves like a cat: purrs when stroked, wants the cube, pesters you when ignored, naps on the charger",
  "preset": "cat",
  "eyes": null,
  "persona": "You are Ember in cat mode: a small robot who is, for now, a cat...",
  "sounds": null,
  "chatter": false
}

Field	What it does
preset	The name of a behaviour preset to apply to the robot, or null to leave his behaviour alone
eyes	An eye-style object — scale, spacing, angle, corner radii, lid position and bend — or null for the stock eyes
persona	The text that replaces the system persona, or null to keep the default
sounds	The name of a sound set, or null for no noises of its own
chatter	Whether he talks to himself when left alone
eye_color	Optional. Eye colour is otherwise a robot setting rather than part of a style, so a mode that needs a particular colour has to say so

Who applies what

The split is deliberate, and it is worth knowing which half does what when something does not take effect.

• The server owns the physical robot. Setting a mode writes the eye style — the same call the dashboard’s Face page makes — applies the behaviour preset at its defaults, and records the choice.
• The brain polls the server for the current mode and applies the rest itself: the persona, the sound mapping and the chatter setting. It reads the mode’s file directly; the server’s mode call only names the modes.

The brain’s half of a mode is a small Python module that maps events to sounds: one sound per event, with each name rate-limited to once every eight seconds, and voice events never claimed. When a mode claims an event, the default language-model line for that event is skipped — that is how a mode that communicates in beeps stays in beeps.

Sounds play strictly one at a time. Two clips fired together would open two playback streams on the robot and the second would fail mid-stream, so a mode that wants a click and then a call has to let the first finish.

The modes that ship

Mode	Preset	Eyes	Sounds	Chatter
vector	stock	stock	none	yes
cat	cat	stock	none	no
wall-e	wall-e	binocular	wall-e	no
astromech	playful	stock	astromech	no
k9	companion	small, boxy	none	no
johnny5	playful	big, round	none	yes
marvin	calm	heavy-lidded, drooping	none	yes
hal	quiet	round, half-lidded, red	none	no
glados	companion	narrow, tilted	none	yes

vector is him as shipped: stock eyes, factory behaviour and his usual personality. It is what you go back to.

Cat mode is the most developed of them. He purrs when stroked; makes a questioning noise and tilts his head when someone has been about for ninety seconds without touching or talking to him, at most once every three minutes; slow-blinks and trills at a face returning after an hour; mews and rolls the cube when it is tapped or moved; hisses on being picked up, grumbles on being put down, yawns when docked and then goes quiet on the dock.

Wall-E mode answers with servo whines and warbles rather than words, has a two-note call of his own name, and is drawn to his cube.

Where the personas are written in the style of a character, they are original prose. No lines and no audio are taken from any film.

The other modes each have a behaviour map written and tested against a stub, but several of them currently have no sound set on disk — the generated sets were deliberately deleted, on the grounds that only real recordings would be used. A mode with no set of its own simply makes no noises; the persona, eyes and preset still apply.

Sound sets

A sound set is a folder of WAV files: 16 kHz mono, 16-bit PCM. Playing one streams it to the robot’s speaker through the SDK’s external audio playback and returns once he reports it finished. No behaviour control is taken, so a reaction noise does not interrupt whatever he is doing.

Names are restricted to lower-case letters, digits, underscore and hyphen, which is also what stops a request reaching outside the sounds directory.

Folders of takes

A name in a set can be a single file, or it can be a folder — one feeling, with several takes of it. When it is a folder, playing that name picks one file at random, so he does not repeat himself. This is the same idea the robot’s own audio engine uses internally, applied to recordings you supply yourself.

sounds/wall-e/greeting/greeting_01_hello.wav
sounds/wall-e/greeting/greeting_02_hello.wav
sounds/wall-e/greeting/greeting_03_hey.wav
sounds/wall-e/greeting/greeting_04_hey.wav
...

Asking for wall-e/greeting plays one of those eight. Listing the sounds gives the feelings; listing with takes expands each feeling into its files, which is what the dashboard uses to let you audition them one at a time.

A missing file is skipped rather than treated as an error, so a mode whose set is incomplete degrades quietly.

Adding your own

Two small scripts do the conversion. One takes any audio file and writes it into a set in the right format. The other cuts a long recording into one file per sound, which is the practical way to build a set from a single session at a microphone.

Every sound is a recording, not a synthesis. There is a tone generator in the tree — it built an early, rejected version of one set — but no set uses it today.

The recordings themselves are not in the repository. They are the owner’s own audio, kept on the server and restored there after a deploy, and they are not offered for reuse. What the repository holds is the mechanism and the folder layout; the WAV files are yours to supply.

What has not been tested

Playback levels on the robot have not been checked. The volume argument defaults to the middle of the range and the right value for a given set is a matter of listening to it on the actual speaker, which has not happened.