HouseFly — a real fly brain, running your house

Why this was rewritten

HouseFly v1 described itself as a “leaky reservoir inspired by fruit-fly motifs”. That framing was doing a lot of work. The brain was a 256-unit reservoir whose recurrent weights, input projections and readouts were all drawn from rng.gauss — and none of which ever changed.

So every motor channel was a fixed random projection of a random projection of your sensors: statistically, a smoothed Gaussian hovering near 0.5. Nothing the fly did depended on anything. Every behaviour you could actually observe — the mode classifier, hunger, phototaxis — was a hand-written if sitting beside the reservoir rather than emerging from it. You could have deleted the reservoir without changing what the integration appeared to do.

It also named the connectome in its docstring and then explicitly declined to use it. And it fired a service call at every configured output every ten seconds with a value that was noise — cover was a supported output domain, so the documented configuration surface included your garage door.

The question was never how to improve the reservoir. It was why there is one.

Give each circuit the input it actually carries

The replacement is not a bigger network. It is a set of circuits that have a known function, each receiving the kind of signal it receives in the animal.

Circuit	What it does in the fly	What it gets from your house
EPG / PEN / PEG / Δ7	Ring attractor holding heading	Integrates the fly's own turns into a compass
ER ring neurons	Visual bearing into the ellipsoid body	Bearing to every card on your dashboard
PFL3	Compares heading to goal, commands a turn	Steering (see the caveat below)
KC → MBON	Associative memory, real plasticity	Learns which parts of the house are good
PAM / PPL1	Reward and punishment teaching signals	Feeding it, and swatting at it
LPLC2 → DNp09/10	Looming detection → escape	Motion sensors firing; your cursor
Projection neurons	Odour identity	The state of your house, as a smell
s-LNv / LNd / DN1	Circadian morning + evening oscillators	Real local time

There is no rule anywhere that says be active at dawn. The morning cells are driven by morning, the evening cells by evening, and arousal is read off their firing rate. The fly is crepuscular because the circuit is.

The ring attractor is in the wiring, not in the code

This is the part worth showing people. Nobody wrote a ring attractor. The connectome contains one.

Take the effective EPG → Δ7 → EPG coupling — using only measured synapse counts and the protocerebral bridge glomerulus labels — and bin it by heading offset:

0° 89 ▉

45° 181 ▉▉▉

90° 919 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉

135° 1791 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉

180° 2025 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ ← peak

225° 1827 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉

270° 921 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉

315° 208 ▉▉▉

Δ7 is inhibitory, so this is local excitation with long-range inhibition — a Mexican hat, 23:1. That is exactly, and only, what a ring attractor needs. Run python3 tools/build_connectome.py and it prints this from the raw data.

The data caught a mistake before the simulation did

The first version of this gave both halves of the protocerebral bridge the same angular order. That produced a much weaker 2.7:1 profile, and a compass that held a heading perfectly well but would only ever turn one way.

The real bridge is a mirror-symmetric double map. Correcting it sharpened the signature from 2.7:1 to 23:1. The connectome said the anatomy was wrong before a single neuron was simulated.

Which way does a turn push the bump? Measure it.

You do not have to assume. It is readable straight out of the synapse counts:

EPG → LEFT  PEN1 → EPG loop shift:  +12.5°
EPG → RIGHT PEN1 → EPG loop shift:  −11.5°
EPG → LEFT  PEG   → EPG loop shift:   −5.6°
EPG → RIGHT PEG   → EPG loop shift:   +0.9°

PEN loops shift the bump in opposite directions per hemisphere. PEG loops do not shift it at all. That is the textbook split between the loop that moves the bump and the loop that holds it — and it fell out of the data.

It has a memory, and the memory is synapses

Learning in Drosophila happens at Kenyon cell → MBON synapses. A Kenyon cell active at the same moment as a dopaminergic neuron in that MBON's compartment gets depressed. That is the whole rule. It is anti-Hebbian, and it is why a fly stops approaching things that turned out to be bad.

HouseFly implements exactly that, on 20,391 real KC→MBON connections. Each MBON's compartment valence is derived from which dopaminergic class dominates its input, rather than hard-coded. Those gains are what persists across restarts — the fly does not forget your house when Home Assistant updates.

What the connectome does not contain

Two things were expected to fall out of the wiring and did not. Both are worth stating plainly, because “we tried to derive it and could not” is a result.

The map of your house

The plan was to read the landmark-bearing → heading mapping off the ER → EPG connectivity. It isn't there: that projection is near-uniform, about 7% modulation depth, with no consistent phase.

This is not a gap in the data. In the real animal that map is learned, in plastic ER→EPG synapses (Fisher et al. 2019; Kim et al. 2019). A naive fly does not have one. So HouseFly doesn't fake it — the compass integrates the fly's own turns, exactly as a real one does in the dark, and it drifts when it cannot see.

PFL3's steering computation

In the animal, PFL3's left-right imbalance is the turn command, because each cell's fan-shaped-body arbor sits about a quarter turn from its bridge arbor, in opposite directions per hemisphere.

We tried it both ways — feeding the goal into the fan-shaped body and letting the connectome do the rest, and imposing the quarter-turn offset explicitly. Neither gave reliable goal-following: the correlation between turn command and the sine of the heading error ranged from +0.03 to −0.54 depending on the trajectory, and the closed loop never converged. That offset is where the arbors physically sit, and synapse counts between cell types do not carry it.

So the goal-seeking controller lives in coordinator.py — four lines of proportional control, labelled as a controller. The fly holds a heading using a connectome-derived ring attractor, which is real, and chooses which heading to hold using arithmetic, which is not. Keeping those visibly separate matters more than having one more thing to claim.

What a real house exposed

The fake ten-room testbed was generated, tidy and metric. Pointed at a live installation instead — Swedish, 30 lights, Bluetooth room-presence sensors, a power meter and an electricity price — the sensory front end turned out to be destroying most of its input before a single neuron saw it.

Every word became the same number

Any state that wasn't numeric returned a constant 0.35. So Allrum, Loft and Kitchen all arrived as the same smell, and the mushroom body could not learn that one room differed from another. The information was gone before it reached a neuron.

Words now pick a glomerulus rather than a magnitude — which is how odour identity actually works: it is carried by which neurons respond, not how hard one does. With 131 glomeruli some states inevitably collide, as they do in a real fly with about 50 for an unbounded number of odours. That is resolved where the animal resolves it: two colliding inputs still produce clearly different Kenyon cell codes — 67 and 39 cells active with 12% overlap — because the projection-neuron-to-Kenyon-cell divergence in the connectome pulls them apart.

One scale cannot serve degrees, watts and currency

tanh(value / 60) put seven temperatures spanning 11–25 °C into a 0.2-wide band, pinned a 2,840 W power sensor flat at 1.0, and turned an electricity price of 0.43 into 0.007. Two of those channels were constants and the third had no resolution left.

Each channel now learns its own range and reports where the current value sits inside it — receptor neurons adapt their gain to the stimulus range they actually receive, which is why you can see indoors and outdoors. It costs two floats per entity and they persist across restarts.

Three smaller ones, all of a kind

abs(value) meant −15 °C and +15 °C were identical readings. On a Swedish install, that sign is the signal.
sun.sun reports above_horizon, which didn't parse as a number — so the sun was a constant.
Dead inputs were silently read as zero. On the install tested, four of nine configured inputs were unknown and nothing said so. A sensor stuck on unknown is not a smell of nothing; it is no smell at all.

Approach, measured properly

LPLC2 — the population that drives escape — fires at an object expanding in the visual field. For a target of size L at range r closing at speed v, the angular size is θ ≈ L/r, so the expansion rate is θ̇ = L·v/r². That r² is the whole character of the response: the same footsteps count for far more at one metre than at five, which is why a real fly leaves it so late and then goes all at once.

A ranging sensor — mmWave radar, ultrasonic, BLE distance — gives r directly and v by differencing, so it delivers exactly the quantity the circuit is built for, by radar instead of by photons. Someone walking in at 1 m/s, sampled at a range of distances:

5.0 m closing θ̇ 0.040 escape 0.0000

3.0 m closing θ̇ 0.111 escape 0.0000

2.2 m closing θ̇ 0.207 escape 0.2714 ▉▉▉▉▉▉▉▉▉▉

2.0 m closing θ̇ 0.250 escape 0.2893 ▉▉▉▉▉▉▉▉▉▉▉

1.4 m closing θ̇ 0.510 escape 0.2627 ▉▉▉▉▉▉▉▉▉▉

0.6 m closing θ̇ 2.778 escape 0.2725 ▉▉▉▉▉▉▉▉▉▉

Walking away produces nothing, because receding is not looming. Standing still produces nothing, because there is no expansion.

Note the shape. Nothing at all until about 2.4 m, then the same full burst at every distance inside it. That is not a rounding artefact, it is the pathway: the giant-fibre escape is a trigger, so the interesting quantity is where the threshold falls, and the r² is what puts it there. Halve the walking speed and the fly lets you get to 1.7 m instead; the response itself does not get any smaller.

This took a correction to get right

The first version of these numbers was wrong in a way that flattered it. The model adds a small resting drive to every neuron, standing in for the parts of the brain that are not modelled — and that drive and the firing threshold happened to be the same number, so the escape pathway sat exactly at its own threshold. Measured from rest, a looming input of 1e-5 produced escape 0.286 and an input of 1.25 produced 0.284. The same burst. θ̇ = v/r² was being computed faithfully and then thrown away, and every claim on this page about the r² was decorative.

The check that was supposed to catch this held the stimulus on for twelve steps, where a weak one simply adapts away — but a real approach is a transient, and that is the case that mattered. DNp09 and its LPLC2 inputs are silent at rest in the animal, which is what makes them a trigger rather than a readout, so the resting drive is now withheld from precisely the circuits already marked phasic. No new constant, no new list. The validation suite now measures the onset response across four decades.

Frame rate is what kills most cameras

Optic flow needs frames close enough together to correspond. The first house this ran on had exactly one camera — a motorway traffic camera whose own photo_time showed it updating every five minutes. At that spacing there is no correspondence between frames at all: flow would be noise, and a looming detector fed from it would fire constantly and mean nothing. The radar in the hallway was the better eye. The same mistake was made again in the simulated radar on the demo box, at five seconds instead of five minutes, and it had the same character: looming is a derivative, so the sample interval decides the shortest approach that can be measured at all.

It has an eye

The ranging-sensor path is honest, but it is not how the animal does it. A fly computes expansion from photons, in the optic lobe, before anything central hears about it — so now it does that too, in the browser, because that is where the frames are.

It is the published circuit rather than a motion detector: photoreceptors on a 32×24 grid (a Drosophila eye has about 750 ommatidia at roughly 5°, so that is the number and not an approximation), a lamina high-pass, Hassenstein–Reichardt correlators giving the four direction-selective layers of the lobula plate, and then 63 LPLC2 cells tiling the field.

The LPLC2 stage is the whole thing. Klapoetke and colleagues showed in 2017 that each cell has four dendritic branches, one in each layer, each offset so it reads motion pointing away from that cell's receptive-field centre — and that the cell needs all four driven at once. That coincidence requirement is the entire selectivity. Getting it wrong is instructive: summing the four branches instead scored a drifting grating at nine times an actual approach, and a disc crossing the view at thirty times, because one large branch carried the sum on its own. A geometric mean cannot be carried by one term. With it, both score exactly zero.

approach, plain background 3.121 ▉▉▉▉▉▉▉▉▉▉▉

approach, cluttered room 0.124 ▉

distant approach (6.0–4.0 m) 0.047

receding 0.000

drifting grating 0.000

object crossing laterally 0.000

full-field flicker 0.000

camera panning across a room 0.000

hand-held camera wandering 0.003

Every number from tools/test_vision.mjs, all against the same approach. The escape threshold is about 0.1 and is enforced by the network, not by the eye.

Three other things had to be measured rather than assumed. Normalising a local response by a field-wide contrast let a small object on a plain wall score a hundred times an approach, so contrast is now measured inside each cell's own receptive field. Taking the peak over 63 cells and 60 frames was measuring the maximum of noise, so each cell integrates over 120 ms first. And a camera panning across a textured room still beat a real approach fourfold — no read-out fixed that, and five were tried, because it is not a read-out problem. It is a self-motion problem, and the fly's own answer works: flow coherence is 0.00 for an approach and 0.76 to 0.84 for a pan, which leaves a gap nobody had to tune a threshold into.

Two limits, stated

Clutter costs it about 25×. Real LPLC2 is also less sensitive in a textured scene, but in practice this works far better against a plain wall than against a bookshelf. It still clears the escape threshold in clutter, at 0.124 against about 0.1, which is not much margin.

An approach during a camera pan is missed entirely, because the self-motion gate cannot tell the difference. A fly has exactly the same blind spot during a saccade.

And a directed escape is not possible with this data pack. LPLC2 is retinotopic, so in the animal the population says where the threat is and the fly turns away from it. Ours cannot: the hemibrain is a hemibrain, and the looming pathway in the pack is 0 left, 130 right, 147 unlabelled, with 1 left and 12 right descending neurons. There is no left population to compare against. The central complex is balanced 23/23, 21/21 and 12/12, so the compass and steering are unaffected — but the escape burst stays undirected, which is at least true to DNp09. The validation suite checks those counts, so the day someone builds a pack from a whole-brain reconstruction, the check fails and this paragraph becomes wrong on purpose.

Nothing leaves the page. Home Assistant receives an expansion rate and an angle about ten times a second, which is roughly what a real LPLC2 population sends down its axons: a magnitude and a retinotopic address, not a picture.

Honest limits

This is a real model of real circuits, and it is still a model.

4,724 neurons of roughly 25,000 in the central brain, and no body, no muscles, no proprioception. Most of the input these cells really receive comes from neurons that aren't in the subnetwork, which is why there is a tonic drive term standing in for the rest of the brain. That term is an assumption.
A rate model, not spikes. One leaky rate unit per neuron, following Shiu et al. 2024. No dendrites, no delays, no channel dynamics.
Synaptic sign comes from predicted transmitter. Mostly reliable — but FlyWire's classifier returns acetylcholine for most ring neurons, which is wrong; they are GABAergic. Where the literature is settled it overrides the prediction, and every override is listed in the build tool.
Two circuits needed a static gain constant. The fan-shaped body and the Kenyon cells are large and almost entirely excitatory with no matching inhibitory population inside the modelled subnetwork, so without it they flood.
Two brains, spliced. Connectivity is hemibrain; positions and transmitters are FlyWire — a different fly. Joined by cell type, which is standard practice and still an approximation.

It is not conscious, it is not an agent, it does not understand your house, and it is not a scientific instrument. It is a small animal's wiring diagram with your sensors plugged into it.

Everything above is checkable

python3 tools/validate.py runs the shipped brain and checks all twenty-nine claims, printing the numbers — including the marginal ones. CI runs it on every push, alongside HACS's own validation action and Home Assistant's hassfest.

a bump forms                               peak 0.223, trough 0.000     PASS
it holds still with no self-motion         0 deg of drift               PASS
turning rotates the bump, right direction  correlation +0.99            PASS
punishment depresses KC→MBON synapses      20,391 plastic synapses      PASS
looming raises descending drive            0.0000 → 0.2452              PASS
  ...and the escape is transient           fell back to 0.0000          PASS
PFL3 cells are not all at the same rate    spread 0.0038                PASS
arousal  03:00 0.17  06:00 1.00  12:00 0.08  18:45 1.00  23:00 0.10     PASS

Try it without installing anything

There is a hosted demo: a real Home Assistant, on a real Vome instance, with a small fake house in it. The link signs you in as a non-admin guest, so you can prod everything without needing an account.

Open the live demo

What to do when you get there

Drag the approach slider from five metres down to half a metre. That feeds the same looming input a ranging sensor would, and the fly bolts. Walking it back out does nothing, because receding is not looming.
Wait ninety seconds. A simulated mmWave radar has someone walking up to it and away again on a cycle, so the demo does something whether or not anyone is watching.
Let it see you. The eye card asks for your camera, runs the optic lobe in your own browser, and shows you the LPLC2 population lighting up. Move a hand towards the lens. Nothing leaves the page.
Toggle the lights and sockets. They are templates on top of toggles; nothing is wired to anything. The fly is watching them and may nudge one.
Try the four it will not touch. Boiler relay, freezer socket, network rack, garage door motor. You can switch them yourself. The fly is refused by name, whatever anyone configures.

If you find it asleep, that is not a fault — it is crepuscular, and the middle of the afternoon is its siesta. It will still bolt at you.

Installing it

HouseFly is a HACS custom repository. In Home Assistant:

HACS → ⋮ → Custom repositories
Repository https://github.com/Vortitron/HouseFly, type Integration
Install HouseFly, then restart Home Assistant
Settings → Devices & Services → Add Integration → HouseFly

Both Lovelace cards register themselves — no resource setup needed:

# Lets the fly out onto this dashboard. Draws nothing itself: it reports
# where your cards are and paints the fly over the top of them.
- type: custom:housefly-overlay
  show_debug: true

# The connectome, with live activity in it.
- type: custom:housefly-brain-card

Somewhere safe to try it

The repository ships a testbed/ — a throwaway Home Assistant with a fake ten-room house: 24 lights, 23 switches, 30 drifting sensors, 15 motion detectors, all template helpers with nothing behind them, plus six entities that exist purely so you can watch the safety layer refuse them.

git clone https://github.com/Vortitron/HouseFly
cd HouseFly/testbed && ./up.sh     # http://localhost:8124

Or skip the setup entirely: Vome hosts Home Assistant instances you can create in a few seconds, which is where the live demo of this runs.

Safety

A fly with write access to your house is a bad idea unless the boundaries are real ones.

Actuation is off by default. A fresh install watches and walks; it touches nothing until you deliberately turn it on.
Never, regardless of configuration: lock, alarm_control_panel, cover, climate, water_heater, humidifier, valve, vacuum, lawn_mower, script, automation, scene.
Refused by name even in an allowed domain: anything containing boiler, freezer, pump, oven, server, alarm, garage, charger, and about twenty more.
It has to land on something to touch it. Actuation is triggered by the fly settling on a card, not by a timer — a few interactions an hour, not thousands.
Hourly budget, per-entity cooldown, a deadband so a jittering brain cannot produce a call storm, and optional quiet hours.

Start it on a spare lamp.

Data and credit

Both datasets are CC-BY 4.0, and both are fetched at build time rather than vendored.

Connectivity, cell types, columnar labels — Janelia hemibrain v1.2. Scheffer, Xu, Januszewski, Lu, Takemura, Hayworth et al. (2020), A connectome and analysis of the adult Drosophila central brain, eLife 9:e57443.
Soma positions, transmitter predictions — FlyWire FAFB. Schlegel, Yin, Bates, Dorkenwald, Eichler, Brooks et al. (2024), Whole-brain annotation and multi-connectome cell typing of Drosophila, Nature 634:139–152.
Modelling approach after Shiu, Sterne, Spiller, Franconville, Sandoval, Zhou et al. (2024), A leaky integrate-and-fire computational model based on the connectome of the entire adult Drosophila brain, Nature 634:210–219.

The flies did the hard part.