HouseFly runs a rate model of 4,724 identified Drosophila neurons, wired together by 126,108 measured synapses, inside Home Assistant. Your sensors go to the neurons that carry that kind of information in the animal. It walks across your dashboard, it startles when something comes at the camera, and โ because that is what a mushroom body is for โ it tells you when your house stops looking like your house.
Two real Home Assistants with fake houses in them, signed in for you as a guest. The first has one fly: everything is a template on top of a toggle, so switch things on and off freely โ then drag the approach slider down and watch it bolt. Four of the switches exist only to be refused. The second has four, on different clocks, awake at different hours, and one of them switches the lights when it feels like it.
There is an easy way to build something like this. Take a recurrent network, seed it with random
weights, call the shape "fly-inspired", then hang the behaviour you want off the side in plain
if statements. It looks alive. You could delete the network and the user would never
know.
So there is one rule: every behaviour comes out of the measured wiring, or it is labelled as not coming out of it.
That rule is why this page is mostly numbers, and why some of them are failures. The ring attractor falls out of the synapse counts rather than being asserted. The escape threshold is where the network sits, not a constant someone picked. Goal seeking, wall avoidance and estimating dawn from the light are controllers rather than biology, and they say so in the code as well as here. Where the model turned out not to do what it appeared to do, that is written down with the measurement that caught it:
Each is now a check that fails if it is ever quietly fixed or quietly broken. A model that always agrees with its own documentation would be the failure mode here, not the goal.
The replacement is not a bigger network. It is a set of circuits that have a known function, each receiving the kind of signal it receives in the animal.
| Circuit | What it does in the fly | What it gets from your house |
|---|---|---|
| EPG / PEN / PEG / ฮ7 | Ring attractor holding heading | Integrates the fly's own turns into a compass |
| ER ring neurons | Visual bearing into the ellipsoid body | Bearing to every card on your dashboard |
| PFL3 | Compares heading to goal, commands a turn | Steering (see the caveat below) |
| KC โ MBON | Associative memory, real plasticity | Learns which parts of the house are good |
| PAM / PPL1 | Reward and punishment teaching signals | Feeding it, and swatting at it |
| LPLC2 โ DNp09/10 | Looming detection โ escape | Motion sensors firing; your cursor |
| Projection neurons | Odour identity | The state of your house, as a smell |
| s-LNv / LNd / DN1 | Circadian morning + evening oscillators | Real local time |
There is no rule anywhere that says be active at dawn. The morning cells are driven by morning, the evening cells by evening, and arousal is read off their firing rate. The fly is crepuscular because the circuit is.
This is the part worth showing people. Nobody wrote a ring attractor. The connectome contains one.
Take the effective EPG โ ฮ7 โ EPG coupling, using only measured synapse counts and the protocerebral bridge glomerulus labels, and bin it by heading offset:
python3 tools/build_connectome.py and it prints this from the raw data.The first version of this gave both halves of the protocerebral bridge the same angular order. That produced a much weaker 2.7:1 profile, and a compass that held a heading perfectly well but would only ever turn one way.
The real bridge is a mirror-symmetric double map. Correcting it sharpened the signature from 2.7:1 to 23:1. The connectome said the anatomy was wrong before a single neuron was simulated.
You do not have to assume. It is readable straight out of the synapse counts:
EPG โ LEFT PEN1 โ EPG loop shift: +12.5ยฐ
EPG โ RIGHT PEN1 โ EPG loop shift: โ11.5ยฐ
EPG โ LEFT PEG โ EPG loop shift: โ5.6ยฐ
EPG โ RIGHT PEG โ EPG loop shift: +0.9ยฐPEN loops shift the bump in opposite directions per hemisphere. PEG loops do not shift it at all. That is the textbook split between the loop that moves the bump and the loop that holds it โ and it fell out of the data.
Learning in Drosophila happens at Kenyon cell โ MBON synapses. A Kenyon cell active at the same moment as a dopaminergic neuron in that MBON's compartment gets depressed. That is the whole rule. It is anti-Hebbian, and it is why a fly stops approaching things that turned out to be bad.
HouseFly implements exactly that, on 20,391 real KCโMBON connections. Each MBON's compartment valence is derived from which dopaminergic class dominates its input, rather than hard-coded. Those gains are what persists across restarts โ the fly does not forget your house when Home Assistant updates.
Two things were expected to fall out of the wiring and did not. Both are worth stating plainly, because โwe tried to derive it and could notโ is a result.
The plan was to read the landmark-bearing โ heading mapping off the ER โ EPG connectivity. It isn't there: that projection is near-uniform, about 7% modulation depth, with no consistent phase.
This is not a gap in the data. In the real animal that map is learned, in plastic ERโEPG synapses (Fisher et al. 2019; Kim et al. 2019). A naive fly does not have one. So HouseFly doesn't fake it โ the compass integrates the fly's own turns, exactly as a real one does in the dark, and it drifts when it cannot see.
In the animal, PFL3's left-right imbalance is the turn command, because each cell's fan-shaped-body arbor sits about a quarter turn from its bridge arbor, in opposite directions per hemisphere.
We tried it both ways โ feeding the goal into the fan-shaped body and letting the connectome do the rest, and imposing the quarter-turn offset explicitly. Neither gave reliable goal-following: the correlation between turn command and the sine of the heading error ranged from +0.03 to โ0.54 depending on the trajectory, and the closed loop never converged. That offset is where the arbors physically sit, and synapse counts between cell types do not carry it.
So the goal-seeking controller lives in coordinator.py โ four lines of proportional
control, labelled as a controller. The fly holds a heading using a connectome-derived ring
attractor, which is real, and chooses which heading to hold using arithmetic, which is not.
Keeping those visibly separate matters more than having one more thing to claim.
The fake ten-room testbed was generated, tidy and metric. Pointed at a live installation instead (Swedish, 30 lights, Bluetooth room-presence sensors, a power meter and an electricity price) the sensory front end turned out to be destroying most of its input before a single neuron saw it.
Any state that wasn't numeric returned a constant 0.35. So
Allrum, Loft and Kitchen all arrived as
the same smell, and the mushroom body could not learn that one room differed
from another. The information was gone before it reached a neuron.
Words now pick a glomerulus rather than a magnitude. That is how odour identity works: it is carried by which neurons respond, not by how hard one does. With 131 glomeruli some states inevitably collide, as they do in a real fly with about 50 for an unbounded number of odours. The animal resolves that downstream, and so does this: two colliding inputs still produce clearly different Kenyon cell codes, 67 and 39 cells active with 12% overlap, because the projection-neuron-to-Kenyon-cell divergence in the connectome pulls them apart.
tanh(value / 60) put seven temperatures spanning 11โ25 ยฐC into a
0.2-wide band, pinned a 2,840 W power sensor flat at 1.0, and turned an
electricity price of 0.43 into 0.007. Two of those channels were constants and
the third had no resolution left.
Each channel now learns its own range and reports where the current value sits inside it โ receptor neurons adapt their gain to the stimulus range they actually receive, which is why you can see indoors and outdoors. It costs two floats per entity and they persist across restarts.
abs(value) meant โ15 ยฐC and +15 ยฐC were identical readings.
On a Swedish install, that sign is the signal.sun.sun reports above_horizon, which didn't parse
as a number โ so the sun was a constant.unknown and nothing said so. A sensor stuck
on unknown is not a smell of nothing; it is no smell at all.LPLC2, the population that drives escape, fires at an object expanding in the visual field. For a target of size L at range r closing at speed v, the angular size is ฮธ โ L/r, so the expansion rate is ฮธฬ = Lยทv/rยฒ. That rยฒ is the whole character of the response: the same footsteps count for far more at one metre than at five, which is why a real fly leaves it so late and then goes all at once.
A ranging sensor (mmWave radar, ultrasonic, BLE distance) gives r directly and v by differencing, so it delivers the quantity the circuit is built for, by radar instead of by photons. Someone walking in at 1 m/s, sampled at a range of distances:
Note the shape. Nothing at all until about 2.4 m, then the same full burst at every distance inside it. That is not a rounding artefact, it is the pathway: the giant-fibre escape is a trigger, so the interesting quantity is where the threshold falls, and the rยฒ is what puts it there. Halve the walking speed and the fly lets you get to 1.7 m instead; the response itself does not get any smaller.
The first version of these numbers was wrong in a way that flattered it. The model adds a small resting drive to every neuron, standing in for the parts of the brain that are not modelled โ and that drive and the firing threshold happened to be the same number, so the escape pathway sat exactly at its own threshold. Measured from rest, a looming input of 1e-5 produced escape 0.286 and an input of 1.25 produced 0.284. The same burst. ฮธฬ = v/rยฒ was being computed faithfully and then thrown away, and every claim on this page about the rยฒ was decorative.
The check that was supposed to catch this held the stimulus on for twelve steps, where a weak one simply adapts away โ but a real approach is a transient, and that is the case that mattered. DNp09 and its LPLC2 inputs are silent at rest in the animal, which is what makes them a trigger rather than a readout, so the resting drive is now withheld from precisely the circuits already marked phasic. No new constant, no new list. The validation suite now measures the onset response across four decades.
Optic flow needs frames close enough together to correspond. The first house this ran on had
exactly one camera โ a motorway traffic camera whose own photo_time showed it updating
every five minutes. At that spacing there is no correspondence between frames at
all: flow would be noise, and a looming detector fed from it would fire constantly and mean
nothing. The radar in the hallway was the better eye. The same mistake was made again in the
simulated radar on the demo box, at five seconds instead of five minutes, and it had the same
character: looming is a derivative, so the sample interval decides the shortest approach that can
be measured at all.
The ranging-sensor path is honest, but it is not how the animal does it. A fly computes expansion from photons, in the optic lobe, before anything central hears about it โ so now it does that too, in the browser, because that is where the frames are.
It is the published circuit rather than a motion detector: photoreceptors on a 32ร24 grid (a Drosophila eye has about 750 ommatidia at roughly 5ยฐ, so that is the number and not an approximation), a lamina high-pass, HassensteinโReichardt correlators giving the four direction-selective layers of the lobula plate, and then 63 LPLC2 cells tiling the field.
The LPLC2 stage does the work. Klapoetke and colleagues showed in 2017 that each cell has four dendritic branches, one per layer, each offset so it reads motion pointing away from that cell's receptive-field centre, and that the cell needs all four driven at once. That coincidence requirement is where the selectivity comes from. Getting it wrong is instructive: summing the four branches scored a drifting grating at nine times a real approach and a disc crossing the view at thirty times, because one large branch carried the sum by itself. A geometric mean cannot be carried by one term. With it, both score zero.
tools/test_vision.mjs, all against the same approach. The
escape threshold is about 0.1 and is enforced by the network, not by the eye.Three other things had to be measured rather than assumed. Normalising a local response by a field-wide contrast let a small object on a plain wall score a hundred times an approach, so contrast is now measured inside each cell's own receptive field. Taking the peak over 63 cells and 60 frames was measuring the maximum of noise, so each cell integrates over 120 ms first. And a camera panning across a textured room still beat a real approach fourfold โ no read-out fixed that, and five were tried, because it is not a read-out problem. It is a self-motion problem, and the fly's own answer works: flow coherence is 0.00 for an approach and 0.76 to 0.84 for a pan, which leaves a gap nobody had to tune a threshold into.
Clutter costs it about 25ร. Real LPLC2 is also less sensitive in a textured scene, but in practice this works far better against a plain wall than against a bookshelf. It still clears the escape threshold in clutter, at 0.124 against about 0.1, which is not much margin.
An approach during a camera pan is missed entirely, because the self-motion gate cannot tell the difference. A fly has exactly the same blind spot during a saccade.
And a directed escape is not possible with this data pack. LPLC2 is retinotopic, so in the animal the population says where the threat is and the fly turns away from it. Ours cannot: the hemibrain is a hemibrain, and the looming pathway in the pack is 0 left, 130 right, 147 unlabelled, with 1 left and 12 right descending neurons. There is no left population to compare against. The central complex is balanced 23/23, 21/21 and 12/12, so the compass and steering are unaffected โ but the escape burst stays undirected, which is at least true to DNp09. The validation suite checks those counts, so the day someone builds a pack from a whole-brain reconstruction, the check fails and this paragraph becomes wrong on purpose.
Nothing leaves the page. Home Assistant receives an expansion rate and an angle about ten times a second, which is roughly what a real LPLC2 population sends down its axons: a magnitude and a retinotopic address, not a picture.
The fair question about all of this is "why". A fly that walks across a dashboard is a pretty picture. Bolting a utility function onto it, so it saves power or learns your habits, would be a rules engine in a fly costume, which is the worse of the two failure modes.
There is an answer in between that does not have to be grafted on. The mushroom body is the largest thing modelled here, and its job is telling familiar from unfamiliar, which happens to be what rules engines are worst at.
Dasgupta, Stevens & Navlakha (2017, Science) showed the Kenyon cell layer is a locality-sensitive hash: a sparse random projection whose codes stay close for similar inputs and far apart for different ones, which is an efficient novelty detector and was published as one. Hattori et al. (2017, Cell) found the circuit that reads it out โ repeated exposure depresses KCโMBON synapses in the ฮฑโฒ3 compartment whether or not anything good or bad happened, so those cells fire hard for something new and barely at all for something met many times.
Both are in the pack. ฮฑโฒ3 is MBON16, MBON17 and MBON28, none of which is an approach/avoid cell. That separation is the reason for using this compartment rather than depressing the valence MBONs and calling the result novelty.
binary_sensor.housefly_unusual is that signal, slowed down until it is worth
saying. It stays quiet until it has learned something, then wants two minutes of sustained
strangeness before it speaks. No training set, no labels, nothing sent anywhere. It
learns what your house is like by living in it. It will never tell you what changed, only
that something has, and that is the part a rule cannot do, because you would have had to write the
rule first.
A sparse hash ticking twice a second on a CPU costs nothing. Asking a language model to look over a whole house costs real money. So let the fly decide when it is worth asking.
HouseFly fires a fly_house_unusual event on the rising edge, carrying the novelty
figure, how long it has held, and a shortlist of suspects โ which configured inputs
the surprise is arriving through, obtained by reading the measured PNโKC wiring backwards from the
cells that are active and have not habituated.
This is not a diagnosis. The Kenyon code is a hash, hashes do not invert, and channels collide whenever there are more entities than glomeruli. Measured on a synthetic input, three of the top four channels were among the eight actually driven and one was not. It narrows the field, which is all you need from something that is about to hand the question to a system that can go and look.
A fly has no concept of your electricity bill. The punishment pathway is real enough: PPL1 dopaminergic neurons depress KCโMBON, which is operant conditioning, so feeding consumption in as something aversive would work mechanically. It still would not save you anything. The actuation budget is sixty calls an hour with deadbands and quiet hours, and the motor output was never a controller. What you would get is a fly that avoids the expensive rooms: a behaviour you can demonstrate, not an energy saving.
The version that works is the one above. Power is just another input channel, so adding a consumption sensor means "unusual" covers it for free. It will tell you when your electricity stops looking like your electricity, which is the failure mode that costs money and the one you cannot write a rule for in advance.
The clock used to peak at a fixed 06:00 and 18:43. That is nobody's daylight, and at this
latitude it is not close for most of the year. It now watches whatever light the house reports โ an
illuminance sensor if there is one, sun.sun's elevation otherwise โ learns where dawn
and dusk actually fall, and moves the morning and evening oscillators there.
This is photoperiod tracking rather than entrainment. Entrainment means a free-running oscillator being pulled into phase by a zeitgeber, and there is no free-running oscillator here: the clock in this model is a function of local time. Moving where the peaks sit is the version of that claim the code can support. Estimating dawn from the light happens in the coordinator, labelled a stand-in rather than dressed up as neural.
Separately and acutely, light drives l-LNv, the arousal-promoting clock cells, which are genuinely light-responsive (Shang et al. 2008). Arousal measured 0.17 in the dark at 00:30 and 0.51 with the light on. The photoperiod shifts when it is active; this makes it active now.
Averaging times of day went wrong twice here. The mean of 23:50 and 00:10 is not midday, so the learned phases are dragged round the circle rather than along a line. The Gaussian needed the same treatment: a dusk learned at 23:30 sits forty-five minutes from 00:15, not three quarters of a day away. Before that fix it reported no evening at all, which only shows up in winter or in a house that gets up late.
This is a real model of real circuits, and it is still a model.
It is not conscious, it is not an agent, it does not understand your house, and it is not a scientific instrument. It is a small animal's wiring diagram with your sensors plugged into it.
python3 tools/validate.py runs the shipped brain and checks every number on this
page, printing them all โ including the marginal ones and the two that are documented failures. CI
runs it on every push, alongside HACS's own validation action and Home Assistant's hassfest, and it
installs Node because the eye's suite runs where the frames do.
a bump forms peak 0.231, trough 0.000 PASS
it holds still with no self-motion 0 deg of drift PASS
turning rotates the bump, right direction correlation +0.98 PASS
a constant turn does NOT keep rotating it an offset, not a velocity PASS
the looming pathway has no left hemisphere 0 left, 130 right PASS
punishment depresses KCโMBON synapses 20,391 plastic synapses PASS
something new registers as novel first sight 0.989 PASS
...and stops being novel with exposure after 400 ticks 0.025 PASS
...with no teaching signal at all valence memory untouched PASS
a trivial expansion triggers nothing 1e-5..1e-2 gave 0.0000 PASS
a camera panning produces nothing textured pan 0.0000 PASS
dawn at 04:00 gives a fly that wakes at 4 peak moved 06:00 โ 04:00 PASS
a tick costs a fraction of what it covers 59 ms of CPU for 2 s of fly PASS
nothing but the fetcher reaches the network 0 offenders PASS
an empty quiet-hours window means never not, as it did, always PASS
135/135 checks passedOn your own machine, and it is not talking to anything. The wiring diagram is a 428 KB file that arrives with the integration; the model that runs on it is Python and numpy on your Home Assistant host. There is no inference service, no account and no API key, and nothing to sign up for โ which is less a privacy stance than a consequence of the thing being small.
That is worth a picture, because code here runs in two different places and the interesting part is what does not cross between them.
Every tick, the coordinator builds the sensory input and hands the network to Home Assistant's
executor, so the event loop is never waiting on it. The inner loop is one np.bincount
over all 126,108 synapses per 50 ms of simulated fly โ forty of those per default two-second
tick, which measured 59 ms of CPU, or about 3% of one core on a 2018 desktop
i7. No GPU, and nothing that would want one โ the entire recurrent step is 2 MB of sparse
arrays and a vector add.
I have not measured it on a Raspberry Pi, so I will not quote a figure for one. What can be said
is that the margin is a factor of thirty-four on the box it was measured on, and that
tools/validate.py checks the ratio rather than the milliseconds โ so if it ever stops
being comfortable on the hardware you run, the suite is where that shows up.
Two files, 428 KB together, sitting in the integration directory next to the Python. They are loaded once per Home Assistant process, expand to about 2 MB of read-only numpy arrays, and are then shared by every fly you configure โ which is why the second fly costs 823 KB rather than another copy of the brain. Restarting re-reads the same file from the same disk. Nothing is fetched, nothing expires, and it works with the network unplugged.
The one exception is honest about itself. If the pack is missing โ an install route that could not carry 428 KB of binary, a pipeline that strips anything that is not source โ it is fetched from the release tag matching the code's own version, and checked against a SHA-256 compiled into the source before anything loads it. A file that does not match is refused rather than used, because every number on this page rests on that pack being the pack.
The looming pathway is the one part that does not run on the host, for the reason given above: a fly computes expansion from photons in the optic lobe, and the photons are in the browser. So the photoreceptors, the T4/T5 correlators and the 63 LPLC2 cells are JavaScript running in the tab, and what crosses back to the brain is an expansion rate and an angle, about ten times a second. No frames, no stills, nothing from which the picture could be reconstructed. Close the tab and that stage simply stops: looming arrives from the ranging and motion paths again, as it did before there was an eye.
The memory is the synapses, so persisting it means persisting numbers: the 20,391 plastic
KCโMBON gains, where it is, how hungry it is, and the sensory ranges and the dawn and dusk it has
worked out for your house โ written to Home Assistant's own .storage every eighty
seconds. There is nowhere else for them to go. Nothing about your
house is uploaded, aggregated or trained on, and the flip side of that is worth saying plainly โ a
fly that learns your house learns only your house, and starts from the measured wiring again
if you delete that file.
There is a hosted demo: a real Home Assistant, on a real Vome instance, with a small fake house in it. The link signs you in as a non-admin guest, so you can prod everything without needing an account.
The second link is a different instance with four flies in one house, which is worth a look for a reason the single-fly demo cannot show: they are not one animal drawn four times. Each has its own name, its own brain, its own memory on disk and its own view of the house, and their clocks are offset so they are awake at different hours. Nothing passes between them except the house โ one turns a light on and the others smell it.
One of them, The Meddler, exists to keep the place from being predictable. It can switch every light in the building, including the ones in the others' territories, and it does so when its own brain decides to rather than on any schedule โ where it has walked, what it has landed on, how awake it is, and whether the safety layer will allow it. Because the hour is part of the Kenyon cell code, a hallway light at three in the morning is a genuinely different smell from the same light at noon.
Whether any of them is awake depends on when you arrive, which is the point rather than a caveat. The arousal and โwho is awakeโ graphs on that page are the thing to look at: four traces that should not line up. Over one night the day fly slept from eight in the evening until half past two while the others covered it.
If you find it asleep, that is not a fault โ it is crepuscular, and the middle of the afternoon is its siesta. It will still bolt at you.
HouseFly is a HACS custom repository. In Home Assistant:
https://github.com/Vortitron/HouseFly, type IntegrationBoth Lovelace cards register themselves โ no resource setup needed:
# Lets the fly out onto this dashboard. Draws nothing itself: it reports
# where your cards are and paints the fly over the top of them.
- type: custom:housefly-overlay
show_debug: true
# The connectome, with live activity in it.
- type: custom:housefly-brain-cardThe repository ships a testbed/ โ a throwaway Home Assistant with a fake ten-room house:
24 lights, 23 switches, 30 drifting sensors, 15 motion detectors, all template helpers with nothing
behind them, plus six entities that exist purely so you can watch the safety layer refuse them.
git clone https://github.com/Vortitron/HouseFly
cd HouseFly/testbed && ./up.sh # http://localhost:8124Or skip the setup entirely: Vome hosts Home Assistant instances you can create in a few seconds, which is where the live demo of this runs.
A fly with write access to your house is a bad idea unless the boundaries are real ones.
lock, alarm_control_panel,
cover, climate, water_heater, humidifier, valve,
vacuum, lawn_mower, script, automation, scene.boiler,
freezer, pump, oven, server, alarm,
garage, charger, and about twenty more.Start it on a spare lamp.
Both datasets are CC-BY 4.0, and both are fetched at build time rather than vendored.
The flies did the hard part.