Why AI still struggles to turn off a light

AI has improved quickly. Yet ask a capable assistant to turn off your lights and it often asks which lights you mean, or takes well over 10 seconds to do it. We have spent years building systems that turn off light bulbs, and this is our account of why that is still hard.

The short version: most smart home AI has picked one of two paths. One is fast and narrow: quick, cheap and low-lag, but it cannot do most of what people ask. The other is slow and capable: it understands far more, but you wait. Most people prefer fast, because it is cheaper and feels responsive. Fast cannot handle the long tail of requests, though, and that long tail is most of what people actually want.

Early 2020s: slot filling and computer speak

Alexa and Google Home were built on finite-state transducers (FSTs) and slot filling. The assistant matched your words against known patterns (turn {state} the {device} in the {room}) and filled in the blanks.

This was fast. A matched request could finish in about 1.3 seconds from the end of speech. It worked well when your words fit a pattern and failed on almost everything else. People learned to speak the way the computer wanted, in short, fixed phrases.

2023: the first agents

Agentic development was just getting started. Teams experimented with switching their assistants to OpenAI models and letting the model call tools. In our experience, tool calls to turn off a light succeeded less than 30% of the time out of the box.

Teams then fine-tuned Llama models and their variants for the job and reached roughly 70–80% accuracy. That was usable but fragile. The data models describing homes were poor, the training sets were thin, and neither reflected how people really talk about their homes.

2024 and 2025: production voice agents

Alexa and Google Home began building agents for real production voice use, and the Model Context Protocol (MCP) gave assistants a common way to reach tools. Voice agents split into two designs:

• Full-duplex streaming with asynchronous work. The assistant keeps talking with you while the work runs in the background.
• Single-shot streaming dispatch. One utterance goes in, and one action comes out.

Both still worked like a zero-shot agent tuned for a single request. The pipeline looks like this:

1. Speech recognition turns the utterance into text.
2. The text becomes an intent signal.
3. The system gathers context: which device heard you, who spoke, and which kinds of functions you probably want (media, lights and so on).
4. The tools and that context go into a prompt, and the model gets one try.

It works reasonably well, but it needs very complex stream processing to reach acceptable latency. Even then it usually takes at least 2–3 seconds, against the 1.3-second target of the old FST systems.

Teams also did a lot of targeted fine-tuning, because turning off a light is more complex than it sounds:

• Which lights? Does the person have rooms set up?
• Which house, and which floor?
• Do they mean the light, the wall switch or the plug the lamp is on?
• Are they even allowed to do that?

Prompt tuning for these cases is flaky, and the target keeps moving.

2026: background agents

The newest voice assistants run a full-duplex conversation while background agents do the work. The conversation feels more responsive, but the work is slower still. Providers share compute across many users, and a virtual machine that has to start cold adds its own delay. Turning off the lights can take 30 seconds to a minute, for roughly the same MCP call that Alexa makes: turn off, type light.

We timed this ourselves. When we asked ChatGPT, through the Port OS MCP, to turn off the kitchen light, it found the light and then asked which of two connections to use. That question is the disambiguation problem in miniature: a fact the system could have settled, handed back to the person.

The models keep getting better, but they are slow. So the smart home tasks where they shine are long-running ones, where a minute of thinking is nothing:

• "Patrol the house with the Roborock while I'm out and tell me if anything looks wrong."
• "Build me a three-hour playlist and play it on the TV."
• "Make the lights fade between blues and greens like the ocean."

2027: where this is going

The long-term outlook is good. Models no longer need fine-tuning for each device task. Fast inference hardware from companies such as Cerebras and Groq, plus better small models, is making responses quicker.

The gaps that remain are not about the model:

• Open datasets for evaluating and training device control are scarce.
• Device APIs are inconsistent and often poorly documented.
• Real devices are hard to test against, which holds accuracy back.

So far, most of the gain in accuracy has been paid for with latency. The agent tries things until it gets an answer it likes. That is painful when it fails: turning off a light can take 10 minutes if the agent gets stuck.

What we are doing about it

Port OS tries to move the hard parts out of the model. It gives the assistant a clean model of your home (rooms, groups, device types, which plug powers which lamp) and one consistent way to control every brand. When the facts are already there, the model does not have to guess, and it does not have to ask you.

To try it with your own assistant, read how to control your smart home with ChatGPT or Claude or see how the Port OS MCP integration works.

All posts