Control your Android TV or Google TV with voice and AI

Want to let your AI control your television? Now you can with the Port OS Television integration for android TV.

Before you start

A TV running Google TV or Android TV, on the same Wi-Fi as your computer.
A computer with the Android platform tools installed, so that adb --version works in your terminal.
A Port OS account you can sign in to on the website.

Install adb on your computer

adb (Android Debug Bridge) is the command-line tool that installs the app and grants its permissions. Install it with the package manager for your system:

Windows (PowerShell): winget install Google.PlatformTools
macOS (Homebrew): brew install --cask android-platform-tools
Linux (Debian or Ubuntu): sudo apt install adb

Or download the platform tools zip for your system from the Android platform tools page, unzip it and run adb from that folder.

Open a new terminal afterwards and check that it works:

Text

adb --version

1. Turn on developer options and wireless debugging

On the TV open Settings > System > About and select Android TV OS build (or Build) seven times until the TV says developer mode is on.
Open Settings > System > Developer options and turn on USB debugging and Wireless debugging.
Open Wireless debugging, note the IP address and port it shows, and choose Pair device with pairing code if the TV asks you to pair first.
On your computer connect to the TV, replacing the address with the one the TV shows:

Text

adb connect <TV-IP>:5555

The TV asks you to allow the computer. Choose Always allow. adb devices lists the TV as device when it works.

2. Install the app

Download the newest app from the latest release of the public portos-android-tv repository, which also holds its source:

Text

https://github.com/portpowered/portos-android-tv/releases/latest/download/portos-android-tv.apk

Then install it on the TV:

Text

adb install -r portos-android-tv.apk

The app is com.portoperatingsystem.tv. It appears with the other apps on the TV home screen.

3. Grant the permissions the app needs

Next we have to grant the app permissions to work on your TV. Default android blocks the agent from doing lots of things like open windows and so this enables the TV controls.

Text

adb shell appops set com.portoperatingsystem.tv SYSTEM_ALERT_WINDOW allow
adb shell cmd notification allow_listener com.portoperatingsystem.tv/com.portoperatingsystem.tv.state.MediaSessionListener
adb shell settings put secure enabled_accessibility_services com.portoperatingsystem.tv/com.portoperatingsystem.tv.control.PortOsAccessibilityService
adb shell settings put secure accessibility_enabled 1

Each of the commands above do the following:

The first lets Port OS open YouTube or Spotify on the TV when you ask.
The second lets the app read the active media session, so Port OS knows what is playing.
The last two switch on the Port OS accessibility service, which presses the remote buttons, taps the screen where Port OS points and takes a picture of the screen when you ask.

Issues with multiple accessibility services

If the TV already uses another accessibility service, run adb shell settings get secure enabled_accessibility_services first. When it prints something other than null, add the Port OS service after the existing value, separated by a colon, instead of replacing it:

Text

adb shell settings put secure enabled_accessibility_services EXISTING_SERVICE:com.portoperatingsystem.tv/com.portoperatingsystem.tv.control.PortOsAccessibilityService

4. Pair the TV with Port OS

Open Port OS on the TV. It shows a code and a QR code.
On your phone or computer, scan the QR code or open the address it shows, sign in to Port OS and approve the code.
The TV shows that it is connected. In Port OS the TV appears under your devices with its name, for example Living Room TV.

5. Try it

Now that you have the agent running, you can ask the agent to do various things like:

launch videos on youtube, and spotify
pause, play, resume and skip videos
play movies that you own
play images that you own.

What the app can do

Install the Port OS app on a Google TV or Android TV, and Port OS can play media on it, change its volume, press its remote buttons, tap its screen, take a picture of its screen and know what is playing. The TV needs no hub and no new device in the house. It does need Google Play Services, because Port OS reaches the app through Firebase Cloud Messaging (FCM), so Fire TV and TVs without Play Services are not supported. The TV must be switched on: an app on a sleeping TV cannot receive commands.

Troubleshooting

The TV does not react to commands. Check that the TV is on and online, then open the Port OS app once. If Port OS says the TV needs to be paired again, the app was reinstalled or its data was cleared: pair it again from step 4.
Commands arrive but nothing opens. The first permission in step 3 is missing. Run the SYSTEM_ALERT_WINDOW command again.
Buttons or screen pictures do nothing. The accessibility service from step 3 is not enabled. Run the two settings put secure commands again (keeping any other service in the list), then open the Port OS app: it says when the service is off.
Port OS does not know what is playing. The second permission in step 3 is missing. Run the allow_listener command again, and check that the component name is printed exactly as above.
adb: failed to connect. Wireless debugging reports a new port each time it is switched on. Read the address from Developer options > Wireless debugging again.
Pairing says the code is not valid. The code expires after a few minutes. Close the app, open it again and approve the new code.
The TV disappeared from Port OS. Delete the old connection in Port OS and pair the TV again.

For AI agents

For AI, you should consider the following when implementing things.

What is playing, and in which app. Read now-playing with a fresh query: endpoint_query with endpointId, expand ["media-player"] and forceDeviceAttributeQuery true. The TV is asked for a new reading and answers within a few seconds. Without forceDeviceAttributeQuery you get the last reading the TV pushed, which can be a minute or more old. now-playing names the app (appId such as com.google.android.youtube.tv, and provider YOUTUBE or SPOTIFY), the title (name), the full portos-media mediaId when the app exposes one (it is left out otherwise), and positionMs.
Play, pause, next, previous, stop. These media-player commands act on whichever app holds the active media session, YouTube or Spotify alike. If media-player pause is ambiguous because the endpoint also has another media-player route, send keypad-input send-key with {"keyCode":"PAUSE"} (or PLAY), which reaches the same session.
Seek and skip. media-player seek with {"positionMilliseconds":90000} jumps to that position of the current item, skip-forward and skip-backward with {"deltaPositionMilliseconds":10000} move that far from where it is now (the distance is required), and start-over with {} goes back to the beginning. They act on the same active session and work only when the playing app lets other apps seek (YouTube and Spotify do). Otherwise the TV logs why and nothing moves, so read now-playing positionMs afterwards. Needs app 0.5.2 or later.
Open an app. application-launcher launch-application with {"catalogVendorId":0,"applicationId":"com.netflix.ninja"} opens that Android package on the TV (catalog 0 means Android package names). The apps the TV can open are in the interface's application-list attribute, as package names only (no labels). An unknown or non-launchable package is not refused by the API: the delivery succeeds and the TV logs APP_NOT_LAUNCHABLE, so check with a snapshot. Needs the route rediscovered after the app updates to 0.5.2.
Check that a command worked. A command succeeds when Google accepts it for delivery, which does not mean the TV acted. Read now-playing again with forceDeviceAttributeQuery true, or take snapshot get-snapshot with responseFormat: BASE64 and look at the screen. play-media is the same: it opens the app on the media, so confirm it is playing the same way.