As of October 2026, AI agents facing the reality of the Android ecosystem have five distinct routes to interact with apps that lack public APIs. Most critical daily appsβ€”food delivery, rideshare, and bankingβ€”remain closed systems, forcing agents to rely on indirect control methods rather than direct programmatic access.

The Native and System-Level Options

The first route is the traditional API or MCP server, available only if the app developer has explicitly published one. The second, and arguably most promising native option, is Android AppFunctions. This experimental feature, requiring Android 16+, allows apps to annotate functions with @AppFunction, enabling direct calls from system agents. Google describes this as 'the mobile equivalent of tools within the Model Context Protocol.' However, adoption is limited to specific system agents like Gemini, which is currently in private preview for this capability.

Gemini and Accessibility Frameworks

For users on Pixel 10 or Galaxy S26 devices in the US and Korea, Gemini’s in-app tasks offer a third route. In beta since February 2026, this feature allows multi-step task execution within apps like DoorDash, Uber, and Instacart. Crucially, the agent stops before the final payment button, requiring human confirmation. The fourth route involves accessibility-based agents like Mobilerun, mobile-use, and AppAgent. These run an observe-act loop on devices connected via ADB, but developers must note that Google Play policy prohibits shipping apps that use AccessibilityService for autonomous planning and execution.

The Cloud Phone and Screen Reading Loop

The fifth route, represented by services like Phonebox, utilizes cloud Android phones driven by agents over MCP, CLI, or API. This method keeps sign-ins persistent between sessions and offloads processing from the user's local machine. Regardless of the route, the core operational loop remains consistent: observe the screen as a list of labeled elements (not just pixels), act by tapping visible text or swiping, and check the new screen state. This text-based observation ensures resilience against UI redesigns, as 'tap Sign in' remains valid even if the button moves.

Key Takeaways

  • Android AppFunctions is the cleanest future path but remains experimental and limited to Android 16+ with low app adoption.
  • Gemini’s in-app tasks require human intervention for final payments and are currently restricted to specific hardware and regions.
  • Accessibility-based agents are effective for development but face strict Google Play policy restrictions against autonomous execution in shipped apps.
  • Cloud-based Android instances offer a flexible workaround for any app, allowing persistent sessions and bypassing local hardware constraints.

The Bottom Line

We are in a transitional era where agents must bridge the gap between closed mobile ecosystems and open automation protocols. Until AppFunctions achieves widespread adoption, cloud-driven Android instances remain the most viable solution for robust, autonomous agent workflows.