Aria's tools, Part 2: The web and other AIs

Part 1 covered Aria’s built-in tools, routing, and follow-through. This part is about the tools that reach outside: a real browser, external tool servers, and other AI services. Each of those is useful, and each is a way for something on the internet to reach back into a home network. So most of this post is about limits.

Its own browser

read_web_page from Part 1 reads a single page. Sometimes that isn’t enough: the answer is a click away, or the page only renders with JavaScript. So Aria can have its own browser: a headless Playwright browser in its own container (Compose profile browser), used through MCP.

The container is set up so a browser can’t become a way in:

• Its own network. The browser container sits on a browser network with internet access and no route to PostgreSQL, Qdrant or the model server.
• Headless, with an in-memory profile. No saved cookies, no logins that persist, and no screenshots returned.
• Read and navigate only. The default tools are navigate, go back, read the page (an accessibility snapshot), click, and wait. No typing, no form filling, no uploads, no script evaluation, no code execution. Those can be added in configuration, but they’re off by default.
• Public addresses only. Every url argument must be a public http(s) address, with every IP it resolves to checked, before the call. After each call, the page’s actual URL is checked again. If a click or a redirect lands on a private address, the browser is reset to about:blank.

That last check is the same address policy as the knowledge fetcher (Knowledge, Part 1), which makes it one rule enforced everywhere Aria touches the web.

The same container does a second job: when the knowledge reader finds a page with no extractable text (a JavaScript app), it renders it in this browser and extracts it again. That uses a fixed script returning the page’s HTML, in its own session, and the model never sees it.

MCP, curated

Aria speaks the Model Context Protocol, so any MCP server can give it tools. But Aria’s rule for MCP is strict, and it’s one of the project rules from the introduction: MCP is curated.

• Only servers listed in configuration are used. Nothing is discovered.
• Only the tools listed for each server are offered. A server that adds a new tool tomorrow doesn’t give Aria a new ability tomorrow.

Servers come from configuration (the Playwright URL, a JSON list of servers with their allowed tools, or the app settings). Aria connects with the official C# MCP SDK and refreshes the allowed tools in the background every five minutes (every 30 seconds while a server is down). MCP tools then go through exactly the same executor as built-in tools: the allow list, the per-turn limit, timeouts, truncation and the “data, not instructions” label.

The System page lists each server, whether it’s connected, and which of its curated tools it offers or lacks. If a server drops, Aria’s self-repair reconnects it (Running it locally, Part 2).

The reason for curation is simple. An MCP server is code I didn’t write, describing tools in its own words to a model that will follow those words. Auto-discovering tools means letting any server decide what Aria can do. Listing them means I do.

Asking another AI

Local models know less than the big hosted ones, and Aria’s knowledge library only has what it has read. Sometimes you ask something it simply doesn’t know.

So Aria can ask an external AI service, Google Gemini first, or any OpenAI-compatible API. It’s built as a last resort, in this order:

1. The ask_external_ai tool is only offered at all when an enabled credential exists.
2. It first searches the knowledge library, and if there’s something relevant, returns that. No external call.
3. Only when nothing is relevant does it ask an external service: your personal credentials first, then global ones, in priority order, skipping any whose limits are reached.
4. Only the question is sent, as written by the model, and the tool asks for it without personal details. Your conversation, memories and name don’t leave.
5. Every answer is saved to the library, as a knowledge document under “Answers from external AI” (with a low source weight), and indexed.

That last step means the next time you ask something similar, the library answers and no external call is made. Aria gets a little less dependent on outside help with every question.

The credentials vault

External services need API keys, and keys don’t belong in .env files or the database in plain text. They live in an encrypted vault:

• AES-256-GCM encryption, in PostgreSQL.
• A master key generated on first start into its own Docker volume, or provided with ARIA_MASTER_KEY. (This is why the backup includes the key volume: without it, the stored credentials can’t be decrypted.)
• The API never returns a key, only its last four characters. Once saved, a key is never shown again.
• Keys are global (shared) or personal (used first for one person).

Each credential has its own rate limits: requests and tokens per minute (tracked in memory, corrected after each call), and per day (stored in the database, so a restart doesn’t reset them). Gemini’s free tier is prefilled: 15 requests a minute, 250,000 tokens a minute, 1,000 requests a day. A 429 from the provider counts as a full minute used.

Settings → Services & credentials manages them and shows usage.

What I learned

• Isolate by network, not by promise. A browser container with no route to the database is safer than any instruction not to go there.
• Check addresses before and after. Validating the URL is half the job. Checking where a click or redirect actually landed is the other half.
• Allow-list tools, never discover them. Curation keeps “what can Aria do?” a question with a fixed answer.
• Make outside help a cache miss. Library first, external AI second, and the answer saved: the expensive path gets used less over time.

Next: Her looks, Part 1, where Aria gets a face, eleven expressions, and an image look that every picture of it is drawn from.