Hermes Agent browser: how it browses, which backend to pick, and how logins work

How the Hermes Agent browser works

The browser is a toolset called browser. With the built-in tools the agent does not look at pixels by default: it asks for a text snapshot of the page, which lists the interactive elements with short reference IDs such as @e1 and @e2, and then acts on those references. The main built-in tools are:

• browser_navigate to open a URL, and browser_snapshot to read the page;
• browser_click, browser_type, browser_scroll, browser_press and browser_back to move around and fill things in;
• browser_vision, which takes a screenshot and has a vision model describe it, for pages where the text snapshot misses the point (charts, odd layouts, some CAPTCHAs);
• browser_console for the page’s console output, and browser_dialog for native alerts and confirmations when the browser is attached through /browser connect or browser.cdp_url;
• browser_cdp, a raw Chrome DevTools Protocol passthrough that only appears when the browser is attached over CDP (see the Chrome section below).

One thing to know early: by default Hermes may offer a different tool instead. When the Browser Use command-line tool can be run, the agent gets a single browser_exec tool that writes and runs Python against the browser, and only in sessions that also have terminal access. If the command-line tool is missing, you get the individual tools listed above. To force them (including in sessions without terminal access), use /browser use off or browser.backend: "off". Camofox always uses the individual tools. The backend choices below apply to both modes, but the tools, how output is handled and some settings differ.

For simply reading a page, the Hermes documentation recommends web_search and web_extract instead: they are faster and cheaper. Reach for the browser when something has to be interacted with.

Which backend to choose

The same tools can run on several different browsers. The choice decides cost, how easily sites recognise a bot, and whether the browser stays signed in. In short:

• Local browser (the default, free). A packaged Chromium driven by the agent-browser command, started on demand. Nothing to pay for, and it works on a server with no screen. It is the plain choice for ordinary pages, and it looks like what it is: an automated headless browser.
• Cloud browsers: Browser Use, Browserbase or Firecrawl. A browser run by a provider that you pay. Upstream documents stealth features, residential proxies and CAPTCHA solving for Browser Use and Browserbase (some Browserbase options need a paid plan), which is the reason to choose them for sites that block automation or for running many sessions at once. The costs are provider credits and session timeouts that follow your plan.
• Camofox. A self-hosted server that wraps Camoufox, a Firefox build with fingerprint spoofing. It is for anti-detection browsing on your own hardware without a cloud account. You run the server yourself, typically in Docker.
• Lightpanda. A very light headless engine for small machines. Choose it in hermes tools under Browser Automation. It has no screenshots, and with the built-in tools Hermes falls back to Chrome for actions it cannot do.
• Your own Chrome over CDP. Hermes attaches to a Chrome, Brave, Chromium or Edge window that you started. Free, and you can watch it work, but it runs on your machine.
• Your real profile. browser.use_real_profile copies your everyday browser profile, with its logins, so the agent browses as you. Convenient, and the riskiest option (see the safety section).

A rough rule: start on the local browser; move to a cloud provider or Camofox when a site blocks you or you need scale; use your own Chrome when you want to watch or need your own session.

One detail catches people out. The cloud provider is set by browser.cloud_provider, which hermes tools writes for you. Once it is set, adding or removing an API key in .env does not switch providers; that file only holds the credentials. A paid Nous Portal subscription can also supply a browser through its Tool Gateway.

Set up the browser step by step

First, make sure the browser toolset is on. The simplest route is the tool picker:

hermes tools

Pick Browser Automation there and choose a backend. It writes the choice to ~/.hermes/config.yaml. To set the toolset list for the interactive terminal directly:

hermes config set platform_toolsets.cli '["hermes-cli", "browser"]'

The reverse also exists: listing browser under agent.disabled_toolsets removes it, which is useful on a locked-down deployment.

The local browser needs Node.js, the agent-browser command and Chromium with its system libraries. Choosing the local browser in hermes tools runs that setup; to install the pieces by hand:

npm install -g agent-browser
agent-browser install --with-deps

For a cloud provider, add its credentials to ~/.hermes/.env and select it in hermes tools:

# Browser Use
BROWSER_USE_API_KEY=your-key

# Browserbase
BROWSERBASE_API_KEY=your-key
BROWSERBASE_PROJECT_ID=your-project-id

# Firecrawl
FIRECRAWL_API_KEY=your-key

For Camofox, point Hermes at the running server and pick Camofox in the same menu:

CAMOFOX_URL=http://localhost:9377

A few config.yaml options for the built-in tools are worth knowing (browser_exec has its own timeout, 300 seconds by default, and is not governed by the snapshot and recording settings):

browser:
  headed: false              # true opens a visible window on a machine with a screen
  inactivity_timeout: 120    # seconds before an idle session is closed
  command_timeout: 30        # base timeout; page loads get at least 60 seconds
  snapshot_threshold: 15000  # characters before a page snapshot is truncated
  record_sessions: false     # true saves sessions as WebM video

Check it with a plain request such as “open example.com and tell me the page heading”. If you are still setting Hermes up, how to install Hermes Agent comes first.

Connect your own Chrome (CDP)

Connecting your own browser gives you free browsing that you can watch live. With the built-in tools it also exposes browser_cdp to the agent (in Browser Use mode the same access sits inside browser_exec). Start a Chromium-family browser with a debugging port and a dedicated data directory:

google-chrome \
  --remote-debugging-port=9222 \
  --user-data-dir=$HOME/.hermes/chrome-debug \
  --no-first-run \
  --no-default-browser-check &

The separate --user-data-dir is not optional on current versions. Since Chrome 136 the debugging flag is ignored for your default profile directory, so the port never opens, and Chrome 144 and later add an “Allow remote debugging?” prompt for the real profile. A dedicated directory avoids both. The catch is that it starts signed out of everything.

Then attach Hermes, in either of two ways:

• In the interactive terminal (hermes or hermes chat), run /browser connect. Hermes tries port 9222, and can launch a detected browser with remote debugging for you. /browser status and /browser disconnect check and detach.
• Or set the endpoint in config.yaml, which also works for a gateway that you run in the background:

browser:
  cdp_url: "http://127.0.0.1:9222"

Note that /browser connect is a slash command for the interactive terminal only. Typed into a Telegram or Discord chat, it reaches the agent as plain text and does nothing. If Hermes runs inside WSL2 while the Chrome you want sits on Windows, the documentation suggests a Windows-side browser MCP server rather than /browser connect.

Signing in to sites the agent needs

By default the local browser starts from a clean profile, so the agent is signed in to nothing. You have four ways to deal with that:

• The Passwords & Logins vault. In an interactive terminal, TUI or desktop session with a browser attached over CDP, ask the agent to log in to a site. When it reaches a sign-in page it has no login for, it asks you in a masked prompt, stores the login encrypted under ~/.hermes/vault/, and fills the password straight into the page. Camofox does not support this fill path. The model never sees the password. Fills are refused unless the page’s origin exactly matches the one saved. Manage entries with hermes vault list, hermes vault add and hermes vault rm.
• Two-factor codes. If you save the authenticator key with the login, Hermes generates the code itself. In an interactive session, a code sent by text or email is typed into a small prompt and entered for you. For passkeys, hardware keys and “tap Approve” prompts, the agent tells you to finish on your device and waits.
• Sign in yourself in a visible window. Set browser.headed: true on a machine with a screen. The window stays open between turns, so you can watch, handle a sign-in challenge or a CAPTCHA by hand, and keep the session warm. Idle sessions are still closed after browser.inactivity_timeout.
• Reuse your real profile or Camofox persistence. browser.use_real_profile: true copies your active profile, cookies and saved logins into a snapshot under ~/.hermes/browser-profile/ and drives that copy. For Camofox, browser.camofox.managed_persistence: true keeps cookies and logins across restarts if your Camofox server supports persistent profiles.

If you already use 1Password or Bitwarden, Hermes detects the op or bw command-line tool when it is installed and signed in, and can fill from those logins after you unlock the manager once. Unattended jobs such as cron cannot answer a prompt, so save the login from an interactive session first. The vault does not protect you from the page itself: once a password is typed into a site, that site has it, as it would if you typed it.

Safety and limits

Network guard. How strictly browser addresses are checked depends on the backend. With the built-in tools, a local browser can reach private addresses on your network, and with a cloud provider Hermes sends private addresses to a local browser instead. Other routes refuse private addresses unless browser.allow_private_urls is switched on, and cloud metadata endpoints stay blocked. After a page loads, the built-in tools check where any redirect ended up. browser_exec does not get this automatic routing. Treat it as a guard against obvious mistakes, not a firewall: if the agent must not reach something on your network, do not run it on that network.

Website blocklist. It is off by default. Switch it on to refuse chosen domains in browser_navigate and the web tools:

security:
  website_blocklist:
    enabled: true
    domains:
      - "admin.example.com"
      - "*.internal.company.com"

Real profile. Upstream describes browser.use_real_profile as a “consent-gated convenience, not an isolation boundary”. Every page the agent visits runs with your real logins. It is off by default; switch it on only when you want the agent acting as you, and consider a separate browser profile for it.

Limits, stated plainly:

• CAPTCHAs and bot checks. A headless local browser is easy to spot. Options are a cloud provider with stealth and CAPTCHA solving, Camofox, or a headed window where you pass the check yourself. browser_vision can read a visual challenge, but do not count on it.
• Downloads. The built-in tools have no download tool. Browser Use mode can set downloads up through the DevTools protocol.
• Long pages are cut off. With the built-in tools, snapshots larger than browser.snapshot_threshold (15,000 characters by default) are truncated; a longer copy, up to 2,000,000 characters, is saved under ~/.hermes/cache/web/ and the agent can read it from there.
• Text, not pixels. The built-in tools work from the accessibility tree, which can miss visual information, so unusual interfaces can confuse them. Browser Use mode can also hand screenshots to a vision-capable model.
• Native dialogs wait for a reply; under the default policy they are dismissed after 300 seconds when the browser is attached over CDP.
• Cloud sessions cost credits and expire according to your plan.

Common problems and where to look

• The agent has no browser tools. The browser toolset is not enabled, or it is listed under disabled_toolsets. Run hermes tools again.
• /browser connect does nothing. You are probably typing it in a messaging chat; use the interactive terminal, or set browser.cdp_url.
• Port 9222 refuses connections. Chrome was started against its default profile directory. Add a dedicated --user-data-dir.
• It keeps hitting a Cloudflare or CAPTCHA page. Switch to a stealth cloud provider or Camofox, or run headed and pass the check by hand.
• I changed an API key and nothing changed. The provider is chosen by browser.cloud_provider; select it in hermes tools.
• The agent seems to miss part of a page. With the built-in tools, the snapshot may have been truncated. Raise browser.snapshot_threshold and start a new session, or ask the agent to read the saved snapshot file.

Everything above runs on a machine you keep switched on, which matters for scheduled browser jobs. The comparison of self-hosting and managed hosting covers that trade-off, and the browser is one tool among many in Hermes Agent skills and the connected apps.

Live Browser on ReadyHermes

On ReadyHermes your own Hermes Agent runs on our servers and has its own browser, so there is no Chrome to install, no debugging port to open and nothing to keep running on your computer. The part that solves the sign-in problem is the Live Browser, a view of that same browser:

1. In the workspace menu, open Bot tools, find the Live Browser card and click Open Live Browser.
2. It opens in a new tab. If your browser blocks it, allow pop-ups for the site and try again.
3. Watch the assistant work, or take over: sign in to a site, or get past a check that the assistant cannot pass on its own.

The Live Browser keeps the assistant’s sign-ins and open pages, so after you sign in once the assistant can carry on from there. Each link works once: if the tab disconnects, return to the card and click Open Live Browser again. The assistant has to be running to open it, and it can take up to a minute to appear.

After at least 15 minutes with nobody watching, the live view can switch back to a lighter background mode to save resources. ReadyHermes looks for browser activity first, but that check cannot see every kind of work. The switch restarts the browser, so a browser step running at that moment may need a retry. The assistant’s browser keeps working in background mode, and opening the Live Browser again brings the live view back.

What is not on offer: the workspace has no browser-backend picker, so there is no switch for a cloud provider, Camofox or your own Chrome. The assistant uses its managed browser, the one you see in the Live Browser. For channels, skills and the rest of what is included, see managed Hermes Agent hosting, and for the desktop side see Hermes Agent desktop.