# Hey Taby Local Integrations — Developer Preview > Canonical context for agents and developers integrating with the local Hey Taby desktop app. This is a Developer Preview. The integration server is disabled by default, runs inside the desktop app, and binds only to the loopback address `127.0.0.1`. It is not a public Hey Taby cloud API. Do not send the REST requests below to `heytaby.com`; send them to the running desktop app's local loopback endpoint. ## Choose the interface - Use REST for scripts, curl, n8n, and workflows that know the exact action. - Use MCP when a local agent should discover typed Hey Taby tools. - curl is an HTTP client, not a separate integration protocol. - Taby's built-in local Brain runs a downloaded Taby-tuned Gemma model through the desktop app's managed `llama.cpp` runtime. It does not require Ollama. - Ollama, LM Studio, and similar model hosts are optional and external. An agent using one of them still reaches Taby through MCP or REST. - Parakeet is Taby's local speech-to-text engine on supported machines and languages. It transcribes speech; it is not the Brain or an integration protocol. REST and MCP use the same validated capability layer. Neither interface exposes arbitrary code execution, raw physical-device commands, uploaded visual assets, or unrestricted outbound HTTP. This is an inbound surface: local scripts, n8n, and MCP agents call Taby. Neither Taby's local nor Cloud Brain is currently an outbound MCP client. It cannot use `taby_brain_ask` to discover or invoke Gmail, n8n, or user-configured MCP servers. Outbound Brain-to-connector access is planned and requires separate connection, credential, consent, approval, audit, and revoke controls. ## Enable and create a connection The primary setup is managed inside the desktop app: 1. Open **Settings > Integrations > Local API & MCP**. 2. Turn on **Allow local connections**. The app-managed server binds only to `127.0.0.1` on fixed port `43123`. 3. Choose **Add connection**, give the local tool or workflow a recognizable name, and enable at least one permission. 4. Save the Bearer token immediately. It is shown only when the connection is created or its token is rotated; Taby stores only a protected hash and a short display hint. 5. Use **Copy MCP setup** or **Copy HTTP example** and place the token in the local client's credential store. Each named connection has eight boolean permissions: - **See Taby's status** (`status.read`) — status, animation discovery, and redacted logs. - **Use digital Taby** (`companion.digital.write`) — digital animations and cards. - **Use physical Taby** (`device.physical.write`) — physical animations and cards. - **Ask for a choice** (`prompts.write`) — create, poll, and cancel two-choice prompts; at least one Taby surface must also be enabled. - **Ask the local Brain** (`brain.read`) — completed read-only Brain answers using local Taby context, including tasks. - **View tasks** (`tasks.read`) — list and get structured tasks. - **Create and change tasks** (`tasks.write`) — create, update, complete, reopen, and restore tasks; this also enables task reads. - **Move tasks to Trash** (`tasks.delete`) — use Taby's recoverable Trash; this also enables task reads. The new-connection defaults enable status, digital Taby, and choice prompts; physical Taby, Brain, and all task permissions start off. Grant only what the named client needs. Settings can test any granted digital or physical surface, replace a token (which immediately invalidates the old token), revoke a connection, and show recent redacted activity attributed to connections. Revocation invalidates that connection without changing other named connections. Settings' activity view is durable and bounded across named connections; authenticated `GET /v1/logs` is a separate per-connection, in-memory action log that resets when the app restarts. Managed endpoints are stable: http://127.0.0.1:43123/v1 http://127.0.0.1:43123/mcp App-managed mode does not publish credentials in an endpoint descriptor. ### Advanced legacy/source-build mode Environment configuration remains available for development and source-build workflows. It is not the normal customer setup. When environment mode is active, Settings labels the server **Managed outside Taby** and its connection controls are unavailable for that app run. HEY_TABY_INTEGRATIONS_ENABLED=1 \ HEY_TABY_INTEGRATION_CLIENT_ID=my-local-agent \ HEY_TABY_INTEGRATION_TOKEN=<32-512-character-local-secret> \ HEY_TABY_INTEGRATION_SCOPES=tasks.read,tasks.write,tasks.delete \ npm run dev The enable value may be `1` or `true`, case-insensitive. The environment task scopes are `tasks.read`, `tasks.write`, and `tasks.delete`; write and delete also carry read. `HEY_TABY_INTEGRATION_PORT` is an optional development override. Environment mode publishes its generated or configured credential and live URLs to the private descriptor: /integrations/taby-local-integrations.json A legacy environment launch without explicit task scopes retains the original eight non-task MCP tools. This broad legacy identity is separate from the permission-driven named connections managed in Settings. The token is a live local credential. Send it as: Authorization: Bearer Never paste a real Bearer token into AI chat, source code, screenshots, workflow exports, or shared logs. Put it in the local client's secret or credential store. The unauthenticated health route is the only exception. Hosted agents, n8n Cloud, and normal Docker containers cannot reach the host's Hey Taby through `127.0.0.1`; that address refers to their own runtime. Use a client or workflow process running directly on the host operating system in the same network namespace as Hey Taby. Default containerized n8n deployments are unsupported in this preview. Do not expose this Developer Preview server to the internet. ## REST API v1 The app-managed HTTP base URL is `http://127.0.0.1:43123/v1`. It already contains `/v1`; do not add a second `/v1` segment. Append `/health`, `/status`, `/animations`, `/logs`, `/animations/play`, `/cards/show`, `/prompts/choice`, `/prompts/`, `/prompts//cancel`, `/brain/ask`, `/tasks`, or a documented `/tasks/` operation. The current REST surface has 15 path templates and 18 method/path operations. For reference, the implemented full server paths are: - `GET /v1/health` — minimal unauthenticated reachability check. - `GET /v1/status` — safe app, companion, target, device, and local-Brain status. - `GET /v1/animations` — approved renderable animation catalog. - `GET /v1/logs?limit=50` — recent redacted integration decisions; limit is 1 through 100. - `POST /v1/animations/play` — request an approved digital or physical animation. - `POST /v1/cards/show` — request a bounded plain-text status card. - `POST /v1/prompts/choice` — create an asynchronous two-choice prompt. - `GET /v1/prompts/:promptId` — poll prompt status and selected choice. - `POST /v1/prompts/:promptId/cancel` — cancel a prompt that is still pending. - `POST /v1/brain/ask` — get a completed answer from Hey Taby's installed local Brain, managed through `llama.cpp`. - `GET /v1/tasks` — list tasks; requires `tasks.read`. - `POST /v1/tasks` — create a task; requires `tasks.write`. - `GET /v1/tasks/:taskId` — read one task by stable UUID; requires `tasks.read`. - `PATCH /v1/tasks/:taskId` — partially update one task; requires `tasks.write`. - `DELETE /v1/tasks/:taskId` — move one task to recoverable Trash; requires `tasks.delete`. - `POST /v1/tasks/:taskId/complete` — complete one task; requires `tasks.write`. - `POST /v1/tasks/:taskId/reopen` — reopen one completed task; requires `tasks.write`. - `POST /v1/tasks/:taskId/restore` — restore one trashed task; requires `tasks.write`. The canonical machine-readable REST contract is: https://www.heytaby.com/openapi.json Successful REST responses use `{ "ok": true, "data": ... }`. Errors use `{ "ok": false, "error": { "code": string, "message": string, "issues"?: array } }`. ## MCP The app-managed MCP URL is `http://127.0.0.1:43123/mcp`, a stateless Streamable HTTP endpoint. Configure it in an MCP client running on the same computer and attach that connection's Bearer token as an Authorization header. Use a real MCP client rather than copying raw JSON-RPC curl calls. MCP tool discovery is permission-driven per named connection. A tool that the connection cannot use is omitted from `tools/list`: - `status.read`: `taby_get_status`, `taby_list_animations`. - `companion.digital.write` or `device.physical.write`: `taby_play_animation`, `taby_show_card`; each call may target only granted surfaces. - `prompts.write` plus a granted surface: `taby_prompt_choice`, `taby_get_prompt_result`, `taby_cancel_prompt`. - `brain.read`: `taby_brain_ask`. - `tasks.read`: `taby_list_tasks`, `taby_get_task`. - `tasks.write`: `taby_create_task`, `taby_update_task`, `taby_complete_task`, `taby_reopen_task`, `taby_restore_task`. - `tasks.delete`: `taby_delete_task`. All permissions together disclose 16 tools. The default Settings selection discloses seven: the two status tools, two display tools, and three prompt tools. Brain and task tools are absent until their permissions are granted. REST uses the same permission boundary and returns 403 for an authenticated connection that lacks the required permission. Important boundary: `brain.read` is separate from direct task permissions, but it allows the read-only Brain to use approved read tools and local Taby context, including task search. Do not grant Brain access to a client that must not read task or other Taby data. `tasks.read`, `tasks.write`, and `tasks.delete` gate only direct structured task tools and REST routes. The MCP action schemas and behavior match the corresponding REST capabilities. Call `taby_list_animations` instead of guessing an animation ID. This preview exposes MCP tools only; it does not currently expose MCP resources or reusable MCP prompt templates. ## Scoped task API Task reads return stable UUIDs and a privacy-safe DTO containing only `id`, `title`, `description`, `tags`, `category`, `status`, structured `deadline`, `estimatedPomodoros`, `completedPomodoros`, `timeSpentSeconds`, `color`, `createdAt`, `updatedAt`, `completedAt`, `deletedAt`, and `trashExpiresAt`. Responses never include linked-note contents, source-note payloads, image payloads or paths, launch items, folder IDs, device-template fields, or local paths. The task response schema is versioned with `schemaVersion: 1`. `GET /v1/tasks` accepts `status=open|completed|trashed|all` (default `open`), an optional text `query` of at most 200 characters, and `limit` from 1 to 100 (default 50). `taby_list_tasks` accepts the same fields. Use a returned stable UUID for later reads and mutations; never identify a task by its title. REST uses `:taskId` from the URL path and rejects a duplicate `taskId` in the JSON body. The corresponding MCP mutation tools take `taskId` in their typed input. Task create accepts: - `title`: required trimmed text, 1 to 200 characters. - `description`: optional string up to 8,000 characters or `null`; blank or whitespace-only text normalizes to `null`. - `tags`: up to 20 strings, each 1 to 40 ASCII letters, numbers, underscores, or hyphens; duplicates are removed case-insensitively. - `category`: `work`, `personal`, `coding`, `learning`, `meeting`, or `null`. - `deadline`: a structured deadline described below, or `null`. - `estimatedPomodoros`: integer 1 through 100, or `null`. - `color`: six-digit `#RRGGBB`, or `null`. - `requestId`: optional retry key. Task update requires a non-empty `patch`. Omitted fields stay unchanged; sending `null` clears a nullable field, and a blank description also normalizes to `null`. It accepts the same writable fields as create, with `title` optional. `expectedUpdatedAt` is an optional offset-qualified ISO 8601 timestamp; the mutation returns `409 task_stale` when the currently read task has a different `updatedAt`. Re-read before retrying a stale edit. Deadlines never use ambiguous free-form strings: - `{ "kind": "date", "date": "YYYY-MM-DD" }` is a valid local calendar date. Taby stores it at the end of that local day and returns the same local date. - `{ "kind": "date_time", "at": "2026-07-31T16:00:00+03:00" }` requires `Z` or an explicit valid UTC offset and a real calendar date and time. Impossible values such as February 30, `24:00`, or invalid offsets are rejected. Responses normalize it to UTC. Create, update, complete, reopen, delete, and restore return `requestId`, `changed`, and the current sanitized task. A repeated already-satisfied mutation can return `changed: false`. `DELETE` is soft deletion only: it moves the task into Taby's recoverable 30-day Trash and returns its `trashExpiresAt`. Permanent deletion is not exposed. Task sync in this preview means client-driven reconciliation, not push. Poll `taby_list_tasks` or `GET /v1/tasks`, retain each stable UUID, and compare `updatedAt`; use `taby_get_task` or `GET /v1/tasks/:taskId` when reconciling a task you already know. The list is a bounded snapshot of at most 100 tasks. There is no updated-since filter, pagination cursor, change feed, event stream, subscription, or webhook. A client must not claim continuous or complete background sync from this preview contract. For REST task mutations, send `Idempotency-Key` and/or body `requestId`. When both are present they must match or the server returns 409. MCP uses `requestId`. Keys are scoped to the configured client and action for five minutes. Identical retries return the retained success or failure outcome without re-executing the mutation. Idempotency and the bounded audit log live only in the current app process. After the five-minute expiry, a caller timeout, or an app restart, reconcile current state before retrying—especially task creation, which could otherwise be duplicated. ## Display action inputs `POST /v1/animations/play` requires `animationId` and accepts optional `targets`, `durationSeconds`, and `requestId`. - `animationId`: 1 to 120 characters; use an ID from `GET /v1/animations`. - `targets`: `digital`, `physical`, or both; defaults to `["digital"]`. - `durationSeconds`: integer from 1 through 30; defaults to 4. - `requestId`: 1 to 120 letters, numbers, dots, underscores, colons, or hyphens. - Animation calls are visual only. Use the card action when visible text is required. `POST /v1/cards/show` requires `title` and accepts optional `subtitle`, `tone`, `animationId`, `targets`, `durationSeconds`, and `requestId`. - Digital cards allow title length 1 to 80 and subtitle length 1 to 160. - If `physical` is requested, title is limited to 18 and subtitle to 32 characters. - `tone` is `neutral`, `progress`, `success`, `warning`, or `error`; default is `neutral`. - `animationId` defaults to `confirmation`; targets and duration use the animation defaults. - Card text is plain bounded text, not Markdown or HTML. Links, remote assets, scripts, and controls are not supported. ## Asynchronous two-choice prompts `POST /v1/prompts/choice` and `taby_prompt_choice` create exactly one prompt with exactly two ordered choices. Creation returns immediately with a `promptId`; it does not wait for presentation or selection. Poll `GET /v1/prompts/:promptId` or `taby_get_prompt_result`, and cancel an obsolete pending prompt with `POST /v1/prompts/:promptId/cancel` or `taby_cancel_prompt`. Prompt input: - `title`: plain text, 1 to 80 characters; 18 maximum when physical is targeted. - `subtitle`: optional plain text, 1 to 160 characters; 32 maximum when physical is targeted. - `choices`: exactly two unique choices, ordered primary then secondary. - Choice `id`: 1 to 64 letters, numbers, dots, underscores, colons, or hyphens. - Choice `label`: plain text, 1 to 32 characters; 12 maximum when physical is targeted. - Choice `effect`: defaults to `{ "type": "none" }`. - `animationId`: approved animation ID; defaults to `confirmation`. Physical prompts require a play-once animation. - `targets`: `digital`, `physical`, or both; defaults to `["digital"]`. - `timeoutSeconds`: integer from 5 through 300; defaults to 30. When physical is targeted, it must also cover the selected animation, the 750 ms input-unlock grace, and at least one actionable second (for example, `boxing` requires 7 seconds). - `requestId`: the same optional five-minute idempotency key used by display actions. - Physical-target text should use printable ASCII in this preview; the current firmware font subset may not render other Unicode glyphs. Allowed choice effects are: - `{ "type": "none" }` - `{ "type": "dismiss" }` - `{ "type": "open_url", "url": "https://example.com/..." }` - `{ "type": "open_taby_section", "section": "tasks" }` `open_url` accepts only HTTP or HTTPS URLs up to 2,048 characters and rejects embedded usernames or passwords. It opens the URL through the operating system only after the person selects the choice; it is not a generic fetch or Brain tool. `open_taby_section` is limited to `home`, `tasks`, `schedule`, `habits`, `reminders`, `stats`, `for_you`, `review`, `settings`, `brain`, or `pages`. `pages` opens the Pages preview; customer builds currently show Coming Soon unless experimental Pages is enabled. In app-managed mode, every prompt belongs to the authenticated named connection that created it. Rotating that connection replaces its token; revoking it invalidates access and cancels its currently pending prompts. This local in-app connection flow is not OAuth 2.1 and does not make prompt copy trustworthy by itself. Prompt UI does not automatically preview an `open_url` host, so connect only trusted local clients and include the source and destination in prompt copy when relevant. Prompt status is `pending`, `selected`, `timed_out`, `cancelled`, or `skipped`. Digital and physical are two surfaces for the same logical prompt; the first valid selection wins. A selected result identifies `choiceId`, `label`, primary/secondary `position`, `surface`, `selectedAt`, and an effect status of `none`, `pending`, `completed`, or `failed`. Selection does not guarantee that an `open_url` or `open_taby_section` effect completed. For example, after the user chooses the secondary DISMISS button on digital Taby, `taby_get_prompt_result` and `GET /v1/prompts/` return a `data.result` shaped exactly like: { "choiceId": "dismiss", "label": "DISMISS", "position": "secondary", "surface": "digital", "selectedAt": "2026-07-28T18:30:00.000Z", "effect": { "type": "dismiss", "status": "none" } } A physical-button selection uses `"surface": "physical"`. The result records the winning selection and the effect outcome; it is not a durable event-history entry. Only one integration prompt may be pending at a time. A second create returns `409 prompt_busy`. Terminal results are retained in memory for up to ten minutes, with at most 100 records in the current app run. Poll them promptly; a timed-out result remains readable during that window. An unknown, pruned, or restart-cleared prompt returns `404 prompt_not_found`. A pending create response is not proof that the prompt was presented or seen. There is no push callback, subscription, change feed, or webhook for prompt results. App restart clears them; integrations must poll while the record is within the ten-minute, 100-record in-memory window. For n8n, create the prompt with an HTTP Request node, store `data.promptId`, use a short Wait node, then GET `/v1/prompts/`. Loop while status is `pending`, stop on a terminal status, and branch on `data.result.choiceId` only when status is `selected`. Cancel the prompt if the workflow no longer needs it. Keep polling below the shared request limit. Callback URLs are not supported and must not be invented. If a caller supplies a `requestId`, reuse it only when retrying the same logical action. Hey Taby retains both success and failure outcomes for the same client, action, input, and key for five minutes in the current process; an identical retry returns that outcome without re-execution. Reusing the key for different input in that window returns HTTP 409. After expiry, timeout, or restart, reconcile current state before retrying a create. ## Acceptance and delivery A successful animation or card response contains `targets.accepted` and `targets.skipped`. An accepted target means the request was validated and handed to the companion runtime. Accepted does not mean delivered, rendered, or seen. Interactive Hey Taby surfaces have priority, queued integration cues expire after 30 seconds, and a physical device can disconnect after acceptance. A response where every target is skipped is still a successful no-op. Inspect skipped reasons and `GET /v1/logs` instead of treating HTTP success as a display receipt. Possible skipped reasons are `digital_disabled`, `physical_disabled`, and `physical_offline`. ## Local Brain `POST /v1/brain/ask` requires a non-empty `prompt` of at most 8,000 characters. Optional `sessionId`, `clientTurnId`, and `requestId` are limited to 160, 200, and 120 characters respectively. The call always uses Hey Taby's installed local Brain through its managed `llama.cpp` runtime, even when normal Taby chat is set to Cloud Brain. It uses domain-level read-only execution mode. It may use approved read tools but cannot create, update, delete, start, navigate, or otherwise mutate Hey Taby data through the AI tool layer. The completed response contains answer text and sanitized provider/action metadata, with `pendingConfirmationId: null`. Normal local AI activity history and the redacted integration audit are still recorded. There is no separate MCP or REST read-later endpoint for the answer; the caller receives it in the completed `taby_brain_ask` or `/v1/brain/ask` response. Granting `brain.read` allows these approved Brain reads even when the same connection lacks direct `tasks.read`. A Brain-enabled client may therefore ask the Brain to search task data and other available Taby context. Treat every Brain-enabled connection as trusted to read that local context. An explicit online-search question may use the existing read-only search path; this is not a generic URL-fetch endpoint. Only one Brain request runs at a time, and a ready local model must already be installed. ## Limits and errors - Authenticated REST and MCP traffic shares a 60-request-per-minute budget. - The unauthenticated health route has a separate 60-request-per-minute budget. - Animation requests: 12 per minute. - Card requests: 10 per minute. - Prompt creates: 6 per minute. - Prompt cancellations: 12 per minute. - Prompt reads count toward the general request budget. - Brain requests: 6 per minute. - Task reads and writes: 60 per minute each; task creation has an additional 20-per-minute limit. - Requests targeting the physical device share a 6-per-minute budget. - JSON request bodies are limited to 32 KiB and have a 10-second arrival timeout. Relevant HTTP errors include 400 invalid input, 401 missing or invalid token, 403 rejected Host or Origin or insufficient connection permission, 404 missing route, prompt, or task, 405 wrong method, 408 body timeout, 409 stale task, idempotency conflict, or unavailable/busy Brain or prompt surface, 413 oversized body, 415 when a request with a JSON body does not use `application/json`, 429 rate limit, and 500 internal failure. ## Current boundaries and planned work | Capability | Availability | Integration contract | | --- | --- | --- | | Companion animations and cards | Developer Preview | Discover and request approved digital or physical reactions plus bounded visible text. | | Choice prompts | Developer Preview | Create, poll, and cancel one two-choice prompt; exact results are short-lived and are not a durable event feed. | | Local Brain | Developer Preview | Return one completed read-only answer from Taby's installed local Brain. | | Tasks | Developer Preview | List, get, create, patch, complete, reopen, move to 30-day Trash, and restore. Reconcile by polling stable IDs and `updatedAt`; there is no change-feed cursor or webhook. | | Reminders | Planned | No MCP tool or REST route yet. | | Notes | Planned | No MCP tool or REST route yet. | | Focus | Planned | No MCP tool or REST route yet. | | Schedule | Planned | No MCP tool or REST route yet. | | Habits | Planned | No MCP tool or REST route yet. | | Custom pages and pins | Planned | No MCP tool, resource, or REST route yet. | | Events and automation | Planned | No durable external event feed, webhook, `/v1/events`, or rule API yet. | The Developer Preview currently supports only the routes and MCP tools listed above. Tasks are the only scoped productivity-data domain available today. Reminders, notes, focus, schedule, habits, custom pages, and pins are planned; they cannot currently be read, synced, created, or updated through MCP or REST. Gmail and other OAuth-backed connectors are planned, not current. Within MCP, Taby currently publishes tools—not MCP resources or reusable prompt templates. Pages and pins will need a bounded, Taby-native resource and UI model rather than arbitrary HTML, scripts, or embedded credentials. If outbound connections ship, either Taby's local or Cloud Brain may use them only after the user grants the connection and its permissions. The shipped Settings flow already provides named local connections, one-time tokens, rotation, revocation, per-connection permissions, tests, and recent redacted activity. OAuth 2.1 remains future work for cloud-backed connectors; it is not used by this loopback-only local connection flow. Generic outbound HTTP actions, user-authored code, remote inbound access, a cloud relay, arbitrary firmware commands, uploaded animations, external event routes, and automation-rule CRUD are not current capabilities. Do not invent note, focus, schedule, reminder, habit, `/v1/events`, `/v1/rules`, page/pin endpoints, Gmail tools, or generic URL-fetch tools. Do not tell users that Taby's Brain can call an external MCP server. Re-check the official documentation and OpenAPI contract when the Developer Preview changes. ## Recommended agent sequence 1. Ask the user to enable **Settings > Integrations > Local API & MCP**. 2. Ask the user to create a named connection with only the required permissions. 3. Have the user save the one-time token directly in the local client's credential store. 4. Use the copied MCP setup or HTTP example for the fixed loopback endpoint. 5. Call `GET /v1/health`, then authenticated `GET /v1/status` when `status.read` is granted. 6. Inspect MCP `tools/list` or the connection's selected permissions before attempting an action. 7. Read tasks and retain their stable IDs before mutating them. 8. Call `GET /v1/animations` before choosing an animation ID. 9. Ask the user before invoking a mutation or digital/physical display action. 10. Reuse a request ID only for a retry of the same logical action in the same app run. 11. Inspect accepted and skipped targets; never claim visual delivery from acceptance alone. 12. For a choice prompt, retain its promptId, poll to a terminal status, and cancel it when obsolete.