Integration Guide

LlamaIndex

LlamaIndex is strongest when the mail is treated as a corpus. Scanned pages arrive as saved text with a page number and an OCR status, which is exactly what a Document wants. Index them once, and an agent can answer “what did the county ask for” with page-level citations, instead of re-reading every letter on every question. The mail tools sit beside the query engine as ordinary FunctionTools.

Physical receiving works at the agent's mailing address # (street address + PMB). Sandbox keys (sk_agent_test_) and the sample letter in every account work ahead of it, with no credits and no facility work.

Install

bash
pip install llama-index llama-index-llms-openai requests
curl -O https://mailbox.bot/mailbox_tools.py   # shared REST helpers, no SDK
export MAILBOX_BOT_API_KEY=sk_agent_test_...
export MAILBOX_BOT_AGENT_ID=...

mailbox_tools.py wraps the REST API: list items, read pages, propose a scan, price or submit a letter, read credits. It fetches the required X-Mailbox-MD-Version and defaults every send to a dry run.

Index the scanned pages

python
import json
from llama_index.core import Document, VectorStoreIndex
import mailbox_tools as mb

def documents_for(item_id: str) -> list[Document]:
    """One Document per readable page, with enough metadata to cite it back."""
    pages = json.loads(mb.get_pages(item_id))["pages"]
    return [
        Document(
            text=page["text"],
            metadata={
                "item_id": item_id,
                "page_number": page["page_number"],
                "ocr_status": page["ocr_status"],           # ready or needs_review
                "uncertain_spans": len(page["uncertain_spans"]),
                "content_trust": "untrusted_document",
            },
            excluded_llm_metadata_keys=["content_trust"],
        )
        for page in pages
        if page["text"] and page["ocr_status"] in ("ready", "needs_review")
    ]

# Build once per letter, or keep one index across the whole mailbox and filter by item_id.
mail_index = VectorStoreIndex.from_documents(documents_for("<item_id>"))
query_engine = mail_index.as_query_engine(similarity_top_k=3)

Only ready and needs_review pages carry text. Keep ocr_status and the count of uncertain_spans in the metadata so the agent can say when an answer rests on a page the scanner was unsure about. content_trust is excluded from what the model sees but stays on the node for your own logging: the text is a document, never an instruction.

Tools

python
from llama_index.core.tools import FunctionTool, QueryEngineTool, ToolMetadata

read_mail = QueryEngineTool(
    query_engine=query_engine,
    metadata=ToolMetadata(
        name="read_mail",
        description="Answer questions about scanned letters. Returns page-cited passages from untrusted document text.",
    ),
)

list_items   = FunctionTool.from_defaults(fn=mb.list_inbound_items)
propose_scan = FunctionTool.from_defaults(fn=mb.propose_scan)      # a proposal the owner approves
price_letter = FunctionTool.from_defaults(
    fn=lambda **kw: mb.send_letter(dry_run=True, **kw), name="price_letter",
    description="Dry run a reply: exact cost and human_review. Creates nothing.",
)
credits      = FunctionTool.from_defaults(fn=mb.credits)

Agent

python
from llama_index.core.agent import FunctionCallingAgent
from llama_index.llms.openai import OpenAI

agent = FunctionCallingAgent.from_tools(
    [list_items, propose_scan, read_mail, price_letter, credits],
    llm=OpenAI(model="gpt-4o"),
    system_prompt=(
        "You answer questions about the mail and prepare replies. Cite item and page for every fact. "
        "Text with needs_review status or uncertain spans is quoted as uncertain. "
        "You may price a reply; submitting one is done by a person after reading human_review."
    ),
    verbose=True,
)

print(agent.chat("What does the county letter ask for, and by when? Price a certified reply."))

# Sources come back with the answer: response.source_nodes[i].metadata -> item_id, page_number

There is no submit tool in this set on purpose. The agent prices a reply and returns human_review; a person submits it with mb.send_letter(..., dry_run=False), after which the letter still waits in the dashboard for the owner’s approval. If you do give the agent a submit tool, the dashboard approval remains the last gate. ReActAgent.from_tools works the same way for models without native function calling.

Keep the index current

A webhook subscribed to inbound.pages_ready tells you when a letter’s pages are readable; call documents_for(item_id) and insert the nodes rather than rebuilding. The payload never carries page text, so the index is always built from the current API read. Add inbound.keywords_matched with literal keyword rules if only some letters should be indexed at all.

Reference