How to connect an LLM to your internal data safely

What this automation does

It gives an AI assistant or agent controlled, audited access to live records in systems like Salesforce, HubSpot, your ERP or a Postgres database, so it can answer questions and draft work using current data instead of what someone pasted into a chat.

1. User asks. A signed-in user asks something like “which of my accounts renew next quarter and have open support escalations?”
2. Identity travels with the request. The access layer receives the user’s identity from single sign-on and resolves what they are allowed to see in each source system.
3. Model picks a tool. The model chooses from a short list of named tools, such as search_accounts, get_open_tickets or get_invoice_status, each with typed parameters.
4. Access layer runs the call. Your code, not the model, calls the source system with the user’s permissions, applies row and field limits, and redacts fields the use case does not need.
5. Model answers. It combines the results into an answer that names the records it used, with links back to the source system.
6. Everything is logged. User, question, tool calls, parameters, records returned and the final answer go to an audit log.

When it is worth building

Build this when people spend time copying data out of systems to ask questions about it, or when you want an AI agent to act on live records rather than static documents. Account reviews, order status questions, vendor lookups and pipeline questions are common starting points.

Do not start here if your need is mostly answering from policies, manuals and wikis. That is a document retrieval problem, covered in how to build a RAG knowledge assistant. And if the question is mostly numbers across tables, natural-language SQL reporting with a read-only warehouse role is simpler. Many vendors also now ship their own AI assistants inside the product. If one covers your questions using the permissions you already set, use it.

Architecture

How to build it

1. Classify the data. List each source and its sensitive fields: salaries, health information, bank details, personal contact data. Decide which fields the assistant never receives, and remove them in the access layer.
2. Start with three to five read tools. Design them around real questions, not around the API. “Get account summary” with a fixed field list beats a generic “query Salesforce” tool.
3. Wire identity end to end. Use delegated tokens where the source supports them. Where it does not, enforce the user’s permissions in the access layer from your identity groups, and document that gap.
4. Cap every result. Limit rows, truncate long text fields and strip attachments by default. Large unfiltered results are both a leakage risk and a cost problem.
5. Contain untrusted content. Emails, ticket bodies, CRM notes and uploaded files can contain text from outsiders. Mark tool results as data, do not let the model render images or links from them, and do not give the same session an outbound tool like “send email” without a confirmation step.
6. Add writes only with previews. Each write tool validates its inputs, shows the user a diff, and runs only after an explicit confirm. Log the confirmation.
7. Build an evaluation suite. Include ordinary questions, permission tests (user A asking for user B’s records), questions that should return nothing, and injection tests with planted instructions in sample records. Run it on every change.

What breaks in production

• Over-privileged service accounts. The quick prototype used an admin key, and it shipped. Anyone with assistant access now has admin read. Audit the credentials before launch.
• Data exfiltration through rendering. A planted instruction asks the model to include a markdown image whose URL carries record data. If your interface renders it, the data leaves. Disable remote image rendering and restrict link domains.
• Tool confusion. With 30 overlapping tools, the model calls the wrong one or loops. Fewer, clearly named tools with good descriptions work better.
• Pagination and silent truncation. The tool returned the first 100 rows and the model answered as if that were everything. Return a “more results exist” flag and have the model say so.
• Logging sensitive data in plain text. The audit log becomes the easiest place to steal data. Apply the same access controls and retention rules to it as to the source.
• Model or prompt changes shifting behavior. A new model version calls tools differently. Pin versions and rerun the evaluation suite before switching.

Timeline and cost

Connecting one or two systems with read-only tools, user-level permissions, audit logging and an evaluation suite usually takes three to five weeks. Adding more sources, write actions or a custom interface extends it to six to eight. Lumeor builds this in the range of $12,000 to $40,000, depending on the number of systems, whether they support delegated access, and how much write capability you need. See what adding AI to existing software costs for the broader picture.

How to measure success

• Permission and injection test pass rate on every release (must be 100 percent before shipping)
• Share of answers where the cited records actually support the answer, from weekly sampling
• Time saved on recurring lookups, such as account prep or order status checks, measured with a few users before and after
• Tool error rate and average tool calls per question, which reveal confusing tool design
• Number of security review findings on the access layer and their time to fix