How to build natural-language SQL reporting

What this automation does

A manager types “what were weekly orders by region for the last eight weeks, excluding returns?” in Slack and gets a table, a one-line summary and the SQL that produced it.

1. Question. A user asks in Slack, Teams or a web page. The system knows who they are and which data domains they can query.
2. Template match. If the question matches a vetted metric, such as revenue, gross margin or active customers, the system uses the reviewed query template and fills in the parameters.
3. Generate. Otherwise, the model writes SQL against a curated set of reporting views, using their descriptions, column notes and example queries.
4. Validate. A SQL parser checks the query: a single SELECT, only allowlisted views and columns, a row limit, no functions on the blocklist. Anything else is rejected and regenerated or refused.
5. Run. The query runs on a read replica under a read-only role with a statement timeout.
6. Answer. The user gets the result as a table or chart, a plain-language summary, the SQL, and a note when the answer came from generated rather than vetted SQL.

When it is worth building

Build it when a few analysts spend their days on one-off pulls, and business users wait days for answers that are a GROUP BY away. It works best when your data already sits in one warehouse or database with reasonably clean tables.

Do not build it on top of a raw transactional schema with hundreds of tables and no documentation. You will spend the budget on wrong answers. Create reporting views first, which is useful anyway. And if your recurring reports are fixed, scheduled ones are simpler, as in automated weekly operations reporting. Ad-hoc questions are where natural-language SQL earns its keep.

Architecture

How to build it

1. Collect 100 real questions. Pull them from analyst request queues and Slack. Note the correct SQL and answer for each. This is your test set.
2. Build the reporting views. Name tables and columns in business terms, apply soft-delete and test-account filters inside the views, and document each column in one line.
3. Create the read-only role. Grant SELECT on the views only, set a statement timeout, and point it at a replica. Test that INSERT, UPDATE, DELETE, DDL and access to base tables all fail.
4. Write vetted templates for key metrics. For each number that appears in board or leadership reports, write and review the query once, with parameters for date range and segment. Route matching questions to these.
5. Build the generation prompt. Include view definitions, column notes, a handful of example question and SQL pairs, and rules such as “always filter by order date, not created date”.
6. Add the parser check. Reject multi-statement queries, unknown objects, missing LIMIT on non-aggregate queries, and anything outside the allowlist. Log every rejection.
7. Evaluate by result, not by SQL text. Run each test question and compare result sets with the expected answer. Different SQL that returns the same numbers is a pass.
8. Show your work. Always return the SQL and say whether a vetted template was used, so analysts can spot-check and users learn to trust the right answers.

What breaks in production

• Plausible but wrong numbers. The query runs and returns a clean table, but joins orders to line items and double-counts revenue. Vetted templates for key metrics and result-based testing are the main defense.
• Ambiguous terms. “Customers” can mean accounts, contacts or anyone who ever ordered. Define each term in the view descriptions and have the model state its interpretation in the answer.
• Fiscal calendars and time zones. “Last quarter” in a company with a non-calendar fiscal year, or dates stored in UTC, quietly shifts results. Provide a calendar view and state the time zone in every answer.
• Runaway queries. A cross join across two large tables. Statement timeouts and warehouse resource limits stop it before it costs money or slows other users.
• Schema changes. An engineer renames a column and the views break, or worse, still run with different meaning. Run the test set nightly and alert on failures.
• Row-level data the user should not see. A regional manager asks about another region’s payroll. Enforce row-level security in the database or separate views per role; do not trust the prompt to filter.

Timeline and cost

A first version over one database with 10–20 curated views, vetted templates for core metrics, a Slack interface and a test set usually takes three to six weeks. Building the reporting views is often half the effort. Lumeor builds this in the range of $12,000 to $35,000, depending on schema complexity, the number of data sources and how much semantic layer work is needed. Ongoing costs are model usage per question plus warehouse compute. For how this fits other AI integration work, see what it costs to add AI to existing software. If you want the model to reach systems beyond the warehouse, read how to connect an LLM to internal data safely.

How to measure success

• Test set accuracy measured on result sets, tracked per release
• Share of questions answered from vetted templates versus generated SQL
• Analyst hours spent on ad-hoc data requests per week
• Median time from question to answer for business users
• Queries rejected by the parser or killed by timeout, reviewed weekly for patterns