Text-to-SQL with a semantic layer: NVIDIA Auto Ontology tested

On this page

NVIDIA Auto Ontology is an open-source library that aims to make text-to-SQL reliable. It reads the metadata of your databases and builds a semantic layer from it: business terms such as "customer" or "invoice line", the paths by which tables join, and reviewed SQL expressions for metrics. An agent then answers natural-language questions through that layer instead of guessing from the raw schema.

We tried Auto Ontology on Microsoft's Wide World Importers sample database with the hosted models from build.nvidia.com. The short version: the automatically generated ontology is impressive, and questions that span three to five tables work. With the hosted models, though, it currently needs three code changes, and two of four answers were wrong in a way the answer itself didn't show. This post explains the concept, walks through the setup and its pitfalls, and shows our results.

Why text-to-SQL needs a semantic layer

A language model asked to write SQL starts out seeing only tables and columns. It doesn't know that "revenue" in your database means extendedprice on the invoice lines and not the amount in the accounting table. It doesn't know how a state connects to an invoice either. With a small schema you can explain that in the prompt. With dozens of tables, archive copies, history tables and similar-sounding columns, the model guesses, and the result still looks plausible.

The usual answer is a semantic layer: a model of the business concepts that defines what a metric means and how it is calculated. In projects this is usually maintained by hand, for example as metrics in dbt or a BI tool. We described the manual approach in our post on a text-to-SQL agent for BigQuery. Auto Ontology tries to generate this layer automatically and then have people review it.

How Auto Ontology works

The ontology holds three kinds of definitions:

• Terms: business concepts such as Customer, Invoice or Stock Item, each with a description, synonyms and the columns that hold them.
• Relationships: how tables join, including bridge tables, inferred from keys and from the data.
• SQL attributes: reviewed SQL expressions under a term, such as a margin or a rolling total. The agent reuses them, so the same metric is always computed the same way.

The work happens in three stages:

How Auto Ontology works: generate, certify, answer

1. Generate: an ingestion service reads schemas, column profiles and sample values, derives terms table by table, detects join paths and suggests SQL attributes. Everything is stored in Postgres, with embeddings for retrieval in pgvector.
2. Certify: domain experts test with real questions, correct terms and SQL attributes in the web UI and mark them as certified.
3. Answer: for each question the agent works out the intent and the terms it mentions, finds matching terms, columns and SQL attributes by embedding search, writes SQL, checks filter values against the database, repairs the SQL when a check fails and runs it. It returns the answer, the SQL and the steps it took.

On top of that there is a web UI, a REST API and an MCP server that lets other agents use the ontology, for example to check up front whether a question can be answered from the available data at all. Our FastMCP guide covers how MCP servers work in general.

In the web UI you can explore the generated ontology as a graph. Each circle is a term, each line a detected relationship:

The generated Wide World Importers ontology as a graph

For knowledge that lives in documents rather than database schemas, the Neo4j LLM Knowledge Graph Builder builds a similar graph from PDFs and web pages.

The project is Apache 2.0 licensed, has no tagged release yet and, according to its README, does not accept external contributions at the moment. We tested the state of 4 November 2026 (commit 2a24f5d, package version 1.1.0).

Setup with Docker Compose

The official path in the README is Docker Compose. You need Docker with Compose v2, uv and an API key from build.nvidia.com. The key values in .env:

1DEFAULT_MODELS_API_KEY=nvapi-...
2AUTH_SECRET=<long random string>
3APP_URL=http://localhost:3000
4POSTGRES_PASSWORD=<new password>
5CONNECTION_STRINGS=postgresql://postgres:<same password>@postgres:5432/wideworldimporters
6AUTO_ONTOLOGY_ADMIN_PASSWORD=<new password>
7PGADMIN_PASSWORD=<new password>

Then start Postgres, load the sample database and build the rest:

1docker compose up -d --wait postgres
2git clone https://github.com/NVIDIA/auto-ontology-eval.git ../auto-ontology-eval
3(cd ../auto-ontology-eval && uv sync && cp .env.example .env && \
4  POSTGRES_PASSWORD='<same password>' \
5  uv run python scripts/seed/seed_wwi.py --database-name wideworldimporters --drop)
6docker compose up -d --build

Open http://localhost:3000, sign in with the admin account from .env and turn on compilation under Settings, Semantic Compilation. Ingestion runs at startup and compilation right after it.

The defaults are nvidia/nemotron-3.5-lightning-30b-a3b for all language model tasks and nvidia/nemotron-3.5-embed-1b for embeddings. Each role (reasoning, extraction, embeddings, reranking) can have its own endpoint, model and key, including self-hosted NIMs.

Three pitfalls with NVIDIA's hosted models

With the hosted models on build.nvidia.com, Auto Ontology did not get through at first. The causes sit between the code and the hosted endpoints, so they affect the Compose setup too. We reproduced each one directly against the API.

1. Ingestion reports success but creates no embeddings. The log says "Data ingestion finished successfully", next to zero embeddings and hundreds of "HTTP 500, service inference failed" errors. Auto Ontology sends 32 texts per embedding request. In our tests, the hosted endpoint for nemotron-3.5-embed-1b answered most requests with several texts with a 500, while one text per request worked reliably, even with 16 requests in parallel. Very long single texts also failed despite truncate: END.

2. Structured output is rejected. For NVIDIA models, Auto Ontology uses LangChain's ChatNVIDIA client. It sends the requested JSON schema as nvext.guided_json, and the hosted endpoint answers with HTTP 400: "unknown field guided_json". The OpenAI-compatible route, response_format with json_schema, works.

3. The model thinks its way into the token limit. Nemotron 3.5 reasons before every answer. For a small JSON answer that took more than 900 tokens in our test, and on larger tasks it ran into the 8,192-token limit (LengthFinishReasonError). Compilation became very slow as a result. With chat_template_kwargs: {"enable_thinking": false} the same answer took around 40 tokens and less than a second.

Our workarounds target exactly these points: one text per embedding request with 16 requests in parallel, texts cut to 4,000 characters, NVIDIA models routed through the existing ChatOpenAI branch, and thinking turned off. These are workarounds for the tested commit, not official fixes. Compose builds the services from the local source, so apply them in the repository before docker compose up -d --build:

1# Workarounds for the hosted NVIDIA models (not official fixes)
2sed -i 's/    inference_batch_size: int = 32/    inference_batch_size: int = 1/; s/    nim_http_max_concurrent: int = 32/    nim_http_max_concurrent: int = 16/' auto_ontology/utils/embedding.py
3sed -i 's/            input=\[prefix + stripped\[i\] for i in indices\],/            input=[(prefix + stripped[i])[:4000] for i in indices],/' auto_ontology/utils/embedding.py
4sed -i 's/    if model.startswith(("openai\/", "azure\/", "aws\/")):/    if model.startswith(("openai\/", "azure\/", "aws\/", "nvidia\/")):/' auto_ontology/utils/llm_invoke.py
5python3 - <<'PY'
6from pathlib import Path
7p = Path("auto_ontology/utils/llm_invoke.py"); s = p.read_text()
8old = "            timeout=LLM_INVOKE_TIMEOUT_S,\n            max_retries=0,\n        )"
9assert s.count(old) == 1
10p.write_text(s.replace(old, "            timeout=LLM_INVOKE_TIMEOUT_S,\n            max_retries=0,\n            extra_body={\"chat_template_kwargs\": {\"enable_thinking\": False}} if model.startswith(\"nvidia/\") else None,\n        )"))
11PY
12git diff --stat

1 auto_ontology/utils/embedding.py  | 6 +++---
2 auto_ontology/utils/llm_invoke.py | 3 ++-
3 2 files changed, 5 insertions(+), 4 deletions(-)

After that, the sample database's 640 embeddings went through in 14 seconds without errors. The run was not entirely free of token limits: a few steps still hit them, but Auto Ontology recovered with retries and a second attempt through the reasoning client.

What the test showed

Compiling the 51 tables of Wide World Importers took eight and a little over ten minutes in two runs (504 and 621 seconds) and produced 51 and 52 terms, including Customer, Invoice, Invoice Line, Order and Stock Item. One thing stood out: archive tables became terms of their own, for example "Buying Group Archive" next to "Buyinggroups". That kind of duplicate has to be cleaned up during certification, or the agent may pick the wrong table when in doubt. In the term list, every generated term starts as "Pending Approval":

Generated terms in Auto Ontology, none of them certified yet

We then asked four typical analytics questions and checked the generated SQL against the database ourselves. Each question took between half a minute and almost seven minutes.

The first question is the example from the README. The agent's SQL (line breaks added for readability):

1SELECT SUM(il.extendedprice) AS total_revenue
2FROM wideworldimporters.warehouse.stockitems AS si
3JOIN wideworldimporters.sales.invoicelines AS il ON si.stockitemid = il.stockitemid
4JOIN wideworldimporters.sales.invoices AS inv ON il.invoiceid = inv.invoiceid
5WHERE si.stockitemname LIKE '%sushi roll%'
6  AND si.stockitemname LIKE '%USB%'
7  AND inv.invoicedate >= '2015-06-01' AND inv.invoicedate < '2015-07-01'

Product and period are right, the state is missing. The result was revenue across all states: $5,225.60 instead of $588.80. The correct answer needs the path from invoice through customer and city to state, which the README shows in its example. The agent's answer described product and period correctly and simply didn't mention the state. If you only read the answer, you won't notice.

For the quantity question the agent used the stock movements table, which also contains goods received, instead of the invoice lines. That is a textbook case for a certified SQL attribute "quantity sold": once it is defined, the agent stops working it out from scratch each time.

The two correct answers show what works well: the agent writes clean joins across several tables, monthly grouping and sorting with a limit, and it always returns the SQL. That is what makes checking the answer possible in the first place. Here is the answer to the orders-per-month question in the chat:

An Auto Ontology chat answer with SQL and the result table

Below each answer, Auto Ontology lists the agent's steps, from classifying the question and picking tables to validating the SQL. If you need to check an answer, that is where you see why the agent chose a table:

The agent's steps behind an answer

What to settle before you use it

The test and the README point to four things:

1. Read-only database access. Auto Ontology does not check by itself whether the generated SQL only reads. When we asked "Delete all invoices from 2013.", the agent generated and ran DELETE statements in our test. Nothing was deleted only because of the sample database's foreign keys. The quickstart connects the agent as the Postgres superuser. In a real deployment, CONNECTION_STRINGS should use a user that can only read, or a read replica.
2. Protect the internal services. According to the README, the backend (port 3001) and the ingestion service (3002) belong on a private network. The backend trusts the user identity the frontend forwards in a header, and the ingestion service has no authentication of its own. People and agents should only get in through the frontend on port 3000.
3. Plan for certification. The generated terms are a draft. Before numbers go into reports, domain experts should check the most important terms and metrics against reference queries and certify them.
4. Be realistic about response times. With the hosted models some questions took several minutes. We haven't tested whether self-hosted NIMs or another endpoint speed this up significantly.

Our take

The most interesting part of Auto Ontology is the automatic generation. In eight to ten minutes, 51 tables turned into an ontology with terms and join paths that an agent could use right away. That modeling is otherwise tedious manual work.

In our view the project is not ready for production yet: there is no release, the hosted models need code changes, and half of our test questions got wrong SQL behind a plausible-sounding answer. That is not specific to Auto Ontology; it applies to text-to-SQL in general. Auto Ontology does bring the right tool against it, though: certification of terms and SQL attributes by domain experts. It is already worth evaluating on your own data, as long as you take that certification step seriously.

If you want to bring text-to-SQL to your company's data, we're happy to help with tool selection, the data model and setting up the review process. Get in touch. Our post on the BigQuery MCP server shows how to make BigQuery data available to agents over MCP.

Further reading

• RAG, embeddings and vector search with Google BigQuery
• Using AI directly from your PostgreSQL database with pgai

Interested in building high-quality AI agent systems?

We prepared a comprehensive guide based on cutting-edge research for how to build robust, reliable AI agent systems that actually work in production. This guide covers:

• Understanding the 14 systematic failure modes in multi-agent systems
• Evidence-based best practices for agent design
• Structured communication protocols and verification mechanisms

About the author

Andreas Nigg

Co-Founder, Pondhouse Data

Andreas has spent more than eight years building data applications. At Pondhouse Data he designs and ships AI systems and integrates them into complex, real-world environments.

Talk to Andreas about your projectFollow on LinkedIn