AI Blog: RAG, AI Agents and LLMs in Practice | Pondhouse Data

Text-to-SQL with a semantic layer: NVIDIA Auto Ontology tested

Thu, 05 Nov 2026 00:00:00 GMT

How NVIDIA Auto Ontology grounds text-to-SQL in an automatically generated semantic layer, what worked in our test and where it still falls short.

Key takeaways

Auto Ontology generates business terms, join paths and SQL expressions from database metadata for a text-to-SQL agent to use.
It built the ontology for the 51-table sample database in eight to ten minutes, with no manual modeling.
With NVIDIA's hosted models it only ran after three small code changes, which we show below.
Two of four answers had wrong SQL that the answer text did not reveal, so expert certification is not optional.
Connect the agent with a database user that can only read.

Confluence to LLM Wiki: Our Self-Maintaining Company Wiki

Tue, 11 Aug 2026 00:00:00 GMT

How we moved 310 Confluence pages into a Git-based company wiki that an LLM agent updates, cross-links and checks, with a human reviewing every change.

Key takeaways

We moved 310 Confluence pages and 720 attachments into a Git repository maintained by an LLM agent.
The first compilation produced roughly 160 active pages and 90 archived pages.
Original sources stay append-only in raw/, while the agent maintains the curated pages in wiki/.
Every active page records a human owner, verification date, sources and a trust status.
The agent proposes changes as one reviewable change set; a human approves it before it is merged.

Copilot Studio: Use SharePoint Lists as a Knowledge Source

Mon, 09 Feb 2026 00:00:00 GMT

Copilot Studio loads SharePoint lists whole. Two agent flow tools fix that: one reads internal column names, one fetches only matching items via OData.

Key takeaways

Copilot Studio loads a SharePoint list as a whole, so large lists flood the agent's context or get truncated.
An agent flow can query a list through the SharePoint REST API with OData filters and return only matching items.
A first tool reads the list's internal column names from the fields endpoint, so no column IDs are hardcoded.
Splitting fetch ids and fetch data into two tools lets the agent retry with a different OData query.
Tell the agent to prefer broad, fuzzy filters over eq on text columns to avoid missing relevant items.

Running GLM-4.7-Flash Locally with OpenCode

Thu, 22 Jan 2026 00:00:00 GMT

Run GLM-4.7-Flash, a 30B MoE coding model, on a 16GB GPU: build llama.cpp for NVIDIA or AMD, serve the Q4 GGUF and connect it to OpenCode.

Key takeaways

GLM-4.7-Flash is a 30B Mixture-of-Experts model with about 3B active parameters, released with open weights under the MIT license.
It scores 59.2% on SWE-bench Verified, versus 22.0% for Qwen3-30B-A3B-Thinking and 34.0% for GPT-OSS-20B.
With the UD-Q4_K_XL quant, GLM-4.7-Flash uses around 12-14GB of VRAM and runs on a 16GB GPU.
llama-server downloads the GGUF from Hugging Face with the -hf flag and serves an OpenAI-compatible API.
In opencode.json, the model ID must match the --alias you pass to llama-server.

Orchestrating Complex Tasks with Microsoft Agent Framework Workflows

Wed, 19 Nov 2025 00:00:00 GMT

Build graph-based Microsoft Agent Framework Workflows in Python: a four-step IT ticket pipeline, the DevUI, and conditional edges for urgent tickets.

Key takeaways

Use a ChatAgent for open-ended conversations and a Workflow for known, repeatable, auditable multi-step processes.
A Workflow is a graph of executors connected by edges, defined with the WorkflowBuilder API.
Executors pass data on with ctx.send_message and publish final results with ctx.yield_output.
Conditional edges send high-priority tickets to an alert node and all others to ticket creation.
The built-in DevUI shows the workflow graph, runs it and lets you inspect each node's inputs and outputs.

Microsoft Agent Framework: Build Enterprise AI Agents in Python

Wed, 05 Nov 2025 00:00:00 GMT

Microsoft Agent Framework explained: tools, workflows, state and checkpointing, then a Python IT helpdesk agent with Dev UI, FastAPI and Docker.

Key takeaways

Microsoft Agent Framework combines AutoGen's multi-agent patterns with Semantic Kernel's enterprise features, for Python and C#.
Its four core components are AI agents, typed tools, graph-based workflows and AgentThread state management.
Checkpointing saves workflow state to durable storage so agents can pause for approvals and resume later.
The built-in Dev UI shows an agent's tool calls and responses in the browser during development.
For production, wrap the agent in a FastAPI endpoint, package it with Docker and add a persistence layer.

BigQuery MCP Server: Google's Remote Server vs. MCP Toolbox, Tested

Mon, 30 Jun 2025 00:00:00 GMT

Connect AI agents to BigQuery via MCP: Google's managed remote server vs. MCP Toolbox 1.16, with setup, guardrails, costs and a 7-question agent test.

Key takeaways

Google runs a managed BigQuery MCP server at bigquery.googleapis.com/mcp; it is available as soon as the BigQuery API is enabled.
MCP Toolbox 1.16 adds what the remote server lacks: a dataset allowlist, a read-only mode and a per-query cap on bytes billed.
With clear instructions our agent answered 7 of 7 questions via the remote server and 6 of 7 via the Toolbox.
After a blocked query the agent once invented a number. Tell it to report tool errors instead of estimating.
Give the agent its own service account with BigQuery Job User and dataset-level Data Viewer, and use short-lived tokens instead of key files.

RAGFlow 1.0 Tutorial: Setup, PDF Parsing and Real Test Results

Fri, 20 Jun 2025 00:00:00 GMT

RAGFlow 1.0 installed with Docker and tested on a 147-page PDF: setup, OpenAI models, DeepDoc vs. a vision LLM parser, RAM usage and answer accuracy.

Key takeaways

RAGFlow 1.0 is a rewrite in Go. The default Docker stack started in 17 seconds and idled at 5.8 GiB RAM.
Switching the document engine from Elasticsearch to Infinity cut idle RAM to about 1.5 GiB in our test.
DeepDoc garbled most text of our 147-page statistics report, so only 2 of 5 test questions got an answer.
With gpt-4.1-mini as the PDF parser, the same report gave clean Markdown tables and 5 correct answers out of 5.
Check the chunk view on a few sample pages before you trust any parser with a new type of document.

LLM Sampling with FastMCP: Using Client LLMs for Scalable AI Workflows in your MCP

Mon, 19 May 2025 00:00:00 GMT

How MCP sampling lets a FastMCP server hand its LLM calls to the client: ctx.sample on the server, a sampling handler on the client, with code.

Key takeaways

With MCP sampling, a server tool asks the client's LLM for a completion instead of calling its own LLM.
The client runs the LLM inference and pays for it, so the MCP server stays lightweight as users grow.
In FastMCP, a tool requests a completion with ctx.sample, passing messages, a system prompt, temperature and max tokens.
FastMCP injects the Context object into any tool function that has a parameter type-hinted as Context.
The client must implement a sampling handler that receives the sampling request and returns the generated text.

Automating MCP Server Creation from OpenAPI and FastAPI

Sun, 04 May 2025 00:00:00 GMT

Generate an MCP server from an existing OpenAPI spec or FastAPI app with FastMCP: route mapping, timeouts, parameter handling and code examples.

Key takeaways

FastMCP.from_openapi turns an OpenAPI specification into an MCP server in a few lines of Python.
FastMCP.from_fastapi converts a FastAPI app into an MCP server and talks to it over the ASGI transport.
Before FastMCP 2.8.0, GET routes became resources or resource templates by default and all other methods became tools. Newer versions turn every route into a tool.
Custom RouteMap objects override the default mapping and are applied before the default rules.
Query parameters with None or empty-string values are dropped; missing path parameters raise a ValueError.

FastMCP Tutorial: Build an MCP Server in Python, Step by Step

Sun, 27 Apr 2025 00:00:00 GMT

What an MCP server is, how it differs from an API, and how to build one in Python with FastMCP: resources, templates, tools, progress reporting and a client.

Key takeaways

An MCP server is a standardized API that lets LLM clients discover and use your tools and data.
FastMCP turns Python functions into MCP resources and tools with decorators, using type hints and docstrings for schemas.
Use STDIO for local tools like Claude Desktop and an HTTP transport for remote MCP servers. This post uses SSE, which has since been replaced by Streamable HTTP.
Resource templates such as data://user-profile/{user_id} pass dynamic values to an MCP resource.
Raising a normal Python exception in a FastMCP tool sends an MCP error response back to the client.

What Is Langfuse? Open-Source LLM Observability, Self-Hosted

Mon, 14 Apr 2025 00:00:00 GMT

Self-host Langfuse with Docker Compose and use it to trace OpenAI calls, group them into sessions, score outputs and version your prompts.

Key takeaways

Langfuse is an MIT-licensed LLM engineering platform for tracing, evaluations, datasets and prompt management.
Importing the OpenAI SDK through langfuse.openai logs every call with latency and cost.
The @observe decorator groups all LLM calls of a function hierarchy into one nested trace.
Session and user IDs group traces into chat replays and per-user cost overviews.
Self-hosting with Docker Compose runs six services, including ClickHouse, MinIO, Redis and Postgres.

Integrating n8n with Open WebUI: Building advanced AI chatbots and workflows

Sun, 30 Mar 2025 00:00:00 GMT

Connect Open WebUI to n8n with a webhook workflow and a Python pipe function, so chat messages trigger n8n workflows. Setup steps and full code.

Key takeaways

An Open WebUI pipe function sends each chat message to an n8n webhook and displays n8n's response.
The n8n Webhook node uses POST, Header Auth and a Respond to Webhook node to return the answer.
A chat message trigger next to the webhook trigger lets you test the n8n workflow on its own.
The pipe's valves hold the n8n webhook URL, the bearer token and the input and response field names.
All the logic lives in the n8n workflow; the Open WebUI pipe only acts as a bridge.

Building High-Quality AI Agent Systems: Best Practices

Sun, 23 Mar 2025 00:00:00 GMT

Why multi-agent LLM systems fail, based on a study of 14 failure modes, and the design practices that make AI agent systems more reliable.

Key takeaways

Multi-agent LLM systems often show only minimal benchmark gains over well-designed single-agent implementations.
A study of five multi-agent frameworks across 150+ tasks identified 14 distinct failure modes.
The failure modes cluster into specification failures, inter-agent misalignment and verification and termination issues.
Every agent output should be verified, by self-verification, cross-verification or a dedicated verification agent.
Start with the simplest system that does the core task and add agents only when they beat a single-agent baseline.

Building Powerful AI Agents with smolagents: A Minimalist Approach

Sat, 15 Mar 2025 00:00:00 GMT

smolagents is Hugging Face's minimal agent library: code agents that act in Python, sandboxed execution, support for any LLM and tips for better agents.

Key takeaways

smolagents is a minimal agent library from Hugging Face whose agent logic fits in about 1,000 lines of code.
Its CodeAgent writes actions as Python code instead of JSON tool calls, which has been shown to need 30% fewer steps.
Agent code runs in a restricted Python interpreter by default, with E2B or Docker sandboxes as alternatives.
Group tools and prefer deterministic functions to cut LLM calls, cost, latency and error risk.
Tool docstrings should spell out input formats, because the LLM only knows what you tell it.

DeepSeek R1 On-Prem Setup: Run Advanced AI Models on Your Hardware with SGLang

Sat, 22 Feb 2025 00:00:00 GMT

Run DeepSeek-R1-Distill-Qwen-32B on two 24GB GPUs with SGLang: install, model download, Docker setup and OpenAI-compatible API calls in Python.

Key takeaways

DeepSeek-R1-Distill-Qwen-32B scores 72.6% on AIME 2024 and a Codeforces rating of 1691, versus 9.3% and 759 for GPT-4o.
The 32B distilled model runs on two 24GB GPUs such as the RTX 3090, with 32GB RAM and 100GB of NVMe storage.
SGLang serves the model with an OpenAI-compatible API on port 30000, so the OpenAI Python client works unchanged.
SGLang supports JSON schema structured output and streaming through the same chat completions endpoint.

Building Better AI Applications with LLM Tracing using Opik

Wed, 12 Feb 2025 00:00:00 GMT

Self-host Opik, Comet's open-source LLM tracing tool, trace OpenAI, PydanticAI and LangChain calls, track costs and score traces with an LLM judge.

Key takeaways

Opik is an open-source, Apache-2 licensed LLM tracing and evaluation platform created by Comet.
You can self-host Opik with Docker Compose or a Helm chart, but it has no built-in authentication.
Wrapping the OpenAI client with track_openai traces every call, including prompt, response, tokens and latency.
Opik tracks costs automatically for OpenAI and Gemini models; token usage and latency are always tracked.
Online evaluation rules let an LLM judge score every production trace, for example for hallucination.

Building AI Agents with n8n: Postgres Tools, Memory and Human Approval

Mon, 10 Feb 2025 00:00:00 GMT

Build a self-hosted AI agent in n8n 2.46: order lookups in Postgres, chat memory, email drafts behind human approval. With screenshots, costs and pitfalls.

Key takeaways

In n8n 2.46 the AI Agent node is a tool-calling loop: a chat model, optional memory and any number of tools as sub-nodes.
Give the agent fixed, parameterized SQL queries with a read-only database role instead of letting the model write SQL.
Human review for tools pauses the run until someone clicks approve; in the n8n chat it needs the response mode Using Response Nodes.
Cast Postgres DATE columns to text, otherwise a timezone like Europe/Vienna shifts them to the previous day.
A four-message conversation with gpt-5.4-mini used about 3,600 input tokens, well under one cent.

Enhancing Self-Made Agent with Memory: Keeping Context for Better Conversations

Sat, 01 Feb 2025 00:00:00 GMT

Add PostgreSQL-backed memory to a self-made Python AI agent: store every interaction, summarize long histories with GPT-4o and feed context into the prompt.

Key takeaways

The memory-enabled agent stores every user input, agent response and tool call in a PostgreSQL table.
Once 20 interactions have piled up since the last summary, GPT-4o is asked to summarize them in under 2,000 words.
Before each query, the latest summary and all newer interactions are added to the prompt as a system message.
Single-turn Q&A, one-off lookups and privacy-sensitive use cases are often better served by a stateless agent.
MemoryAgent subclasses the multi-step agent, so the tools and the iteration loop stay the same.

Advanced Self-Made Agent: Building a Smarter, Multi-Step AI Agent

Sat, 25 Jan 2025 00:00:00 GMT

Extend a framework-free Python AI agent with an AgentState enum, a multi-step reasoning loop and clarifying questions to the user. Full code included.

Key takeaways

The advanced agent tracks its progress with an AgentState enum: THINKING, DONE, ERROR and NEED_MORE_INFO.
A loop repeats the think-and-act step up to max_iterations times before the agent stops.
When information is missing, the agent asks the user a clarifying question instead of failing.
Each LLM response has to name one action: TOOL_CALL, ASK_USER or FINAL_ANSWER.
The agent reuses the query_database and search_wikipedia tools from the framework-free tutorial.