How to create LLM tools from any Python SDK using langchain-autotools

On this page

langchain-autotools is a community-maintained library that creates LLM tools from an existing Python SDK. You pass it an SDK client object, choose which of the SDK's functions to expose with CrudControls, and hand the resulting operations to a LangChain agent. That saves you from writing and describing a wrapper function for every API call. It has one notable limitation: it can only call the SDK's top-level functions, which we show with an Azure Blob Storage example.

Giving LLMs access to our internal and external APIs and databases is one of the most promising ways to use LLMs in business. Chatbot applications like ChatGPT are great, but that's where the productivity gains start. They get much higher once we integrate these models into our existing workflows, or build new workflows around them.

The established concept for letting LLMs interact with other technologies is called Tools. Tools are a good concept, but it's not that straightforward to design them so that LLMs can work with them. And at the end of the day, creating them is a lot of work.

langchain-autotools integrates with the well-known Langchain SDK, and its goal is to make creating tools for LLMs as easy as possible.

What are LLM tools?

In their purest form, LLM tools are just functions that run after an LLM decides to use them.

The process is fairly simple:

1. Create a runtime environment where you can run code, for example a Python environment.

2. Create a function that, for example, interacts with an API or a database. The function can have parameters, which the LLM provides later.

3. Describe in natural language what the function does. Also describe its parameters and return value.

4. When you want the LLM to execute a task, send it your task prompt plus the descriptions of all tools (think: functions). The LLM then decides whether it can fulfill the prompt without a tool, or whether it needs one.

5. If the LLM decides to use a tool, it tells you which one. You then call your function with the parameters the LLM provided. In pseudo Python code, this is just an if statement:

1## Note: This is example code, not real one
2def database_tool(query):
3    database_client = new Client()
4    result = database_client.query(query)
5    return result
6
7system_prompt = """You are a system to answer questions. You get a list of
8tools which you can use to answer the question.
9Either answer the users question directly, or answer with the tool to user.
10If you need a tool, answer with the following format. If required by the
11tool, make sure to provide the tool parameters.
12
13{"tool": "database_tool", "parameters": {"query": "select count(*) from table"}}
14
15Tools:
16
171. Database Tool: name: database_tool
18parameters:
19- query: The database PostgreSQL query to run
20"""
21
22response = LLM.prompt("How many users are in our database?")
23
24if json.loads(response)["tool"]:
25    tool = json.loads(response)["tool"]
26    if tool == "database_tool":
27        ## Call your tool function
28        result = database_tool(json.loads(response)["parameters"]["query"])

To summarize: a tool is nothing more than a function with a good description, so that the LLM knows what it does and how to use it.

If you deal with many external systems, you can probably see the problem already: wrapping all of them into functions and writing a good description for each one is cumbersome. (For a framework-free look at how agents call tools, see our guide on building AI agents without frameworks.)

Many external systems nowadays provide SDKs, and those SDKs already come with extensive documentation. Why not use that documentation to create tools for our LLMs?

How to create LLM tools with langchain-autotools

We can, and that's what langchain-autotools is for. It combines the Langchain Agent concept with the SDK's documentation, so that agents can use any SDK as their tool.

With langchain-autotools, creating a tool takes four steps:

1. Instantiate a client object of the SDK you want to use.

2. Create an autotool with that client.

3. Create an agent that uses a predefined prompt and the autotool.

4. Execute the prompt with your agent.

In pseudocode, this looks like this:

1database_client = new Client(secrets) # Your 'normal' SDK client creation
2client = {"client": database_client}
3
4autotool = AutoToolWrapper(client=client)
5
6llm = ChatOpenAI(model="gpt-4o", temperature=0)
7prompt = hub.pull("hwchase17/structured-chat-agent") # This pulls a predefined prompt from the langchain prompt hub
8agent = create_structured_chat_agent(llm, autotool.operations, prompt)
9agent_executor = AgentExecutor(
10    agent=agent,
11    tools=autotool.operations, # Contains the descriptions of the functions your SDK provides
12    verbose=True, # How much information you want to see in the logs
13    handle_parsing_errors=True,
14    max_iterations=5,
15)
16
17agent_executor.invoke(
18    {
19        "input": "How many users are in our database?",
20    }
21)

The tedious part, wrapping your SDK functions into well-described functions, is gone.

The parameters of the AgentExecutor:

• agent is the agent object you created, using an LLM as well as a prompt.
• prompt: You can supply your own prompt or use a predefined one. The Langchain prompt hub is an excellent place to find predefined prompts, and Langchain makes them easy to use: just call hub.pull("author/prompt-name").
• tools: A list of tools to supply. autotool.operations contains a list of all available functions of your SDK client.
• handle_parsing_errors defines how the agent reacts when a tool can't be invoked correctly because the LLM's answer is not in the expected format, or more generally whenever a runtime error happens after an LLM response. By default, execution is aborted and an exception is raised. Setting this to true sends the error message along with the original task to the LLM again, and the execution is repeated. This is a surprisingly robust way of dealing with such errors. You can also provide your own error handling functions; read more in the Langchain documentation.
• max_iterations is the maximum number of iterations the agent may use for the task. One iteration is one LLM call. If you want the LLM to fetch your database user count, the first call will most likely result in a tool call. In the second iteration, you present the result to the agent, which decides whether the task is finished. This task therefore needs 2 iterations.

Specifying the functions to expose to the LLM

SDKs typically provide a lot of functions. Sending all of them to the LLM would probably overload its context window, or at least cause unnecessary costs. Most of the time, we only need two or three functions anyway.

langchain-autotools lets you filter the functions you expose to the LLM with CrudControls. CrudControls define CRUD verbs, and only functions that match these verbs are exposed.

You can define the following CRUD verbs:

• create
• read
• update
• delete

For example, to expose only the query and queryMany functions and the drop_database function, you could define:

1crud_controls = CrudControls(
2    read = True, # You need to enable each CRUD verb you want to use
3    read_list="query, queryMany", # Define multiple functions per CRUD operations using comma separation
4    delete = True,
5    delete_list="drop_database"
6)

Note that you need to explicitly enable each CRUD verb you want to use. Providing a list of functions per verb is not enough.

The available parameters of CrudControls are:

• read: Enable the read verb
• read_list: Comma-separated list of function-names to expose to the LLM
• create: Enable the create verb
• create_list: Comma-separated list of function-names to expose to the LLM
• update: Enable the update verb
• update_list: Comma-separated list of function-names to expose to the LLM
• delete: Enable the delete verb
• delete_list: Comma-separated list of function-names to expose to the LLM

To use the CrudControls, pass them to the AutoToolWrapper:

1autotool = AutoToolWrapper(client=client, crud_controls=crud_controls)

Example: a langchain-autotools agent for Azure Blob Storage

Now for a real-world example with langchain-autotools. Say I have an Azure Blob Storage account where I keep data needed for some of my customer service requests.

We could build a small customer service LLM agent that fetches documents from the Blob Storage when needed and answers questions using them. The agent therefore needs two capabilities:

• Listing files in our blob storage
• Downloading files and extracting the content

This example also demonstrates one of the limitations of the autotools library. But first things first.

Start by installing the required libraries:

1pip install azure-storage-blob azure-identity langchain-autotools langchain==0.2.16 langchain_openai==0.1.25

Get the connection string from the Access keys page of your Azure storage account:

Connection string on the Access keys page of an Azure storage account

Load the required dependencies and initialize the Azure Blob Storage client:

1import os
2from azure.identity import DefaultAzureCredential
3from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
4
5from langchain_autotools import AutoToolWrapper, CrudControls
6from langchain import hub
7from langchain_core.tools import tool
8
9from langchain.agents import AgentExecutor, create_structured_chat_agent
10from langchain_openai import ChatOpenAI
11
12os.environ["AZURE_STORAGE_CONNECTION_STRING"] = "<your connection string>"
13blob_service_client = BlobServiceClient.from_connection_string(os.environ["AZURE_STORAGE_CONNECTION_STRING"])
14container_client = blob_service_client.get_container_client("blogphdata")

Next, define the autotools client and the CRUD controls. You can find the correct function names of the Azure Blob Storage SDK in the documentation.

1client = {"client": container_client}
2crud_controls = CrudControls(
3    read=True,
4    read_list="list_blobs,download_blob",
5)

Then initialize autotools and Langchain as described in the previous section:

1autotool = AutoToolWrapper(client=client, crud_controls=crud_controls)
2
3prompt = hub.pull("hwchase17/structured-chat-agent")
4
5
6llm = ChatOpenAI(model="gpt-4o", temperature=0)
7
8tools = autotool.operations
9print(tools)
10agent = create_structured_chat_agent(llm, tools, prompt)
11agent_executor = AgentExecutor(agent=agent,
12                               tools=tools,
13                               verbose=True,
14                               handle_parsing_errors=True,
15                               max_iterations=10)
16
17agent_executor.invoke(
18    {
19        "input": "How much is the price of the product 'Kopfhörer'? Use your tools to find the answer in one of the blobs"
20    }
21)

Most likely, your agent didn't respond with the correct answer, even though the answer was in one of the blobs. The agent's output shows why:

1{
2  "action": "download_blob",
3  "action_input": {
4    "blob": "test.txt"
5  }
6}
7"<azure.storage.blob._download.StorageStreamDownloader object at 0x7f99640ae350>"
8
9> Finished chain.
10
11{'input': "How much is the price of the product 'Kopfhörer'? Use your tools to find the answer in one of the blobs",
12 'output': 'Agent stopped due to iteration limit or time limit.'}

The download_blob function returns a StorageStreamDownloader, which is not the content of the blob.

You would need to call download_blob('blobname').readall() to get the actual text content. That isn't possible with langchain-autotools at the moment: you can only call top-level functions of the SDK. To work around this, we create and describe our own function:

1@tool
2def download_blob(blob_name):
3    """Download a blob from the container and return the text content"""
4
5    data = container_client.download_blob(blob_name).readall()
6    return data

The @tool decorator turns the function into a Langchain tool.

We can combine the automatically created autotool functions with our own function like this:

1crud_controls = CrudControls(
2    read=True,
3    read_list="list_blobs", # remove the download_blob function, as we'll create our own
4)
5autotool = AutoToolWrapper(client=client, crud_controls=crud_controls)
6
7tools = autotool.operations + [download_blob]
8agent = create_structured_chat_agent(llm, tools, prompt)
9agent_executor = AgentExecutor(agent=agent,
10                               tools=tools,
11                               verbose=True,
12                               handle_parsing_errors=True,
13                               max_iterations=10)

If we run the agent_executor again, we get the correct answer, provided it is in one of the blobs.

langchain-autotools is a quick way to give an agent access to an SDK's simple, top-level functions. Expect to write a few custom @tool functions for anything that needs chained calls, like reading a downloaded blob.

If you want the same API available to any MCP client instead of a single LangChain agent, FastMCP can generate an MCP server from an OpenAPI spec or a FastAPI app.

Further reading

• Extract structured data for LLMs from any website
• Chat with your confluence using LLMs
• How to fine-tune your very own LLM
• Creating an MCP server with FastMCP
• Building AI agents with smolagents
• Building a multi-step AI agent without frameworks

Interested in building high-quality AI agent systems?

We prepared a comprehensive guide based on cutting-edge research for how to build robust, reliable AI agent systems that actually work in production. This guide covers:

• Understanding the 14 systematic failure modes in multi-agent systems
• Evidence-based best practices for agent design
• Structured communication protocols and verification mechanisms

About the author

Andreas Nigg

Co-Founder, Pondhouse Data

Andreas has spent more than eight years building data applications. At Pondhouse Data he designs and ships AI systems and integrates them into complex, real-world environments.

Talk to Andreas about your projectFollow on LinkedIn