GPT-5 Hits Ceiling

Tutorial of the week

Building a Smart Search for Your Company Docs: A Practical Approach

If you've ever been frustrated searching through your company's documentation, here's a practical solution: Let's turn your knowledge base into a searchable system that actually understands questions and provides relevant answers. While the provided approach is technically just ordinary RAG - the used combination of tools is interesting as it basically allows to integrate dozens of different documentation systems, using the outlined setup.

The Tools That Make It Work

This setup uses three main components:

• Airbyte: Pulls your content from existing systems
• PGVector: A PostgreSQL extension that handles the search
• Langchain: Connects everything and manages the AI interaction

What Makes This Approach Different

The key advantage here is simplicity. Thanks to Airbyte's new Python library, you can now connect to your data sources with minimal setup – no complex infrastructure required. While our example uses Confluence, the same approach works just as well with Notion, SharePoint, or other documentation systems. With Airbyte providing literally hundreds of connectors, there is a good chance that you’ll be able to connect your knowledge base using the demonstrated approach!

How It Actually Works

1. Pull in your documentation using Airbyte
2. Process and index the content
3. Store it efficiently in PostgreSQL and pgvector
4. When someone asks a question: Find relevant documents Use AI to formulate a clear answer Return results based on your actual documentation

Why This Matters

This approach solves a straightforward problem: making existing documentation more accessible. Instead of building a new knowledge base or changing how teams work, it enhances what you already have. The setup is particularly useful for organizations that maintain extensive documentation but struggle with quick information retrieval.

Airbyte's extensive connector library lets you tap into these sources without writing custom integration code - Whether it's internal wikis, support tickets, or project documentation spread across different tools.

Read the full hands-on step-by-step guide in the link below:

For more details, visit our step-by-step guideChat with your Confluence: A Step-by-Step Guide using Airbyte, Langchain and PGVectorwww.pondhouse-data.com/blog/chat-with-your-confluence-using-airbyte →

Tool of the week

LibreChat: OpenSource, privacy-friendly ChatGPT alternative

As companies increasingly rely on AI chatbots, many face two critical challenges with ChatGPT: high subscription costs ($20-60 per user monthly) and data privacy concerns, particularly regarding OpenAI's data usage policies for training their models.

LibreChat offers a practical solution as a self-hosted, open-source alternative. It mirrors ChatGPT's familiar interface while adding several business-critical features:

Key Features & Advantages:

• 💰 Cost BenefitsZero per-user subscription feesFull control over AI provider selection and usageFlexible scaling without licensing restrictions
• 🔒 Privacy & SecuritySelf-hosted infrastructureComplete data sovereigntyMultiple authentication options (LDAP, Azure AD, AWS Cognito)No external data sharing for model training
• 🔗 Integration & FlexibilitySupport for multiple AI providers (OpenAI, Anthropic, Google, AWS)OpenAI Assistants API integrationCustom endpoint support for specialized AI modelsRAG (Retrieval Augmented Generation) capabilities for document analysis
• 🛠️ Advanced FeaturesMulti-modal support (text, images, code)Customizable prompt library with variablesConversation forking for team collaborationBuilt-in image generation via DALL-E 3"Artifacts" feature for live code/diagram renderingFile handling and analysis capabilities

Current Limitation: While LibreChat excels in features and privacy, it currently lacks granular user permissions - though this is prioritized on their roadmap.

Perfect for: Organizations that need ChatGPT's capabilities but require data privacy, cost control, and infrastructure flexibility. Particularly valuable for enterprises in regulated industries or those handling sensitive information.

More features in our deep-diveWhat is Libre Chat? The best alternative to Open WebUI and ChatGPTwww.pondhouse-data.com/blog/introduction-to-libre-chat →

Top News

Leak suggests: Diminishing returns with GPT-5 - are LLMs hitting their ceiling?

What happened?

• OpenAI's new model "Orion" shows signs of diminishing returns in traditional AI scaling
• Model reached GPT-4 level performance already at 20% of training, but further improvements proved more modest than previously
• General improvements were smaller than previous generational leaps (GPT-3 to GPT-4)

This has prompted OpenAI to create a dedicated "foundations" team to explore new development approaches as high-quality training data becomes scarcer.

The company still plans to release Orion in early 2025, potentially under a new naming convention, marking a strategic pivot in their AI development roadmap.

⚡ Why it matters:
The AI industry may be hitting a ceiling with conventional scaling approaches as high-quality training data becomes scarce, forcing AI leaders to rethink development strategies. On the other hands, this might be good news for any AI application practitioners, as it gives room to regroup, breath and apply the already outstanding models currently available.

📅 What's next:
Orion is scheduled for release in early 2025, pending safety testing completion.

Read the full leak here.

Tip of the week

Cutting LLM costs in half by using batch APIs

LLM costs have already come down significantly in the last years and months. However there is one method we see too little adopted: Using batch inference.

Almost every major LLM provider (OpenAI, Anthropic, Google - also self-hosting servers) offers batch processing capabilities. Check your provider's documentation for "Batch API" or "Async Processing" options.

What is Batch Inference?
Instead of sending individual API requests one by one, batch inference - or processing - allows you to bundle multiple LLM requests together and process them asynchronously.

Benefits:

• Mostly 50% cost reduction compared to regular API calls
• Significantly higher rate limits
• Guaranteed 24-hour turnaround time
• Perfect for non-time-critical tasks

Ideal Use Cases:

• Content classification
• Document processing
• Data analysis
• Bulk content generation

How it works:

Below you’ll find the steps required to use the batch processing API of OpenAI. No matter which AI provider you use - they all work similarly.

1. Create your batch file. It’s simply a JSONL file of your request messages

1. Upload this file to the provider's API

1. Create the batch job

1. Wait until the job is done

1. Get the results