Open WebUI Integration¶
Hayhooks provides seamless integration with Open WebUI, enabling you to use Haystack pipelines and agents as chat completion backends with full feature support.
Overview¶
Open WebUI integration allows you to:
- Use Haystack pipelines as OpenAI-compatible chat backends
- Support streaming responses in real-time
- Send status events to enhance user experience
- Intercept tool calls for better feedback
Getting Started¶
Prerequisites¶
- Open WebUI instance running
- Hayhooks server running
- Pipeline with chat completion support
Configuration¶
1. Install Open WebUI¶
Please follow the Open WebUI installation guide to install Open WebUI.
We recommend using Docker to install Open WebUI.
A quick command to install Open WebUI using Docker is:
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -e WEBUI_AUTH=False -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main
This will start Open WebUI on local port 3000, with no authentication, and with the data stored in the open-webui
volume. It's the easiest way to get started.
2. Disable Auto-generated Content¶
Open WebUI automatically generates content for your pipelines. More precisely, it calls your pipelines to generate tags, title and follow-up messages. Depending on your pipeline, this may not be suitable for this use case.
We recommend disabling those features as a starting point, then you can enable them if you need them.
Go to Admin Settings → Interface and turn off the following features:
3. Add Hayhooks as an OpenAI compatible API endpoint¶
You have two options to connect Hayhooks to Open WebUI:
Option 1: Direct Connection (Recommended)¶
First, enable Direct Connections in Open WebUI.
Go to Admin Settings → Connections and enable Direct Connections:
Then go to Settings → Connections and add a new connection:
- API Base URL:
http://localhost:1416
- API Key:
any-value
(or leave it empty, it's not used by Hayhooks)
Option 2: OpenAI API Connection¶
Alternatively, you can add Hayhooks as an additional OpenAI API Connection from Admin Settings → Connections:
In both cases, remember to fill a random value as API key (Hayhooks doesn't require authentication).
Pipeline Implementation¶
To make your pipeline work with Open WebUI, implement the run_chat_completion
or run_chat_completion_async
method in your PipelineWrapper
. See the OpenAI Compatibility guide for detailed implementation examples.
Non-Streaming Example¶
Here's how a non-streaming chat completion looks in Open WebUI:
Streaming Example¶
With streaming enabled, responses appear in real-time:
Open WebUI Events¶
Hayhooks supports sending events to Open WebUI for enhanced user experience:
Available Events¶
- status: Show progress updates and loading indicators
- message: Append content to the current message
- replace: Replace the current message content entirely
- notification: Show toast notifications (info, success, warning, error)
- source: Add references, citations, or code execution results
Event Implementation¶
from typing import AsyncGenerator, List
from hayhooks import async_streaming_generator, get_last_user_message, BasePipelineWrapper
from hayhooks.open_webui import create_status_event, create_message_event, OpenWebUIEvent
class PipelineWrapper(BasePipelineWrapper):
async def run_chat_completion_async(self, model: str, messages: List[dict], body: dict) -> AsyncGenerator[str | OpenWebUIEvent, None]:
# Indicate loading
yield create_status_event("Processing your request...", done=False)
question = get_last_user_message(messages)
try:
# Stream model output alongside events
result = async_streaming_generator(
pipeline=self.pipeline,
pipeline_run_args={"prompt_builder": {"query": question}},
)
# Optional UI hint
yield create_message_event("✍️ Generating response...")
async for chunk in result:
yield chunk
yield create_status_event("Request completed successfully", done=True)
except Exception as e:
yield create_status_event("Request failed", done=True)
yield create_message_event(f"Error: {str(e)}")
raise
Here's how Open WebUI events enhance the user experience:
Tool Call Interception¶
For agent pipelines, you can intercept tool calls to provide real-time feedback:
def on_tool_call_start(tool_name: str, arguments: dict, tool_id: str):
"""Called when a tool call starts"""
print(f"Tool call started: {tool_name}")
def on_tool_call_end(tool_name: str, arguments: dict, result: dict, error: bool):
"""Called when a tool call ends"""
print(f"Tool call ended: {tool_name}, Error: {error}")
class PipelineWrapper(BasePipelineWrapper):
def run_chat_completion(self, model: str, messages: List[dict], body: dict) -> Generator:
return streaming_generator(
pipeline=self.pipeline,
pipeline_run_args={"messages": messages},
on_tool_call_start=on_tool_call_start,
on_tool_call_end=on_tool_call_end,
)
Here's an example of tool call interception in action:
OpenAPI Tool Server¶
Hayhooks can serve as an OpenAPI Tool Server for Open WebUI, exposing its core API endpoints as tools that can be used directly from the chat interface.
OpenAPI Tool Servers are a standard way to integrate external tools and data sources into LLM agents using the widely-adopted OpenAPI specification. This approach offers several advantages:
- Standard Protocol: Uses the established OpenAPI specification - no proprietary protocols to learn
- Easy Integration: If you build REST APIs today, you're already familiar with the approach
- Secure: Built on HTTP/REST with standard authentication methods (OAuth, JWT, API Keys)
- Flexible Deployment: Can be hosted locally or externally without vendor lock-in
Since Hayhooks exposes its OpenAPI schema at /openapi.json
, Open WebUI can automatically discover and integrate all available Hayhooks endpoints as tools.
Setup¶
- Go to Settings → Tools in Open WebUI
- Add OpenAPI Tool Server:
- Name: Hayhooks
- URL:
http://localhost:1416/openapi.json
Available Tools¶
Once configured, the following Hayhooks operations become available as tools in your Open WebUI chat:
- Deploy Pipeline: Deploy new pipelines from your chat interface
- Undeploy Pipeline: Remove existing pipelines
- Run Pipeline: Execute deployed pipelines with parameters
- Get Status: Check the status and list of all deployed pipelines
This enables you to manage your entire Hayhooks deployment directly through natural language conversations.
Example: Deploy a Haystack pipeline from open-webui
chat interface¶
Here's a video example of how to deploy a Haystack pipeline from the open-webui
chat interface:
Example: Chat with Website¶
Here's a complete example for a website chat pipeline:
from typing import AsyncGenerator, List
from haystack import Pipeline
from haystack.components.fetchers import LinkContentFetcher
from haystack.components.converters import HTMLToDocument
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIChatGenerator
from hayhooks import BasePipelineWrapper, async_streaming_generator, get_last_user_message
class PipelineWrapper(BasePipelineWrapper):
def setup(self) -> None:
fetcher = LinkContentFetcher()
converter = HTMLToDocument()
prompt_builder = PromptBuilder(
template="Based on this content: {{documents}}\nAnswer: {{query}}"
)
llm = OpenAIChatGenerator(
model="gpt-4o",
streaming_callback=lambda x: None
)
self.pipeline = Pipeline()
self.pipeline.add_component("fetcher", fetcher)
self.pipeline.add_component("converter", converter)
self.pipeline.add_component("prompt_builder", prompt_builder)
self.pipeline.add_component("llm", llm)
self.pipeline.connect("fetcher.content", "converter")
self.pipeline.connect("converter.documents", "prompt_builder.documents")
self.pipeline.connect("prompt_builder", "llm")
async def run_chat_completion_async(self, model: str, messages: List[dict], body: dict) -> AsyncGenerator:
question = get_last_user_message(messages)
# Extract URLs from messages or use defaults
urls = ["https://haystack.deepset.ai"] # Default URL
return async_streaming_generator(
pipeline=self.pipeline,
pipeline_run_args={
"fetcher": {"urls": urls},
"prompt_builder": {"query": question}
},
)
Troubleshooting¶
Common Issues¶
- Connection Failed
- Verify Hayhooks server is running
- Check API URL in Open WebUI settings
-
Ensure correct port (1416 by default)
-
No Response
- Check if pipeline implements
run_chat_completion
- Verify pipeline is deployed
-
Check server logs for errors
-
Streaming Not Working
- Ensure
streaming_callback
is set on generator - Check if
run_chat_completion_async
is implemented - Verify Open WebUI streaming is enabled
Debug Commands¶
# Check Hayhooks status
hayhooks status
# Check deployed pipelines
curl http://localhost:1416/status
# Test chat completion endpoint (OpenAI-compatible)
curl -X POST http://localhost:1416/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "my_pipeline",
"messages": [{"role": "user", "content": "test message"}],
"stream": false
}'
# Test streaming chat completion
curl -X POST http://localhost:1416/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "my_pipeline",
"messages": [{"role": "user", "content": "test message"}],
"stream": true
}'
Next Steps¶
- OpenAI Compatibility - Implementation details for chat completion methods
- Open WebUI Events Example - Advanced UI integration patterns