smart_toy
psychology AI & Local Inference

Adding Gemini and AI to Your GPU-Powered Laptop

Connect Google's free Gemini API to Open WebUI with Docker, Ollama, and GPU acceleration.

"How I set up a local AI workstation with Open WebUI, Ollama, and Gemini API access -- all running through Docker on a GPU-powered Windows laptop with WSL integration."

To connect the free Gemini API to your Open WebUI, you need to configure Open WebUI to use the Gemini API endpoint. Since Open WebUI is designed to be compatible with the OpenAI API, you can leverage Gemini's OpenAI-compatible endpoint. This guide walks through the entire process from installation to configuration.

The setup also covers installing Open WebUI and Ollama using Docker with GPU support on Windows, and integrating everything with WSL (Ubuntu) so you can use your AI tools from anywhere.

key

Step 1: Obtain a Gemini API Key

First, you need to get your free API key from Google AI Studio:

  • check_circle Go to the Google AI Studio website.
  • check_circle Create a project if you haven't already.
  • check_circle Navigate to the "API key" section.
  • check_circle Click "Create API key" and copy the generated key. Keep this key secure.
settings

Step 2: Configure Open WebUI

There are several ways to connect the Gemini API endpoint to Open WebUI:

tune

Method 1: OpenAI API Connection Settings

  • arrow_right Open your Open WebUI interface and go to Admin settings.
  • arrow_right Navigate to Settings > Connections.
  • arrow_right Look for "Manage OpenAI API Connections".
  • arrow_right Add a new connection or edit an existing one.

Set the API URL to Gemini's OpenAI-compatible endpoint:

https://generativelanguage.googleapis.com/v1beta/openai/

Enter the Gemini API key you obtained in Step 1. You may need to manually add specific Gemini models (e.g., gemini-2.0-flash) in the model management section.

terminal

Method 2: Environment Variables

When running Open WebUI via Docker, set these environment variables:

Environment 
OPENAI_API_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/
OPENAI_API_KEY=your-gemini-api-key
hub

Method 3: Using a Proxy Service (LiteLLM)

You can use a proxy service like LiteLLM, which has built-in support for Gemini. Configure LiteLLM with your Gemini API key and then point Open WebUI to the LiteLLM API endpoint. Alternatively, set up your own proxy using a Docker container that forwards requests to the Gemini API.

model_training

Step 3: Add Gemini Models

After configuring the API endpoint, you may need to manually add the Gemini models you want to use within Open WebUI:

  • check_circle Go to the model management section (under Admin or Settings).
  • check_circle Add model names such as gemini-2.0-flash or gemini-1.5-pro-latest.
warning
info

Important Considerations

sync_alt

API Compatibility

Gemini's OpenAI-compatible API may not support all features or parameters. Refer to the Gemini API documentation for compatibility details.

check_box

Model Availability

Ensure the Gemini models you want to use are available through the OpenAI-compatible endpoint.

shield

Safety Filters

The Gemini API has its own safety filters, which may behave differently from other models you've used in Open WebUI.

speed

Rate Limits

The free Gemini API has rate limits. Consult the Google AI Studio documentation for details.

view_in_ar

Installing Open WebUI & Ollama with Docker (GPU Support)

For the full local AI experience, install Open WebUI and Ollama using Docker with GPU passthrough. This works on Windows with WSL integration, allowing you to use your AI tools from anywhere.

info

Docker with GPU support on Windows requires WSL 2 with an Ubuntu distribution and the NVIDIA Container Toolkit installed. Make sure your GPU drivers are up to date and Docker Desktop has WSL 2 backend enabled.

With WSL integration, Docker containers can access your NVIDIA GPU directly, enabling local model inference through Ollama alongside the cloud-based Gemini API through Open WebUI. This gives you the best of both worlds: fast local models for quick tasks and powerful cloud models for complex reasoning.

play_circle

Getting Started

By following these steps, you can connect the free Gemini API to your Open WebUI and start experimenting with Google's latest models within the familiar Open WebUI interface. Remember to consult the Open WebUI and Gemini API documentation for the most up-to-date instructions and configuration options.

The combination of local Ollama models and cloud Gemini API gives you a powerful, flexible AI workstation -- all running on your GPU-powered laptop.

Open WebUI provides a self-hosted AI interface compatible with both local Ollama models and cloud APIs like Gemini. For more details, visit the Open WebUI documentation.