What is the VertexAI model garden - a practical example: Deploying Llama 3.1 and Claude 3.5 Sonnet

On this page

Google Cloud's Vertex AI Model Garden is a catalog of pre-trained AI models inside Vertex AI that you can deploy with a few clicks. It includes LLMs like Claude 3.5 Sonnet and Llama 3.1, multi-modal models like CLIP and image generation models like Stable Diffusion. Models are either offered as a pay-per-request API service or deployed on a dedicated virtual machine that Google manages for you.

The Model Garden is part of the broader Vertex AI ecosystem and aims to make integrating AI capabilities into applications simpler. It builds on Google's strong Google Cloud Platform, and it comes with a large collection of pretrained models out of the box.

In this article, we look at what Vertex AI Model Garden offers and how it can speed up the development of AI-powered applications. Then we walk through hands-on examples: Claude 3.5 Sonnet and Llama 3.1 through the API service, CLIP on a virtual machine and a model straight from Hugging Face.

What is the Google Vertex AI Model Garden

Google Vertex AI Model Garden is a platform within Google Cloud's Vertex AI suite for working with AI models. It offers a large collection of pre-trained models from Google, OpenAI, Mistral, Anthropic, Meta and selected open-source communities, so you can add AI capabilities to your applications without building the model infrastructure yourself.

Features of the Vertex AI Model Garden

The platform sorts its models into foundation models, fine-tunable models and task-specific solutions, and you can browse them by use case. When we took the screenshot below, the overview listed 86 foundation models and 56 fine-tunable models.

Some models can be customized: you can fine-tune them with your own data to adapt them to specific tasks.

Deployment integrates with Google Cloud services, so models can be deployed as API endpoints inside your existing Google Cloud infrastructure.

Along with Google's proprietary models, the platform includes open-source models, which widens the choice.

The user interface has filtering options to help you find the right model, and deploying a model takes a few clicks.

Overall, the Model Garden makes AI models easier to access and manage. It's a good foundation for businesses and developers who want to use AI without investing in model operations.

The Vertex AI Model Garden overview with foundation and fine-tunable models

Why use the Vertex AI Model Garden to deploy models?

Before getting into the details of how to use the Model Garden, let's look at why anyone would use it.

A large model repository

Vertex AI Model Garden has a large collection of models from Google, from other commercial providers like OpenAI and Mistral, and from the open-source community. It includes foundation models, fine-tunable models and task-specific solutions for a wide range of applications and industries. Having this many models in one place makes it easier to find the right one. It also greatly reduces the complexity of your deployment pipeline: there is one single place for all the models you need.

Customization and fine-tuning

Vertex AI Model Garden supports model customization and fine-tuning. You can adapt pre-trained models to your requirements using AI Studio, the Vertex AI API or custom notebooks. For specific use cases this can improve model performance considerably, without requiring deep machine learning expertise. (For the basics, see our guide on how to finetune LLMs.)

Integration with Google Cloud services

Vertex AI Model Garden is integrated with Google Cloud's other services. You can deploy models quickly as API endpoints on Google Cloud's infrastructure, and they scale with demand.

A simple deployment process

The user interface makes deployment accessible even to people with little AI experience. With one-click deployment and pre-configured settings for many models, deploying a model takes far less time and effort than doing it yourself.

Security and compliance

Models deployed in Vertex AI benefit from Google Cloud's security features and compliance certifications. That matters for organizations that handle sensitive data and must meet strict regulatory requirements.

Potential cost savings

Because the Model Garden provides the whole deployment pipeline, you spend less on maintaining the "auxiliary infrastructure" around a model deployment. For some models, the Model Garden also offers an API service with simple pay-per-use pricing, which can cut costs significantly for some users. Having cheap and expensive models behind one API also makes it easier to route each request to the cheapest model that can handle it.

In plain terms, the Vertex AI Model Garden is a one-stop shop for AI model deployments. Having many models in one place is convenient, and it greatly reduces the effort needed for security, data privacy and compliance.

Two ways to deploy models in the Vertex AI Model Garden

Enough theory, let's get started. There are two ways to deploy models with the Vertex AI Model Garden.

The first is the more traditional way, and the one most models support: deploying the model on a virtual machine. You select the model you want, click 'deploy', and the Model Garden automatically creates a virtual machine and an API endpoint that serves the model. Most of the Llama and Mistral models are served this way.

You are not responsible for maintaining the virtual machine; Google does that. But you pay for a full virtual machine by the hour.

The second, and arguably more convenient, way is the Model Garden's API service. Here you simply activate the model you want to use. Google gives you an API token, which you use to send requests to the model endpoint. You pay per request.

Using Claude 3.5 Sonnet through the Model Garden API service

Let's start with the API service. Claude 3.5 Sonnet, the latest version of Claude at the time of writing, has been getting raving reviews, so we'll use it as the example.

1. Navigate to the Vertex AI Model Garden.

2. Use the search box to look for the model you want, Claude 3.5 in our example.

   Searching for Claude 3.5 Sonnet in the Model Garden

3. On the next screen, click "Enable" to enable the model. You only need to do this the first time you use this specific model.

   Enable Claude 3.5 Sonnet

4. For some models, Google asks you to fill out a form to get access. That's because Google only acts as the hosting provider for these third-party companies. Anthropic, for example, approves usage of Claude 3.5 on a per-user basis. So fill out the form if required and wait for confirmation.

5. After the form, you'll see the pricing information. Accept the terms and conditions and click "Agree". You don't pay anything yet, only when you use the model.

   Claude 3.5 Sonnet pricing in the Model Garden

6. On the final screen, click "Manage on Vertex AI". That's all there is to it.

On the next screen, Vertex AI shows you how to use the model endpoint. For example, in Python you can create chat completions like this:

1from anthropic import AnthropicVertex
2
3LOCATION="europe-west4" # or "europe-west4"
4
5client = AnthropicVertex(region=LOCATION, project_id="PROJECT_ID")
6
7message = client.messages.create(
8  max_tokens=1024,
9  messages=[
10    {
11      "role": "user",
12      "content": "Send me a recipe for banana bread.",
13    }
14  ],
15  model="claude-3-5-sonnet@20240620",
16)
17print(message.model_dump_json(indent=2))

Using Llama 3.1 on the Model Garden API service

Using the Llama 3.1 API service on the Model Garden works the same way and is even easier.

Navigate to the Vertex AI Model Garden Llama 3.1 API Service. Click "Enable", and that's it.

As with Claude 3.5 Sonnet, you now have access to the model endpoint and can send requests to it. The Model Garden explains clearly how to use the endpoint.

Instructions for calling the Llama 3.1 API service with curl

Deploying a CLIP model on a virtual machine in the Model Garden

After two API service deployments, let's deploy a model on a virtual machine.

1. Navigate to the Vertex AI Model Garden.

2. Search for the CLIP model. CLIP can classify images without first being trained on labeled data.

   Searching for CLIP in the Model Garden

3. Click "Deploy". A side panel opens.

4. In the side panel:

   • provide a name for your model and model endpoint
   • select the virtual machine size. You mainly choose how many and which GPUs you want to use. The Model Garden only offers sizes that fit the model, so whatever you select, the model will work on that machine.

   Deployment side panel for CLIP with machine type selection

5. After you click "Deploy", the resources are provisioned. This can take a while.

Once the resources are provisioned, go to "My endpoints". At the top right, select the "Region" you chose during deployment. There you'll find your deployed model endpoint.

The deployed CLIP model listed under My models

Click the model name to get more information about the deployed model and how to use it.

If you'd rather run open models on infrastructure you control, see our guide on self-hosting an LLM with vLLM and Caddy, or our walkthrough of running DeepSeek R1 on your own hardware with SGLang.

Deploying models from Hugging Face on the Google Vertex AI Model Garden

As the last example, we deploy a model from the Hugging Face model hub.

Yes, the Model Garden can deploy models directly from Hugging Face, which makes the pool of supported models almost endless.

1. At the very top of the Model Garden interface, click "Deploy from Hugging Face". A side panel opens.

2. Enter the Hugging Face model name and your Hugging Face API token, and specify the size of the virtual machine you want to use.

   The Deploy from Hugging Face side panel

3. Click "Deploy". Once again, that's all you need to do.

If you don't have a Hugging Face API token yet, you can get one by

• Signing up for an account
• Creating a token in your Hugging Face profile. The token looks like hf_xxxxxxx.

Summary: what the Vertex AI Model Garden offers

The Google Vertex AI Model Garden makes advanced AI models much easier to access and deploy. For businesses and developers who want to use AI without the usual work of deploying and managing models, it's an attractive option.

What we took away from using it:

• It's a central repository of very different AI models, from language models like Claude 3.5 Sonnet and Llama 3.1 to specialized models like CLIP, all behind one interface.
• You can choose between API service deployments with pay-per-use pricing (Claude 3.5, Llama 3.1) and virtual machine deployments (CLIP), which give you more control and potentially better compliance options.
• Because all models run on Google Cloud's infrastructure, organizations deal with one provider's security and compliance instead of many.
• Even for complex models, deployment takes a few clicks.

There are things to watch out for, such as potential lock-in to the Google Cloud ecosystem and the hourly cost of virtual machine deployments you forget to shut down. Still, for many use cases the variety of models and the ease of use make the Model Garden a good choice. If you want to build agents on top of these models in Google Cloud, read our introduction to the Vertex AI Agent Builder.

Further reading

• What is the Vertex AI Agent Builder?
• How to finetune your very own LLM
• Advanced RAG: Recursive Retrieval
• RAG, embeddings and vector search with Google BigQuery
• Running GLM-4.7-Flash locally with OpenCode

Interested in building high-quality AI agent systems?

We prepared a comprehensive guide based on cutting-edge research for how to build robust, reliable AI agent systems that actually work in production. This guide covers:

• Understanding the 14 systematic failure modes in multi-agent systems
• Evidence-based best practices for agent design
• Structured communication protocols and verification mechanisms

About the author

Andreas Nigg

Co-Founder, Pondhouse Data

Andreas has spent more than eight years building data applications. At Pondhouse Data he designs and ships AI systems and integrates them into complex, real-world environments.

Talk to Andreas about your projectFollow on LinkedIn