ChatGoogleGenerativeAI chat models. For detailed documentation of all ChatGoogleGenerativeAI features and configurations head to the API reference.
Overview
Integration details
| Class | Package | Local | Serializable | PY support | Downloads | Version |
|---|---|---|---|---|---|---|
| ChatGoogleGenerativeAI | @langchain/google-genai | ❌ | ✅ | ✅ |  |  |
Model features
See the links in the table headers below for guides on how to use specific features.| Tool calling | Structured output | JSON mode | Image input | Audio input | Video input | Token-level streaming | Token usage | Logprobs |
|---|---|---|---|---|---|---|---|---|
| ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
Setup
You can access Google’sgemini and gemini-vision models, as well as other
generative models in LangChain through ChatGoogleGenerativeAI class in the
@langchain/google-genai integration package.
Credentials
Get an API key here: https://ai.google.dev/tutorials/setup Then set theGOOGLE_API_KEY environment variable:
Installation
The LangChainChatGoogleGenerativeAI integration lives in the @langchain/google-genai package:
Instantiation
Now we can instantiate our model object and generate chat completions:Invocation
Chaining
We can chain our model with a prompt template like so:Safety Settings
Gemini models have default safety settings that can be overridden. If you are receiving lots of “Safety Warnings” from your models, you can try tweaking the safety_settings attribute of the model. For example, to turn off safety blocking for dangerous content, you can import enums from the@google/generative-ai package, then construct your LLM as follows:
Tool calling
Tool calling with Google AI is mostly the same as tool calling with other models, but has a few restrictions on schema. The Google AI API does not allow tool schemas to contain an object with unknown properties. For example, the following Zod schemas will throw an error:const invalidSchema = z.object({ properties: z.record(z.unknown()) });
and
const invalidSchema2 = z.record(z.unknown());
Instead, you should explicitly define the properties of the object field. Here’s an example:
Built in Google Search Retrieval
Google also offers a built in search tool which you can use to ground content generation in real-world information. Here’s an example of how to use it:Code Execution
Google Generative AI also supports code execution. Using the built inCodeExecutionTool, you can make the model generate code, execute it, and use the results in a final completion:
Context Caching
Context caching allows you to pass some content to the model once, cache the input tokens, and then refer to the cached tokens for subsequent requests to reduce cost. You can create aCachedContent object using GoogleAICacheManager class and then pass the CachedContent object to your ChatGoogleGenerativeAIModel with enableCachedContent() method.
- Context caching supports both Gemini 1.5 Pro and Gemini 1.5 Flash. Context caching is only available for stable models with fixed versions (for example, gemini-1.5-pro-001). You must include the version postfix (for example, the -001 in gemini-1.5-pro-001).
- The minimum input token count for context caching is 32,768, and the maximum is the same as the maximum for the given model.
Gemini Prompting FAQs
As of the time this doc was written (2023/12/12), Gemini has some restrictions on the types and structure of prompts it accepts. Specifically:- When providing multimodal (image) inputs, you are restricted to at most 1 message of “human” (user) type. You cannot pass multiple messages (though the single human message may have multiple content entries)
- System messages are not natively supported, and will be merged with the first human message if present.
- For regular chat conversations, messages must follow the human/ai/human/ai alternating pattern. You may not provide 2 AI or human messages in sequence.
- Message may be blocked if they violate the safety checks of the LLM. In this case, the model will return an empty response.