ChatGroq

This will help you get started with Groq chat models. For detailed documentation of all ChatGroq features and configurations head to the API reference. For a list of all Groq models, visit this link.

Overview

Integration details

Class	Package	Local	Serializable	JS support	Downloads	Version
ChatGroq	langchain-groq	❌	beta	✅

Model features

Tool calling	Structured output	JSON mode	Image input	Audio input	Video input	Token-level streaming	Native async	Token usage	Logprobs
✅	✅	✅	❌	❌	❌	✅	✅	✅	✅

Setup

To access Groq models you’ll need to create a Groq account, get an API key, and install the langchain-groq integration package.

Credentials

Head to the Groq console to sign up to Groq and generate an API key. Once you’ve done this set the GROQ_API_KEY environment variable:

import getpass
import os

if "GROQ_API_KEY" not in os.environ:
    os.environ["GROQ_API_KEY"] = getpass.getpass("Enter your Groq API key: ")

To enable automated tracing of your model calls, set your LangSmith API key:

# os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")
# os.environ["LANGSMITH_TRACING"] = "true"

Installation

The LangChain Groq integration lives in the langchain-groq package:

%pip install -qU langchain-groq

Instantiation

Now we can instantiate our model object and generate chat completions.

Reasoning Format

If you choose to set a reasoning_format, you must ensure that the model you are using supports it. You can find a list of supported models in the Groq documentation.

from langchain_groq import ChatGroq

llm = ChatGroq(
    model="deepseek-r1-distill-llama-70b",
    temperature=0,
    max_tokens=None,
    reasoning_format="parsed",
    timeout=None,
    max_retries=2,
    # other params...
)

Invocation

messages = [
    (
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ),
    ("human", "I love programming."),
]
ai_msg = llm.invoke(messages)
ai_msg

AIMessage(content="J'aime la programmation.", additional_kwargs={'reasoning_content': 'Okay, so I need to translate the sentence "I love programming." into French. Let me think about how to approach this. \n\nFirst, I know that "I" in French is "Je." That\'s straightforward. Now, the verb "love" in French is "aime" when referring to oneself. So, "I love" would be "J\'aime." \n\nNext, the word "programming." In French, programming is "la programmation." But wait, in French, when you talk about loving an activity, you often use the definite article. So, it would be "la programmation." \n\nPutting it all together, "I love programming" becomes "J\'aime la programmation." That sounds right. I think that\'s the correct translation. \n\nI should double-check to make sure I\'m not missing anything. Maybe I can think of similar phrases. For example, "I love reading" is "J\'aime lire," but when it\'s a noun, like "I love music," it\'s "J\'aime la musique." So, yes, using "la programmation" makes sense here. \n\nI don\'t think I need to change anything else. The sentence structure in French is Subject-Verb-Object, just like in English, so "J\'aime la programmation" should be correct. \n\nI guess another way to say it could be "J\'adore la programmation," using "adore" instead of "aime," but "aime" is more commonly used in this context. So, sticking with "J\'aime la programmation" is probably the best choice.\n'}, response_metadata={'token_usage': {'completion_tokens': 346, 'prompt_tokens': 23, 'total_tokens': 369, 'completion_time': 1.447541218, 'prompt_time': 0.000983386, 'queue_time': 0.009673684, 'total_time': 1.448524604}, 'model_name': 'deepseek-r1-distill-llama-70b', 'system_fingerprint': 'fp_e98d30d035', 'finish_reason': 'stop', 'logprobs': None}, id='run--5679ae4f-f4e8-4931-bcd5-7304223832c0-0', usage_metadata={'input_tokens': 23, 'output_tokens': 346, 'total_tokens': 369})

print(ai_msg.content)

J'aime la programmation.

Chaining

We can chain our model with a prompt template like so:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a helpful assistant that translates {input_language} to {output_language}.",
        ),
        ("human", "{input}"),
    ]
)

chain = prompt | llm
chain.invoke(
    {
        "input_language": "English",
        "output_language": "German",
        "input": "I love programming.",
    }
)

AIMessage(content='The translation of "I love programming" into German is "Ich liebe das Programmieren." \n\n**Step-by-Step Explanation:**\n\n1. **Subject Pronoun:** "I" translates to "Ich."\n2. **Verb Conjugation:** "Love" becomes "liebe" (first person singular of "lieben").\n3. **Gerund Translation:** "Programming" is translated using the infinitive noun "Programmieren."\n4. **Article Usage:** The definite article "das" is included before the infinitive noun for natural phrasing.\n\nThus, the complete and natural translation is:\n\n**Ich liebe das Programmieren.**', additional_kwargs={'reasoning_content': 'Okay, so I need to translate the sentence "I love programming." into German. Hmm, let\'s break this down. \n\nFirst, "I" in German is "Ich." That\'s straightforward. Now, "love" translates to "liebe." Wait, but in German, the verb conjugation depends on the subject. Since it\'s "I," the verb would be "liebe" because "lieben" is the infinitive, and for first person singular, it\'s "liebe." \n\nNext, "programming" is a gerund in English, which is the -ing form. In German, the equivalent would be the present participle, which is "programmierend." But wait, sometimes in German, they use the noun form instead of the gerund. So maybe it\'s better to say "Ich liebe das Programmieren." Because "Programmieren" is the infinitive noun form, and it\'s commonly used in such contexts. \n\nLet me think again. "I love programming" could be directly translated as "Ich liebe Programmieren," but I\'ve heard both "Programmieren" and "programmierend" used. However, "Ich liebe das Programmieren" sounds more natural because it uses the definite article "das" before the infinitive noun. \n\nAlternatively, if I use "programmieren" without the article, it\'s still correct but maybe a bit less common. So, to make it sound more natural and fluent, including the article "das" would be better. \n\nTherefore, the correct translation should be "Ich liebe das Programmieren." That makes sense because it\'s similar to saying "I love (the act of) programming." \n\nI think that\'s the most accurate and natural way to express it in German. Let me double-check some examples. If someone says "I love reading," in German it\'s "Ich liebe das Lesen." So yes, using "das" before the infinitive noun is the correct structure. \n\nSo, putting it all together, "I love programming" becomes "Ich liebe das Programmieren." That should be the right translation.\n'}, response_metadata={'token_usage': {'completion_tokens': 569, 'prompt_tokens': 18, 'total_tokens': 587, 'completion_time': 2.511255685, 'prompt_time': 0.001466702, 'queue_time': 0.009628211, 'total_time': 2.512722387}, 'model_name': 'deepseek-r1-distill-llama-70b', 'system_fingerprint': 'fp_87eae35036', 'finish_reason': 'stop', 'logprobs': None}, id='run--4d5ee86d-5eec-495c-9c4e-261526cf6e3d-0', usage_metadata={'input_tokens': 18, 'output_tokens': 569, 'total_tokens': 587})

API reference

For detailed documentation of all ChatGroq features and configurations head to the API reference.