> ## Documentation Index
> Fetch the complete documentation index at: https://portkey-docs-add-third-party-integration-issues-fixes.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Fireworks

> Use Fireworks for chat, vision, embeddings, and image generation with advanced grammar and JSON modes through Portkey.

## Quick Start

Get started with Fireworks in under 2 minutes:

<CodeGroup>
  ```python Python icon="python" theme={null}
  from portkey_ai import Portkey

  # 1. Install: pip install portkey-ai
  # 2. Add @fireworks-ai provider in model catalog
  # 3. Use it:

  portkey = Portkey(api_key="PORTKEY_API_KEY")

  response = portkey.chat.completions.create(
      model="@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
      messages=[{"role": "user", "content": "Hello!"}]
  )

  print(response.choices[0].message.content)
  ```

  ```js Javascript icon="square-js" theme={null}
  import Portkey from 'portkey-ai'

  // 1. Install: npm install portkey-ai
  // 2. Add @fireworks-ai provider in model catalog
  // 3. Use it:

  const portkey = new Portkey({
      apiKey: "PORTKEY_API_KEY"
  })

  const response = await portkey.chat.completions.create({
      model: "@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
      messages: [{ role: "user", content: "Hello!" }]
  })

  console.log(response.choices[0].message.content)
  ```

  ```python OpenAI Py icon="python" theme={null}
  from openai import OpenAI
  from portkey_ai import PORTKEY_GATEWAY_URL

  # 1. Install: pip install openai portkey-ai
  # 2. Add @fireworks-ai provider in model catalog
  # 3. Use it:

  client = OpenAI(
      api_key="PORTKEY_API_KEY",  # Portkey API key
      base_url=PORTKEY_GATEWAY_URL
  )

  response = client.chat.completions.create(
      model="@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
      messages=[{"role": "user", "content": "Hello!"}]
  )

  print(response.choices[0].message.content)
  ```

  ```js OpenAI JS icon="square-js" theme={null}
  import OpenAI from "openai"
  import { PORTKEY_GATEWAY_URL } from "portkey-ai"

  // 1. Install: npm install openai portkey-ai
  // 2. Add @fireworks-ai provider in model catalog
  // 3. Use it:

  const client = new OpenAI({
      apiKey: "PORTKEY_API_KEY",  // Portkey API key
      baseURL: PORTKEY_GATEWAY_URL
  })

  const response = await client.chat.completions.create({
      model: "@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
      messages: [{ role: "user", content: "Hello!" }]
  })

  console.log(response.choices[0].message.content)
  ```

  ```sh cURL icon="square-terminal" theme={null}
  # 1. Add @fireworks-ai provider in model catalog
  # 2. Use it:

  curl https://api.portkey.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "x-portkey-api-key: $PORTKEY_API_KEY" \
    -d '{
      "model": "@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
      "messages": [{"role": "user", "content": "Hello!"}]
    }'
  ```
</CodeGroup>

## Add Provider in Model Catalog

Before making requests, add Fireworks to your Model Catalog:

1. Go to [**Model Catalog → Add Provider**](https://app.portkey.ai/model-catalog/providers)
2. Select **Fireworks**
3. Enter your [Fireworks API key](https://fireworks.ai/api-keys)
4. Name your provider (e.g., `fireworks-ai`)

<Card title="Complete Setup Guide" icon="book" href="/product/model-catalog">
  See all setup options and detailed configuration instructions
</Card>

***

## Fireworks Capabilities

### Embeddings

Generate embeddings using Fireworks models:

<CodeGroup>
  ```python Python theme={null}
  from portkey_ai import Portkey

  portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

  embeddings = portkey.embeddings.create(
      model="thenlper/gte-large",
      input="create vector representation on this sentence"
  )

  print(embeddings.data[0].embedding)
  ```

  ```javascript Node.js theme={null}
  import Portkey from 'portkey-ai';

  const portkey = new Portkey({
      apiKey: 'PORTKEY_API_KEY',
      provider: '@fireworks-ai'
  });

  const embeddings = await portkey.embeddings.create({
      model: "thenlper/gte-large",
      input: "create vector representation on this sentence"
  });

  console.log(embeddings.data[0].embedding);
  ```
</CodeGroup>

### Vision Models

Process images with vision models:

<CodeGroup>
  ```python Python theme={null}
  from portkey_ai import Portkey

  portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

  response = portkey.chat.completions.create(
      model="accounts/fireworks/models/firellava-13b",
      messages=[{
          "role": "user",
          "content": [
              {"type": "text", "text": "Can you describe this image?"},
              {
                  "type": "image_url",
                  "image_url": {
                      "url": "https://images.unsplash.com/photo-1582538885592-e70a5d7ab3d3"
                  }
              }
          ]
      }]
  )

  print(response.choices[0].message.content)
  ```

  ```javascript Node.js theme={null}
  import Portkey from 'portkey-ai';

  const portkey = new Portkey({
      apiKey: 'PORTKEY_API_KEY',
      provider: '@fireworks-ai'
  });

  const response = await portkey.chat.completions.create({
      model: "accounts/fireworks/models/firellava-13b",
      messages: [{
          role: "user",
          content: [
              { type: "text", text: "Can you describe this image?" },
              {
                  type: "image_url",
                  image_url: {
                      url: "https://images.unsplash.com/photo-1582538885592-e70a5d7ab3d3"
                  }
              }
          ]
      }]
  });

  console.log(response.choices[0].message.content);
  ```
</CodeGroup>

### Image Generation

Generate images with Stable Diffusion:

<CodeGroup>
  ```python Python theme={null}
  from portkey_ai import Portkey
  import base64
  from io import BytesIO
  from PIL import Image

  portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

  image = portkey.images.generate(
      model="accounts/fireworks/models/stable-diffusion-xl-1024-v1-0",
      prompt="An orange elephant in a purple pond"
  )

  Image.open(BytesIO(base64.b64decode(image.data[0].b64_json))).save("generated.png")
  ```

  ```javascript Node.js theme={null}
  import Portkey from 'portkey-ai';
  import fs from 'fs';

  const portkey = new Portkey({
      apiKey: 'PORTKEY_API_KEY',
      provider: '@fireworks-ai'
  });

  const image = await portkey.images.generate({
      model: "accounts/fireworks/models/stable-diffusion-xl-1024-v1-0",
      prompt: "An orange elephant in a purple pond"
  });

  const imageData = image.data[0].b64_json;
  fs.writeFileSync("generated.png", Buffer.from(imageData, 'base64'));
  ```
</CodeGroup>

### Function Calling

Use function calling with Fireworks models:

<Card title="Function Calling Guide" icon="function" href="/guides/getting-started/function-calling">
  Explore function calling examples and best practices
</Card>

### Grammar Mode

Fireworks lets you define [formal grammars](https://en.wikipedia.org/wiki/Formal%5Fgrammar) to constrain model outputs. You can use it to force the model to generate valid JSON, speak only in emojis, or anything else. ([Originally created by GGML](https://github.com/ggerganov/llama.cpp/tree/master/grammars))

Grammar mode is set with the `response_format` param. Just pass your grammar definition with `{"type": "grammar", "grammar": grammar_definition}`

Let's say you want to classify patient requests into 3 pre-defined classes:

<CodeGroup>
  ```python Python theme={null}
  from portkey_ai import Portkey

  portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

  patient_classification = """
  root      ::= diagnosis
  diagnosis ::= "flu" | "dengue" | "malaria"
  """

  response = portkey.chat.completions.create(
      model="accounts/fireworks/models/llama-v3-70b-instruct",
      messages=[{"role": "user", "content": "Patient has fever and chills"}],
      response_format={"type": "grammar", "grammar": patient_classification}
  )

  print(response.choices[0].message.content)
  ```

  ```javascript Node.js theme={null}
  import Portkey from 'portkey-ai';

  const portkey = new Portkey({
      apiKey: 'PORTKEY_API_KEY',
      provider: '@fireworks-ai'
  });

  const patient_classification = `
  root      ::= diagnosis
  diagnosis ::= "flu" | "dengue" | "malaria"
  `;

  const response = await portkey.chat.completions.create({
      model: "accounts/fireworks/models/llama-v3-70b-instruct",
      messages: [{ role: "user", content: "Patient has fever and chills" }],
      response_format: { type: "grammar", grammar: patient_classification }
  });

  console.log(response.choices[0].message.content);
  ```
</CodeGroup>

<Card title="Fireworks Grammar Guide" icon="brackets-curly" href="https://readme.fireworks.ai/docs/structured-output-grammar-based">
  Learn more about grammar mode and examples
</Card>

### JSON Mode

Force JSON output with optional schema validation:

<CodeGroup>
  ```python Python theme={null}
  from portkey_ai import Portkey
  from pydantic import BaseModel
  from typing import List

  portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

  class Recipe(BaseModel):
      title: str
      description: str
      steps: List[str]

  response = portkey.chat.completions.create(
      model="accounts/fireworks/models/llama-v3-70b-instruct",
      messages=[{"role": "user", "content": "Give me a recipe for making Ramen"}],
      response_format={
          "type": "json_object",
          "schema": Recipe.schema_json()
      }
  )

  print(response.choices[0].message.content)
  ```

  ```javascript Node.js theme={null}
  import Portkey from 'portkey-ai';

  const portkey = new Portkey({
      apiKey: 'PORTKEY_API_KEY',
      provider: '@fireworks-ai'
  });

  const response = await portkey.chat.completions.create({
      model: "accounts/fireworks/models/llama-v3-70b-instruct",
      messages: [{ role: "user", content: "Give me a recipe for making Ramen" }],
      response_format: {
          type: "json_object",
          schema: {
              type: "object",
              properties: {
                  title: { type: "string" },
                  description: { type: "string" },
                  steps: { type: "array" }
              }
          }
      }
  });

  console.log(response.choices[0].message.content);
  ```
</CodeGroup>

<Card title="Fireworks JSON Mode Guide" icon="brackets-curly" href="https://readme.fireworks.ai/docs/structured-response-formatting">
  Learn more about JSON mode
</Card>

***

## Supported Models

<Card title="Fireworks Models" icon="list" href="https://readme.fireworks.ai/docs/querying-text-models">
  View the complete list of models available on Fireworks
</Card>

***

## Next Steps

<CardGroup cols={2}>
  <Card title="Gateway Configs" icon="sliders" href="/product/ai-gateway">
    Add fallbacks, load balancing, and more
  </Card>

  <Card title="Observability" icon="chart-line" href="/product/observability">
    Monitor and trace your Fireworks requests
  </Card>

  <Card title="Prompt Library" icon="book" href="/product/prompt-engineering-studio">
    Manage and version your prompts
  </Card>

  <Card title="Metadata" icon="tag" href="/product/observability/metadata">
    Add custom metadata to requests
  </Card>
</CardGroup>

For complete SDK documentation:

<Card title="SDK Reference" icon="code" href="/api-reference/sdk/list">
  Complete Portkey SDK documentation
</Card>