How does Inkling Small differ from Inkling?

Size and knowledge. Inkling Small has 276 billion total parameters and 12 billion active, against 975 billion and 41 billion for Inkling. Inkling Small matches or beats Inkling on coding, reasoning, and tool-use evaluations, and trails it on factual recall, scoring 20.6% on SimpleQA Verified against 43.9%.

How did Inkling Small score on coding benchmarks?

80.2% on SWE-bench Verified, 55.9% on SWE-bench Pro (public), 64.7% on Terminal-Bench 2.1, and 48.7% on SciCode. Each result sits at or above Inkling on the same evaluation.

What input types does Inkling Small accept?

Text, images, and audio, with text output. Images are encoded through a hierarchical patch encoder and audio through discrete token encoding, and the decoder processes all three together.

How does Inkling Small handle documents and charts?

Inkling Small can crop, zoom, and inspect images programmatically, which helps when the detail that answers a question is small. Inkling Small scores 77.4% on CharXiv reasoning questions, or 81.3% with a Python tool, and 74.0% on MMMU Pro.

How does controllable thinking effort work?

You set effort per request, from minimal to maximum. Higher effort spends more thinking tokens and raises scores on hard tasks; lower effort cuts cost and latency. Published benchmark results are reported at maximum effort.

Does AI Gateway support Zero Data Retention for Inkling Small?

Yes, Zero Data Retention is available for this model. Turn it on team-wide from the dashboard or per request with `zeroDataRetention: true`. Zero Data Retention is offered on a per-provider basis. See https://vercel.com/docs/ai-gateway/capabilities/zdr for details.

Are the weights for Inkling Small open?

Yes. Thinking Machines released Inkling Small with open weights under the Apache 2.0 license, which permits commercial use and modification.

How do I call Inkling Small through AI Gateway?

Set the model to `thinkingmachines/inkling-small` in the AI SDK, Chat Completions API, Responses API, Messages API, or other API formats, from TypeScript or Python. AI Gateway handles authentication, retries, and failover across `baseten`, `deepinfra`, `togetherai`, `thinkingmachines`.

What does Inkling Small cost?

Current rates appear in the pricing panel on this page. AI Gateway mirrors provider pricing with no markup and adds no platform fee on inference, including on Bring Your Own Key requests.

Thinking Machines

Inkling Small

Inkling Small is the smaller model in the Inkling family at 276 billion total parameters and 12 billion active. It matches or beats Inkling on many coding, reasoning, and tool-use evaluations, reasons natively over images and audio, and supports a context window of 1M tokens.

View API reference

Input and output price: Prices from: Input $0.50, Output $1.20, Per 1M tokens
24h uptime: Loading AI Gateway uptime

Get API key Read the docs

import { streamText } from 'ai'

const result = streamText({
  model: 'thinkingmachines/inkling-small',
  prompt: 'Why is the sky blue?'
})

Read docs

Overview Playground Providers Uptime Status Throughput Latency API Similar About FAQ

Copy link to headingPlayground

Try out Inkling Small by Thinking Machines. Usage is billed to your team at API rates. Free users (those who haven't made a payment) get $5 of credits every 30 days.

Inkling Small

Copy link to headingProviders

Route requests across multiple providers. Copy a provider slug to set your preference. Visit the docs for more info. Using a provider means you agree to their terms, listed under Legal.

Provider

Context	Max Output	Latency	Throughput	Input	Output	Cache	Web Search	Capabilities	ZDR	No Training	Regional Inference	Free Tier	Release Date

Baseten

Legal:Terms•Privacy

1.3 s

131 tps

$0.50/M

$1.20/M

Read$0.10/M

—

07/30/2026

DeepInfra

Legal:Terms•Privacy

0.3 s

133 tps

$0.58/M

$1.44/M

Read$0.12/M

—

07/30/2026

Together AI

Legal:Terms•Privacy

0.3 s

322 tps

$0.50/M

$1.20/M

Read$0.10/M

—

07/30/2026

Thinking Machines

Legal:Terms•Privacy

64K

1.6 s

296 tps

$0.30/M

$1.20/M

Read$0.06/M

—

07/30/2026

Getting started

Call Inkling Small through AI Gateway with the AI SDK generateText and streamText functions, or through the OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages APIs by changing the base URL. AI Gateway authenticates the request and routes it to an available provider.

Install the AI SDK (pnpm add ai dotenv), create an API key from the API Keys page, and set it as AI_GATEWAY_API_KEY in your environment. Full setup is covered in the text generation quickstart.

index.ts

import { generateText } from 'ai';
import 'dotenv/config';

async function main() {
  const result = await generateText({
    model: 'thinkingmachines/inkling-small',
    prompt: 'Why is the sky blue?',
  });

  console.log(result.text);
}

main().catch(console.error);

Top-level parameters

The same Inkling Small request in each API format AI Gateway supports.

top-level-params.ts

import { generateText } from 'ai';
import 'dotenv/config';

async function main() {
  const result = await generateText({
    model: 'thinkingmachines/inkling-small',
    system: 'You are a concise technical assistant.',
    prompt: 'Summarize the tradeoffs between static generation and SSR.',
    maxOutputTokens: 1024,
  });

  console.log(result.text);
}

main().catch(console.error);

Standard parameters like prompt, messages, temperature, and tools work as documented in the AI SDK docs. These are the parameters with model-specific behavior.

Parameter	Type	Required	Description
`model`	`string`	Yes	Model ID in the form `creator/model`, e.g. `thinkingmachines/inkling-small`. AI Gateway routes the request to an available provider.
`maxOutputTokens`	`number`	No	Hard cap on generated tokens. Inkling Small supports up to `1,000,000` output tokens. Reasoning tokens count toward this limit.
`reasoning`	`'provider-default' \| 'none' \| 'minimal' \| 'low' \| 'medium' \| 'high' \| 'xhigh'`	No	Provider-agnostic reasoning effort, available in AI SDK 7 or later. Maps to the provider’s native reasoning configuration; reasoning settings under `providerOptions` take precedence when both are set. See the Reasoning section below.
`providerOptions`	`Record<string, JSONValue>`	No	AI Gateway routing options under `gateway`, plus any provider-native options under the provider’s own namespace — see the table below.

Input limits

Input	Formats	Sources	Max count	Max size	Limits
Text	—	—	—	—	Prompt and response share the 1M-token context window
Image	—	URL, base64, Uint8Array	—	—	Sent as image parts in messages; counts as input tokens
PDF	—	URL, base64, Uint8Array	—	—	Sent as file parts in messages; counts as input tokens

Provider options

Set AI Gateway routing options under providerOptions.gateway. For provider-specific options, pass them under the provider’s namespace as documented by the AI SDK.

Learn more in the AI SDK provider docs.

provider-options.ts

import { generateText } from 'ai';
import 'dotenv/config';

async function main() {
  const result = await generateText({
    model: 'thinkingmachines/inkling-small',
    prompt: 'Why is the sky blue?',
    providerOptions: {
      gateway: {
        only: ['baseten', 'deepinfra'],
      },
    },
  });

  console.log(result.text);
}

main().catch(console.error);

These AI Gateway routing options apply to every model. Provider-specific options pass through under the provider’s own namespace (for example providerOptions.anthropic) exactly as documented by the AI SDK.

Parameter	Type	Required	Description
`providerOptions.gateway.only`	`string[]`	No	Restrict routing to these provider slugs. Requests fail over only within the listed providers.
`providerOptions.gateway.order`	`string[]`	No	Preferred provider order. Listed providers are tried first; unlisted providers remain available as fallbacks.
`providerOptions.gateway.sort`	`'cost' \| 'ttft' \| 'tps'`	No	Rank candidate providers by price, time to first token, or tokens per second instead of the default routing order.
`providerOptions.gateway.zeroDataRetention`	`boolean`	No	Route only to providers with a zero-data-retention policy for this model.

Routing across providers

AI Gateway serves the same model through multiple providers and fails over automatically. order expresses a preference while keeping every provider eligible; only is a hard allowlist — if none of the listed providers are available the request fails instead of falling back.

Options under a provider's own namespace (for example providerOptions.anthropic) are forwarded to that provider with the request. Providers ignore option namespaces that don't apply to them, so it is safe to set provider options alongside gateway routing options.

Reasoning

AI Gateway bridges reasoning across every API format. The AI SDK exposes a provider-agnostic top-level reasoning level (none, minimal, low, medium, high, or xhigh); the Chat Completions and Responses formats take the same effort under reasoning.effort; and the Anthropic Messages format uses a native thinking token budget. Whichever you send, the gateway maps it to the target model’s native configuration, converting between effort levels and token budgets as needed. Reasoning-related settings under providerOptions take full precedence over the top-level reasoning value and are never merged. Reasoning tokens typically count toward your output-token usage, though how they’re reported and billed varies by provider.

Learn more in the AI Gateway reasoning guide.

reasoning.ts

import { generateText } from 'ai';
import 'dotenv/config';

async function main() {
  const result = await generateText({
    model: 'thinkingmachines/inkling-small',
    prompt: 'Explain the Monty Hall problem step by step.',
    reasoning: 'high',
  });

  console.log(result.text);
}

main().catch(console.error);

Image input

Send images alongside text as message parts. Images count as input tokens.

image-input.ts

import { generateText } from 'ai';
import 'dotenv/config';

async function main() {
  const result = await generateText({
    model: 'thinkingmachines/inkling-small',
    messages: [
      {
        role: 'user',
        content: [
          { type: 'text', text: 'Describe this image.' },
          { type: 'image', image: 'https://example.com/photo.jpg' },
        ],
      },
    ],
  });

  console.log(result.text);
}

main().catch(console.error);

PDF input

Attach PDFs as file parts. Their contents count as input tokens.

pdf-input.ts

import { generateText } from 'ai';
import 'dotenv/config';

async function main() {
  const result = await generateText({
    model: 'thinkingmachines/inkling-small',
    messages: [
      {
        role: 'user',
        content: [
          { type: 'text', text: 'Summarize this document.' },
          {
            type: 'file',
            mediaType: 'application/pdf',
            data: 'https://example.com/document.pdf',
          },
        ],
      },
    ],
  });

  console.log(result.text);
}

main().catch(console.error);

Tool calling

Expose tools the model can call. Define each tool’s inputs with a Zod schema.

tool-calling.ts

import { generateText, tool } from 'ai';
import { z } from 'zod';
import 'dotenv/config';

async function main() {
  const result = await generateText({
    model: 'thinkingmachines/inkling-small',
    prompt: 'What is the weather in San Francisco?',
    tools: {
      getWeather: tool({
        description: 'Get the current weather for a location',
        inputSchema: z.object({ location: z.string() }),
        execute: async ({ location }) => ({ location, temperatureC: 18 }),
      }),
    },
  });

  console.log(result.text);
}

main().catch(console.error);

Copy link to headingMore models by Thinking Machines

Model

Context	Latency	Throughput	Input	Output	Cache	Web Search	Capabilities	Providers	ZDR	No Training	Free Tier	Release Date

thinkingmachines/inkling

262K

0.3 s

239 tps

$1/M

$4.05/M

Read$0.17/M

—

07/15/2026

Copy link to headingAbout Inkling Small

Inkling Small became available on AI Gateway on July 30, 2026. Inkling Small is a 42-layer decoder-only transformer with a sparse Mixture-of-Experts (MoE) backbone: 276 billion total parameters, 12 billion active per token, and each token routed to 6 of 256 experts plus 2 shared experts. Attention mixes local and global layers, images enter through a hierarchical patch encoder, audio through discrete token encoding, and the context window is 1M tokens. Thinking Machines released the weights under the Apache 2.0 license.

On agentic coding, Inkling Small scores 80.2% on SWE-bench Verified, 55.9% on SWE-bench Pro (public), 64.7% on Terminal-Bench 2.1, and 48.7% on SciCode. Each of those sits at or above Inkling, which scores 77.6%, 54.3%, 63.8%, and 46.1%. Tool use follows the same pattern: 54.4% on Toolathlon Verified against Inkling's 45.5%, and 79.6% on the public split of MCP Atlas. On reasoning it scores 89.5% on GPQA Diamond, 90.2% on HMMT February 2026, 31.6% on Humanity's Last Exam text-only, and 40.1% on ARC-AGI-2.

The gap is knowledge. Inkling Small scores 20.6% on SimpleQA Verified where Inkling scores 43.9%, and it trails on the AA Omniscience index and on Tau 3 Banking, a multi-turn domain agent evaluation. Audio results sit close behind Inkling at 90.1% on VoiceBench, 77.0% on MMAU, and 54.9% on Audio MC. Treat Inkling Small as a capable reasoner with a smaller store of memorized facts, and give it a retrieval path when questions turn factual.

Vision is a practical strength. Inkling Small scores 74.0% on MMMU Pro and 77.4% on CharXiv reasoning questions, rising to 81.3% when it crops, zooms, and inspects images programmatically. That helps most on documents, forms, and charts where the detail that answers the question is small.

Controllable thinking effort runs from minimal to maximum, so you can trade answer quality against cost and latency per request. Sweep the setting across representative traffic rather than fixing it up front.

Inkling Small is compatible with Zero Data Retention on AI Gateway. Turn it on team-wide from the dashboard, or per request with zeroDataRetention: true, and AI Gateway routes only to providers that delete prompts and responses after each request. Set the model to thinkingmachines/inkling-small in the AI SDK, Chat Completions API, Responses API, Messages API, or other API formats, from TypeScript or Python. To use Inkling Small in a coding agent, run vercel ai-gateway coding-agents setup, then select thinkingmachines/inkling-small in the agent's model configuration.

Copy link to headingWhat To Consider When Choosing a Provider

Configuration: Size shows up in world knowledge, not in reasoning or coding. Inkling Small scores 20.6% on SimpleQA Verified against Inkling's 43.9%, so anything that depends on recalling specific facts needs retrieval or web search alongside the model. Reasoning, coding, and tool-use scores hold up, and several of them land above Inkling's.
Zero Data Retention: Zero Data Retention is available for this model. It is offered on a per-provider and model basis. See the documentation for details.
Authentication: AI Gateway authenticates requests using an API key or OIDC token. You do not need to manage provider credentials directly.

Copy link to headingWhen to Use Inkling Small

Best for

High-Volume Coding Agents: Agentic coding and tool use, with 80.2% on SWE-bench Verified
Tool Orchestration Pipelines: 54.4% on Toolathlon Verified and 79.6% on the public MCP Atlas split
Document and Chart Analysis: Programmatic cropping and zooming to read small visual detail
Compact Multimodal Apps: Native text, image, and audio input in a smaller model than Inkling
Effort-Tuned Latency Budgets: Thinking effort set from minimal to maximum on each request
Zero Data Retention Routing: Requests routed only to providers that delete prompts and responses

Consider alternatives when

World Knowledge Questions: 20.6% on SimpleQA Verified against Inkling's 43.9% is a real recall gap
Strongest Audio Results: Inkling scores higher on VoiceBench, MMAU, and Audio MC
Multi-Turn Domain Agents: Inkling scores higher on Tau 3 Banking and on broader factual evaluations
Frontier Coding Ceiling: Closed frontier models still lead on SWE-bench Verified and Terminal-Bench 2.1

Copy link to headingConclusion

Inkling Small keeps Inkling's coding, reasoning, tool use, and multimodal input in a model a quarter the size, and gives up world knowledge to get there. Use Inkling Small for coding agents, tool pipelines, and document work, and add retrieval when the questions turn factual.

Copy link to headingFrequently Asked Questions

How does Inkling Small differ from Inkling?
Size and knowledge. Inkling Small has 276 billion total parameters and 12 billion active, against 975 billion and 41 billion for Inkling. Inkling Small matches or beats Inkling on coding, reasoning, and tool-use evaluations, and trails it on factual recall, scoring 20.6% on SimpleQA Verified against 43.9%.
How did Inkling Small score on coding benchmarks?
80.2% on SWE-bench Verified, 55.9% on SWE-bench Pro (public), 64.7% on Terminal-Bench 2.1, and 48.7% on SciCode. Each result sits at or above Inkling on the same evaluation.
What input types does Inkling Small accept?
Text, images, and audio, with text output. Images are encoded through a hierarchical patch encoder and audio through discrete token encoding, and the decoder processes all three together.
How does Inkling Small handle documents and charts?
Inkling Small can crop, zoom, and inspect images programmatically, which helps when the detail that answers a question is small. Inkling Small scores 77.4% on CharXiv reasoning questions, or 81.3% with a Python tool, and 74.0% on MMMU Pro.
How does controllable thinking effort work?
You set effort per request, from minimal to maximum. Higher effort spends more thinking tokens and raises scores on hard tasks; lower effort cuts cost and latency. Published benchmark results are reported at maximum effort.
Can I use Inkling Small in a coding agent?
Yes. Run vercel ai-gateway coding-agents setup to connect your agents to AI Gateway, then select thinkingmachines/inkling-small in the agent's model configuration. See the coding agents guide at https://vercel.com/docs/ai-gateway/coding-agents.
Does AI Gateway support Zero Data Retention for Inkling Small?
Yes, Zero Data Retention is available for this model. Turn it on team-wide from the dashboard or per request with zeroDataRetention: true. Zero Data Retention is offered on a per-provider basis. See https://vercel.com/docs/ai-gateway/capabilities/zdr for details.
Are the weights for Inkling Small open?
Yes. Thinking Machines released Inkling Small with open weights under the Apache 2.0 license, which permits commercial use and modification.
How do I call Inkling Small through AI Gateway?
Set the model to thinkingmachines/inkling-small in the AI SDK, Chat Completions API, Responses API, Messages API, or other API formats, from TypeScript or Python. AI Gateway handles authentication, retries, and failover across baseten, deepinfra, togetherai, thinkingmachines.
What does Inkling Small cost?
Current rates appear in the pricing panel on this page. AI Gateway mirrors provider pricing with no markup and adds no platform fee on inference, including on Bring Your Own Key requests.

Your use is subject to Thinking Machines's Terms & Privacy Policies.

Agent Stack

Core Platform

Tools

Learn

Build

Explore

Inkling Small

Copy link to headingPlayground

Copy link to headingProviders

Copy link to headingUptime24 hours

Copy link to headingThroughput24 hours

Copy link to headingLatency24 hours

Getting started

Top-level parameters

Input limits

Provider options

Routing across providers

Reasoning

Image input

PDF input

Tool calling

Copy link to headingMore models by Thinking Machines

Copy link to headingAbout Inkling Small

Copy link to headingWhat To Consider When Choosing a Provider

Copy link to headingWhen to Use Inkling Small

Best for

Consider alternatives when

Copy link to headingConclusion

Copy link to headingFrequently Asked Questions