Hugging Face

Hugging Face Inference Providers is a single, OpenAI-compatible router that gives you access to hundreds of open-weight models served by many upstream infrastructure providers (Cerebras, Groq, Together, Fireworks, SambaNova, Fal, Replicate, Novita, and more) behind one token and one base URL.

In Odock, a Hugging Face provider behaves like any other upstream: you store one Hugging Face token as an encrypted provider key, create models that map your client-facing model name to a Hub model id, and call them through the gateway with a virtual API key.

Connection

The token is encrypted before storage and resolved only when the gateway calls upstream, exactly like every other provider key.

Endpoints

Hugging Face exposes an OpenAI-compatible surface for chat plus task-specific routes for everything else. Odock maps these to native gateway endpoints under /v1/huggingface/....

• Chat / models are OpenAI-shaped and hit router.huggingface.co/v1/... — point an OpenAI SDK at the gateway's /v1/huggingface base URL.
• Per-task routes proxy to the model's chosen inference provider — hf-inference/models/{model} by default, or {provider}/{providerModelId} when the model's slug carries a :provider suffix (see Choosing the task provider).
• /v1/huggingface/jobs exists only for the return_job async mode.

Chat and model listing are OpenAI-shaped, so an application can target them by pointing an OpenAI SDK at the Odock gateway /v1/huggingface base URL. The per-task routes are JSON in, JSON out: the request body is {"model": "...", "inputs": ...} (plus optional task parameters), and binary output (images, video, audio) comes back as a base64 envelope:

{ "model": "...", "mime_type": "image/png", "data": "<base64>", "encoding": "base64" }

JSON outputs (embedding vectors, ASR transcription) pass through unchanged. For the endpoint model, see Endpoints.

Choosing the task provider

Per-task routing depends on which inference provider serves the model. Select it with a :provider suffix on the model's upstream slug (the same idea as the chat suffix, but resolved by Odock rather than the chat router):

The provider name is whatever the model's page lists under Inference Providers (fal-ai, replicate, nebius, together, novita, …). Routing is generic — one engine handles all of them:

• hf-inference (default) — the synchronous, HF-normalized task API. Best for embeddings/ASR and any model hf-inference hosts. No Hub lookup.
• Any other provider — Odock resolves the model's provider model id from the Hub (inferenceProviderMapping, cached) and routes to router.huggingface.co/{provider}/{providerModelId} with the normalized request mapped to the common provider keys (prompt for generation, audio_url for ASR, inputs for embeddings). Then it:
  1. auto-detects async jobs — if the provider returns a queue handle (request_id / status_url / response_url), Odock polls server-side until it completes, so the client still makes one blocking call (bounded by the model's request timeout);
  2. downloads returned media — if the result is a URL (image/video/audio), Odock fetches it and base64-envelopes the bytes; JSON results (embeddings, transcription) pass through.

This covers providers that follow the common router patterns. Provider-specific quirks (e.g. fal-ai's async queue hint) live in a small data table in the provider package — adding a provider's quirk is a one-line change, not new routing code.

The client-facing model name stays clean (e.g. black-forest-labs/FLUX.1-dev); only the upstream slug carries the suffix. The catalog's image/video sample models already ship with :fal-ai.

Model id & provider selection

A Hugging Face model is identified by its Hub id (<org>/<model>, e.g. deepseek-ai/DeepSeek-R1). You can steer which upstream provider serves the request with a suffix on the model id:

Set this in the model's upstream slug when you add a model, so your client-facing model name stays stable while you tune cost/latency upstream.

Usage & billing

Hugging Face passes the upstream provider's rate through with no markup, and it ships a generous free tier plus PRO/Team credits. How Odock meters a call depends on the task:

• Chat returns a standard OpenAI usage object (prompt_tokens, completion_tokens, total_tokens), metered per token like any OpenAI-compatible model. Configure per-token pricing as usual — see Model pricing.
• Per-task routes (feature extraction, text-to-image, text-to-video, speech-to-text) return no usage object, so Odock derives a deterministic count of output items and bills it through the generic per-unit billing dimension (units / unit_kind) — the same mechanism used for Mistral OCR pages. The charge appears under units_cost_usd on the usage record.

To price a per-task model, set its pricing to a per-unit rate with the matching unit_kind (the model catalog ships defaults for the sample models above). For example, $0.03 per image with unit_kind: images bills two generated images as $0.06.

GET /v1/huggingface/models returns per-provider pricing, context length, latency, and throughput, which is useful when deciding how to price an Odock model that wraps a Hub model.

Async delivery modes (text-to-video)

Some providers run long tasks (notably text-to-video) on an async queue: submit returns a job id, and the result is fetched after the render completes. Odock supports two delivery modes, set with the PROVIDER_HUGGINGFACE_ASYNC_MODE variable in the gateway's .env (default wait):

In return_job mode, a text-to-video call returns:

{ "request_id": "019f…", "status": "IN_QUEUE", "provider": "fal-ai",
  "async_mode": "return_job", "poll_endpoint": "/v1/huggingface/jobs" }

Poll it with your virtual API key (Odock proxies the authenticated status/result fetch and downloads the media when ready):

curl https://your-odock-gateway/v1/huggingface/jobs \
  -H "Authorization: Bearer sk_your_virtual_key" \
  -H "Content-Type: application/json" \
  -d '{"model":"<your video model>","request_id":"019f…"}'
# in progress -> {"status":"IN_PROGRESS"}
# done        -> {"model":"…","mime_type":"video/mp4","data":"<base64>","encoding":"base64"}

The unit is billed once at submit; the /v1/huggingface/jobs status polls create no usage record at all (so repeated polling doesn't flood Usage Records with zero-cost rows). Sync tasks (image, embeddings, ASR) are unaffected by this setting — only async queue tasks use it.

Gateway environment variables

These are read from the gateway's .env at startup. The org's stored provider key and per-model upstream overrides take precedence per request — these are deployment-wide defaults.

# odock-server/.env
#PROVIDER_HUGGINGFACE_BASE_URL=https://router.huggingface.co
#PROVIDER_HUGGINGFACE_TIMEOUT=300s
#PROVIDER_HUGGINGFACE_ASYNC_MODE=wait

Hugging Face MCP server

Separately from inference, Hugging Face runs an official MCP server at https://huggingface.co/mcp (streamable HTTP, authenticated with an HF token). It exposes tools to search and explore the Hub (models, datasets, Spaces, papers), search the documentation in natural language, and run community Gradio Spaces as callable tools.

You add this to Odock as a normal MCP server record — not as a provider — so the same governance applies (access grants, tool rules, scoping, pricing, budgets, guardrails, usage records):

See Add an MCP server manually and Streamable HTTP transport. The server is open source at huggingface/hf-mcp-server.

Workflow

Tutorial: set up Hugging Face and send your first prompt

This walks the whole path end to end — from creating a Hugging Face token to prompting a chat model and running a per-unit task through Odock.

1. Create a Hugging Face token

2. Configure the provider in Odock

3. Prompt a chat model

Point the OpenAI SDK at the gateway's /v1/huggingface base URL — chat is fully OpenAI-compatible.

from openai import OpenAI

client = OpenAI(
    base_url="https://your-odock-gateway/v1/huggingface",
    api_key="sk_your_virtual_key",  # an Odock virtual API key, not your HF token
)

resp = client.chat.completions.create(
    model="openai/gpt-oss-120b",          # your Odock model name
    messages=[{"role": "user", "content": "In one sentence, what is an API gateway?"}],
)
print(resp.choices[0].message.content)

The equivalent cURL:

curl https://your-odock-gateway/v1/huggingface/chat/completions \
  -H "Authorization: Bearer sk_your_virtual_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Streaming works too — add "stream": true (and stream_options: {include_usage: true} to get token usage on the final chunk).

4. Run a per-unit task

The task routes are plain JSON. Each request body is {"model": "...", "inputs": ...} plus optional parameters.

Feature-extraction (embeddings) — billed per input:

curl https://your-odock-gateway/v1/huggingface/feature-extraction \
  -H "Authorization: Bearer sk_your_virtual_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sentence-transformers/all-MiniLM-L6-v2",
    "inputs": ["an API gateway routes requests", "embeddings are vectors"]
  }'
# -> [[...], [...]]   (2 inputs -> 2 units billed)

Text-to-image — billed per image; binary output is base64-enveloped:

curl https://your-odock-gateway/v1/huggingface/text-to-image \
  -H "Authorization: Bearer sk_your_virtual_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "black-forest-labs/FLUX.1-dev",
    "inputs": "a serene lake at sunset, photorealistic"
  }'
# -> { "model": "...", "mime_type": "image/png", "data": "<base64>", "encoding": "base64" }

Decode the image in your client:

import base64, httpx

r = httpx.post(
    "https://your-odock-gateway/v1/huggingface/text-to-image",
    headers={"Authorization": "Bearer sk_your_virtual_key"},
    json={"model": "black-forest-labs/FLUX.1-dev", "inputs": "a red bicycle"},
    timeout=120,
)
env = r.json()
open("out.png", "wb").write(base64.b64decode(env["data"]))

Speech-to-text (ASR) — billed per transcription; inputs is an audio URL or base64 data URI:

curl https://your-odock-gateway/v1/huggingface/speech-to-text \
  -H "Authorization: Bearer sk_your_virtual_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/whisper-large-v3",
    "inputs": "https://example.com/audio.flac"
  }'
# -> { "text": "..." }

Text-to-video — billed per clip. Video generation is slow (minutes), so behavior depends on the async mode:

curl https://your-odock-gateway/v1/huggingface/text-to-video \
  -H "Authorization: Bearer sk_your_virtual_key" \
  -H "Content-Type: application/json" \
  -d '{"model":"genmo/mochi-1-preview","inputs":"a timelapse of clouds at sunset"}'

# wait mode (default): blocks until done -> {"mime_type":"video/mp4","data":"<base64>",...}
# return_job mode: returns immediately -> {"request_id":"…","status":"IN_QUEUE","poll_endpoint":"/v1/huggingface/jobs"}
#   then poll until ready:
#   curl .../v1/huggingface/jobs -H "Authorization: Bearer sk_…" \
#     -d '{"model":"genmo/mochi-1-preview","request_id":"…"}'

5. Verify billing

Open Usage Records in your organisation. Chat calls show token costs; task calls show the per-unit charge — the Tokens card reports units / unit_kind and the Billing card reports units_cost_usd. See Record details.

Troubleshooting