Shared serving

Shared serving connects two sides of the marketplace:

• Guests call models other teams host — no GPU to rent, nothing to deploy, billed per token.
• Owners open their deployment’s spare throughput and are credited for every token guests spend — a discount on the deployment they’re already paying for.

Using shared models (guest)

Serverless › Shared lists every model currently open for guest traffic. Each entry shows the per-token price and a live availability state. Pick one to see exactly how to call it — View code gives a ready-to-paste snippet, Try in Playground works without any setup.

• Your existing workspace API key works — nothing new to create.
• Billing is per token, charged to workspace credit at the listed input/output rates per 1M tokens. This is the one place Serverless bills per token rather than per GPU-hour (how deployments are billed).
• Call it through the shared endpoint with the model id shown on the page:

curl https://inference.meshive.ai/v1/chat/completions \
  -H "Authorization: Bearer $MESHIVE_API_KEY" \
  -d '{"model": "shared/<model-id>", "messages": [{"role": "user", "content": "Hi"}]}'

GET /v1/models on the same endpoint lists everything currently available — handy for SDKs.

What guests should expect

Shared capacity is spare capacity — the catalog reflects what owners have open right now:

You might see	It means	Do
Available / Degraded / Unavailable state	Live pool health for that model	Degraded = slower but succeeding; retry a timeout
429 — Shared pool is busy right now	Guest demand exceeds idle capacity — temporary, not an outage	Retry shortly (the Playground retries automatically)
402 — You’re out of credit	Shared usage draws on workspace credit	Add credit
503 — This model just went offline	Every provider left the pool for that model	Notify me on the model row — your key and code stay valid when it returns

Guest requests are isolated: per-request engine logs are off, so not even the deployment’s owner sees your request contents.

Sharing your deployment (owner)

Eligible deployments show a Sharing section with a Share idle capacity toggle — in the deploy panel and in Edit deployment. Currently eligible: deployments of official text (LLM) models on vLLM; other deployments don’t show the section.

Turning it on opens a one-time consent covering the ground rules:

• Your requests always come first — guests only use idle capacity.
• Guest traffic never scales your deployment — autoscaling follows your own load only.
• 100% of what guests pay is credited to your balance.
• Guests run on this deployment’s hardware, and per-request engine logs are turned off — even you won’t see guests’ request details.

Enabling performs a rolling restart of your replicas to activate priority scheduling. With a single replica, the endpoint is briefly unavailable (up to a few minutes while the model reloads); sharing starts automatically once the restart completes — the card shows Sharing: Activating…, then a Shared badge.

Earnings appear on the deployment card as Earned this month, credited to your balance. Turn sharing off anytime in Edit deployment — the deployment simply stops serving guest traffic.

Note

Guest-side prices are platform-managed and may move with the market; your credit for guest usage is always 100% of what guests paid.