Changelog

Permissions per Model on the Inter-2 Routes

Grant each Inter-2 model on its own

An API key can now carry a permission for exactly one model on one route: interhumanai.upload.inter-2-audio for POST /v2/upload/analyze with model=inter-2-audio, interhumanai.stream.inter-2 for a /v2/stream/analyze session on Inter-2, and likewise interhumanai.upload.inter-2, interhumanai.upload.inter-2-deep and interhumanai.stream.inter-2-audio. A key granted one of them can do that and nothing else on /v2, and holding any interhumanai.upload.<model> permission reads the account’s upload jobs. Request the same strings on POST /v1/auth and POST /v1/client_tokens to mint a token for a subset of your key’s models, which is how a browser gets Inter-2 Audio on the stream and nothing more. The existing interhumanai.upload and interhumanai.stream permissions are the Inter-1 (v1) ones; they keep working on /v2 for now, and we will tell you before that changes.

Name the model when you open a stream

A /v2/stream/analyze session now names its model in the connection URL: wss://api.interhuman.ai/v2/stream/analyze?model=inter-2-audio. Leave the parameter out and the session opens on inter-2. Your credential needs the permission for the model you name, and session.ready lists, under supported_session_config_options.model, the models you may switch to here. Switching in the settings message to a model your credential lacks is answered with an error message (ih2003) and the session simply continues on its current model. Both first-party SDKs (0.18.0) add the permissions to Scope and a model option to the v2 stream client.

Choose the Model on the Inter-2 Stream, and Stream Audio Alone

Pick the model for a live session

A /v2/stream/analyze session now chooses its model in the settings message: send {"model": "inter-2-audio"} after session.ready and the session is analyzed by Inter-2 Audio, which reads the voice alone. Leave model out and the session runs Inter-2 on the video exactly as before. The models a deployment serves are listed in session.ready under supported_session_config_options.model, and session.updated echoes the one in force. Choose the model before you send media; it is fixed for the session after that.

Stream an audio-only recording

With inter-2-audio your recording no longer needs a video track. Send an audio-only WebM or MP4 as you would send video, or a recording that carries video too, and the model hears the audio. Every signal it reports carries modality: ["audio"].

Send raw audio frames

Live-audio clients can skip the container. Declare the shape once in the settings message and send signed 16-bit PCM frames as binary messages:

{
  "model": "inter-2-audio",
  "audio_format": { "encoding": "pcm_s16le", "sample_rate": 16000, "channels": 1 }
}

Frames can be any size and cut at any byte boundary, at 16 kHz or 24 kHz, mono. The API gathers them into the same rolling windows a video session gets and sends the same messages back. Both first-party SDKs (0.17.0) add the model and audio_format settings and a sendAudio / send_audio method for the frames.

Audio Analysis by Inter-2 Audio, as an Upload Job

Analyze an audio file with Inter-2 Audio

POST /v2/upload/analyze accepts an audio file — or a video with an audio track — and analyzes it with the Inter-2 Audio model: engagement and social signals read from the voice alone, per window, each signal with a rationale. Send the file and model=inter-2-audio as a multipart request with the same credential and interhumanai.upload permission the upload API already uses. Accepted files are wav, flac, mp3, m4a, ogg, and webm or mp4 that carry audio, at least 3 seconds long and at most 32 MB.

The analysis runs as a job

Unlike POST /v1/upload/analyze, the request does not wait for the analysis. It answers 202 at once with a job envelope: a job_id, a status of queued, and a status_url. Read GET /v2/upload/jobs/{job_id} until status is completed, when result holds one entry per window — its span, an engagement status, and the signals read over it — or failed, when error says why in the API’s standard error shape. Results stay readable until the envelope’s expires_at.To keep things to one request for a short file, add wait_seconds: the API holds the request open for up to that long and answers 200 with the finished job when it completes in time, else the 202 above.

curl -X POST https://api.interhuman.ai/v2/upload/analyze \
  -H "Authorization: Bearer $API_KEY" \
  -F "file=@call.wav" -F "model=inter-2-audio" -F "wait_seconds=60"

Both first-party SDKs (0.16.0) add submit, get_job / getJob and wait_for_job / waitForJob on the upload client.inter-2 and inter-2-deep are accepted values for model that this route does not serve yet; they answer ih4020 until their models are available.

A Clear Error for Streams That Never Decode

A stream that never decodes now tells you

Video arrives as media units — a cluster in WebM, a fragment in MP4. A clip that ends partway through a unit measures as zero length; the API holds those bytes and joins them to the clip that finishes the unit, so nothing is lost at the seam.A session that never finishes a unit at all — a byte stream cut at arbitrary offsets and never rejoined, or a session whose initialization segment never arrived — now ends with one ih6010 error message, instead of staying open and producing nothing. A session that has sent its header, or decoded anything, is never closed for this: it keeps running and recovers as soon as a unit completes.If you see ih6010, check that the sending client emits whole media fragments — for MediaRecorder, send its dataavailable blobs as they come rather than re-splitting them at your own byte offsets — and that any initialization segment is sent before the media that depends on it.

Stream Analysis with the Inter-2 Model

A second stream endpoint, analyzed by Inter-2

WS /v2/stream/analyze analyzes a live video feed with the Inter-2 model. It works exactly like WS /v1/stream/analyze: the same credential and interhumanai.stream permission, the same session.ready limits and session settings, the same video clips in, and the same stream of signal.*, engagement.updated, conversation_quality.updated, coverage.* and error messages out, in the same order — including the graceful session.close handshake. Only the model reading the video differs, so an application built for the v1 stream can switch by changing the path.

const ws = new WebSocket("wss://api.interhuman.ai/v2/stream/analyze", [
  "access_token",
  clientToken,
]);

Both first-party SDKs open it with a single option: client.stream({ apiVersion: "v2" }) in TypeScript and client.stream(api_version="v2") in Python.Where the Inter-2 model is not yet available, a v2 session is refused right after the handshake with one error message (ih1003) and close code 1013, and nothing is analyzed or billed.

Clear Errors for Streams Without Video

A stream with no video now tells you

WS /v1/stream/analyze and WS /v0/realtime/analyze analyze the picture in your stream. When a session’s recording carries no video track, the session now sends one error message with the code ih5001 and stops analyzing, rather than returning results with nothing in them.The usual cause is a recorder started from an audio-only source — a getUserMedia({ audio: true }) call with no video constraint, or a camera whose permission was never granted. The message says what is missing and what to do about it, and analysis resumes on its own once a recording with video arrives. Nothing the session skipped is billed.This is a requirement on the recording, not on each clip: a continuation clip carrying raw media with no header of its own is still fine.

Analyzing audio-only media in a realtime session

To analyze a stream that has no video track, select audio analysis on its own:

{ "analysis_groups": ["audio"] }

The visual group reads the picture, so it produces nothing from audio-only media. Leaving it selected and then sending a stream with no video is reported as ih5001 and stops the session’s analysis, the audio tracks included — a session never returns part of what you selected without telling you. Narrowing analysis_groups to ["audio"] resumes analysis from the next window.

SDK Attribution in API Telemetry

The API now records which SDK a request came from

Requests and live sessions can identify the client SDK that made them, so we can report SDK adoption, see which versions are in use, and give better support when a problem turns out to be version-specific.

HTTP requests may send X-Interhuman-SDK: <sdk-name>/<semver>, e.g. typescript/0.13.0.
WebSocket handshakes (WS /v1/stream/analyze, WS /v0/realtime/analyze) may carry sdk=<sdk-name>&sdk_version=<semver> on the URL, because a browser WebSocket cannot set custom handshake headers. The Sec-WebSocket-Protocol: access_token, <credential> authentication contract is unchanged.

If you use the first-party SDKs, @interhumanai/sdk and interhumanai 0.13.0 send this for you — upgrade and there is nothing else to do. If you call the API directly, nothing changes: the metadata is optional, and requests without it are supported exactly as before.The metadata is SDK identity and version only. It carries no device, OS, runtime, hostname, application, or end-user information, and because any caller can send any value, we treat it as a self-declared hint: it never affects authentication, authorization, scopes, quotas, or billing, and a missing, malformed, or unrecognized value is ignored rather than rejected.

A Dedicated Permission for the Realtime API

The Realtime API has its own permission

Realtime sessions authorize with interhumanai.realtime. Give the keys that open realtime sessions that permission, then request the scope when you mint credentials: pass "scopes": ["interhumanai.realtime"] to POST /v1/auth or POST /v1/client_tokens.One key can hold several permissions, so a single key can cover upload, stream, and realtime. A credential without interhumanai.realtime is refused at the WebSocket handshake with ih2003 (close code 4003).

Realtime Recommendations

Live guidance while the conversation is happening

The Realtime API can generate periodic recommendations for the responding speaker, grounded in the social signals detected so far and the transcript you supply.On WS /v0/realtime/analyze:

Set realtime_recommendation_instructions in the session config to describe your use case — the goal, the domain, and the shape of the output you want. A non-empty value turns recommendations on.
Set realtime_recommendation_frequency to pace them: high every 10 seconds of analyzed video, medium every 20 seconds, low every 30 seconds.
Listen for realtime_recommendation.generated, which carries the guidance text and the analyzed interval it covers.

A recommendation is emitted once all three conditions hold: you have supplied instructions, at least one signal has been detected, and at least one transcript has been received since the previous recommendation.

Visual-Coverage Notices on the Stream and Realtime APIs

Know when a window’s video coverage was partial

The informational coverage.degraded message reports analysis windows that decoded materially less video than the window span while the audio ran to the end — the shape a static screen share or a long-GOP encoder produces when its keyframe interval exceeds the analysis window.Those windows are still analyzed, using the audio plus whatever video decoded, and are billed normally. The notice lists the affected time ranges so your application knows the visual signals over that stretch drew on partial video.This applies to WS /v1/stream/analyze and WS /v0/realtime/analyze.

Signal Modality

Every signal names where its evidence came from

Signals carry a modality list naming the analysis modalities that detected them:

{ "type": "agreement", "start": 0.0, "end": 10.0, "probability": "high", "modality": ["video"] }

Signals from the Upload and Stream APIs carry video. A Realtime session runs several analyses at once, so its signals name every one that detected them — ["audio", "visual"] when both did.Use it to tell your users which evidence a signal rests on, or to weight signals that several modalities agree on.

Visual Tension Signal

The Realtime API detects `tension`

The Realtime API reports the signal tension, available when the visual analysis group is active for the session.

Stream Shutdown Grace Period

Close a stream session without losing the last windows

The stream WebSocket endpoint supports a caller-initiated graceful shutdown handshake. Send session.close when you have finished sending video and the API acknowledges with session.closing, including a maximum drain timeout. It then finishes analyzing the video it already accepted, ends still-active signals, sends session.ended, and closes the connection cleanly.

Inter-1 Streaming

Inter-1 goes streaming

The Inter-1 Streaming API is now available. The same behavioral analysis Inter-1 already delivers on upload — social signals with rationales, engagement, and the conversation quality — now runs on live video over WebSocket while the conversation is still happening.

Highlights

Live behavioral analysis: stream video chunks to wss://api.interhuman.ai/v1/stream/analyze and receive typed events (signal.detected, signal.ended, engagement.updated, and more) as state changes unfold.
Full Inter-1 capability on live video: the same social signals Inter-1 reports on upload, with structured rationales, engagement tracking, and optional five-dimension Conversation Quality Index scores — not a reduced streaming subset.
Predictable session contract: session.ready declares server limits up front; an optional session config message opts into additional analyses before you send the first frame.
Low-latency sliding windows: stream chunks at whatever size fits your pipeline; analysis runs on overlapping sliding windows with ordered, concurrent processing — when the queue backs up, dropped windows are reported so clients never miss silent gaps.
Production-minded billing and lifecycle: you are billed only for seconds actually analyzed and delivered; on disconnect, in-flight work is cancelled and active signals receive an implicit signal.ended.

Explore streaming

Authentication Simplification

Direct API key authentication for requests

Interhuman supports a simple authentication path for API integrations. You can send your API key directly in the Authorization header on requests, including POST /v1/upload/analyze.

Highlights

One-step auth for integrations: call API endpoints directly with Authorization: Bearer <api_key>.
Lower setup overhead: no required key exchange step before your first upload request.

General Availability

Interhuman V1 is live

Interhuman V1 is now generally available. This release stabilizes the core integration path: authenticate, upload a video, and receive structured analysis with predictable error handling.

Highlights

Self-serve onboarding: create an account and generate API keys directly at platform.interhuman.ai.
Stable API contract: V1 authentication and upload-analysis flows are stable for production use.
Reliable processing: consistent analysis completion across real-world video uploads.
Production-ready errors: standardized error payloads (error_id, correlation_id, link, message) for faster debugging and stronger recovery logic.
Faster setup path: quickstart and codealong docs that get you from API key to first successful analysis quickly.

Product Updates

Grant each Inter-2 model on its own

Name the model when you open a stream

Pick the model for a live session

Stream an audio-only recording

Send raw audio frames

Analyze an audio file with Inter-2 Audio

The analysis runs as a job

A stream that never decodes now tells you

A second stream endpoint, analyzed by Inter-2

A stream with no video now tells you

Analyzing audio-only media in a realtime session

The API now records which SDK a request came from

The Realtime API has its own permission

Live guidance while the conversation is happening

Know when a window’s video coverage was partial

Every signal names where its evidence came from

The Realtime API detects `tension`

Close a stream session without losing the last windows

Inter-1 goes streaming

Highlights

Explore streaming

Direct API key authentication for requests

Highlights

Interhuman V1 is live

Highlights

Explore V1

​Grant each Inter-2 model on its own

​Name the model when you open a stream

​Pick the model for a live session

​Stream an audio-only recording

​Send raw audio frames

​Analyze an audio file with Inter-2 Audio

​The analysis runs as a job

​A stream that never decodes now tells you

​A second stream endpoint, analyzed by Inter-2

​A stream with no video now tells you

​Analyzing audio-only media in a realtime session

​The API now records which SDK a request came from

​The Realtime API has its own permission

​Live guidance while the conversation is happening

​Know when a window’s video coverage was partial

​Every signal names where its evidence came from

​The Realtime API detects tension

​Close a stream session without losing the last windows

​Inter-1 goes streaming

​Highlights

​Explore streaming

​Direct API key authentication for requests

​Highlights

​Interhuman V1 is live

​Highlights

​Explore V1

Grant each Inter-2 model on its own

Name the model when you open a stream

Pick the model for a live session

Stream an audio-only recording

Send raw audio frames

Analyze an audio file with Inter-2 Audio

The analysis runs as a job

A stream that never decodes now tells you

A second stream endpoint, analyzed by Inter-2

A stream with no video now tells you

Analyzing audio-only media in a realtime session

The API now records which SDK a request came from

The Realtime API has its own permission

Live guidance while the conversation is happening

Know when a window’s video coverage was partial

Every signal names where its evidence came from

The Realtime API detects `tension`

Close a stream session without losing the last windows

Inter-1 goes streaming

Highlights

Explore streaming

Direct API key authentication for requests

Highlights

Interhuman V1 is live

Highlights

Explore V1