STT WebSocket

Real-time Speech-to-Text WebSocket API for streaming audio transcription

STT WebSocket API

Real-time Speech-to-Text transcription via WebSocket streaming with support for 99+ languages and telephony integration.

🚀 Quick Start (Copy & Paste)

const WebSocket = require('ws');

// 1. Your API key
const API_KEY = 'sk_live_your_api_key';

// 2. Connect
const ws = new WebSocket(`wss://api.60db.ai/ws/stt?apiKey=${API_KEY}`);

// 3. Handle messages
ws.on('message', (data) => {
  const msg = JSON.parse(data);

  // Authenticated? Start session!
  if (msg.connection_established) {
    console.log('✅ Authenticated');
    ws.send(JSON.stringify({
      type: 'start',
      languages: ['en'],
      config: { encoding: 'mulaw', sample_rate: 8000, continuous_mode: true }
    }));
  }

  // Session ready? Send audio!
  if (msg.type === 'connected') {
    console.log('✅ Ready! Send audio now');

    // Send dummy audio (480 bytes every 60ms)
    let count = 0;
    const interval = setInterval(() => {
      ws.send(Buffer.alloc(480, 0xff));
      if (++count >= 83) {  // 5 seconds
        clearInterval(interval);
        ws.send(JSON.stringify({ type: 'stop' }));
      }
    }, 60);
  }

  // Got text!
  if (msg.type === 'transcription' && msg.is_final) {
    console.log('📝', msg.text);
  }

  // Done!
  if (msg.type === 'session_stopped') {
    console.log('✅ Complete! Cost:', msg.billing_summary.total_cost);
    ws.close();
  }
});

That’s it! You’ll see:

• ✅ Authenticated
• ✅ Ready! Send audio now
• 📝 Hello world (transcribed text)
• ✅ Complete! Cost: $0.000043

---

📖 How It Works (5 Simple Steps)

1. Connect with your API key
2. Send { type: "start", ... } to begin session
3. Stream audio data (binary chunks)
4. Receive text transcriptions in real-time
5. Stop with { type: "stop" } when done

---

Endpoint

Authentication

Query parameter authentication:

Example:

ws://api.60db.ai/ws/stt?apiKey=sk_live_your_api_key

Connection Details

Property	Value
Protocol	WebSocket (RFC 6455)
Frame types	Binary (telephony) or Text/JSON (browser)
Ping/keepalive	Server sends WebSocket pings every 30s (timeout 10s)

Session Lifecycle

Client                          Server
  │                               │
  │──── TCP/TLS connect ─────────►│
  │◄─── {"connecting": true} ─────│  authenticating...
  │◄─── connection_established ──│  authenticated
  │                               │
  │──── {"type":"start", ...} ───►│
  │◄─── {"type":"connected"} ─────│  session ready
  │                               │
  │──── audio frames / messages ─►│
  │◄─── {"type":"speech_started"} │  VAD detected voice
  │◄─── {"type":"transcription"}  │  is_final=true, speech_final=true
  │                               │
  │──── {"type":"stop"} ─────────►│
  │◄─── {"type":"session_stopped"}│

Client → Server Messages

start — Begin session

Sent once after connection is established. Must be sent before any audio.

Request

{
  "type": "start",
  "languages": ["en", "hi"],
  "config": {
    "encoding": "mulaw",
    "sample_rate": 8000,
    "utterance_end_ms": 500,
    "continuous_mode": true,
    "interim_results_frequency": 300,
    "no_speech_threshold": 0.60
  }
}

Parameters:

audio — JSON audio chunk (browser mode)

Request

{
  "type": "audio",
  "audio": "<base64-encoded Int16 PCM or μ-law bytes>",
  "encoding": "linear",
  "sample_rate": 48000,
  "timestamp": 1700000000000
}

Fields:

Binary frame — raw μ-law audio (telephony mode)

Send a raw WebSocket binary frame with μ-law bytes, no JSON wrapper. The server auto-detects this as telephony mode on the first binary frame.

Recommended chunk size: 480 bytes = 60ms at 8kHz
Twilio default: 160 bytes = 20ms — batch 3 chunks into 60ms before sending

config — Change language mid-session

Request

{
  "type": "config",
  "languages": ["hi"],
  "continuous_mode": true
}

Both languages and continuous_mode are optional; include only fields you want to change. Send "languages": null to revert to auto-detect.

stop — End session

Request

{
  "type": "stop"
}

Server processes any remaining audio buffer, sends session_stopped, then closes.

test — Ping / latency check

Request

{
  "type": "test",
  "message": "ping",
  "timestamp": 1700000000000
}

Server echoes test_response with the same timestamp for round-trip measurement.

Server → Client Messages

connecting — Authentication in progress

Response

{
  "connecting": true,
  "message": "Authenticating...",
  "timestamp": 1775465918269
}

connection_established — Authentication successful

Response

{
  "connection_established": {
    "service": "stt",
    "user_id": 43,
    "credit_balance": 9.97,
    "workspace": "default"
  }
}

Fields:

service · string

Service name: "stt"

user_id · integer

Your user ID

credit_balance · number

Available credits

workspace · string

Workspace name

connected — After start message is processed

Response

{
  "type": "connected",
  "server_info": {
    "server_type": "Modular Sentence-based STT",
    "device": "cuda",
    "model": "large-v3",
    "processing_mode": "sentence_based_modular",
    "supported_languages": { "en": "English", "hi": "Hindi" },
    "total_languages": 99,
    "features": {
      "sentence_based_processing": true,
      "real_time_streaming": true,
      "telephony_support": true,
      "unicode_support": true,
      "mixed_language_support": true
    }
  }
}

speech_started — VAD detected voice activity

Response

{
  "type": "speech_started",
  "timestamp": 1700000000.123
}

Use this for barge-in: interrupt TTS playback when this arrives. Fired after 2 consecutive VAD-positive chunks (~64ms of confirmed speech).

transcription — Transcription result

All results (interim and final) share the same transcription type — differentiate with flags.

Final result (is_final=true, speech_final=true):

Response

{
  "type": "transcription",
  "text": "Hello, how are you?",
  "confidence": 0.87,
  "language": "en",
  "language_name": "EN",
  "is_final": true,
  "speech_final": true,
  "is_partial": false,
  "sentence_id": 3,
  "processing_mode": "sentence_complete",
  "duration": 1.82,
  "latency": 0.43,
  "timestamp": 1700000000.456,
  "words": [
    { "word": "Hello", "start": 0.0, "end": 0.32, "probability": 0.98 },
    { "word": "how", "start": 0.35, "end": 0.52, "probability": 0.94 }
  ],
  "utterance_end_ms": 1820
}

Empty speech_final signal (text="", is_final=true, speech_final=true): Sent when audio was detected but transcription was rejected (silence, hallucination, low confidence, wrong language). Client should reset its state on this message and not treat it as an error.

Response

{
  "type": "transcription",
  "text": "",
  "confidence": 0.0,
  "is_final": true,
  "speech_final": true,
  "processing_mode": "speech_end_no_result",
  "timestamp": 1700000000.789
}

Interim result (is_final=false, speech_final=false) — only sent when interim_results_frequency is set:

Response

{
  "type": "transcription",
  "text": "Hello how",
  "confidence": 0.72,
  "language": "en",
  "is_final": false,
  "speech_final": false,
  "is_partial": true
}

Use interims only for barge-in word-count checks. Never send interim text to the LLM — a final with is_final=true, speech_final=true will follow.

Response Fields:

text · string

Transcribed text. Empty string = speech-end-no-result signal.

confidence · number

0.0–1.0. Telephony typically 0.35–0.75; browser 0.55–0.95.

language · string

Detected language code e.g. "en".

language_name · string

Uppercase language code e.g. "EN".

is_final · boolean

true = complete utterance transcription or end-of-speech signal.

speech_final · boolean

true when speech segment fully processed. Always matches is_final for finals.

is_partial · boolean

true for interim results only.

sentence_id · integer

Monotonically increasing counter per session.

duration · number

Duration (seconds) of the audio segment transcribed.

latency · number

Seconds from processing start to result ready (excludes queue time).

words · array

Word-level timestamps [{word, start, end, probability}]. Present on finals when available.

utterance_end_ms · integer

Timestamp (ms) of last word in the utterance.

processing_mode · string

Internal mode string — useful for debugging.

language_changed — After config message changes language

Response

{
  "type": "language_changed",
  "language": "Multi-language: HI",
  "language_code": ["hi"]
}

mode_changed — After config message changes continuous_mode

Response

{
  "type": "mode_changed",
  "continuous_mode": true,
  "mode_name": "continuous",
  "silence_threshold": 0.5
}

session_stopped — After stop is processed

Response

{
  "type": "session_stopped",
  "billing_summary": {
    "total_duration_seconds": 5.2,
    "total_cost": 0.000043,
    "characters_transcribed": 42
  }
}

error — Processing error

Response

{
  "type": "error",
  "error": "Audio processing error: ...",
  "timestamp": 1700000000.0
}

test_response — Reply to test ping

Response

{
  "type": "test_response",
  "message": "pong - Sentence-based STT ready",
  "timestamp": 1700000000000,
  "processing_mode": "sentence_based"
}

Complete Example

Audio Requirements

Property	Telephony (μ-law)	Browser (PCM)
Encoding	mulaw (8-bit)	linear (16-bit)
Sample Rate	8000 Hz	16000, 24000, 44100, 48000 Hz
Chunk Size	480 bytes (60ms)	960-1920 bytes (60-120ms)
Channels	Mono (1 channel)	Mono (1 channel)

Supported Languages

The service supports 99+ languages via Whisper auto-detection. Common languages include:

Code	Language	Code	Language
en	English	hi	Hindi
bn	Bengali	es	Spanish
fr	French	de	German
gu	Gujarati	ta	Tamil
te	Telugu	kn	Kannada
ml	Malayalam	mr	Marathi
pa	Punjabi	ar	Arabic

Pricing

• Rate: $0.00000833 per second
• Minimum: $0.01 per session
• Billing: Per second of audio processed

Related

• TTS WebSocket - Text-to-Speech endpoint
• WebSocket Quick Start - Get started guide
• WebSocket Playground - Test in browser