AssemblyAI

AssemblyAI

This connector uses the AssemblyAI speech-to-text API: asynchronous transcription of audio and video files (api.assemblyai.com/v2), Speech Understanding and the LLM Gateway for questions about transcripts (llm-gateway.assemblyai.com/v1), and temporary tokens for real-time streaming (streaming.assemblyai.com/v3). The EU region uses the same hosts with eu. in front.

Setup (2 minutes)

1. Sign up at https://www.assemblyai.com/dashboard/signup (new accounts get $50 of free credit for transcription and Speech Understanding).
2. In the dashboard open API Keys, copy a key and paste it into ASSEMBLYAI_API_KEY. It is sent as the raw Authorization header value (no Bearer).
3. Set ASSEMBLYAI_DOMAIN to assemblyai.com (US, default) or eu.assemblyai.com (EU data residency: audio, transcripts and prompts are processed and stored in the EU). Transcripts made in one region are not visible from the other. On the EU LLM Gateway only Claude and Gemini models are available.

Costs: assemblyai_submit_transcript is billed per hour of audio, pro-rated to the second: Universal-3.5 Pro $0.21/h, Universal-2 $0.15/h; add-ons speaker labels $0.02/h, keyterms or prompt $0.05/h (Universal-3.5 Pro), medical mode $0.15/h; multichannel bills every channel. assemblyai_understand_transcript is billed per task. assemblyai_ask_transcript is billed per input and output token at the model's price (see assemblyai_list_llm_models, USD per million tokens) and is not covered by the free credit. Failed transcripts are free. Reads, assemblyai_list_llm_models and assemblyai_create_streaming_token are free (a streaming session itself is billed by connection time). Confirm with the user before transcribing long files.

Workflow

1. assemblyai_submit_transcript with audio_url, a public direct link to the file (a cloud storage link, a podcast MP3; not a YouTube page). Local files cannot be uploaded through this connector. Turn on what is needed: speaker_labels (who said what), language_detection or language_code, keyterms_prompt for names, redact_pii with redact_pii_policies, summarization or auto_chapters, entity_detection, sentiment_analysis, speech_understanding for translation or speaker names.
2. assemblyai_get_transcript with the returned id every few seconds until status is completed (or error, with the reason in error). Processing usually takes a fraction of the audio length.
3. Read: text and utterances (speaker, text, start, end) from assemblyai_get_transcript; assemblyai_get_sentences and assemblyai_get_paragraphs for timed segments; assemblyai_get_subtitles for SRT or VTT; assemblyai_search_words to find when something was said; assemblyai_get_redacted_audio for the beeped audio after redact_pii_audio.
4. Analyse: assemblyai_ask_transcript with a prompt that contains {{ transcript }} (summaries, action items, questions; this replaces the retired LeMUR API); assemblyai_understand_transcript to translate or identify speakers on a finished transcript. Pick a model with assemblyai_list_llm_models.
5. Manage: assemblyai_list_transcripts (newest first, filter by status or created_on; for older ones pass before_id = the last id of the page) and assemblyai_delete_transcript when the user asks to remove one (permanent).
6. Real time: assemblyai_create_streaming_token returns a short-lived token for a browser or device to open the streaming WebSocket (wss://streaming.assemblyai.com/v3/ws?token=...) without the API key.

Reading answers: all times (start, end, audio_start_from, word timestamps) are in milliseconds, except audio_duration, which is in seconds. confidence is 0 to 1. Speakers are labelled A, B, C within one transcript. Dates such as created are ISO 8601 UTC. Word-level timestamps are removed from assemblyai_get_transcript, assemblyai_get_sentences and assemblyai_get_paragraphs to keep answers small; very long transcripts are cut at about 600 KB.

Errors: 401 "Authentication error, API token missing/invalid" (wrong key); 400 with error naming a bad parameter (e.g. a feature not available for the language or model); 404 transcript not found (wrong id or other region); a transcript with status error and "Download error" means the audio_url could not be fetched (not public, or a web page instead of a file); 429 too many concurrent requests. On the streaming token, an invalid key answers 404 "Invalid API key".

Out of scope: uploading local files (/v2/upload needs a raw binary body), the synchronous Sync and Dictation APIs, text to speech, and saved prompts and tool calling on the LLM Gateway.

Switched off at install: assemblyai_delete_transcript; they cannot be undone, so an admin switches them on when the team needs them.