Verbalyx Speech → transcript → analysis → retrieval Q&A Connecting…

Speech recognition · LLM summarisation · retrieval-augmented generation

Analyse what is said in a video, then question it

The assistant extracts the audio track of a YouTube video or an uploaded file, transcribes it (Whisper, or Sarvam for Hinglish), asks Mistral for a title, summary, action items, decisions and open questions, and indexes the transcript so you can ask questions answered from retrieved passages. It analyses speech only, not the images in the video.

Preview appears here once you add a link or file

Processing

Idle

idle
  1. Audiodownload / convert, 10-min chunks—
  2. TranscribeWhisper or Sarvam—
  3. Titlefirst 2,000 chars—
  4. Summarymap-reduce over chunks—
  5. Extractactions, decisions, questions—
  6. Indexembed + store for Q&A—
Server log for this job 
No job yet.

Generated title

No analysis yet

LLM output · mistral-small-latest

The summary appears here as soon as that stage finishes.

Ask the transcript

Each question retrieves the 4 most similar transcript chunks and sends them, with the question, to the LLM. The answer is shown next to the passages it was given.

Questions unlock after indexingAnalyse a video first. Answers are restricted to the transcript; the model is told to say when the information is not there.

How it works

  1. Transcriptfull text from Whisper (WHISPER_MODEL, default small) or Sarvam saaras:v2.5
  2. ChunkRecursiveCharacterTextSplitter, 500 characters, 50 overlap
  3. Embedall-MiniLM-L6-v2 sentence embeddings, CPU
  4. StoreChroma, persisted in vector_db/, collection meeting_transcript (cleared before each new video)
  5. Retrievesimilarity search, top k = 4 chunks
  6. Generatemistral-small-latest (set by MISTRAL_MODEL), T = 0.3, prompted to answer only from the retrieved context

Analysis outputs

The summary is map-reduce: 3,000-character chunks (200 overlap) are summarised separately, then combined into bullet points. The title uses only the first 2,000 characters. Action items, decisions and open questions each send the full transcript in a single prompt.

Limitations

  • No timestamps or speakers. Only plain text is kept from transcription, so answers cannot point to a moment in the video.
  • One index at a time. Analysing a new video replaces the previous index, so questions always refer to the most recent video. Any retrieved passage not found in the current transcript would be flagged in red.
  • Long recordings. Extraction prompts include the whole transcript and may exceed the model's context window.
  • LLM outputs can be wrong. Summaries and extracted items are generated text; check them against the transcript.
  • Speech only. On-screen text, slides and visuals are not analysed.