Skip to content
Star on GitHub
ConversationVoice capture
Style
Accent
Sound

Built by DesignMe, free to use under the MIT licence.

Expert design and dev support for startups and enterprises.

Book a call

Voice capture

Talk to the agent, then fix what it misheard before sending. Free and MIT licensed, as HTML or a typed React component, in two styles with light and dark mode.

$npx designme add voice-capture
Listening0:00

01

Install

Add it to your project with the CLI.

HTML, CSS and JavaScript

$npx designme add voice-capture

React

$npx designme add voice-capture --react

The CLI copies the source into your project, so you own it and can change anything. The HTML version lands in ./designme with the shared core and a demo.html. The React version lands in src/components/designme and needs React 19; it works in Next.js, Vite and Astro.

02

About

What it is for and what it does.

Voice capture is the panel a person talks into when they speak a request to an agent. It shows a live waveform, a clock and the transcript as it is heard, highlights the words the speech model was unsure of, and asks the person to check them before the text is sent.

  • When people speak tasks to your agent, on a phone or hands-free, and transcription mistakes would send the agent the wrong way.
  • When your speech-to-text service returns alternatives for low-confidence words and you want people to pick the right one.
  • When people need to pause, cancel or record again without losing track of what was sent.
03

Usage

Import it and drive it from your own state.

Import

import { VoiceCapture } from '@/components/designme/voice-capture/VoiceCapture';

Example

The demo above as a React component, 46 lines you can copy.

'use client';
// The animated preview from designme.dev, rebuilt from the React component: an example of driving it from your own state.
import { useEffect, useRef, useState } from 'react';
import { VoiceCapture, type HeardWord, type VoiceCaptureState } from './VoiceCapture';
import { useScript } from '../lib/useScript';

/* ---------- Voice capture: a sentence is heard word by word, two words it was unsure of get a highlight ---------- */
const SENTENCE = 'Move the yearly toggle above the cards and make the save badge say 20% off for annual plans'.split(' ');
const UNSURE: Record<number, string[]> = { 3: ['toggle', 'toddle', 'tunnel'], 10: ['save', 'sale', 'saving'] };

export default function VoiceCaptureDemo() {
  // Record again starts the whole take over, like replaying the card on the site
  const [take, setTake] = useState(0);
  return <Take key={take} onRecordAgain={() => setTake(t => t + 1)} />;
}

function Take({ onRecordAgain }: { onRecordAgain: () => void }) {
  const [words, setWords] = useState<Array<string | HeardWord>>([]);
  const [state, setState] = useState<VoiceCaptureState>('listening');
  const now = useRef(state);
  useEffect(() => { now.current = state; });
  // a stand-in for the microphone: each word makes noise for the next five samples
  const speak = useRef(0);
  const level = () => { if (speak.current <= 0) return 0; speak.current--; return .2 + Math.random() * .8; };

  useScript(async ({ wait }) => {
    await wait(700);
    for (let i = 0; i < SENTENCE.length; i++) {
      while (now.current === 'paused') await wait(100);
      if (now.current !== 'listening') return;
      speak.current = 5;
      const w = SENTENCE[i], alternatives = UNSURE[i];
      setWords(ws => [...ws, alternatives ? { text: w, alternatives } : w]);
      await wait(170 + Math.random() * 150);
    }
    await wait(900);
    if (now.current === 'listening' || now.current === 'paused') setState('review');
  });

  return (
    <div style={{ width: '100%' }}>
      <VoiceCapture words={words} level={level} state={state} onStateChange={setState} onRecordAgain={onRecordAgain} />
    </div>
  );
}
04

Props

Everything VoiceCapture takes.

PropTypeWhat it does
wordsrequiredArray<string | HeardWord>The words heard so far, in order. Add to the end as they arrive and each one fades in
stateVoiceCaptureStatelistening and paused record, review asks the person to check the words, sent and cancelled are the end. Leave it out to let the buttons move it along
defaultStateVoiceCaptureState
onStateChange(state: VoiceCaptureState) => void
secondsnumberSeconds recorded. Leave it out to let it count while listening
levelnumber | (() => number)The input level from 0 to 1, drawn into the wave about 20 times a second. Pass a function to read the microphone on every sample (for example from an AnalyserNode) without re-rendering
statusstringThe status line. Defaults to Listening, Paused, Check what I heard, Sent or Cancelled
onSend(text: string) => voidCalled with the checked transcript when the person presses Done in review
onFix(index: number, text: string) => voidCalled when the person picks one of a word's alternatives
onRecordAgain() => voidCalled by Record again, once the capture is sent or cancelled. Clear your words and start listening again
classNamestring
05

Code

The full source, as the CLI installs it.

VoiceCapture.tsxReact · 224 lines
'use client';
import { useEffect, useRef, useState, type CSSProperties, type KeyboardEvent as ReactKeyboardEvent } from 'react';
import { cx } from '../lib/cx';
import { useControllable, useIsoLayoutEffect } from '../lib/hooks';
import { Icon } from '../lib/icons';
import { Swap, Tip } from '../lib/motion';
import '../designme.css';
import './voice-capture.css';

export type VoiceCaptureState = 'listening' | 'paused' | 'review' | 'sent' | 'cancelled';

/** A word the speech model was not sure about */
export interface HeardWord {
  text: string;
  /** What it might have been instead. Highlights the word and lets the person pick one */
  alternatives?: string[];
}

export interface VoiceCaptureProps {
  /** The words heard so far, in order. Add to the end as they arrive and each one fades in */
  words: Array<string | HeardWord>;
  /** listening and paused record, review asks the person to check the words, sent and cancelled are the end.
      Leave it out to let the buttons move it along */
  state?: VoiceCaptureState;
  defaultState?: VoiceCaptureState;
  onStateChange?: (state: VoiceCaptureState) => void;
  /** Seconds recorded. Leave it out to let it count while listening */
  seconds?: number;
  /** The input level from 0 to 1, drawn into the wave about 20 times a second. Pass a function to read the
      microphone on every sample (for example from an AnalyserNode) without re-rendering */
  level?: number | (() => number);
  /** The status line. Defaults to Listening, Paused, Check what I heard, Sent or Cancelled */
  status?: string;
  /** Called with the checked transcript when the person presses Done in review */
  onSend?: (text: string) => void;
  /** Called when the person picks one of a word's alternatives */
  onFix?: (index: number, text: string) => void;
  /** Called by Record again, once the capture is sent or cancelled. Clear your words and start listening again */
  onRecordAgain?: () => void;
  className?: string;
}

const STATUS: Record<VoiceCaptureState, string> = { listening: 'Listening', paused: 'Paused', review: 'Check what I heard', sent: 'Sent', cancelled: 'Cancelled' };
const clock = (s: number) => `${Math.floor(s / 60)}:${String(s % 60).padStart(2, '0')}`;

/** Speech to text that shows what it heard as it hears it, and lets the person fix the words it was unsure of. */
export function VoiceCapture({ words, state, defaultState = 'listening', onStateChange, seconds, level = 0, status, onSend, onFix, onRecordAgain, className }: VoiceCaptureProps) {
  const [now, setState] = useControllable(state, defaultState, onStateChange);
  const live = now === 'listening' || now === 'paused', ended = now === 'sent' || now === 'cancelled';
  const root = useRef<HTMLDivElement>(null);
  const wave = useRef<HTMLCanvasElement>(null);
  const pop = useRef<HTMLDivElement>(null);
  // once recording stops the box keeps the height it had while recording: the wave stays, still, and the words stay put
  const liveHeight = useRef<number | undefined>(undefined);
  const [held, setHeld] = useState<number>();
  useIsoLayoutEffect(() => { if (live && root.current) liveHeight.current = root.current.offsetHeight; });
  useIsoLayoutEffect(() => { setHeld(live ? undefined : liveHeight.current); }, [live]);

  // the clock counts while listening, unless the parent passes seconds
  const [count, setCount] = useState(0);
  useEffect(() => {
    if (seconds !== undefined || now !== 'listening') return;
    const id = setInterval(() => setCount(c => c + 1), 1000);
    return () => clearInterval(id);
  }, [seconds, now]);

  // the wave: one sample every 50ms, drawn as bars that scroll left while it records. Once recording stops it shows the
  // whole recording, still, in quieter bars, so the box keeps its height and every part stays where it was
  const samples = useRef<number[]>([]);
  const levelRef = useRef(level), stateRef = useRef(now);
  useEffect(() => { levelRef.current = level; stateRef.current = now; });
  useEffect(() => {
    const cv = wave.current, el = root.current; if (!cv || !el) return;
    // the colours are read a few times a second, not every frame: reading them makes the browser recheck the page's styles
    let cols: string[] | null = null, colsAt = -1e9;
    const colours = () => { const t = performance.now(); if (!cols || t - colsAt > 400) { const cs = getComputedStyle(el); cols = ['--dm-fg2', '--dm-faint', '--dm-muted'].map(k => cs.getPropertyValue(k).trim()); colsAt = t; } return cols; };
    const paint = (vals: number[], still: boolean) => {
      const dpr = devicePixelRatio || 1, w = cv.clientWidth, h = cv.clientHeight; if (!w) return;
      if (cv.width !== w * dpr) { cv.width = w * dpr; cv.height = h * dpr; }
      const x = cv.getContext('2d'); if (!x) return;
      x.setTransform(dpr, 0, 0, dpr, 0, 0); x.clearRect(0, 0, w, h);
      const [on, off, rec] = colours(), n = vals.length;
      for (let i = 0; i < n; i++) {
        const v = vals[i], px = i * 5 + 2, bh = v ? Math.max(4, v * (h - 6)) : 2;
        x.globalAlpha = still ? (v ? .6 : 1) : Math.min(1, i / 14, (n - i) / 3 + .2);
        x.fillStyle = v ? (still ? rec : on) : off;
        x.beginPath();
        if (x.roundRect) x.roundRect(px, h / 2 - bh / 2, 2.5, bh, 1.2); else x.rect(px, h / 2 - bh / 2, 2.5, bh);
        x.fill();
      }
    };
    const bars = () => Math.floor(cv.clientWidth / 5);
    if (!live) {
      // the recording without the silence either side, fitted to the width: each bar is the loudest moment it covers
      const still = () => {
        const s = samples.current, n = bars(); const a = s.findIndex(v => v); let z = s.length - 1; while (z > a && !s[z]) z--;
        const rec = a < 0 ? [] : s.slice(Math.max(0, a - 2), z + 3);
        paint(Array.from({ length: n }, (_, i) => { const from = Math.floor(i * rec.length / n), to = Math.max(from + 1, Math.floor((i + 1) * rec.length / n)); return Math.max(0, ...rec.slice(from, to).map(v => v || 0)); }), true);
      };
      // drawn again only when its width or the theme changes
      const again = () => { cols = null; still(); };
      still();
      const ro = new ResizeObserver(again); ro.observe(cv);
      const mo = new MutationObserver(again); mo.observe(document.documentElement, { attributes: true, attributeFilter: ['data-theme', 'data-style', 'class'] });
      const mq = matchMedia('(prefers-color-scheme: dark)'); mq.addEventListener('change', again);
      return () => { ro.disconnect(); mo.disconnect(); mq.removeEventListener('change', again); };
    }
    // the wave is drawn only while it is on screen
    let raf = 0, last = 0, seen = true;
    const io = 'IntersectionObserver' in window ? new IntersectionObserver(e => { seen = e[e.length - 1].isIntersecting; }) : null;
    io?.observe(cv);
    const draw = (t: number) => {
      raf = requestAnimationFrame(draw);
      if (!seen) return;
      if (t - last > 50) {
        last = t;
        const l = levelRef.current, v = stateRef.current === 'listening' ? (typeof l === 'function' ? l() : l) : 0;
        samples.current.push(Math.max(0, Math.min(1, v))); if (samples.current.length > 300) samples.current.shift();
      }
      const s = samples.current, n = bars();
      paint(Array.from({ length: n }, (_, i) => s[s.length - n + i] || 0), false);
    };
    raf = requestAnimationFrame(draw);
    return () => { cancelAnimationFrame(raf); io?.disconnect(); };
  }, [live]);

  // words it was unsure of open a small "Did you mean" menu under themselves
  const [fixes, setFixes] = useState<Record<number, string>>({});
  // the menu keeps its place and words while it fades out, so only `open` flips when it closes
  const [menu, setMenu] = useState<{ index: number; alts: string[]; style: CSSProperties }>({ index: -1, alts: [], style: {} });
  const [menuOpen, setMenuOpen] = useState(false);
  // the keyboard: Enter or Space on a word opens its menu with the focus on the first fix; the arrows move through the fixes
  // and wrap, Home and End jump, and Escape closes the menu back to the word
  const keyOpen = useRef(false), inMenu = useRef(false);
  const wordEl = (i: number) => root.current?.querySelector<HTMLElement>(`.dm-low[data-i="${i}"]`);
  useIsoLayoutEffect(() => {
    if (menuOpen) { if (keyOpen.current) { keyOpen.current = false; pop.current?.querySelector<HTMLElement>('.dm-mi')?.focus({ preventScroll: true }); } return; }
    if (inMenu.current) { inMenu.current = false; wordEl(menu.index)?.focus({ preventScroll: true }); }
  }, [menuOpen]); // eslint-disable-line react-hooks/exhaustive-deps
  const menuKey = (e: ReactKeyboardEvent) => {
    const all = Array.from(pop.current?.querySelectorAll<HTMLElement>('.dm-mi') ?? []), i = all.indexOf(document.activeElement as HTMLElement);
    const go = ({ ArrowDown: all[(i + 1) % all.length], ArrowUp: all[(i - 1 + all.length) % all.length], Home: all[0], End: all[all.length - 1] } as Record<string, HTMLElement | undefined>)[e.key];
    if (go) { e.preventDefault(); go.focus({ preventScroll: true }); }
    else if (e.key === 'Tab') { inMenu.current = false; wordEl(menu.index)?.focus({ preventScroll: true }); setMenuOpen(false); }
  };
  const show = (index: number, el: HTMLElement) => {
    const box = root.current; if (!box) return;
    const hr = box.getBoundingClientRect(), lr = el.getBoundingClientRect();
    setMenu({ index, alts: (words[index] as HeardWord).alternatives ?? [], style: { left: lr.left - hr.left, top: lr.bottom - hr.top + 6 } });
    setMenuOpen(true);
  };
  useEffect(() => {
    if (!menuOpen) return;
    // the click that opened the menu, or one on another unsure word, is not a click outside. Opening the menu
    // re-renders the word without its tip, so that click's target may already have left the page
    const click = (e: MouseEvent) => { const t = e.target as Element; if (t.isConnected && !pop.current?.contains(t) && !(root.current?.contains(t) && t.closest('.dm-low'))) setMenuOpen(false); };
    const key = (e: KeyboardEvent) => { if (e.key === 'Escape') setMenuOpen(false); };
    document.addEventListener('click', click); document.addEventListener('keydown', key);
    return () => { document.removeEventListener('click', click); document.removeEventListener('keydown', key); };
  }, [menuOpen]);
  const fix = (index: number, text: string) => { setFixes(f => ({ ...f, [index]: text })); onFix?.(index, text); setMenuOpen(false); };

  const text = () => words.map((w, i) => fixes[i] ?? (typeof w === 'string' ? w : w.text)).join(' ');
  const done = () => {
    if (live) setState('review');
    else if (now === 'review') { setState('sent'); onSend?.(text()); }
  };
  const again = () => { setCount(0); setFixes({}); setMenuOpen(false); samples.current = []; onRecordAgain?.(); setState('listening'); };

  return (
    <div data-dm="" ref={root} className={cx('dm-vcx dm-panel', now === 'paused' && 'dm-paused', !live && 'dm-done', now === 'cancelled' && 'dm-gone', held && 'dm-held', className)} style={held ? { minHeight: held } : undefined}>
      <div className="dm-vcx-top">
        <span className="dm-vcx-dot" />
        <Swap className="dm-vcx-st">{status ?? STATUS[now]}</Swap>
        <span className="dm-vcx-t dm-num dm-muted">{clock(seconds ?? count)}</span>
      </div>
      <canvas ref={wave} className="dm-vcx-wave" aria-hidden="true" />
      <p className="dm-vcx-tx" aria-live="polite">
        {words.map((w, i) => {
          const word = typeof w === 'string' ? { text: w } : w;
          // cancelled: the words stay, greyed and plain, so it is clear what was not sent
          if (!word.alternatives?.length || now === 'cancelled') return <span key={i} className="dm-w">{fixes[i] ?? word.text} </span>;
          const fixed = fixes[i] !== undefined;
          const low = (
            <span className={cx('dm-low', fixed && 'dm-fixed')} role="button" tabIndex={0} aria-haspopup="menu" aria-expanded={menuOpen && menu.index === i} data-i={i}
              onClick={e => show(i, e.currentTarget)} onKeyDown={e => { if (e.key === 'Enter' || e.key === ' ') { e.preventDefault(); keyOpen.current = true; show(i, e.currentTarget); } }}>
              {fixes[i] ?? word.text}
            </span>
          );
          // once recording stops, each unsure word says why it is marked; the tip steps aside while its menu is open
          return <span key={i} className="dm-w">{!live && !menuOpen ? <Tip label={fixed ? 'Fixed' : 'Not sure about this word'}>{low}</Tip> : low} </span>;
        })}
      </p>
      <div className="dm-vcx-acts">
        {ended ? (
          <>
            <span className="dm-muted" style={{ fontSize: 12.5 }}>{now === 'sent' ? 'Sent to the agent' : 'Nothing was sent'}</span>
            <span className="dm-sp" />
            <button type="button" className="dm-pill dm-ghost dm-sm dm-vcx-again" onClick={again}><Icon name="refresh" size="xs" />Record again</button>
          </>
        ) : (
          <>
            {live && (
              <Tip label={now === 'paused' ? 'Resume' : 'Pause'}>
                <button type="button" className="dm-ib dm-fill dm-vcx-pause" aria-label={now === 'paused' ? 'Resume' : 'Pause'} onClick={() => setState(now === 'paused' ? 'listening' : 'paused')}>
                  <Icon name={now === 'paused' ? 'mic' : 'pause'} />
                </button>
              </Tip>
            )}
            <span className="dm-sp" />
            <button type="button" className="dm-pill dm-ghost dm-sm dm-vcx-x" onClick={() => { setMenuOpen(false); setState('cancelled'); }}>Cancel</button>
            <button type="button" className="dm-pill dm-primary dm-sm dm-vcx-done" onClick={done}>Done</button>
          </>
        )}
      </div>
      <div ref={pop} className={cx('dm-pop dm-vcx-fix', menuOpen && 'dm-open')} role="menu" aria-label="Did you mean" style={menu.style} inert={!menuOpen} aria-hidden={!menuOpen}
        onKeyDown={menuKey} onFocus={() => { inMenu.current = true; }} onBlur={e => { if (!pop.current?.contains(e.relatedTarget as Node)) inMenu.current = false; }}>
        <div className="dm-ph">Did you mean</div>
        {menu.alts.map(a => <button type="button" key={a} role="menuitem" tabIndex={-1} className="dm-mi" onClick={() => fix(menu.index, a)}>{a}</button>)}
      </div>
    </div>
  );
}
voice-capture.cssReact · 26 lines
/* Generated from designme.dev by scripts/sync.mjs. Edit the site, then sync. */
/* ---------- Voice capture ---------- */
/* one column: the status line, the wave, the words and the buttons, a gap apart. Once recording stops the wave stays, still,
   as the whole recording, so the box keeps its height and every part stays where it was */
.dm-vcx{padding:var(--dm-s-4);position:relative;display:flex;flex-direction:column;gap:var(--dm-s-3)}
/* held (review, sent, cancelled): the parts stay at the top and the buttons at the foot, not centred in the height */
.dm-vcx.dm-held{justify-content:flex-start}
.dm-vcx.dm-held .dm-vcx-tx{flex:1}
.dm-vcx-top{display:flex;align-items:center;gap:var(--dm-s-2);font-size:13px}
.dm-vcx-dot{width:8px;height:8px;border-radius:50%;background:var(--dm-bad);animation:dm-vpulse 1.4s var(--dm-ease-io) infinite}
@keyframes dm-vpulse{0%,100%{box-shadow:0 0 0 0 color-mix(in srgb,var(--dm-bad) 35%,transparent)}50%{box-shadow:0 0 0 5px transparent}}
.dm-vcx.dm-paused .dm-vcx-dot,.dm-vcx.dm-done .dm-vcx-dot{animation:none;background:var(--dm-faint)}
.dm-vcx-t{margin-left:auto;font-size:12.5px}
.dm-vcx-wave{display:block;width:100%;height:48px}
/* room for two lines from the start, so the first words don't sit over an empty band and a short request never moves */
.dm-vcx-tx{margin:0;min-height:calc(2em * 1.6);font-size:14px;line-height:1.6;color:var(--dm-fg)}
.dm-vcx-tx .dm-w{animation:dm-lvw .45s var(--dm-ease) both}
@keyframes dm-lvw{from{opacity:0;filter:blur(3px)}to{opacity:1;filter:none}}
.dm-vcx-tx .dm-low{padding:1px 2px;margin:0 -2px;background:var(--dm-hl);-webkit-box-decoration-break:clone;box-decoration-break:clone;cursor:pointer;transition:background .25s}
@media (hover:hover) and (pointer:fine){.dm-vcx-tx .dm-low:hover{background:color-mix(in srgb,var(--dm-hl) 70%,var(--dm-py))}}
.dm-vcx-tx .dm-low.dm-fixed{background:var(--dm-hl-ok)}
/* cancelled: the words stay, greyed, so it is clear what was not sent */
.dm-vcx.dm-gone .dm-vcx-tx{color:var(--dm-muted)}
.dm-vcx-acts{display:flex;align-items:center;gap:var(--dm-s-2);min-height:32px}
.dm-vcx-fix{width:220px}
voice-capture.htmlHTML · 17 lines
<div class="card" data-c="voice-capture">
  <div class="stage-in">
    <div class="vcx panel">
      <div class="vcx-top"><span class="vcx-dot"></span><span class="vcx-st">Listening</span><span class="vcx-t num muted">0:00</span></div>
      <canvas class="vcx-wave" aria-hidden="true"></canvas>
      <p class="vcx-tx" aria-live="polite"></p>
      <div class="vcx-acts">
        <button class="ib fill vcx-pause" aria-label="Pause" data-tip="Pause"><svg class="i s"><use href="#i-pause"/></svg></button>
        <span class="sp"></span>
        <button class="pill ghost sm vcx-x">Cancel</button>
        <button class="pill primary sm vcx-done">Done</button>
      </div>
      <div class="pop vcx-fix" role="menu"></div>
    </div>
  </div>
</div>
voice-capture.cssHTML · 25 lines
/* ---------- Voice capture ---------- */
/* one column: the status line, the wave, the words and the buttons, a gap apart. Once recording stops the wave stays, still,
   as the whole recording, so the box keeps its height and every part stays where it was */
.vcx{padding:var(--s-4);position:relative;display:flex;flex-direction:column;gap:var(--s-3)}
/* held (review, sent, cancelled): the parts stay at the top and the buttons at the foot, not centred in the height */
.vcx.held{justify-content:flex-start}
.vcx.held .vcx-tx{flex:1}
.vcx-top{display:flex;align-items:center;gap:var(--s-2);font-size:13px}
.vcx-dot{width:8px;height:8px;border-radius:50%;background:var(--bad);animation:vpulse 1.4s var(--ease-io) infinite}
@keyframes vpulse{0%,100%{box-shadow:0 0 0 0 color-mix(in srgb,var(--bad) 35%,transparent)}50%{box-shadow:0 0 0 5px transparent}}
.vcx.paused .vcx-dot,.vcx.done .vcx-dot{animation:none;background:var(--faint)}
.vcx-t{margin-left:auto;font-size:12.5px}
.vcx-wave{display:block;width:100%;height:48px}
/* room for two lines from the start, so the first words don't sit over an empty band and a short request never moves */
.vcx-tx{margin:0;min-height:calc(2em * 1.6);font-size:14px;line-height:1.6;color:var(--fg)}
.vcx-tx .w{animation:lvw .45s var(--ease) both}
@keyframes lvw{from{opacity:0;filter:blur(3px)}to{opacity:1;filter:none}}
.vcx-tx .low{padding:1px 2px;margin:0 -2px;background:var(--hl);-webkit-box-decoration-break:clone;box-decoration-break:clone;cursor:pointer;transition:background .25s}
@media (hover:hover) and (pointer:fine){.vcx-tx .low:hover{background:color-mix(in srgb,var(--hl) 70%,var(--py))}}
.vcx-tx .low.fixed{background:var(--hl-ok)}
/* cancelled: the words stay, greyed, so it is clear what was not sent */
.vcx.gone .vcx-tx{color:var(--muted)}
.vcx-acts{display:flex;align-items:center;gap:var(--s-2);min-height:32px}
.vcx-fix{width:220px}
voice-capture.jsHTML · 71 lines
DM.def('voice-capture', (r, c) => {
  r.innerHTML = r.innerHTML.replace('{pause}', I('pause'));
  const box = $('.vcx', r), tx = $('.vcx-tx', r), cv = $('.vcx-wave', r), st = $('.vcx-st', r), pop = $('.vcx-fix', r);
  const words = 'Move the yearly toggle above the cards and make the save badge say 20% off for annual plans'.split(' ');
  const low = { 3:['toggle','toddle','tunnel'], 10:['save','sale','saving'] };
  let samples = [], speak = 0, last = 0, paused = false, secs = 0, done = false;
  c.every(1000, () => { if (!paused && !done){ secs++; $('.vcx-t', r).textContent = Math.floor(secs / 60) + ':' + String(secs % 60).padStart(2, '0'); } });
  // the wave scrolls while it records. Once recording stops it is the whole recording, still, in quieter bars, drawn again
  // only when its width or colours change. The colours are read a few times a second, not every frame
  let cols = null, colsAt = -1e9, drawn = '';
  c.loop(t => {
    if (!done && t - last > 50){ last = t; samples.push(!paused && speak > 0 ? .2 + Math.random() * .8 : 0); if (samples.length > 300) samples.shift(); if (speak > 0) speak--; }
    const dpr = devicePixelRatio || 1, w = cv.clientWidth, h = cv.clientHeight; if (!w) return;
    const now = performance.now(); if (!cols || now - colsAt > 400){ const cs = getComputedStyle(r); cols = ['--fg2', '--faint', '--muted'].map(k => cs.getPropertyValue(k).trim()); colsAt = now; }
    if (done){ const key = w + cols.join(); if (key === drawn) return; drawn = key; }
    if (cv.width !== w * dpr){ cv.width = w * dpr; cv.height = h * dpr; }
    const x = cv.getContext('2d'); x.setTransform(dpr, 0, 0, dpr, 0, 0); x.clearRect(0, 0, w, h);
    const n = Math.floor(w / 5);
    let vals;
    if (done){
      // the recording without the silence either side, fitted to the width: each bar is the loudest moment it covers
      let a = samples.findIndex(v => v), z = samples.length - 1; while (z > a && !samples[z]) z--;
      const rec = a < 0 ? [] : samples.slice(Math.max(0, a - 2), z + 3);
      vals = Array.from({ length:n }, (_, i) => { const from = Math.floor(i * rec.length / n), to = Math.max(from + 1, Math.floor((i + 1) * rec.length / n)); return Math.max(0, ...rec.slice(from, to).map(v => v || 0)); });
    } else vals = Array.from({ length:n }, (_, i) => samples[samples.length - n + i] || 0);
    for (let i = 0; i < n; i++){ const v = vals[i], px = i * 5 + 2; x.globalAlpha = done ? (v ? .6 : 1) : Math.min(1, i / 14, (n - i) / 3 + .2);
      x.fillStyle = v ? cols[done ? 2 : 0] : cols[1];
      const bh = v ? Math.max(4, v * (h - 6)) : 2; x.beginPath(); x.roundRect ? x.roundRect(px, h / 2 - bh / 2, 2.5, bh, 1.2) : x.rect(px, h / 2 - bh / 2, 2.5, bh); x.fill(); }
  });
  const pb = $('.vcx-pause', r);
  pb.onclick = () => { paused = !paused; box.classList.toggle('paused', paused); pb.innerHTML = I(paused ? 'mic' : 'pause'); pb.dataset.tip = paused ? 'Resume' : 'Pause'; swap(st, paused ? 'Paused' : 'Listening'); };
  function finish(){
    // the box keeps the height it had while recording: the wave stays, still, and the words stay where they were
    hold(box); done = true; box.classList.add('done'); swap(st, 'Check what I heard'); pb.remove();
    $$('.low', tx).forEach(l => { l.dataset.tip = 'Not sure about this word'; });
  }
  const settle = t => { $('.vcx-acts', r).innerHTML = `<span class="muted" style="font-size:12.5px">${t}</span><span class="sp"></span><button class="pill ghost sm vcx-again">${I('refresh','xs')}Record again</button>`; $('.vcx-again', r).onclick = () => DM.mount(r.closest('.card')); };
  $('.vcx-done', r).onclick = () => { if (!done) finish(); else { swap(st, 'Sent'); settle('Sent to the agent'); } };
  $('.vcx-x', r).onclick = () => { shutFix(); if (!done) hold(box); done = true; $$('.low', tx).forEach(l => l.replaceWith(l.textContent)); pb.remove(); box.classList.add('done', 'gone'); swap(st, 'Cancelled'); settle('Nothing was sent'); };
  // a word it isn't sure of opens a short menu of fixes: with a click, or with Enter or Space, which puts the focus on the
  // first fix. The arrows move through them and wrap, Home and End jump, and Escape closes the menu back to the word
  let fixFor = null;
  const fixes = () => $$('.mi', pop);
  const shutFix = () => { const back = pop.contains(document.activeElement) && fixFor; pop.classList.remove('open'); pop.inert = true; if (fixFor) fixFor.setAttribute('aria-expanded', 'false'); fixFor = null; if (back && back.isConnected) back.focus({ preventScroll:true }); };
  function openFix(l, key){
    const alts = low[l.dataset.i];
    pop.innerHTML = '<div class="ph">Did you mean</div>' + alts.map(a => `<button class="mi" role="menuitem" tabindex="-1" data-close data-w="${a}">${a}</button>`).join('');
    hideTip(); const hr = box.getBoundingClientRect(), lr = l.getBoundingClientRect();
    pop.style.left = (lr.left - hr.left) + 'px'; pop.style.top = (lr.bottom - hr.top + 6) + 'px'; pop.inert = false; pop.classList.add('open');
    if (fixFor && fixFor !== l) fixFor.setAttribute('aria-expanded', 'false'); fixFor = l; l.setAttribute('aria-expanded', 'true');
    fixes().forEach(b => b.onclick = ev => { ev.stopPropagation(); l.textContent = b.dataset.w; l.classList.add('fixed'); l.dataset.tip = 'Fixed'; shutFix(); });
    if (key) fixes()[0].focus({ preventScroll:true });
  }
  pop.inert = true;
  c.on(tx, 'click', e => { const l = e.target.closest('.low'); if (!l) return; openFix(l, e.isTrusted && !e.detail); e.stopPropagation(); });
  c.on(tx, 'keydown', e => { const l = e.target.closest('.low'); if (!l || (e.key !== 'Enter' && e.key !== ' ')) return; e.preventDefault(); openFix(l, true); });
  c.on(pop, 'keydown', e => { const all = fixes(), i = all.indexOf(document.activeElement), go = { ArrowDown:all[(i + 1) % all.length], ArrowUp:all[(i - 1 + all.length) % all.length], Home:all[0], End:all[all.length - 1] }[e.key];
    if (go){ e.preventDefault(); go.focus({ preventScroll:true }); } else if (e.key === 'Escape'){ e.preventDefault(); e.stopPropagation(); shutFix(); } else if (e.key === 'Tab'){ const l = fixFor; shutFix(); if (l) l.focus({ preventScroll:true }); } });
  c.on(document, 'click', e => { if (!pop.contains(e.target) && pop.classList.contains('open')) shutFix(); });
  c.run(async () => {
    await c.wait(700);
    for (let i = 0; i < words.length; i++){
      while (paused){ await c.wait(100); }
      if (done) return;
      speak = 5; tx.insertAdjacentHTML('beforeend', low[i] ? `<span class="w"><span class="low" data-i="${i}" role="button" tabindex="0" aria-haspopup="menu" aria-expanded="false">${words[i]}</span> </span>` : `<span class="w">${words[i]} </span>`);
      await c.wait(170 + Math.random() * 150);
    }
    await c.wait(900); if (!done) finish();
  });
});
06

Questions

What people ask about it.

Does the component use the microphone or do speech recognition?
No. You run the recording and transcription, add words to words as they arrive, and pass the input level as level. The component draws them and handles review.
How do I mark a word the speech model wasn't sure about?
Pass it as an object, like { text: 'toddle', alternatives: ['toggle', 'tunnel'] }. It gets a highlight, and after recording stops its tooltip says it is not sure about the word.
What text does onSend receive?
The heard words joined with spaces, with every word the person fixed replaced by their pick.
07

More in Conversation

Everything for asking, answering and replying, by text or by voice.

Live answerAnswers that stream in with numbered sources and next steps.
ComposerMessage input with attachments, what to work on, a mode and dictation.
Thread panelSwitch between agent threads and reply in place.
Artifact cardA file the agent made, with its versions, Open, Copy and Download.
Rich text editorA minimal editor with an inline AI menu that streams a suggestion to accept or discard.
Tool callEach tool the agent calls, with its input and output.
Voice orbA hands-free voice chat, with an orb that shows who is talking.
Answer action barCopy, regenerate, rate and listen to an answer from one row under it.
All 15 in Conversation →