ConversationVoice capture
Built by DesignMe, free to use under the MIT licence.
Expert design and dev support for startups and enterprises.
Book a callVoice capture
Talk to the agent, then fix what it misheard before sending. Free and MIT licensed, as HTML or a typed React component, in two styles with light and dark mode.
Install
Add it to your project with the CLI.
HTML, CSS and JavaScript
npx designme add voice-captureReact
npx designme add voice-capture --reactThe CLI copies the source into your project, so you own it and can change anything. The HTML version lands in ./designme with the shared core and a demo.html. The React version lands in src/components/designme and needs React 19; it works in Next.js, Vite and Astro.
About
What it is for and what it does.
Voice capture is the panel a person talks into when they speak a request to an agent. It shows a live waveform, a clock and the transcript as it is heard, highlights the words the speech model was unsure of, and asks the person to check them before the text is sent.
- When people speak tasks to your agent, on a phone or hands-free, and transcription mistakes would send the agent the wrong way.
- When your speech-to-text service returns alternatives for low-confidence words and you want people to pick the right one.
- When people need to pause, cancel or record again without losing track of what was sent.
- It moves through five states: listening, paused, review, sent and cancelled. The buttons move it along by themselves, or you control it with
stateandonStateChange. - Words passed in
wordsfade in as they arrive. A word given withalternativesis highlighted and opens a "Did you mean" menu, and picking one callsonFix. - Done first moves from recording to review, then a second Done sends the checked text, with fixes applied, to
onSend. - The waveform samples
levelabout every 50 milliseconds. Pass a function to read the microphone directly, and it records silence while paused. Once recording stops, the wave stays as the whole recording, still, so the panel keeps its height. - The clock counts up while listening unless you pass
seconds, and the status line defaults to Listening, Paused, Check what I heard, Sent or Cancelled. - Cancelling keeps the words on show, greyed. Once the capture is sent or cancelled, a Record again button clears the fixes and the clock and calls
onRecordAgain.
- The transcript is an
aria-live="polite"region, so new words are announced. - Unsure words are focusable
role="button"elements witharia-haspopup="menu", and Enter or Space opens their menu with the focus on the first fix. - The "Did you mean" menu uses
role="menu"andmenuitem, the arrow keys move through it and wrap, Home and End jump, and Escape or a click outside closes it (Escape gives the focus back to the word). - The Pause button is labelled Pause or Resume to match the state, and the waveform canvas is
aria-hidden.
Usage
Import it and drive it from your own state.
Import
import { VoiceCapture } from '@/components/designme/voice-capture/VoiceCapture';Example
The demo above as a React component, 46 lines you can copy.
'use client';
// The animated preview from designme.dev, rebuilt from the React component: an example of driving it from your own state.
import { useEffect, useRef, useState } from 'react';
import { VoiceCapture, type HeardWord, type VoiceCaptureState } from './VoiceCapture';
import { useScript } from '../lib/useScript';
/* ---------- Voice capture: a sentence is heard word by word, two words it was unsure of get a highlight ---------- */
const SENTENCE = 'Move the yearly toggle above the cards and make the save badge say 20% off for annual plans'.split(' ');
const UNSURE: Record<number, string[]> = { 3: ['toggle', 'toddle', 'tunnel'], 10: ['save', 'sale', 'saving'] };
export default function VoiceCaptureDemo() {
// Record again starts the whole take over, like replaying the card on the site
const [take, setTake] = useState(0);
return <Take key={take} onRecordAgain={() => setTake(t => t + 1)} />;
}
function Take({ onRecordAgain }: { onRecordAgain: () => void }) {
const [words, setWords] = useState<Array<string | HeardWord>>([]);
const [state, setState] = useState<VoiceCaptureState>('listening');
const now = useRef(state);
useEffect(() => { now.current = state; });
// a stand-in for the microphone: each word makes noise for the next five samples
const speak = useRef(0);
const level = () => { if (speak.current <= 0) return 0; speak.current--; return .2 + Math.random() * .8; };
useScript(async ({ wait }) => {
await wait(700);
for (let i = 0; i < SENTENCE.length; i++) {
while (now.current === 'paused') await wait(100);
if (now.current !== 'listening') return;
speak.current = 5;
const w = SENTENCE[i], alternatives = UNSURE[i];
setWords(ws => [...ws, alternatives ? { text: w, alternatives } : w]);
await wait(170 + Math.random() * 150);
}
await wait(900);
if (now.current === 'listening' || now.current === 'paused') setState('review');
});
return (
<div style={{ width: '100%' }}>
<VoiceCapture words={words} level={level} state={state} onStateChange={setState} onRecordAgain={onRecordAgain} />
</div>
);
}
Props
Everything VoiceCapture takes.
| Prop | Type | What it does |
|---|---|---|
wordsrequired | Array<string | HeardWord> | The words heard so far, in order. Add to the end as they arrive and each one fades in |
state | VoiceCaptureState | listening and paused record, review asks the person to check the words, sent and cancelled are the end. Leave it out to let the buttons move it along |
defaultState | VoiceCaptureState | |
onStateChange | (state: VoiceCaptureState) => void | |
seconds | number | Seconds recorded. Leave it out to let it count while listening |
level | number | (() => number) | The input level from 0 to 1, drawn into the wave about 20 times a second. Pass a function to read the microphone on every sample (for example from an AnalyserNode) without re-rendering |
status | string | The status line. Defaults to Listening, Paused, Check what I heard, Sent or Cancelled |
onSend | (text: string) => void | Called with the checked transcript when the person presses Done in review |
onFix | (index: number, text: string) => void | Called when the person picks one of a word's alternatives |
onRecordAgain | () => void | Called by Record again, once the capture is sent or cancelled. Clear your words and start listening again |
className | string |
Code
The full source, as the CLI installs it.
VoiceCapture.tsxReact · 224 lines
'use client';
import { useEffect, useRef, useState, type CSSProperties, type KeyboardEvent as ReactKeyboardEvent } from 'react';
import { cx } from '../lib/cx';
import { useControllable, useIsoLayoutEffect } from '../lib/hooks';
import { Icon } from '../lib/icons';
import { Swap, Tip } from '../lib/motion';
import '../designme.css';
import './voice-capture.css';
export type VoiceCaptureState = 'listening' | 'paused' | 'review' | 'sent' | 'cancelled';
/** A word the speech model was not sure about */
export interface HeardWord {
text: string;
/** What it might have been instead. Highlights the word and lets the person pick one */
alternatives?: string[];
}
export interface VoiceCaptureProps {
/** The words heard so far, in order. Add to the end as they arrive and each one fades in */
words: Array<string | HeardWord>;
/** listening and paused record, review asks the person to check the words, sent and cancelled are the end.
Leave it out to let the buttons move it along */
state?: VoiceCaptureState;
defaultState?: VoiceCaptureState;
onStateChange?: (state: VoiceCaptureState) => void;
/** Seconds recorded. Leave it out to let it count while listening */
seconds?: number;
/** The input level from 0 to 1, drawn into the wave about 20 times a second. Pass a function to read the
microphone on every sample (for example from an AnalyserNode) without re-rendering */
level?: number | (() => number);
/** The status line. Defaults to Listening, Paused, Check what I heard, Sent or Cancelled */
status?: string;
/** Called with the checked transcript when the person presses Done in review */
onSend?: (text: string) => void;
/** Called when the person picks one of a word's alternatives */
onFix?: (index: number, text: string) => void;
/** Called by Record again, once the capture is sent or cancelled. Clear your words and start listening again */
onRecordAgain?: () => void;
className?: string;
}
const STATUS: Record<VoiceCaptureState, string> = { listening: 'Listening', paused: 'Paused', review: 'Check what I heard', sent: 'Sent', cancelled: 'Cancelled' };
const clock = (s: number) => `${Math.floor(s / 60)}:${String(s % 60).padStart(2, '0')}`;
/** Speech to text that shows what it heard as it hears it, and lets the person fix the words it was unsure of. */
export function VoiceCapture({ words, state, defaultState = 'listening', onStateChange, seconds, level = 0, status, onSend, onFix, onRecordAgain, className }: VoiceCaptureProps) {
const [now, setState] = useControllable(state, defaultState, onStateChange);
const live = now === 'listening' || now === 'paused', ended = now === 'sent' || now === 'cancelled';
const root = useRef<HTMLDivElement>(null);
const wave = useRef<HTMLCanvasElement>(null);
const pop = useRef<HTMLDivElement>(null);
// once recording stops the box keeps the height it had while recording: the wave stays, still, and the words stay put
const liveHeight = useRef<number | undefined>(undefined);
const [held, setHeld] = useState<number>();
useIsoLayoutEffect(() => { if (live && root.current) liveHeight.current = root.current.offsetHeight; });
useIsoLayoutEffect(() => { setHeld(live ? undefined : liveHeight.current); }, [live]);
// the clock counts while listening, unless the parent passes seconds
const [count, setCount] = useState(0);
useEffect(() => {
if (seconds !== undefined || now !== 'listening') return;
const id = setInterval(() => setCount(c => c + 1), 1000);
return () => clearInterval(id);
}, [seconds, now]);
// the wave: one sample every 50ms, drawn as bars that scroll left while it records. Once recording stops it shows the
// whole recording, still, in quieter bars, so the box keeps its height and every part stays where it was
const samples = useRef<number[]>([]);
const levelRef = useRef(level), stateRef = useRef(now);
useEffect(() => { levelRef.current = level; stateRef.current = now; });
useEffect(() => {
const cv = wave.current, el = root.current; if (!cv || !el) return;
// the colours are read a few times a second, not every frame: reading them makes the browser recheck the page's styles
let cols: string[] | null = null, colsAt = -1e9;
const colours = () => { const t = performance.now(); if (!cols || t - colsAt > 400) { const cs = getComputedStyle(el); cols = ['--dm-fg2', '--dm-faint', '--dm-muted'].map(k => cs.getPropertyValue(k).trim()); colsAt = t; } return cols; };
const paint = (vals: number[], still: boolean) => {
const dpr = devicePixelRatio || 1, w = cv.clientWidth, h = cv.clientHeight; if (!w) return;
if (cv.width !== w * dpr) { cv.width = w * dpr; cv.height = h * dpr; }
const x = cv.getContext('2d'); if (!x) return;
x.setTransform(dpr, 0, 0, dpr, 0, 0); x.clearRect(0, 0, w, h);
const [on, off, rec] = colours(), n = vals.length;
for (let i = 0; i < n; i++) {
const v = vals[i], px = i * 5 + 2, bh = v ? Math.max(4, v * (h - 6)) : 2;
x.globalAlpha = still ? (v ? .6 : 1) : Math.min(1, i / 14, (n - i) / 3 + .2);
x.fillStyle = v ? (still ? rec : on) : off;
x.beginPath();
if (x.roundRect) x.roundRect(px, h / 2 - bh / 2, 2.5, bh, 1.2); else x.rect(px, h / 2 - bh / 2, 2.5, bh);
x.fill();
}
};
const bars = () => Math.floor(cv.clientWidth / 5);
if (!live) {
// the recording without the silence either side, fitted to the width: each bar is the loudest moment it covers
const still = () => {
const s = samples.current, n = bars(); const a = s.findIndex(v => v); let z = s.length - 1; while (z > a && !s[z]) z--;
const rec = a < 0 ? [] : s.slice(Math.max(0, a - 2), z + 3);
paint(Array.from({ length: n }, (_, i) => { const from = Math.floor(i * rec.length / n), to = Math.max(from + 1, Math.floor((i + 1) * rec.length / n)); return Math.max(0, ...rec.slice(from, to).map(v => v || 0)); }), true);
};
// drawn again only when its width or the theme changes
const again = () => { cols = null; still(); };
still();
const ro = new ResizeObserver(again); ro.observe(cv);
const mo = new MutationObserver(again); mo.observe(document.documentElement, { attributes: true, attributeFilter: ['data-theme', 'data-style', 'class'] });
const mq = matchMedia('(prefers-color-scheme: dark)'); mq.addEventListener('change', again);
return () => { ro.disconnect(); mo.disconnect(); mq.removeEventListener('change', again); };
}
// the wave is drawn only while it is on screen
let raf = 0, last = 0, seen = true;
const io = 'IntersectionObserver' in window ? new IntersectionObserver(e => { seen = e[e.length - 1].isIntersecting; }) : null;
io?.observe(cv);
const draw = (t: number) => {
raf = requestAnimationFrame(draw);
if (!seen) return;
if (t - last > 50) {
last = t;
const l = levelRef.current, v = stateRef.current === 'listening' ? (typeof l === 'function' ? l() : l) : 0;
samples.current.push(Math.max(0, Math.min(1, v))); if (samples.current.length > 300) samples.current.shift();
}
const s = samples.current, n = bars();
paint(Array.from({ length: n }, (_, i) => s[s.length - n + i] || 0), false);
};
raf = requestAnimationFrame(draw);
return () => { cancelAnimationFrame(raf); io?.disconnect(); };
}, [live]);
// words it was unsure of open a small "Did you mean" menu under themselves
const [fixes, setFixes] = useState<Record<number, string>>({});
// the menu keeps its place and words while it fades out, so only `open` flips when it closes
const [menu, setMenu] = useState<{ index: number; alts: string[]; style: CSSProperties }>({ index: -1, alts: [], style: {} });
const [menuOpen, setMenuOpen] = useState(false);
// the keyboard: Enter or Space on a word opens its menu with the focus on the first fix; the arrows move through the fixes
// and wrap, Home and End jump, and Escape closes the menu back to the word
const keyOpen = useRef(false), inMenu = useRef(false);
const wordEl = (i: number) => root.current?.querySelector<HTMLElement>(`.dm-low[data-i="${i}"]`);
useIsoLayoutEffect(() => {
if (menuOpen) { if (keyOpen.current) { keyOpen.current = false; pop.current?.querySelector<HTMLElement>('.dm-mi')?.focus({ preventScroll: true }); } return; }
if (inMenu.current) { inMenu.current = false; wordEl(menu.index)?.focus({ preventScroll: true }); }
}, [menuOpen]); // eslint-disable-line react-hooks/exhaustive-deps
const menuKey = (e: ReactKeyboardEvent) => {
const all = Array.from(pop.current?.querySelectorAll<HTMLElement>('.dm-mi') ?? []), i = all.indexOf(document.activeElement as HTMLElement);
const go = ({ ArrowDown: all[(i + 1) % all.length], ArrowUp: all[(i - 1 + all.length) % all.length], Home: all[0], End: all[all.length - 1] } as Record<string, HTMLElement | undefined>)[e.key];
if (go) { e.preventDefault(); go.focus({ preventScroll: true }); }
else if (e.key === 'Tab') { inMenu.current = false; wordEl(menu.index)?.focus({ preventScroll: true }); setMenuOpen(false); }
};
const show = (index: number, el: HTMLElement) => {
const box = root.current; if (!box) return;
const hr = box.getBoundingClientRect(), lr = el.getBoundingClientRect();
setMenu({ index, alts: (words[index] as HeardWord).alternatives ?? [], style: { left: lr.left - hr.left, top: lr.bottom - hr.top + 6 } });
setMenuOpen(true);
};
useEffect(() => {
if (!menuOpen) return;
// the click that opened the menu, or one on another unsure word, is not a click outside. Opening the menu
// re-renders the word without its tip, so that click's target may already have left the page
const click = (e: MouseEvent) => { const t = e.target as Element; if (t.isConnected && !pop.current?.contains(t) && !(root.current?.contains(t) && t.closest('.dm-low'))) setMenuOpen(false); };
const key = (e: KeyboardEvent) => { if (e.key === 'Escape') setMenuOpen(false); };
document.addEventListener('click', click); document.addEventListener('keydown', key);
return () => { document.removeEventListener('click', click); document.removeEventListener('keydown', key); };
}, [menuOpen]);
const fix = (index: number, text: string) => { setFixes(f => ({ ...f, [index]: text })); onFix?.(index, text); setMenuOpen(false); };
const text = () => words.map((w, i) => fixes[i] ?? (typeof w === 'string' ? w : w.text)).join(' ');
const done = () => {
if (live) setState('review');
else if (now === 'review') { setState('sent'); onSend?.(text()); }
};
const again = () => { setCount(0); setFixes({}); setMenuOpen(false); samples.current = []; onRecordAgain?.(); setState('listening'); };
return (
<div data-dm="" ref={root} className={cx('dm-vcx dm-panel', now === 'paused' && 'dm-paused', !live && 'dm-done', now === 'cancelled' && 'dm-gone', held && 'dm-held', className)} style={held ? { minHeight: held } : undefined}>
<div className="dm-vcx-top">
<span className="dm-vcx-dot" />
<Swap className="dm-vcx-st">{status ?? STATUS[now]}</Swap>
<span className="dm-vcx-t dm-num dm-muted">{clock(seconds ?? count)}</span>
</div>
<canvas ref={wave} className="dm-vcx-wave" aria-hidden="true" />
<p className="dm-vcx-tx" aria-live="polite">
{words.map((w, i) => {
const word = typeof w === 'string' ? { text: w } : w;
// cancelled: the words stay, greyed and plain, so it is clear what was not sent
if (!word.alternatives?.length || now === 'cancelled') return <span key={i} className="dm-w">{fixes[i] ?? word.text} </span>;
const fixed = fixes[i] !== undefined;
const low = (
<span className={cx('dm-low', fixed && 'dm-fixed')} role="button" tabIndex={0} aria-haspopup="menu" aria-expanded={menuOpen && menu.index === i} data-i={i}
onClick={e => show(i, e.currentTarget)} onKeyDown={e => { if (e.key === 'Enter' || e.key === ' ') { e.preventDefault(); keyOpen.current = true; show(i, e.currentTarget); } }}>
{fixes[i] ?? word.text}
</span>
);
// once recording stops, each unsure word says why it is marked; the tip steps aside while its menu is open
return <span key={i} className="dm-w">{!live && !menuOpen ? <Tip label={fixed ? 'Fixed' : 'Not sure about this word'}>{low}</Tip> : low} </span>;
})}
</p>
<div className="dm-vcx-acts">
{ended ? (
<>
<span className="dm-muted" style={{ fontSize: 12.5 }}>{now === 'sent' ? 'Sent to the agent' : 'Nothing was sent'}</span>
<span className="dm-sp" />
<button type="button" className="dm-pill dm-ghost dm-sm dm-vcx-again" onClick={again}><Icon name="refresh" size="xs" />Record again</button>
</>
) : (
<>
{live && (
<Tip label={now === 'paused' ? 'Resume' : 'Pause'}>
<button type="button" className="dm-ib dm-fill dm-vcx-pause" aria-label={now === 'paused' ? 'Resume' : 'Pause'} onClick={() => setState(now === 'paused' ? 'listening' : 'paused')}>
<Icon name={now === 'paused' ? 'mic' : 'pause'} />
</button>
</Tip>
)}
<span className="dm-sp" />
<button type="button" className="dm-pill dm-ghost dm-sm dm-vcx-x" onClick={() => { setMenuOpen(false); setState('cancelled'); }}>Cancel</button>
<button type="button" className="dm-pill dm-primary dm-sm dm-vcx-done" onClick={done}>Done</button>
</>
)}
</div>
<div ref={pop} className={cx('dm-pop dm-vcx-fix', menuOpen && 'dm-open')} role="menu" aria-label="Did you mean" style={menu.style} inert={!menuOpen} aria-hidden={!menuOpen}
onKeyDown={menuKey} onFocus={() => { inMenu.current = true; }} onBlur={e => { if (!pop.current?.contains(e.relatedTarget as Node)) inMenu.current = false; }}>
<div className="dm-ph">Did you mean</div>
{menu.alts.map(a => <button type="button" key={a} role="menuitem" tabIndex={-1} className="dm-mi" onClick={() => fix(menu.index, a)}>{a}</button>)}
</div>
</div>
);
}
voice-capture.cssReact · 26 lines
/* Generated from designme.dev by scripts/sync.mjs. Edit the site, then sync. */
/* ---------- Voice capture ---------- */
/* one column: the status line, the wave, the words and the buttons, a gap apart. Once recording stops the wave stays, still,
as the whole recording, so the box keeps its height and every part stays where it was */
.dm-vcx{padding:var(--dm-s-4);position:relative;display:flex;flex-direction:column;gap:var(--dm-s-3)}
/* held (review, sent, cancelled): the parts stay at the top and the buttons at the foot, not centred in the height */
.dm-vcx.dm-held{justify-content:flex-start}
.dm-vcx.dm-held .dm-vcx-tx{flex:1}
.dm-vcx-top{display:flex;align-items:center;gap:var(--dm-s-2);font-size:13px}
.dm-vcx-dot{width:8px;height:8px;border-radius:50%;background:var(--dm-bad);animation:dm-vpulse 1.4s var(--dm-ease-io) infinite}
@keyframes dm-vpulse{0%,100%{box-shadow:0 0 0 0 color-mix(in srgb,var(--dm-bad) 35%,transparent)}50%{box-shadow:0 0 0 5px transparent}}
.dm-vcx.dm-paused .dm-vcx-dot,.dm-vcx.dm-done .dm-vcx-dot{animation:none;background:var(--dm-faint)}
.dm-vcx-t{margin-left:auto;font-size:12.5px}
.dm-vcx-wave{display:block;width:100%;height:48px}
/* room for two lines from the start, so the first words don't sit over an empty band and a short request never moves */
.dm-vcx-tx{margin:0;min-height:calc(2em * 1.6);font-size:14px;line-height:1.6;color:var(--dm-fg)}
.dm-vcx-tx .dm-w{animation:dm-lvw .45s var(--dm-ease) both}
@keyframes dm-lvw{from{opacity:0;filter:blur(3px)}to{opacity:1;filter:none}}
.dm-vcx-tx .dm-low{padding:1px 2px;margin:0 -2px;background:var(--dm-hl);-webkit-box-decoration-break:clone;box-decoration-break:clone;cursor:pointer;transition:background .25s}
@media (hover:hover) and (pointer:fine){.dm-vcx-tx .dm-low:hover{background:color-mix(in srgb,var(--dm-hl) 70%,var(--dm-py))}}
.dm-vcx-tx .dm-low.dm-fixed{background:var(--dm-hl-ok)}
/* cancelled: the words stay, greyed, so it is clear what was not sent */
.dm-vcx.dm-gone .dm-vcx-tx{color:var(--dm-muted)}
.dm-vcx-acts{display:flex;align-items:center;gap:var(--dm-s-2);min-height:32px}
.dm-vcx-fix{width:220px}
voice-capture.htmlHTML · 17 lines
<div class="card" data-c="voice-capture">
<div class="stage-in">
<div class="vcx panel">
<div class="vcx-top"><span class="vcx-dot"></span><span class="vcx-st">Listening</span><span class="vcx-t num muted">0:00</span></div>
<canvas class="vcx-wave" aria-hidden="true"></canvas>
<p class="vcx-tx" aria-live="polite"></p>
<div class="vcx-acts">
<button class="ib fill vcx-pause" aria-label="Pause" data-tip="Pause"><svg class="i s"><use href="#i-pause"/></svg></button>
<span class="sp"></span>
<button class="pill ghost sm vcx-x">Cancel</button>
<button class="pill primary sm vcx-done">Done</button>
</div>
<div class="pop vcx-fix" role="menu"></div>
</div>
</div>
</div>
voice-capture.cssHTML · 25 lines
/* ---------- Voice capture ---------- */
/* one column: the status line, the wave, the words and the buttons, a gap apart. Once recording stops the wave stays, still,
as the whole recording, so the box keeps its height and every part stays where it was */
.vcx{padding:var(--s-4);position:relative;display:flex;flex-direction:column;gap:var(--s-3)}
/* held (review, sent, cancelled): the parts stay at the top and the buttons at the foot, not centred in the height */
.vcx.held{justify-content:flex-start}
.vcx.held .vcx-tx{flex:1}
.vcx-top{display:flex;align-items:center;gap:var(--s-2);font-size:13px}
.vcx-dot{width:8px;height:8px;border-radius:50%;background:var(--bad);animation:vpulse 1.4s var(--ease-io) infinite}
@keyframes vpulse{0%,100%{box-shadow:0 0 0 0 color-mix(in srgb,var(--bad) 35%,transparent)}50%{box-shadow:0 0 0 5px transparent}}
.vcx.paused .vcx-dot,.vcx.done .vcx-dot{animation:none;background:var(--faint)}
.vcx-t{margin-left:auto;font-size:12.5px}
.vcx-wave{display:block;width:100%;height:48px}
/* room for two lines from the start, so the first words don't sit over an empty band and a short request never moves */
.vcx-tx{margin:0;min-height:calc(2em * 1.6);font-size:14px;line-height:1.6;color:var(--fg)}
.vcx-tx .w{animation:lvw .45s var(--ease) both}
@keyframes lvw{from{opacity:0;filter:blur(3px)}to{opacity:1;filter:none}}
.vcx-tx .low{padding:1px 2px;margin:0 -2px;background:var(--hl);-webkit-box-decoration-break:clone;box-decoration-break:clone;cursor:pointer;transition:background .25s}
@media (hover:hover) and (pointer:fine){.vcx-tx .low:hover{background:color-mix(in srgb,var(--hl) 70%,var(--py))}}
.vcx-tx .low.fixed{background:var(--hl-ok)}
/* cancelled: the words stay, greyed, so it is clear what was not sent */
.vcx.gone .vcx-tx{color:var(--muted)}
.vcx-acts{display:flex;align-items:center;gap:var(--s-2);min-height:32px}
.vcx-fix{width:220px}
voice-capture.jsHTML · 71 lines
DM.def('voice-capture', (r, c) => {
r.innerHTML = r.innerHTML.replace('{pause}', I('pause'));
const box = $('.vcx', r), tx = $('.vcx-tx', r), cv = $('.vcx-wave', r), st = $('.vcx-st', r), pop = $('.vcx-fix', r);
const words = 'Move the yearly toggle above the cards and make the save badge say 20% off for annual plans'.split(' ');
const low = { 3:['toggle','toddle','tunnel'], 10:['save','sale','saving'] };
let samples = [], speak = 0, last = 0, paused = false, secs = 0, done = false;
c.every(1000, () => { if (!paused && !done){ secs++; $('.vcx-t', r).textContent = Math.floor(secs / 60) + ':' + String(secs % 60).padStart(2, '0'); } });
// the wave scrolls while it records. Once recording stops it is the whole recording, still, in quieter bars, drawn again
// only when its width or colours change. The colours are read a few times a second, not every frame
let cols = null, colsAt = -1e9, drawn = '';
c.loop(t => {
if (!done && t - last > 50){ last = t; samples.push(!paused && speak > 0 ? .2 + Math.random() * .8 : 0); if (samples.length > 300) samples.shift(); if (speak > 0) speak--; }
const dpr = devicePixelRatio || 1, w = cv.clientWidth, h = cv.clientHeight; if (!w) return;
const now = performance.now(); if (!cols || now - colsAt > 400){ const cs = getComputedStyle(r); cols = ['--fg2', '--faint', '--muted'].map(k => cs.getPropertyValue(k).trim()); colsAt = now; }
if (done){ const key = w + cols.join(); if (key === drawn) return; drawn = key; }
if (cv.width !== w * dpr){ cv.width = w * dpr; cv.height = h * dpr; }
const x = cv.getContext('2d'); x.setTransform(dpr, 0, 0, dpr, 0, 0); x.clearRect(0, 0, w, h);
const n = Math.floor(w / 5);
let vals;
if (done){
// the recording without the silence either side, fitted to the width: each bar is the loudest moment it covers
let a = samples.findIndex(v => v), z = samples.length - 1; while (z > a && !samples[z]) z--;
const rec = a < 0 ? [] : samples.slice(Math.max(0, a - 2), z + 3);
vals = Array.from({ length:n }, (_, i) => { const from = Math.floor(i * rec.length / n), to = Math.max(from + 1, Math.floor((i + 1) * rec.length / n)); return Math.max(0, ...rec.slice(from, to).map(v => v || 0)); });
} else vals = Array.from({ length:n }, (_, i) => samples[samples.length - n + i] || 0);
for (let i = 0; i < n; i++){ const v = vals[i], px = i * 5 + 2; x.globalAlpha = done ? (v ? .6 : 1) : Math.min(1, i / 14, (n - i) / 3 + .2);
x.fillStyle = v ? cols[done ? 2 : 0] : cols[1];
const bh = v ? Math.max(4, v * (h - 6)) : 2; x.beginPath(); x.roundRect ? x.roundRect(px, h / 2 - bh / 2, 2.5, bh, 1.2) : x.rect(px, h / 2 - bh / 2, 2.5, bh); x.fill(); }
});
const pb = $('.vcx-pause', r);
pb.onclick = () => { paused = !paused; box.classList.toggle('paused', paused); pb.innerHTML = I(paused ? 'mic' : 'pause'); pb.dataset.tip = paused ? 'Resume' : 'Pause'; swap(st, paused ? 'Paused' : 'Listening'); };
function finish(){
// the box keeps the height it had while recording: the wave stays, still, and the words stay where they were
hold(box); done = true; box.classList.add('done'); swap(st, 'Check what I heard'); pb.remove();
$$('.low', tx).forEach(l => { l.dataset.tip = 'Not sure about this word'; });
}
const settle = t => { $('.vcx-acts', r).innerHTML = `<span class="muted" style="font-size:12.5px">${t}</span><span class="sp"></span><button class="pill ghost sm vcx-again">${I('refresh','xs')}Record again</button>`; $('.vcx-again', r).onclick = () => DM.mount(r.closest('.card')); };
$('.vcx-done', r).onclick = () => { if (!done) finish(); else { swap(st, 'Sent'); settle('Sent to the agent'); } };
$('.vcx-x', r).onclick = () => { shutFix(); if (!done) hold(box); done = true; $$('.low', tx).forEach(l => l.replaceWith(l.textContent)); pb.remove(); box.classList.add('done', 'gone'); swap(st, 'Cancelled'); settle('Nothing was sent'); };
// a word it isn't sure of opens a short menu of fixes: with a click, or with Enter or Space, which puts the focus on the
// first fix. The arrows move through them and wrap, Home and End jump, and Escape closes the menu back to the word
let fixFor = null;
const fixes = () => $$('.mi', pop);
const shutFix = () => { const back = pop.contains(document.activeElement) && fixFor; pop.classList.remove('open'); pop.inert = true; if (fixFor) fixFor.setAttribute('aria-expanded', 'false'); fixFor = null; if (back && back.isConnected) back.focus({ preventScroll:true }); };
function openFix(l, key){
const alts = low[l.dataset.i];
pop.innerHTML = '<div class="ph">Did you mean</div>' + alts.map(a => `<button class="mi" role="menuitem" tabindex="-1" data-close data-w="${a}">${a}</button>`).join('');
hideTip(); const hr = box.getBoundingClientRect(), lr = l.getBoundingClientRect();
pop.style.left = (lr.left - hr.left) + 'px'; pop.style.top = (lr.bottom - hr.top + 6) + 'px'; pop.inert = false; pop.classList.add('open');
if (fixFor && fixFor !== l) fixFor.setAttribute('aria-expanded', 'false'); fixFor = l; l.setAttribute('aria-expanded', 'true');
fixes().forEach(b => b.onclick = ev => { ev.stopPropagation(); l.textContent = b.dataset.w; l.classList.add('fixed'); l.dataset.tip = 'Fixed'; shutFix(); });
if (key) fixes()[0].focus({ preventScroll:true });
}
pop.inert = true;
c.on(tx, 'click', e => { const l = e.target.closest('.low'); if (!l) return; openFix(l, e.isTrusted && !e.detail); e.stopPropagation(); });
c.on(tx, 'keydown', e => { const l = e.target.closest('.low'); if (!l || (e.key !== 'Enter' && e.key !== ' ')) return; e.preventDefault(); openFix(l, true); });
c.on(pop, 'keydown', e => { const all = fixes(), i = all.indexOf(document.activeElement), go = { ArrowDown:all[(i + 1) % all.length], ArrowUp:all[(i - 1 + all.length) % all.length], Home:all[0], End:all[all.length - 1] }[e.key];
if (go){ e.preventDefault(); go.focus({ preventScroll:true }); } else if (e.key === 'Escape'){ e.preventDefault(); e.stopPropagation(); shutFix(); } else if (e.key === 'Tab'){ const l = fixFor; shutFix(); if (l) l.focus({ preventScroll:true }); } });
c.on(document, 'click', e => { if (!pop.contains(e.target) && pop.classList.contains('open')) shutFix(); });
c.run(async () => {
await c.wait(700);
for (let i = 0; i < words.length; i++){
while (paused){ await c.wait(100); }
if (done) return;
speak = 5; tx.insertAdjacentHTML('beforeend', low[i] ? `<span class="w"><span class="low" data-i="${i}" role="button" tabindex="0" aria-haspopup="menu" aria-expanded="false">${words[i]}</span> </span>` : `<span class="w">${words[i]} </span>`);
await c.wait(170 + Math.random() * 150);
}
await c.wait(900); if (!done) finish();
});
});
Questions
What people ask about it.
- Does the component use the microphone or do speech recognition?
- No. You run the recording and transcription, add words to
wordsas they arrive, and pass the input level aslevel. The component draws them and handles review. - How do I mark a word the speech model wasn't sure about?
- Pass it as an object, like
{ text: 'toddle', alternatives: ['toggle', 'tunnel'] }. It gets a highlight, and after recording stops its tooltip says it is not sure about the word. - What text does
onSendreceive? - The heard words joined with spaces, with every word the person fixed replaced by their pick.
More in Conversation
Everything for asking, answering and replying, by text or by voice.