How Carver works
Carver is a single continuous speech VAE with a lightweight router. Each latent frame is scored by its waveform reconstruction error; a top-k gate keeps the hardest frames and drops the rest, and the kept frames are repacked before decoding. A multi-rate masking curriculum trains one set of weights to operate across a continuous frame-rate–distortion curve, so the same model spans full rate down to aggressive compression. At inference, an auto mode picks a per-utterance rate that drops silence and redundant frames first while protecting unpredictable unvoiced sounds.
Reconstruction comparison
Same LibriSpeech test-clean utterances, scored with one pipeline for every
system.
Latent Rate = frame rate × latent dim (Carver full rate =
25 Hz × 64 = 1600), a direct measure of latent size.
auto uses the realized per-utterance keep rate.
| Method | Latent Rate (Hz×dim) | Listen |
|---|
Rate explorer
Drag the keep-rate slider and listen. The stacked panel shows, top to bottom, the waveform, the per-frame routing score (the router's reconstruction-difficulty signal), the log-mel spectrogram, and the MFA-aligned word and phone tiers. Pink bands mark the frames dropped at the current rate. Watch them fall on silence and steady vowel interiors first, while consonant onsets and fricatives are kept.
The phone tier is colored by voicing (silence / unvoiced / voiced, see the legend beneath the panel). Readouts (top-right): kept = fraction of frames retained (k/T); mel = multi-scale log-mel L1 error vs. ground truth (lower is better); SI-SDR = scale-invariant signal-to-distortion ratio in dB (higher is better).