Neural speech coding is a rapidly developing topic, where state-of-the-art approaches now exhibit superior compression performance than conventional methods. Despite significant progress, existing methods still have limitations in preserving and reconstructing fine details for optimal reconstruction, especially at low bitrates. In this study, we introduce SuperCodec, a neural speech codec that achieves state-of-the-art performance at low bitrates. It employs a novel back projection method with selective feature fusion for augmented representation. Specifically, we propose to use Selective Up-sampling Back Projection (SUBP) and Selective Down-sampling Back Projection (SDBP) modules to replace the standard up- and down-sampling layers at the encoder and decoder, respectively. Experimental results show that our method outperforms the existing neural speech codecs operating at various bitrates. Specifically, our proposed method can achieve higher quality reconstructed speech at 1 kbps than Lyra V2 at 3.2 kbps and Encodec at 6 kbps.
Use the sliders below to select bitrate from 1kbps to 3kbps operation.
Target | SuperCodec | Encodec(ours/pretrained) | Others |
---|---|---|---|
p226_267_mic2 | SuperCodec-1000bps SuperCodec-2000bps SuperCodec-3000bps SuperCodec-6000bps | Encodec-1000bps(ours) Encodec-2000bps(ours) Encodec-3000bps(ours) Encodec-6000bps(ours) Encodec-3000bps(pretrained) Encodec-6000bps(pretrained) | Lyra-v1-3kbps Lyra-v2-3.2kbps Lyra-v2-6kbps Opur-6kbps Opus-9kbps Speex-4kbps |
p233_024_mic2 | SuperCodec-1000bps SuperCodec-2000bps SuperCodec-3000bps SuperCodec-6000bps | Encodec-1000bps(ours) Encodec-2000bps(ours) Encodec-3000bps(ours) Encodec-6000bps(ours) Encodec-3000bps(pretrained) Encodec-6000bps(pretrained) | Lyra-v1-3kbps Lyra-v2-3.2kbps Lyra-v2-6kbps Opur-6kbps Opus-9kbps Speex-4kbps |
p236_185_mic2 | SuperCodec-1000bps SuperCodec-2000bps SuperCodec-3000bps SuperCodec-6000bps | Encodec-1000bps(ours) Encodec-2000bps(ours) Encodec-3000bps(ours) Encodec-6000bps(ours) Encodec-3000bps(pretrained) Encodec-6000bps(pretrained) | Lyra-v1-3kbps Lyra-v2-3.2kbps Lyra-v2-6kbps Opur-6kbps Opus-9kbps Speex-4kbps |
p262_006_mic2 | SuperCodec-1000bps SuperCodec-2000bps SuperCodec-3000bps SuperCodec-6000bps | Encodec-1000bps(ours) Encodec-2000bps(ours) Encodec-3000bps(ours) Encodec-6000bps(ours) Encodec-3000bps(pretrained) Encodec-6000bps(pretrained) | Lyra-v1-3kbps Lyra-v2-3.2kbps Lyra-v2-6kbps Opur-6kbps Opus-9kbps Speex-4kbps |
p360_363_mic2 | SuperCodec-1000bps SuperCodec-2000bps SuperCodec-3000bps SuperCodec-6000bps | Encodec-1000bps(ours) Encodec-2000bps(ours) Encodec-3000bps(ours) Encodec-6000bps(ours) Encodec-3000bps(pretrained) Encodec-6000bps(pretrained) | Lyra-v1-3kbps Lyra-v2-3.2kbps Lyra-v2-6kbps Opur-6kbps Opus-9kbps Speex-4kbps |
p362_019_mic2 | SuperCodec-1000bps SuperCodec-2000bps SuperCodec-3000bps SuperCodec-6000bps | Encodec-1000bps(ours) Encodec-2000bps(ours) Encodec-3000bps(ours) Encodec-6000bps(ours) Encodec-3000bps(pretrained) Encodec-6000bps(pretrained) | Lyra-v1-3kbps Lyra-v2-3.2kbps Lyra-v2-6kbps Opur-6kbps Opus-9kbps Speex-4kbps |
p374_163_mic2 | SuperCodec-1000bps SuperCodec-2000bps SuperCodec-3000bps SuperCodec-6000bps | Encodec-1000bps(ours) Encodec-2000bps(ours) Encodec-3000bps(ours) Encodec-6000bps(ours) Encodec-3000bps(pretrained) Encodec-6000bps(pretrained) | Lyra-v1-3kbps Lyra-v2-3.2kbps Lyra-v2-6kbps Opur-6kbps Opus-9kbps Speex-4kbps |
p376_007_mic2 | SuperCodec-1000bps SuperCodec-2000bps SuperCodec-3000bps SuperCodec-6000bps | Encodec-1000bps(ours) Encodec-2000bps(ours) Encodec-3000bps(ours) Encodec-6000bps(ours) Encodec-3000bps(pretrained) Encodec-6000bps(pretrained) | Lyra-v1-3kbps Lyra-v2-3.2kbps Lyra-v2-6kbps Opur-6kbps Opus-9kbps Speex-4kbps |