Credits
Audio Magic is a browser wrapper around other people’s research. The speech recognition, noise suppression and translation models below were built and published by the researchers and engineers named here, and released under licences that let anyone use them. This page is where they get the credit, in the citation format each project asks for.
Artwork
The mage — the mark in every masthead here, and the face of Audio Magic — was drawn by Gobby, an artist and musician from the Bronx.
Speech recognition
Transcription runs OpenAI’s Whisper, a transformer trained on 680,000 hours of multilingual audio.
[1]
Whisper
MITTranscribing speech to text and subtitles
Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust Speech Recognition via Large-Scale Weak Supervision. arXiv:2212.04356.+−BibTeX
@article{radford2022whisper, title = {Robust Speech Recognition via Large-Scale Weak Supervision}, author = {Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya}, journal = {arXiv preprint arXiv:2212.04356}, year = {2022} }
Speech synthesis
Generate Speech runs Kokoro, an 82-million-parameter voice model small enough to send to your browser instead of sending your text to a server. Clone a Voice runs Chatterbox, which copies a voice from a few seconds of it.
[2]
Chatterbox
MITCloning a voice from a short recording and speaking text in it
Resemble AI. 2025. Chatterbox-TTS. Model weights and code, released under the MIT licence.+−BibTeX
@misc{chatterboxtts2025, author = {{Resemble AI}}, title = {{Chatterbox-TTS}}, year = {2025}, howpublished = {\url{https://github.com/resemble-ai/chatterbox}}, note = {GitHub repository} }[3]
Kokoro
Apache-2.0Generating spoken audio from text
hexgrad. 2025. Kokoro-82M. Model weights, released under the Apache 2.0 licence.[4]
StyleTTS 2
MITThe synthesis architecture Kokoro is built on
Yinghao Aaron Li, Cong Han, Vinay S. Raghavan, Gavin Mischler, and Nima Mesgarani. 2023. StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models. In Advances in Neural Information Processing Systems (NeurIPS). arXiv:2306.07691.+−BibTeX
@inproceedings{li2023styletts2, title = {{StyleTTS} 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models}, author = {Li, Yinghao Aaron and Han, Cong and Raghavan, Vinay S. and Mischler, Gavin and Mesgarani, Nima}, booktitle = {Advances in Neural Information Processing Systems (NeurIPS)}, year = {2023} }[5]
iSTFTNet
Apache-2.0The vocoder that turns Kokoro’s output into a waveform
Takuhiro Kaneko, Kou Tanaka, Hirokazu Kameoka, and Shogo Seki. 2022. iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transform. In Proceedings of ICASSP. arXiv:2203.02395.+−BibTeX
@inproceedings{kaneko2022istftnet, title = {{iSTFTNet}: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transform}, author = {Kaneko, Takuhiro and Tanaka, Kou and Kameoka, Hirokazu and Seki, Shogo}, booktitle = {Proc. IEEE ICASSP 2022}, year = {2022} }[6]
phonemize
MITConverting text into the phonemes Kokoro reads, without a GPL dependency
hans00. 2025. phonemize: a pure-JavaScript rule-based grapheme-to-phoneme converter.
Speech enhancement
Both noise-removal engines pair a neural network with classical signal processing — which is what makes them fast enough to run on your own machine.
[7]
RNNoise
BSD-3-ClauseFast background noise removal
Jean-Marc Valin. 2018. A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement. In Proceedings of the IEEE International Workshop on Multimedia Signal Processing (MMSP). arXiv:1709.08243.+−BibTeX
@inproceedings{valin2018rnnoise, title = {A Hybrid {DSP}/Deep Learning Approach to Real-Time Full-Band Speech Enhancement}, author = {Valin, Jean-Marc}, booktitle = {Proceedings of the IEEE International Workshop on Multimedia Signal Processing (MMSP)}, year = {2018} }[8]
DeepFilterNet3
MIT or Apache-2.0Higher-quality background noise removal
Hendrik Schröter, Tobias Rosenkranz, Alberto N. Escalante-B., and Andreas Maier. 2023. DeepFilterNet: Perceptually Motivated Real-Time Speech Enhancement. In Proceedings of INTERSPEECH. arXiv:2305.08227.+−BibTeX
@inproceedings{schroeter2023deepfilternet3, title = {{DeepFilterNet}: Perceptually Motivated Real-Time Speech Enhancement}, author = {Schr{\"o}ter, Hendrik and Rosenkranz, Tobias and Escalante-B., Alberto N. and Maier, Andreas}, booktitle = {Proceedings of INTERSPEECH}, year = {2023} }[9]
DeepFilterNet
MIT or Apache-2.0The deep-filtering framework DeepFilterNet3 is built on
Hendrik Schröter, Alberto N. Escalante-B., Tobias Rosenkranz, and Andreas Maier. 2022. DeepFilterNet: A Low Complexity Speech Enhancement Framework for Full-Band Audio based on Deep Filtering. In Proceedings of ICASSP.+−BibTeX
@inproceedings{schroeter2022deepfilternet, title = {{DeepFilterNet}: A Low Complexity Speech Enhancement Framework for Full-Band Audio based on Deep Filtering}, author = {Schr{\"o}ter, Hendrik and Escalante-B., Alberto N. and Rosenkranz, Tobias and Maier, Andreas}, booktitle = {ICASSP 2022 - IEEE International Conference on Acoustics, Speech and Signal Processing}, year = {2022} }
Music source separation
Splitting a song back into vocals, drums, bass and everything else is done by Demucs — a hybrid model that works in both the spectrogram and the waveform domain.
[10]
HT-Demucs (Demucs v4)
MITSeparating a song into stems, and removing vocals
Simon Rouard, Francisco Massa, and Alexandre Défossez. 2023. Hybrid Transformers for Music Source Separation. In Proceedings of ICASSP. arXiv:2211.08553.+−BibTeX
@inproceedings{rouard2023hybrid, title = {Hybrid Transformers for Music Source Separation}, author = {Rouard, Simon and Massa, Francisco and D{\'e}fossez, Alexandre}, booktitle = {ICASSP 23}, year = {2023} }[11]
Hybrid Demucs
MITThe hybrid spectrogram/waveform design HT-Demucs builds on
Alexandre Défossez. 2021. Hybrid Spectrogram and Waveform Source Separation. In Proceedings of the ISMIR 2021 Workshop on Music Source Separation.+−BibTeX
@inproceedings{defossez2021hybrid, title = {Hybrid Spectrogram and Waveform Source Separation}, author = {D{\'e}fossez, Alexandre}, booktitle = {Proceedings of the ISMIR 2021 Workshop on Music Source Separation}, year = {2021} }
Sound event detection
Detect Sounds recognises a sound you teach it by standing on Google’s YAMNet — a network trained on AudioSet, two million human-labelled clips from YouTube — and learning only the last step from your own examples.
[12]
YAMNet
Apache-2.0Turning audio into the embeddings a custom sound detector is trained on
Shawn Hershey, Sourish Chaudhuri, Daniel P. W. Ellis, Jort F. Gemmeke, Aren Jansen, R. Channing Moore, Manoj Plakal, Devin Platt, Rif A. Saurous, Bryan Seybold, Malcolm Slaney, Ron J. Weiss, and Kevin Wilson. 2017. CNN Architectures for Large-Scale Audio Classification. In Proceedings of ICASSP. arXiv:1609.09430.+−BibTeX
@inproceedings{hershey2017cnn, title = {{CNN} Architectures for Large-Scale Audio Classification}, author = {Hershey, Shawn and Chaudhuri, Sourish and Ellis, Daniel P. W. and Gemmeke, Jort F. and Jansen, Aren and Moore, R. Channing and Plakal, Manoj and Platt, Devin and Saurous, Rif A. and Seybold, Bryan and Slaney, Malcolm and Weiss, Ron J. and Wilson, Kevin}, booktitle = {2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, year = {2017} }[13]
AudioSet
CC BY 4.0 (ontology and labels)The labelled corpus YAMNet learned what sounds are made of from
Jort F. Gemmeke, Daniel P. W. Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R. Channing Moore, Manoj Plakal, and Marvin Ritter. 2017. Audio Set: An ontology and human-labeled dataset for audio events. In Proceedings of ICASSP, New Orleans, LA.+−BibTeX
@inproceedings{gemmeke2017audioset, title = {Audio Set: An ontology and human-labeled dataset for audio events}, author = {Gemmeke, Jort F. and Ellis, Daniel P. W. and Freedman, Dylan and Jansen, Aren and Lawrence, Wade and Moore, R. Channing and Plakal, Manoj and Ritter, Marvin}, booktitle = {Proc. IEEE ICASSP 2017}, address = {New Orleans, LA}, year = {2017} }[14]
MobileNets
Apache-2.0The depthwise-separable convolution architecture YAMNet is built on
Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv:1704.04861.+−BibTeX
@article{howard2017mobilenets, title = {{MobileNets}: Efficient Convolutional Neural Networks for Mobile Vision Applications}, author = {Howard, Andrew G. and Zhu, Menglong and Chen, Bo and Kalenichenko, Dmitry and Wang, Weijun and Weyand, Tobias and Andreetto, Marco and Adam, Hartwig}, journal = {arXiv preprint arXiv:1704.04861}, year = {2017} }
Speaker diarization
Label Speakers is two networks and a clustering step: pyannote’s segmentation model finds when each voice is active in a ten-second window, and a speaker-embedding network turns those excerpts into vectors close enough to group. Neither the segmentation nor the identity comes from one model alone.
[15]
pyannote powerset segmentation
MITFinding when each person is speaking, overlap included
Alexis Plaquet and Hervé Bredin. 2023. Powerset multi-class cross entropy loss for neural speaker diarization. In Proceedings of INTERSPEECH 2023.+−BibTeX
@inproceedings{Plaquet23, author = {Plaquet, Alexis and Bredin, Herv{\'e}}, title = {{Powerset multi-class cross entropy loss for neural speaker diarization}}, booktitle = {Proc. INTERSPEECH 2023}, year = {2023} }[16]
pyannote.audio
MITThe diarization pipeline this tool reimplements in the browser
Hervé Bredin. 2023. pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe. In Proceedings of INTERSPEECH 2023.+−BibTeX
@inproceedings{Bredin23, author = {Bredin, Herv{\'e}}, title = {{pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe}}, booktitle = {Proc. INTERSPEECH 2023}, year = {2023} }[17]
WeSpeaker
Apache-2.0The ResNet34 speaker embedding model, one of the two voice comparisons offered
Hongji Wang, Chengdong Liang, Shuai Wang, Zhengyang Chen, Binbin Zhang, Xu Xiang, Yanlei Deng, and Yanmin Qian. 2023. Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkit. In Proceedings of ICASSP. arXiv:2210.17016.+−BibTeX
@inproceedings{wang2023wespeaker, title = {Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkit}, author = {Wang, Hongji and Liang, Chengdong and Wang, Shuai and Chen, Zhengyang and Zhang, Binbin and Xiang, Xu and Deng, Yanlei and Qian, Yanmin}, booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, year = {2023} }[18]
ERes2Net
Apache-2.0The other voice comparison model, the faster of the two
Yafeng Chen, Siqi Zheng, Hui Wang, Luyao Cheng, Qian Chen, and Jiajun Qi. 2023. An Enhanced Res2Net with Local and Global Feature Fusion for Speaker Verification. In Proceedings of INTERSPEECH 2023. arXiv:2305.12838.+−BibTeX
@inproceedings{chen2023eres2net, title = {An Enhanced {Res2Net} with Local and Global Feature Fusion for Speaker Verification}, author = {Chen, Yafeng and Zheng, Siqi and Wang, Hui and Cheng, Luyao and Chen, Qian and Qi, Jiajun}, booktitle = {Proc. INTERSPEECH 2023}, year = {2023} }[19]
3D-Speaker
Apache-2.0The toolkit ERes2Net’s released checkpoint was trained and published with
Yafeng Chen, Siqi Zheng, Hui Wang, Luyao Cheng, Tinglong Zhu, Rongjie Huang, Chong Deng, Qian Chen, Shiliang Zhang, Wen Wang, and Xihao Li. 2024. 3D-Speaker-Toolkit: An Open-Source Toolkit for Multimodal Speaker Verification and Diarization. arXiv:2403.19971.+−BibTeX
@article{chen20243dspeaker, title = {{3D-Speaker-Toolkit}: An Open-Source Toolkit for Multimodal Speaker Verification and Diarization}, author = {Chen, Yafeng and Zheng, Siqi and Wang, Hui and Cheng, Luyao and Zhu, Tinglong and Huang, Rongjie and Deng, Chong and Chen, Qian and Zhang, Shiliang and Wang, Wen and Li, Xihao}, journal = {arXiv preprint arXiv:2403.19971}, year = {2024} }[20]
VoxCeleb
CC BY 4.0The corpus both speaker embedding models learned to tell voices apart from
Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. 2017. VoxCeleb: a large-scale speaker identification dataset. In Proceedings of INTERSPEECH 2017. arXiv:1706.08612.+−BibTeX
@inproceedings{nagrani2017voxceleb, title = {{VoxCeleb}: a large-scale speaker identification dataset}, author = {Nagrani, Arsha and Chung, Joon Son and Zisserman, Andrew}, booktitle = {Proc. INTERSPEECH 2017}, year = {2017} }
Machine translation
Translated transcripts come from Helsinki-NLP’s OPUS-MT models, trained with the Marian toolkit.
[21]
OPUS-MT
Apache-2.0Translating transcripts into other languages
Jörg Tiedemann and Santhosh Thottingal. 2020. OPUS-MT – Building open translation services for the World. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation (EAMT), pages 479–480.+−BibTeX
@inproceedings{tiedemann-thottingal-2020-opus, title = {{OPUS-MT} {--} Building open translation services for the World}, author = {Tiedemann, J{\"o}rg and Thottingal, Santhosh}, booktitle = {Proceedings of the 22nd Annual Conference of the European Association for Machine Translation}, pages = {479--480}, year = {2020} }[22]
Marian NMT
MITThe toolkit the OPUS-MT models were trained with
Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, and Alexandra Birch. 2018. Marian: Fast Neural Machine Translation in C++. In Proceedings of ACL 2018, System Demonstrations, pages 116–121.+−BibTeX
@inproceedings{junczys-dowmunt-etal-2018-marian, title = {{M}arian: Fast Neural Machine Translation in {C}++}, author = {Junczys-Dowmunt, Marcin and Grundkiewicz, Roman and Dwojak, Tomasz and Hoang, Hieu and Heafield, Kenneth and Neckermann, Tom and Seide, Frank and Germann, Ulrich and Aji, Alham Fikri and Bogoychev, Nikolay and Martins, Andr{\'e} F. T. and Birch, Alexandra}, booktitle = {Proceedings of ACL 2018, System Demonstrations}, pages = {116--121}, year = {2018} }
Audio codecs
Browsers decode MP3 but will not encode it, so writing an MP3 takes someone else’s encoder.
[23]
LAME (via lamejs)
LGPL-3.0Encoding MP3 files in Convert Audio
The LAME project, https://lame.sourceforge.net. JavaScript port: lamejs by Alex Zhukov (@zhuker), a rewrite of jump3r-code, itself a rewrite of libmp3lame. Used unmodified as @breezystack/lamejs 1.2.7.
Colour
Make a Spectrogram is our own signal processing, but the colour ramps it maps loudness onto are somebody else’s research. Heat, Ice, Grayscale and anything built in the custom editor are not — they are ours, and are absent here for that reason.
[24]
Viridis, Magma, Inferno and Plasma
CC0-1.0 (public domain dedication)The perceptually uniform palettes in Make a Spectrogram
Nathaniel J. Smith and Stéfan van der Walt. 2015. A Better Default Colormap for Matplotlib. SciPy 2015. Viridis additionally credited to Eric Firing.[25]
Cividis
CC0-1.0 (public domain dedication)The colour-vision-deficiency palette in Make a Spectrogram
Jamie R. Nuñez, Christopher R. Anderton, and Ryan S. Renslow. 2018. Optimizing colormaps with consideration for color vision deficiency to enable accurate interpretation of scientific data. PLOS ONE 13(7): e0199239.[26]
Turbo
Apache-2.0The rainbow palette in Make a Spectrogram
Anton Mikhailov, Google LLC. 2019. Turbo, An Improved Rainbow Colormap for Visualization. Published as a 256-entry sRGB lookup table.[27]
Cubehelix
Free to use, citation requestedThe greyscale-safe palette in Make a Spectrogram
D. A. Green. 2011. A colour scheme for the display of astronomical intensity images. Bulletin of the Astronomical Society of India, 39, 289.
Running models in the browser
None of the above would run client-side without the work that compiled these runtimes to WebAssembly.
[28]
Transformers.js
Apache-2.0Running the Whisper and OPUS-MT models in a Web Worker
Joshua Lochner and the Hugging Face team. Transformers.js: State-of-the-art machine learning for the web.[29]
ONNX Runtime
MITExecuting the models via WebAssembly
ONNX Runtime developers. 2021. ONNX Runtime: Cross-platform accelerated machine learning.[30]
rnnoise-wasm
Apache-2.0The WebAssembly build of RNNoise we vendor
Tim Guggenmos (@timephy). rnnoise-wasm.[31]
demucs.cpp
MITThe C++/WebAssembly engine that runs HT-Demucs in Separate Stems
Sevag H (@sevagh). demucs.cpp — a C++17 inference library for the Demucs v3/v4 hybrid transformer music source separation models, built on Eigen and compiled to WebAssembly.
Typefaces
The site is set in Inter, Rasmus Andersson’s grotesque drawn for screen text at small sizes. Measured values — the figure’s axes and captions — stay in Computer Modern’s typewriter cut, which Donald Knuth designed with TeX and which holds a column of numbers in line because every character is the same width.
[32]
Inter
SIL OFL-1.1Every letterform on the site except the figure annotations
Rasmus Andersson. Inter — a typeface carefully crafted and designed for computer screens.[33]
Computer Modern Unicode
SIL OFL-1.1The monospaced figure annotations — axis chrome, captions, and measured values
Andrey V. Panov. Computer Modern Unicode fonts — Type 1 and OpenType conversions of Donald E. Knuth’s Computer Modern family, extended to Unicode coverage.
Corrections
Every citation above was checked against the primary source, but if your work is listed incorrectly — or is used here and missing entirely — that is a bug worth reporting, and it will be fixed.