Chun-Yi Kuan
"Silence, I discover, is something you can actually hear." — Kafka on the Shore
Hello! I am a Ph.D. student at the National Taiwan University (NTU), where I am a member of the Speech Processing and Machine Learning (SPML) Lab advised by Prof. Hung-yi Lee. My research focuses on building trustworthy audio-aware large language models, with an emphasis on hallucination, abstention, and robust audio-language alignment. I am also interested in controllable audio generation, including instruction-guided text-to-audio and text-to-speech systems.
News
- 2026.06 I made a little book game — Guess My Bookshelf.
- 2026.06 Excited to share that our paper, Improving Text-to-Audio Instruction Following via Fine-Grained Feedback from Audio-Aware Large Language Models, has been accepted as a long paper at INTERSPEECH 2026 🇦🇺. See you in Sydney, Australia! 🐨
Show older news (1)
- 2025.05 Our paper Teaching Audio-Aware Large Language Models What Does Not Hear: Mitigating Hallucinations through Synthesized Negative Samples was accepted to Interspeech 2025 🇳🇱.
Selected Publications
-
Improving Text-to-Audio Instruction Following via Fine-Grained Feedback from Audio-Aware Large Language Models
TL;DRUses fine-grained feedback from audio-aware LLMs to make text-to-audio models follow instructions more faithfully.
@inproceedings{kuan2026improving, title = {Improving Text-to-Audio Instruction Following via Fine-Grained Feedback from Audio-Aware Large Language Models}, author = {Kuan, Chun-Yi and Kim, Siwon and Kim, Byeonggeun and Kim, Suyoun and Lu, Bo-Ru and Tang, Qingming and Gandhe, Ankur and Lee, Hung-yi and Kao, Chieh-Chi and Wang, Chao}, booktitle = {Interspeech 2026}, year = {2026}, url = {https://arxiv.org/abs/2607.13408}, } -
AQAScore: Evaluating Semantic Alignment in Text-to-Audio Generation via Audio Question Answering
TL;DRScores text-to-audio alignment from an audio-aware LLM's confidence in answering 'Yes' to targeted questions, catching fine-grained mismatches that similarity metrics like CLAPScore miss.
@article{kuan2026aqascore, title = {AQAScore: Evaluating Semantic Alignment in Text-to-Audio Generation via Audio Question Answering}, author = {Kuan, Chun-Yi and Chang, Kai-Wei and Lee, Hung-yi}, journal = {arXiv preprint arXiv:2601.14728}, year = {2026}, url = {https://arxiv.org/abs/2601.14728}, } -
Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Models
TL;DRThe systematic study of uncertainty estimation for audio-aware LLMs, finding that semantic and verification-based methods win on general reasoning but their advantage breaks down on hallucination and unanswerable-question benchmarks.
@article{kuan2026walking, title = {Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Models}, author = {Kuan, Chun-Yi and Huang, Wei-Ping and Lee, Hung-yi}, journal = {arXiv preprint arXiv:2604.25591}, year = {2026}, url = {https://arxiv.org/abs/2604.25591}, } -
From Alignment to Advancement: Bootstrapping Audio-Language Alignment with Synthetic Data
TL;DRBootstraps audio–language alignment with synthetic data to push audio-aware LLMs from basic alignment toward stronger reasoning.
@article{kuan2025alignment, title = {From Alignment to Advancement: Bootstrapping Audio-Language Alignment with Synthetic Data}, author = {Kuan, Chun-Yi and Lee, Hung-yi}, journal = {IEEE Transactions on Audio, Speech and Language Processing}, year = {2025}, volume = {33}, pages = {4604--4619}, doi = {10.1109/TASLPRO.2025.3626233}, url = {https://arxiv.org/abs/2505.20166}, } -
Teaching Audio-Aware Large Language Models What Does Not Hear: Mitigating Hallucinations through Synthesized Negative Samples
TL;DRCurbs hallucinations in audio-aware LLMs by teaching them what is NOT in the audio using synthesized negative samples.
@inproceedings{kuan2025teaching, title = {Teaching Audio-Aware Large Language Models What Does Not Hear: Mitigating Hallucinations through Synthesized Negative Samples}, author = {Kuan, Chun-Yi and Lee, Hung-yi}, booktitle = {Interspeech 2025}, pages = {2073--2077}, year = {2025}, organization = {ISCA}, doi = {10.21437/Interspeech.2025-324}, url = {https://arxiv.org/abs/2505.14518}, } -
Gender Bias in Instruction-Guided Speech Synthesis Models
TL;DRAudits and quantifies gender bias in instruction-guided speech synthesis models.
@inproceedings{kuan2025gender, title = {Gender Bias in Instruction-Guided Speech Synthesis Models}, author = {Kuan, Chun-Yi and Lee, Hung-yi}, booktitle = {Findings of the Association for Computational Linguistics: NAACL 2025}, pages = {5402--5428}, year = {2025}, publisher = {Association for Computational Linguistics}, doi = {10.18653/v1/2025.findings-naacl.298}, url = {https://aclanthology.org/2025.findings-naacl.298/}, } -
Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoning
TL;DRProbes whether audio-LLMs truly 'hear' via multi-task assessment and stepwise audio reasoning to reduce hallucinations.
@inproceedings{kuan2025can, title = {Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoning}, author = {Kuan, Chun-Yi and Lee, Hung-yi}, booktitle = {ICASSP 2025 -- 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, pages = {1--5}, year = {2025}, organization = {IEEE}, doi = {10.1109/ICASSP49660.2025.10888384}, url = {https://arxiv.org/abs/2410.16130}, } -
Towards General-Purpose Text-Instruction-Guided Voice Conversion
TL;DRA first step toward general-purpose voice conversion controlled by free-form text instructions.
@inproceedings{kuan2023towards, title = {Towards General-Purpose Text-Instruction-Guided Voice Conversion}, author = {Kuan, Chun-Yi and Li, Chen-An and Hsu, Tsu-Yuan and Lin, Tse-Yang and Chung, Ho-Lam and Chang, Kai-Wei and Chang, Shuo-Yiin and Lee, Hung-yi}, booktitle = {2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)}, pages = {1--8}, year = {2023}, organization = {IEEE}, doi = {10.1109/ASRU57964.2023.10389672}, url = {https://arxiv.org/abs/2309.14324}, }
Footprints
Conferences I attended in person · a fading paw is where I’m headed next.