Accepted at Interspeech 2026 🐨 · Long Paper

Improving Text-to-Audio Instruction Following via Fine-Grained Feedback from Audio-Aware Large Language Models

Chun-Yi Kuan♣,1, Siwon Kim2, Byeonggeun Kim2, Suyoun Kim2, Bo-Ru Lu2, Qingming Tang2, Ankur Gandhe2, Hung-yi Lee1, Chieh-Chi Kao2, Chao Wang2
1National Taiwan University, Taiwan · 2Amazon, USA
♣Work done during internship at Amazon  ·  chunyi.kuan.tw@gmail.com
Paper ↗

Abstract

Recent text-to-audio models generate high-quality audio, but often fail to follow realistic instructions involving multiple sound events and explicit temporal order. This gap stems from evaluation and training signals that emphasize global similarity or perceptual quality, with little supervision on event completeness and ordering. We propose an instruction-level evaluation and training framework that uses audio-aware large language models (ALLMs) as judges to explicitly verify target event presence and temporal relations in generated audio. We show modern ALLMs can reliably perform these judgments on multiple audio understanding benchmarks, and use their fine-grained feedback to construct preference pairs for direct preference optimization. To evaluate temporally structured generation, we introduce Sound Scene Story Benchmark (S3Bench), a narrative benchmark with multi-event temporal progression. Experiments consistently improve sound event completeness and temporal ordering across existing benchmarks and S3Bench.

Overview of the proposed framework: a frozen reference text-to-audio model generates candidate audio; an audio-aware LLM judge verifies sound event existence and temporal order to construct preference data; the text-to-audio model is then fine-tuned for better instruction following.
Overview of the proposed framework. Click the figure to enlarge.

Highlights 🐾

  • Gap: Modern text-to-audio models generate high-quality audio, yet often miss sound events or scramble their temporal order under multi-event instructions.
  • Verify: Audio-aware LLMs (ALLMs) serve as judges, performing instruction-level verification of sound event presence and temporal relations.
  • Optimize: The judges’ fine-grained feedback constructs preference pairs for Direct Preference Optimization (RLAIF-style training).
  • Benchmark: Sound Scene Story Benchmark (S3Bench), with narrative instructions and multi-event temporal progression.
  • Result: Consistent gains in event completeness and temporal ordering on existing benchmarks and S3Bench, with no trade-off in audio quality.

Below, we present:
(1) Side-by-side comparisons of generated audio samples across multiple benchmarks, highlighting how our method improves sound event completeness and temporal ordering compared to baseline systems;
(2) audio samples across multiple DPO iterations, demonstrating the progressive performance gains achieved through iterative preference optimization; and
(3) model-based evaluation using Meta AudioBox Aesthetics. Our model maintains comparable production quality while achieving stronger instruction-following ability, indicating no trade-off between instruction following and audio quality.

The audio samples and results shown here were independently reproduced and regenerated by the author. The work itself builds on publicly available models, datasets, and open-source components (see the full disclaimer ↓).

Note: The audio examples below are independently regenerated using public/open-source components and are not outputs from the original internship experiments. Read the full disclaimer ↓

Loading benchmarks data...

References

    1. H. Liu, Y. Yuan, X. Liu, X. Mei, Q. Kong, Q. Tian, Y. Wang, W. Wang, Y. Wang, and M. D. Plumbley, “AudioLDM 2: Learning Holistic Audio Generation With Self-Supervised Pretraining,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, pp. 2871–2883, 2024.
    2. Z. Evans, J. D. Parker, C. J. Carr, Z. Zukowski, J. Taylor, and J. Pons, “Stable Audio Open,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025, pp. 1–5.
    3. C.-Y. Hung, N. Majumder, Z. Kong, A. Mehrish, A. Zadeh, C. Li, R. Valle, B. Catanzaro, and S. Poria, “TangoFlux: Super Fast and Faithful Text-to-Audio Generation with Flow Matching and CLAP-Ranked Preference Optimization,” arXiv preprint arXiv:2412.21037, 2024.
    4. J. Hai, Y. Xu, H. Zhang, C. Li, H. Wang, M. Elhilali, and D. Yu, “EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformer,” arXiv preprint arXiv:2409.10819, 2024.
    5. N. Majumder, C.-Y. Hung, D. Ghosal, W.-N. Hsu, R. Mihalcea, and S. Poria, “Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization,” arXiv preprint arXiv:2404.09956, 2024.
    6. C.-Y. Kuan and H.-Y. Lee, “From Alignment to Advancement: Bootstrapping Audio-Language Alignment With Synthetic Data,” IEEE Transactions on Audio, Speech and Language Processing, vol. 33, pp. 4604–4619, 2025.
    7. J. Xu et al., “Qwen2.5-Omni Technical Report,” arXiv preprint arXiv:2503.20215, 2025.
    8. A. Tjandra et al., “Meta AudioBox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Sound,” arXiv preprint arXiv:2502.05139, 2025.
Disclaimer

For confidentiality reasons, audio samples generated during the internship project are not released on this page. The work itself builds on publicly available models, datasets, and open-source components. The audio examples provided here were independently regenerated using those public components and the implementation details described in the paper and the publicly released GitHub repository. No internal code, model checkpoints, generated samples, prompts, logs, or non-public training data from the internship project were used.

These examples are intended only to illustrate the qualitative evaluation format and the type of instruction-following behavior studied in the paper. They should not be interpreted as outputs from the original internship experiments or as an official company demo.