ETD

• Group Relative Policy Optimization
• Reinforcement Learning
• Symbolic Music Generation

Controlling the stylistic behavior of symbolic music generation models remains a difficult problem. Although symbolic representations such as MIDI are editable and compact, models that generate symbolic music do not always reliably follow requested stylistic attributes such as mood, genre, or musical character. This thesis investigates whether feedback computed from the rendered audio output can be used to guide and improve symbolic music generation.

The thesis proposes an audio-reward fine-tuning framework that connects symbolic music generation with audio-domain evaluation models. In this framework, a text-to-MIDI model generates MIDI from a prompt, the MIDI is rendered into audio, and the resulting audio is evaluated using several reward signals. These rewards are then used to fine-tune the symbolic generation model with Group Relative Policy Optimization, a reinforcement-learning method that compares groups of generated outputs and updates the model toward higher-reward generations.

The framework is applied to MIDI-LLM, a text-to-MIDI generation model. The reward function combines three complementary audio-domain signals: prompt alignment, which measures how well the rendered audio matches the input text prompt; learned musical-quality estimation, which encourages outputs that are more coherent; and reference-audio similarity, which encourages generated outputs to resemble target recordings or styles.

The approach is evaluated through focused style-adaptation experiments, automatic metrics, qualitative examples, and a human listening test. The results show that audio-reward fine-tuning can steer symbolic music generation toward target styles in controlled settings.

However, the experiments also reveal important limitations. Reward-based optimization is sensitive to the prompt distribution, reward configuration, and training setup. In broader or less focused settings, training can become unstable, and generated outputs may become invalid or less diverse.