用 logits 偏置让预训练音乐 Transformer 实时与人合奏
Got a pretrained music transformer to jam with me live, no fine-tuning, just steering the sampling [P]
作者用预训练符号音乐模型 Anticipatory Music Transformer（music-medium-800k）做实时二重奏，不微调，只在采样前对 logits 加偏置：压低调外音、抬高和弦音与张力音、压低手部附近琴键避免抢音，停止演奏时把输出推向旋律音区，并对延续上一乐句音程略作加权。
I wanted to see how far you can get making a pretrained symbolic music model "listen" to a live player without training it. Turns out pretty far.
Setup: I play a MIDI keyboard, the Anticipatory Music Transformer (Thickstun et al., music-medium-800k) plays a second piano. While I play I track my key, the chord I'm holding and my tempo, and add biases to the logits before sampling. Out-of-key notes get pushed down, chord tones and tensions get pushed up, the keys right around my hands get pushed down so it doesn't step on me, and when I stop it gets pulled up to where a melody would be. There's also a small boost for continuing the intervals of my last phrase, which makes it sound like it's answering.
It samples about a second ahead, and when I change chord I throw away what it had planned and resample. Runs on a Jetson Orin Nano.
Video + code: https://github.com/RiccardoCataldi/duetto
The offsets are all hand-tuned by ear, which feels a bit dumb. Has anyone tried something more principled for this, like a small learned conditioning head or guidance-style tricks on a symbolic model?
submitted by /u/GGAllin43
[link] [留言]
来源：r/MachineLearning · reddit.com