Info

This is a pretrained transformer autoregressive model, built on the Qwen3 architecture. It is trained to predict sequences of notes each having a pitch, volume, length and starting time. These sequences is written in text and the model predict individual characters.

A note in this format consists of about 7 letters and the model has a maximum sequence length of 8192 letters.

This format does not include the instrument, so we will play the sequences of notes on a piano. The model was trained 20 epochs on Monster-midi dataset containing over 400 000 midi files.

Non cherry picked examples. All generation uses 30 seconds of initial context of each melody and samples are made with:

Temperature: 0.8
Top p: 1.0
Seed = 4
Other parameters are set as default

The model sampled until 8192 tokens were reached, some generations contain multiple melodies("*" replaced with pause), and some seem to be able to continue for longer.

Prompt	Generation
Alla turca
Dancing queen*
Let it go
Gimme gimme
Fur Elise
Johansson
Pirates of the caribean
Undertale
Stereo Madness

*This melody exceeds 25 notes/second, predictions can be worse.

Quick start

The model needs an initial context to make predictions, to make predictions from melodies from scratch we can use the start "*" as a prompt which is the separator of melodies in the dataset.

from transformers import Qwen3ForCausalLM
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"
model = Qwen3ForCausalLM.from_pretrained("qwrt/Melodimodell-67M").to(device)

prompt="""*"""
gen_len=8192-len(prompt)
print("börjar genereringen")
seed = torch.tensor(list(prompt.encode()), dtype=torch.long)[None].to(device)
out = model.generate(seed, max_new_tokens=gen_len,
                                     do_sample=True, temperature=0.8, top_p=1.0)
text_formated_midi=bytes(out[0].tolist()).decode("utf-8", errors="replace")

output

5C5f22 4F4e23 3A3e24 4A3e24 3C2f26
4A5e14 3F4c15 4F3d16 4C3e17 4G5e48 3H4e49 4F5e49 2G5f49
4D6e28 3G4e29 2H4e30
3H5e5 4F6e6 3C4f7 3D4e7 3G4f48 4E6e48
...

To convert the generated string to a playable midi file, we start by installing mido:

!pip install mido

Then, use this script for conversion.

import mido
from huggingface_hub import hf_hub_download


file_path = hf_hub_download(
    repo_id="qwrt/Monster_textmidis_filtered",
    filename="stringtomidi.py",
    repo_type="dataset"
)


spec = importlib.util.spec_from_file_location("parse_custom",file_path)
modul = importlib.util.module_from_spec(spec)
spec.loader.exec_module(modul)

text_formated_midi=text_formated_midi.strip("*") #Remove the first star so to not get an empty string
generated_songs=text_formated_midi.split("*")
first_song=generated_songs[0]

if(len(generated_songs)==1): #remove the last incomplete note of an unfinished melody
    all_notes=first_song.split(" ")
    all_notes.pop()
    first_song=" ".join(all_notes)

events = modul.parse_custom(first_song)
modul.build_midi(events, "example.mid")

You can then use an online tool to convert this midi to a playable mp3 file. Here, the website https://www.freeconvert.com/midi-to-mp3 was used.

More Samples

Here are some non-cherry-picked generation using different melodies as a prompt. 30s initial context, temperature=0.9 and Seed=4 for all. Higher temperatures often yields more creative songs since it often increases probabilities for the song to take "new paths". Again the generation is cut after the models maximum context, ensuring that the model can see the whole melody at every position: