Spaces:

---

facebook

/

bouquet

Like 37

Running

Model Submission: Fine-Tuned EuroLLM-9B for Genoese (ITA <-> LIJ)

Base Model: utter-project/EuroLLM-9B-Instruct
Fine-Tuning Method: QLoRA with Prompt Loss Masking
Evaluation Benchmark: BOUQuET (benchmark_sentence_level)

Submitted Runs & Metrics Summary:

1. eurollm_ita2gen (Italian -> Genoese):

   • Hyperparameters: QLoRA (r = 64, α = 128)
   • chrF++: 50.80
   • MetricX-24: 7.76
   • GlotLID: 0.9355
   • GlotLID x MetricX: 0.4200
   • Training Data: Parallel Sentences Dataset (24,664 pairs)

2. eurollm_gen2ita_plus (Genoese -> Italian):

   • Hyperparameters: QLoRA (r = 32, α = 64)
   • chrF++: 56.90
   • MetricX-24: 2.45
   • GlotLID: 0.9787
   • GlotLID x MetricX: 0.6887
   • Training Data: Parallel Sentences + Dictionary Entries (24,664 pairs + 23,482 words)

Hi @matteof22 !
Thanks a lot for your submission!
At the moment, we are not yet sure how to best integrate into the leaderboard such partial submissions (i.e. models evaluated with only one or a few directions), and whether it would be sustainable for us to host the complete translation and scoring outputs (as opposed to the small csv files with aggregated scores only).
But I am letting you know that we are aware of this submission, and will reflect it in the leaderboard, one way or another!