laion/Humaneness-Voice-Small-DPO-LoRAs

Humaneness Voice Small — six S3 DPO LoRAs

This repository contains one exact S3 base weight and six separate DPO LoRA adapters. They are preference-tuning experiments, not different backbone models and not a claim that DPO improved listening quality. The S3 base is the original-learning-rate curriculum checkpoint at step 32,337 from Humaneness Voice Small. Its semantic model was initialized from Qwen3-0.6B pretrained language-model weights, then trained for speech; it was not initialized from random weights. It has a Qwen3-0.6B semantic path, an approximately 112.76M-parameter hierarchical Talker, and predicts 12 MOSS Audio Tokenizer v2 acoustic codebook tokens per 80 ms audio frame. All six adapters use exactly this S3 base, not S10.

base/S3/model_bf16.pt is the full, unmerged bf16 S3 weight. Its SHA-256 is 60079723bf797a81107def1e6fc479e5353b3cc51f3780f98ea65a9f5c109f97, identical to the source repository's checkpoints/S3/model_bf16.pt at revision 5de86032771c37a3008d14e78ee7cdb39368e2a8. Each adapters/*.adapter.pt contains only LoRA tensors and minimal architecture metadata, with no optimizer state and no source-audio records. The adjacent JSON files provide source checkpoint and export hashes.

Adapter	Rank / alpha	Completed updates	Preference mix	Training-pair count
original_rank064	64 / 128	5,589	Original five-family mix	715,402
original_rank128	128 / 256	5,589	Same original mix	715,402
rebalanced_rank064	64 / 128	3,493	Six-task capped/rebalanced mix	447,179
rebalanced_rank128	128 / 256	3,493	Same rebalanced mix	447,179
subset_pitch_quality_p2_rank016	16 / 32	1,138	S5 pitch + SIDON quality + transcript-compatible CFG p2 only	145,667
subset_pitch_quality_p2_rank064	64 / 128	1,138	Same three-family subset	145,667

The original mix uses real-speech stop, voice-profile emotion, voice-profile truncation, transcript-compatible CFG/p2 length and S5 pitch pairs. The rebalanced mix caps each of real-speech stop, emotion and truncation at 100,000, and includes all 55,995 eligible CFG/p2 length, 66,805 S5 pitch, and 24,379 SFT-3 quality-repair pairs. The new three-family subset selects 55,415 CFG-p2-length, 66,147 S5-pitch and 24,105 SIDON-quality training pairs, with 1,512 UID-disjoint validation pairs. It excludes the real-speech stop, voice-emotion and voice-truncation tasks. Here CFG p2 means transcript-compatible p2_len, not text-free p2_emox. Training examples share the same prompt and reference on both preference branches. The objective is length-normalized stop-plus-12-codebook DPO, beta 2.0, with 0.02 chosen-NLL anchor. LoRA targets semantic and Talker attention/MLP projections; the base weight and codec heads are frozen. All six adapters used 32 GPUs, effective batch 128, peak learning rate 5e-5, 5% warmup and cosine decay to 10% of peak over one pass. Files under dpo_code/ include the exact trainers, manifest builders, Slurm scripts and adapter attachment code; code/ and assets/ contain the S3 architecture/inference components.

Loading and inference

Use the supplied code/ model construction/inference utilities and dpo_code/load_adapter.py. Download this repository, then load base/S3/model_bf16.pt into the S3 architecture and attach one adapter. For example, after setting up the dependencies and tokenizer described by the base model card:

import json, sys, torch
from pathlib import Path
repo = Path("/path/to/Humaneness-Voice-Small-DPO-LoRAs")
sys.path.insert(0, str(repo / "code"))
sys.path.insert(0, str(repo / "dpo_code"))
from large_talker import build_fresh
from load_adapter import attach_adapter
schema = json.loads((repo / "assets/score_schema.json").read_text())
model, config = build_fresh(schema)
model.load_state_dict(torch.load(repo / "base/S3/model_bf16.pt", map_location="cpu", weights_only=True), strict=True)
model.tie_weights()
attach_adapter(model, repo / "adapters/rebalanced_rank064.adapter.pt")
model.eval()
# Continue with code/infer.py or code/acting_challenge_eval.py for generation.

The code constructs the Qwen configuration using the upstream Qwen3-0.6B environment before replacing all model parameters with this repository's exact S3 checkpoint. The MOSS v2 audio codec is required to render codec tokens as waveform. This is research code, not a pip-packaged turnkey pipeline.

Evidence and limits

Held-out pair classification improved on real-speech stop, voice-profile emotion and truncation for the original and rebalanced runs, but pair accuracy is not the same as generated-audio quality. The rebalanced run includes no disjoint pair-level pitch or quality validation: all released pitch and quality pairs were used for training. S5 pitch preferences are mostly confounded with signal-quality differences, so a pitch preference win alone does not establish better speaker identity. Select a model using independent audio comparisons and listening, not training loss or pair accuracy alone.

The frozen 50-scene/11-format, matched-seed acting comparison of the original four adapters is published separately as the evaluation dataset and the side-by-side listening Space. The new rank-16/rank-64 subset adapters are not yet included in that public comparison. The evaluation table covers WER, genuineness, vocal-burst blending, clip-duration diagnostics and prompt-condition breakdowns. Forced sentence alignment is not yet complete, so the clip-duration diagnostic must not be interpreted as a validated per-sentence timing score. The adapter comparison is intended to identify trade-offs, not to assert a global winner from one scalar.

Three-family subset experiment: preliminary evidence, not a quality recommendation

The two subset runs completed on 30 September 2026. Their final adapter-only files are adapters/subset_pitch_quality_p2_rank016.adapter.pt and adapters/subset_pitch_quality_p2_rank064.adapter.pt; adjacent JSON files record source-checkpoint hashes and provenance. Exact training code, the packed-manifest summary, metrics and run log are under dpo_code/pitch_quality_p2/. There were 1,138 optimizer updates with adapter-plus-optimizer checkpoints during training; the published adapter-only files omit optimizer state. The released dpo_code/load_adapter.py supports all three ranks, 16, 64 and 128.

The only generated-audio check so far is a 36-output, one-scene canary per subset adapter, matched to the S3 base by prompt, text, seed, reference and language. Mean Parakeet WER was 0.0160 for S3, 0.0236 for rank 16, and 0.1028 for rank 64 (lower is better). Mean vocal-burst blend was 6.220, 5.479, and 5.936 respectively (higher is better). Genuineness predictor scores increased 1.556 → 1.665 → 1.792, but this does not establish audible quality improvement. These 36 samples represent only one acting scene; no full 50-scene evaluation, human listening verdict or sentence-level timing study has been completed for the subset adapters. In particular, do not assume rank 64 improves intelligibility. The retained CFG-p2 length pairs still use the older length-normalized preference objective, which has a known shared-prefix likelihood loophole; this release is a data-mixture ablation, not an objective repair. The frozen evaluator also exposes target_seconds + 5 s as its Tokens: prompt field, so its clip-duration error is not a valid timing-following score.

License and provenance

Released under CC BY 4.0, consistent with the base model. Training mixes are described by the included records/code and the source DPO dataset collection. Cite and respect each underlying dataset's attribution and usage terms. Speech can reproduce training artifacts, accent and speaker traits, and the output metrics do not guarantee identity fidelity, consent, suitability, or safety for a particular use.