2024-2025Technique
MGS — MultiGumbelSampling
Yet-Unpublished LLM Regularization Technique
★Compatible with UNA for Additive Gains
MGS is MultiGumbelSampling — a regularization technique introducing Gumbel-sampled noise across signal paths during SFT/RLHF training. Combinable with UNA (UNAMGS releases) for additive performance gains.
- Client
- Independent — Juanako.AI
- Role
- Sole author
- Duration
- 2024-2025
- Team
- Solo
Outcomes
5
Public Releases
4
UNAMGS Releases
1.5B to 7B
Model Sizes
Multiple
#1 Positions
Highlights
- •Compatible with UNA — UNAMGS combines both
- •Operates on different network paths than UNA
- •First public release: Oct 2024
Selected Releases & Leaderboard Positions
| Date | Model | Size | Base | 1.5B | 3B | 7B | 34B | 70B+ |
|---|---|---|---|---|---|---|---|---|
| 30-Oct-2024 | Cybertron v4 MGS | 7B | Qwen2.5 | · | · | 🏆 | · | · |
| 07-Nov-2024 | MiniClaus UNAMGS | 1.5B | Qwen2.5 | 🏆 | · | · | · | · |
| 21-Nov-2024 | Cybertron v4 UNAMGS | 7B | Qwen2.5 | · | · | 🏆 | · | · |
| 04-Nov-2024 | Pancho v1 3B UNAMGS | 3B | Qwen2.5 | · | 🏆 | · | · | · |
| 03-Feb-2025 | MiniClaus UNAMGS GRPO | 1.5B | Qwen2.5 | 🏆 | · | · | · | · |
🏆 marks the size tier reached on the leaderboard
transformersdeepspeedaccelerateaxolotltorchwandbsftrhlfdistributed-training