2023-2024Technique
UNA — Uniform Neural Alignment
Yet-Unpublished LLM SFT/RLHF Technique
★8 Public Releases, Multiple #1 Positions
UNA is Uniform Neural Alignment — a transformers architecture change introducing an auxiliary loss, applied as a patch to HuggingFace Transformers models. Operates during SFT and RLHF training. Applicable to attention layers, MLP layers, or both. Memory intensive but compatible with LoRA. Training data does not need to be novel, but must not have been previously overfitted. Applied across Mistral, Intel, Yi/Smaug, Qwen2.5, LLaMA 1 & 2, Pythia, and Luxa architectures.
- Client
- Independent — Juanako.AI
- Role
- Sole author · Method, training, releases
- Duration
- 2023 & 2024
- Team
- Solo
Outcomes
18
Public Releases
4+
Base Architectures
1.5B to 34B
Model Sizes
Multiple
#1 Positions
Highlights
- •Consistent positive delta over base models
- •Multiple #1 leaderboard positions
- •Applicable to different network layers
Selected Releases & Leaderboard Positions
9 of 18 total releases
| Date | Model | Size | Base | 1.5B | 3B | 7B | 34B | 70B+ |
|---|---|---|---|---|---|---|---|---|
| 28-Nov-2023 | Juanako 7B UNA | 7B | Mistral | · | · | 🏆 | · | · |
| 02-Dec-2023 | Cybertron v1 | 7B | Mistral | · | · | 🏆 | · | · |
| 05-Dec-2023 | Cybertron v2 | 7B | Mistral | · | · | 🏆 | 🏆 | 🏆 |
| 09-Dec-2023 | Xaberius 34B v1beta | 34B | Yi | · | · | · | 🏆 | 🏆 |
| 11-Jan-2024 | UNA-TheBeagle v1 | 7B | Mistral | · | · | 🏆 | 🏆 | 🏆 |
| 04-Feb-2024 | UNA-SimpleSmaug 34B | 34B | Yi/Smaug | · | · | · | 🏆 | · |
| 30-Oct-2024 | Cybertron v4 MGS | 7B | Qwen2.5 | · | · | 🏆 | · | · |
| 07-Nov-2024 | MiniClaus UNAMGS | 1.5B | Qwen2.5 | 🏆 | · | · | · | · |
| 21-Nov-2024 | Cybertron v4 UNAMGS | 7B | Qwen2.5 | · | · | 🏆 | · | · |
🏆 marks the size tier reached on the leaderboard
transformersdeepspeedaccelerateaxolotltorchwandbsftrhlfdistributed-training