Hy4 preview is a new-generation Mixture-of-Experts (MoE) flagship model developed by the Tencent Hy Team. The model comprises 770B total parameters, of which 49B are activated per token. Model supports 1M tokens context length.

• Hy4-Preview on HuggingFace
• Hy4-Preview on ModelScope

Native FP8 weights are published as well.

Benchmarks:

Known issues with the preview model: spends longer than necessary reasoning through complex tasks, and a tendency to over-verify its own work.

Open-weights frontier models will continue to drop until RAM prices improve 😅