RL Post-Training on Macs
July 2026Post-training an 8B MoE with GRPO on 14 consumer Macs and one B200, decoupled over R2; held-out pass@1 on agentic search more than doubled, from 29% to 63%
Posts
Reducing blocking communication in DiLoCo by factoring synchronization into a non-blocking parameter mixing and a minimal blocking gradient mixing
We replicated Meta's full post-training pipeline on a single compute node with one researcher, matching Llama-3.2-1B-Instruct quality
Pluralis' Multi-party Training Stack
January 2026A deep dive into our library built for fault-tolerant multi-party distributed training
Asynchronous Pipeline Parallelism
November 2025Nesterov Method for Asynchronous Pipeline Parallel Optimization
A Third Path: Protocol Learning
March 2025Developing the true open-source AI
Two enormous, previously disparate fields converge and a path towards the largest models to ever be trained is opened
Decentralized Training Looms
July 2024Collaborative Training of foundation models is closer to actualization than broadly understood. The popular view that low bandwidth node-to-node connections render this infeasible is incorrect