ICML Workshop: Protocol Learning

  InterContinental Grand Hotel, Seoul Parnas  Seoul, South Korea

Pluralis Research brought together academic and industry leaders during ICML to convene on the emerging paradigm of Protocol Learning - decentralized, communication-efficient, model-parallel training of foundation models.

View Luma Event

Workshop

Researchers working on related topics joined us for a series of talks, followed by a selection of posters tackling various parts of the decentralized training stack. The workshop was organized in collaboration with Professor Namhoon Lee and POSTECH.

Protocol Learning

Training frontier foundation models today demands massive, co-located clusters of high-end GPUs - accessible only to a handful of the most well-resourced organizations. Protocol Learning removes this co-location requirement, enabling multi-participant training of foundation models across open, permissionless networks of globally distributed compute, where no single participant has, or can ever obtain, a full copy of the model.

This requires solving hard open problems in low-bandwidth model parallelism, asynchronous distributed optimization, supporting heterogeneous hardware, fault-tolerant training systems, Byzantine robustness, and trustless verification. The workshop convened the researchers advancing these building blocks to define the challenges ahead and chart a research roadmap for training the next generation of community-owned frontier models with self-sustaining economics.

Talks

Lightning Talks

Poster Sessions

FRESCO: A Novel Consistency Control for Asynchronous Pipeline Parallel Training

Dongyeop Lee

Photos

  • Attendees watching a presentation at the ICML Protocol Learning workshop
  • Speaker presenting research at the ICML Protocol Learning workshop
  • Workshop attendees gathered for a Protocol Learning talk
  • A speaker addressing attendees at the Protocol Learning workshop
  • Protocol Learning workshop presentation in Seoul
  • Collective Sovereignty talk at the ICML Protocol Learning workshop
  • Speaker concluding a talk at the ICML Protocol Learning workshop
  • Researcher presenting at the ICML Protocol Learning workshop
  • Lightning talk at the ICML Protocol Learning workshop
  • Optimization research talk at the Protocol Learning workshop
  • Distributed learning presentation at the Protocol Learning workshop
  • Attendees viewing posters at the ICML Protocol Learning workshop
  • Researchers discussing a poster at the Protocol Learning workshop
  • Workshop attendees gathered around the poster displays
  • A full room of attendees at the ICML Protocol Learning workshop

Back to Events