Petals - Open source tool for running LLMs collaboratively

Petals  Features

Open source tool for running large language models like BLOOM-176B collaboratively — you load a small part of the model, then team up with people serving the other parts to run inference or fine-tuning.