AI hardware startup Cerebras Systems has released seven open-source versions of large language models that researchers and others can access for free.
The GPT-based AI models were trained on Cerebras' Andromeda AI supercomputer.
- The supercomputer can deliver 1 exaflop of AI computing power, or 1 million trillion calculations per second, rivaling the world's top supercomputer.
- Andromeda is made up of Cerebras' own CS-2 AI machines containing the WSE-2 chip, which runs AI software and is the largest computer chip ever built.
- The generative AI models released to the public are among the first to be trained on systems that don't use GPUs, such as Nvidia's. It only took a few weeks, according to the startup.
- The models range from 111 million parameters to 13 billion parameters, indicating their ability to perform complex functions. By comparison, ChatGPT has 175 billion parameters.
- Cerebras said it's sharing the models, weights, and training recipe via the industry standard Apache 2.0 license. The models are on Hugging Face and GitHub.
- Sunnyvale, California-based Cerebras has raised $720M in venture funding to date.
