SambaNova Systems recently achieved an important goal with their Samba-CoE v0.2 Large Language Model.
This model, boasting an astounding 330 tokens per second performance, outshines several notable models from competitors, such as Databricks’ DBRX (just released yesterday), MistralAI’s Mixtral-8x7B and Grok-1 by Elon Musk’s xAI among many others.
What makes this achievement truly impressive is the efficiency of this model; it reaches such speeds without compromising precision, and only 8 sockets are necessary to run instead of alternatives requiring 576 sockets and lower bit rates.
Indeed, during our tests of the LLM, it responded swiftly and accurately to our inputs; an answer about the Milky Way galaxy took just one second with 330.42 tokens being exchanged per answer provided.
Questioning quantum computing generated an equally fast and comprehensive response, totalling 332.56 tokens delivered within one second.
Efficiency Enhancements
SambaNova’s focus on using fewer sockets while still offering high bit rates suggests significant advancements in computing efficiency and model performance.
Samba-CoE v0.3 will soon be made available through their partnership with LeptonAI and will represent further progress and innovation.
SambaNova Systems notes that their advancements are founded upon open-source models from Samba-1 and the Sambaverse, employing an innovative method for model merging.
This methodology not only underpins the current version, but it also provides a scalable and innovative framework for future developments.
Comparing Samba-CoE v0.2 against other models such as GoogleAI’s Gemma-7B, MistralAI’s Mixtral-8x7B, Meta’s Llama2-70B and Alibaba Group Qwen 72B Falcon 180B as well as BigScience BLOOM-176B highlights its competitive edge in this space.
This announcement is sure to spark lively discussions in the AI and machine learning communities, prompting inquiries about efficiency, performance, and future of AI model development.
SambaNova Systems was established in Palo Alto, California by three co-founders – Kunle Olukotun, Rodrigo Liang, and Christopher Re – in 2017.
SambaNova began by producing custom AI hardware chips, but has quickly expanded into offering machine learning services and an enterprise AI training, development, and deployment platform known as the SambaNova Suite by early 2023. Most recently in April 2019, an ambitious AI model called “Samba-1,” comprised of 50 smaller models in an “Expert Composition,” was also revealed.
This shift from hardware-focused startup to full-service AI innovator is testament to our founders’ desire for creating accessible AI technologies.
As SambaNova finds its niche within AI, it has also established itself as an formidable rival to established giants like Nvidia; successfully raising a $676 million Series D with a valuation over $5 billion in 2021.
Today, Nvidia competes against both emerging AI chip startups like Groq and more established entities like Nvidia.
