SambaNova Systems recently made headlines for achieving an outstanding milestone with their Samba-CoE v0.2 Large Language Model (LLM).
This model, operating at an impressive 330 tokens per second, outdistances several notable models from competitors including Databricks’ recently unveiled DBRX model and MistralAI’s Mixtral-8x7B as well as Grok-1 by Elon Musk’s xAI; among many others.
What makes this achievement particularly noteworthy is the efficiency of the model; it achieves these speeds without compromising precision, while only needing 8 sockets to run compared to alternatives requiring 576 sockets and offering lower bit rates.
Indeed, in our tests of the LLM, it responded quickly and accurately to our inputs – in one second for 425-word answers about the Milky Way galaxy, it produced responses at an amazing rate of 330.42 tokens!
Questioning quantum computing brought forth an equally impressive response – 332.56 tokens delivered within one second!
Efficiency Advances
SambaNova’s focus on using fewer sockets while maintaining higher bit rates suggests significant advances in computing efficiency and model performance.
LeptonAI also teased an upcoming release of Samba-CoE v0.3 as evidence of ongoing progress and innovation.
SambaNova Systems points out that these advancements are built upon open source models from Samba-1 and the Sambaverse, employing an exclusive method for model assembly and merging.
This methodology not only forms the basis of our current version but also suggests a flexible and innovative strategy for future developments.
Comparison with models such as GoogleAI’s Gemma-7B, MistralAI’s Mixtral-8x7B, Meta’s Llama2-70B, Alibaba Group Qwen-72B Falcon 180B Falcon-180B and BigScience BLOOM-176B demonstrate Samba-CoE v0.2’s competitive edge in its field.
This announcement is likely to create significant interest among AI and machine learning practitioners, prompting discussions around efficiency, performance, and the future of AI model development.
SambaNova Systems was established in Palo Alto, California in 2017 by three co-founders – Kunle Olukotun, Rodrigo Liang and Christopher Re.
Beginning as an AI hardware chip manufacturer, SambaNova quickly expanded to offer machine learning services and an enterprise AI training, development, and deployment platform known as the SambaNova Suite by early 2023. Later that same year came Samba-1: an 1-trillion parameter AI model constructed out of 50 smaller models to form a “Composition of Experts.”
This transition from hardware-centric startup to full-service AI innovator illustrates the founders’ dedication to building scalable, accessible AI technologies.
SambaNova has established its presence within AI by carving out its niche and challenging established giants like Nvidia for market leadership, raising a $676 million Series D round at a valuation of over $5 billion by 2021.
Today, Nvidia faces competition from other AI chip startups such as Groq and Nvidia itself.
