Meta Llama 3.1: The World's Largest Open-Source AI Model

Meta has made moves to position itself as an AI leader through the release of Llama 3.1, its latest, and world’s largest, open-source AI model. 

Speaking of the announcement on a blog post, Meta CEO Mark Zuckerberg said the move was precipitating what he sees as the way the industry is going, comparing it to how operating system Linux is now standard for things like cloud and mobile devices due to its open source nature. 

“I believe the Llama 3.1 release will be an inflection point in the industry where most developers begin to primarily use open source,” Mark remarked. 

This move Meta, who recently dumped their Metaverse project to invest their main attention into AI, signals a significant shift in the AI landscape, potentially challenging the dominance of closed-source models like OpenAI's GPT-4 and Anthropic's Claude 3.5 Sonnet.

The Llama model makeup 

Llama 3.1 is the next generation of Meta's Llama language model series, Llama 3, and represents a significant update and improvement over previous versions.

The selling points the tech titan is trying to push are twofold - open source and scale. 

Being open source allows developers to make systems based on their specific needs, deploy them, and even share them with the public or even clients.

Then, there is the model capacity.  Llama 3.1 boasts a staggering 405 billion parameters, making it one of the largest open-source models in recent years, and is trained using 16,000 Nvidia H100 GPUs on a dataset of 15 trillion tokens.

The size and training of the model directly benefit developers building AI systems in several key ways. 

Firstly, the larger parameter count allows the model to capture more complex patterns and relationships in data. 

Second, the training on 15 trillion tokens across diverse datasets means the model has broad knowledge and capabilities that developers can leverage without needing to train from scratch. 

Llama 3.1 demonstrates impressive capabilities across various tasks. The model can handle coding, basic maths problems, and document summarisation in eight languages. While it's a text-only model, its 128,000-token context window allows for processing longer text passages, enhancing its utility in chatbot applications and document analysis.

Mark claimed that the open source nature allows developers to run inference on Llama 3.1 at roughly 50% the cost of using closed models like GPT-4o. 

Meta's Llama is also available for use via its Meta AI assistant, although with some limitations.

This cost-efficiency, combined with the model's open-source nature, allows organisations to train, fine-tune, and distil their own models without relying on closed vendors or compromising data security.

OpenAI v open AI

Meta is partnering with companies like Amazon, Databricks, and NVIDIA to support developers in fine-tuning and distilling their own models. 

This ecosystem approach aims to make Llama the industry standard, potentially democratising AI technology and bringing its benefits to a wider audience.

“Open source will ensure that more people around the world have access to the benefits and opportunities of AI, that power isn’t concentrated in the hands of a small number of companies,” said Mark. 

The debate between open-source and closed AI models mainly circle around a number of concerns. 

Where open models are accused of being insecure, closed models are sometimes dubbed too rigid. 

As the dust settles on this latest development, one thing is clear: the AI landscape is evolving rapidly. Whether Llama 3.1 will indeed surpass ChatGPT in user numbers by the end of the year, as Mark predicts, remains to be seen. What's certain is that the push towards open-source AI is gaining momentum.

******

Make sure you check out the latest edition of AI Magazine and also sign up to our global conference series - Tech & AI LIVE 2024

******

AI Magazine is a BizClik brand