Meta releases new model LLaMA to roam a world of public data

Meta introduces LLaMA and says it shows it is possible to train language models using publicly available datasets without resorting to proprietary datasets

Meta - the company formerly known as Facebook - has thrown its generative AI model hat into the ring in the form of LLaMA, the Large Language Model Meta AI. 

In a blog post, Meta states that this foundational large language model will be released to help advance research in the field of AI. The company says it has committed itself to open science, and LLaMA is a part of that. 

Smaller and more performant models like LLaMA allow researchers without the substantial infrastructure required to study them, democratising access to this essential and fast-changing field, says Meta.

“We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets,” say the report authors.

Guidelines required for responsible AI

The LLaMA model takes a sequence of words as an input and predicts the next word to generate text recursively, much like other large language models. The model was trained on text from the 20 languages with the most speakers, focusing on those with Latin and Cyrillic alphabets. 

Researchers believe that the entire AI community, including policymakers, academics, civil society, and industry, must collaborate to develop clear guidelines on responsible AI and large language models. 

Meta hopes that releasing LLaMA will promote further research in these crucial areas. Access to the model will be granted on a case-by-case basis for research use cases.

“There is still more research that needs to be done to address the risks of bias, toxic comments, and hallucinations in large language models,” write the authors. “Like other models, LLaMA shares these challenges. As a foundation model, LLaMA is designed to be versatile and can be applied to many different use cases, versus a fine-tuned model that is designed for a specific task. 

“By sharing the code for LLaMA, other researchers can more easily test new approaches to limiting or eliminating these problems in large language models.”


Featured Articles

Data & the IoT key to enabling smart cities of the future

Data is the lifeblood of smart cities like Barcelona, transforming everything from shopping and transportation to autonomous driving and augmented reality

ServiceNow & Nvidia to build enterprise-grade generative AI

ServiceNow and Nvidia announced a partnership to develop powerful, enterprise-grade generative AI capabilities that can transform business processes

IBM and SAP accelerating the rate of AI innovation

IBM aims to provide SAP customers with a better user experience, faster decision-making and greater insights to help transform their business processes

Google I/O: Google shares details of Duet AI collaborator


45% of executives state ChatGPT has increased AI investment

AI Strategy

Top 10 quantum computing companies globally in 2023