Meta releases new model LLaMA to roam a world of public data

Meta introduces LLaMA and says it shows it is possible to train language models using publicly available datasets without resorting to proprietary datasets

Meta - the company formerly known as Facebook - has thrown its generative AI model hat into the ring in the form of LLaMA, the Large Language Model Meta AI. 

In a blog post, Meta states that this foundational large language model will be released to help advance research in the field of AI. The company says it has committed itself to open science, and LLaMA is a part of that. 

Smaller and more performant models like LLaMA allow researchers without the substantial infrastructure required to study them, democratising access to this essential and fast-changing field, says Meta.

“We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets,” say the report authors.

Guidelines required for responsible AI

The LLaMA model takes a sequence of words as an input and predicts the next word to generate text recursively, much like other large language models. The model was trained on text from the 20 languages with the most speakers, focusing on those with Latin and Cyrillic alphabets. 

Researchers believe that the entire AI community, including policymakers, academics, civil society, and industry, must collaborate to develop clear guidelines on responsible AI and large language models. 

Meta hopes that releasing LLaMA will promote further research in these crucial areas. Access to the model will be granted on a case-by-case basis for research use cases.

“There is still more research that needs to be done to address the risks of bias, toxic comments, and hallucinations in large language models,” write the authors. “Like other models, LLaMA shares these challenges. As a foundation model, LLaMA is designed to be versatile and can be applied to many different use cases, versus a fine-tuned model that is designed for a specific task. 

“By sharing the code for LLaMA, other researchers can more easily test new approaches to limiting or eliminating these problems in large language models.”

Share

Featured Articles

US to Form AI Task Force to Confront AI Threats to Safety

The United States aims to form an AI task force to explore AI safety and possible regulations, amid a global debate over developing responsible technology

Wipro to Advance Enterprise Gen AI Adoption with IBM watsonx

Wipro and IBM are expanding their partnership to launch a Gen AI platform, aiming to provide easy and secure AI adoption solutions for businesses

Dr Joy Buolamwini: Helping Tech Giants Recognise AI Biases

With a longstanding commitment to AI ethics, Dr Joy Buolamwini continues to hold AI developers accountable as they work to tackle machine learning biases

Big Tech Companies Agree to Tackle AI Election Fraud

AI Strategy

Microsoft to Offer AI Support to Europe Amid Economic Slump

AI Strategy

Microsoft: Digitally Transforming India via AI Upskilling

AI Strategy