NEWS  AI  July 26, 2024  Reading time: 3 Minute(s)

Max (RS editor)

Meta has introduced its latest AI breakthrough: Llama 3.1 405B. This monumental large language model (LLM) boasts a staggering 405 billion parameters and a substantial 750 GB size, positioning it as one of the largest and most capable models ever released. With its expanded 128K token input window, Llama 3.1 405B competes fiercely with leading AI models such as Anthropic Claude 3.5 Sonnet and OpenAI GPT-4o. Unlike many proprietary, paid models, Meta’s Llama 3.1 is available for royalty-free use, allowing researchers and developers to customize and deploy it on their high-powered Nvidia GPUs.

INTENSIVE CREATION PROCESS. Meta’s creation of Llama 3.1 405B required immense computational resources. Utilizing up to 16,384 H100 GPUs with a 700W thermal design power (TDP) on the Meta Grand Teton AI server platform, Meta generated the model’s 3.8 x 10^25 FLOPs. Training spanned over 54 days, consuming 39.3 million GPU hours. The total electricity consumption is estimated to exceed 11 gigawatt-hours (GWh), resulting in a release of approximately 11,390 tons of CO2-equivalent greenhouse gases.

The process encountered significant challenges, with GPU failures accounting for 57.3% of downtime and faulty GPUs contributing an additional 30.1% of the delay. Despite these hurdles, Meta’s efficient use of resources and cutting-edge technology led to the successful development of this advanced model.

EMPHASIZING SAFETY AND PERFORMANCE. Llama 3.1 405B demonstrates notable advancements in safety compared to its competitors. Extensive training across various domains—including cybersecurity, child safety, and chemical and biological attacks—alongside the use of Llama Guard 3 for input and output text filtering, has enhanced the model’s safety features. However, the model’s training data showed limited foreign language documents, potentially increasing the risk of unsafe responses in languages such as Portuguese and French.

In performance evaluations, Llama 3.1 405B achieved scores ranging from 51.1% to 96.6% on college and graduate-level tests, aligning with its peers, Claude 3.5 Sonnet and GPT-4o. Despite these promising results, human-graded real-life tests revealed that GPT-4o provided superior answers 52.9% more frequently than Llama 3.1. While Llama 3.1 is restricted to knowledge up to December 2023, it can access the latest information online via Brave Search, perform mathematical computations with Wolfram Alpha, and solve coding problems through a Python interpreter.

HARDWARE AND ACCESSIBILITY REQUIREMENTS. Researchers eager to deploy Llama 3.1 405B locally will need to invest in substantial hardware. The model requires 750 GB of storage and a configuration of at least eight Nvidia A100 GPUs or equivalents, providing two MP16 nodes and 810 GB GPU VRAM for inference, alongside 1 TB of RAM. For those with less demanding needs, Meta offers smaller versions of the model: Llama 3.1 8B and 70B. The Llama 3.1 8B variant, with only 16 GB of GPU VRAM requirement, can be effectively run on high-performance systems such as Nvidia 4090-equipped laptops, delivering performance comparable to GPT-3.5 Turbo.

For general users who wish to interact with top-tier AI without extensive hardware, apps from companies like Anthropic offer convenient solutions on Android and iOS platforms.

 IMAGES CREDITS: META