ServiceNow open sources Fast-LLM in a bid to help enterprises train AI models 20% quicker

For businesses, training a large language model (LLM) is one of the most expensive and time-consuming tasks. ServiceNow's new open-source methodology, which promises to train 20% faster and save businesses time and money, might have a significant impact.

Within the organization, the Fast-LLM technology has already been developed, assisting ServiceNow in expediting its own LLM training initiatives. ServiceNow released its StarCoder 2 LLM earlier this year, with assistance from Fast-LLM.Additionally, StarCoder is an open source project that benefits from Hugging Face, Nvidia, and other contributors. Fast-LLM is also used by ServiceNow for fine-tuning tasks and for massive, trillion-token continuous pre-training from pre-existing models.

Anyone can use Fast-LLM to help speed up AI training, including fine tuning procedures, because it is an open source technology. With only minor setup adjustments, it is intended to be a drop-in replacement for an existing AI training pipeline. With a number of advancements in data parallelism and memory management, the new open source project seeks to set itself apart from popular AI training frameworks, such as the open-source PyTorch.

“Twenty percent can be a huge saving in terms of both dollars and time and the overall CO2 footprint when you're dealing with compute clusters that cost hundreds of millions and training runs that cost millions of dollars,” Nicolas Chapados, vice president of research at ServiceNow, told VentureBeat.

The advancements that allow Fast-LLM to speed up AI training

The difficulty of more effectively training AI is well understood by the AI sector.A panel at VentureBeat Transform 2024 covered that very topic and provided ideas for expanding infrastructure.

The goal of the Fast-LLM technique is to maximize the effectiveness of already available training resources, not to scale infrastructure.

Chapados clarified, “We thoroughly examined every operation required to train large language models, particularly transformer-based large language models.” “We carefully optimize how the models themselves use the memory as well as how the compute is distributed to the individual cores within the GPU.”

Two key developments that help set Fast-LLM apart from the competition are the source of its competitive advantage. The first is the computation ordering method used by Fast-LLM, which establishes the sequence in which calculations are performed throughout an AI training run. According to Chapados, Fast-LLM employs a novel method known as “Breadth-First Pipeline Parallelism” by ServiceNow.

“The way that computation is scheduled, both within a single GPU and across multiple GPUs, is the fundamental scientific innovation,” Chapados stated.

Memory management is the subject of the second significant invention. Memory fragments over time in massive training processes. This implies that as training goes on, memory gradually breaks down. Because of the fragmentation, training clusters are unable to effectively use all of the memory that is available.

“When training those large language models, we have been extremely careful in how we design Fast LLM to almost completely eliminate the problem of memory fragmentation,” Chapados stated.

How businesses may expedite training with Fast-LLM today

The Fast-LLM framework is meant to be easily navigable while retaining enterprise-level functionality. It interfaces with current distributed training configurations and serves as a drop-in substitute for PyTorch environments.

According to Chapados, “it's just a simple configuration file that lets you specify all the architectural details that matter for any model developer or researcher.”

Faster training operations have several advantages and can let businesses try new things.

According to Chapados, “it reduces the risk of large training runs.” “Because they won't be worried about the cost, it gives users, researchers, and model builders a little more drive to train larger runs.”

As an open source project, it is anticipated that Fast-LLM would grow more quickly in the future and gain from outside contributions. With StarCoder, ServiceNow has already achieved success using that strategy.

In terms of using this paradigm, Chapados stated, “We really want to be very, very transparent and responsive to the community contributions.””Our goal is to scale this, and we're still getting early feedback about what people like and can do with it.”