South Korea’s most popular AI voice assistant, GiGA Genie, converses with 8 million people every day.
Telecom company KT’s AI-powered speakerphone can control TVs, offer real-time traffic updates and perform a host of other voice-based home assistance tasks. He mastered his conversational skills in the very complex Korean language through large language models (LLM) – machine learning algorithms capable of recognizing, understanding, predicting and generating human languages based on huge sets of textual data.
The Korean language, known as Hangul, reliably appears in lists of the world’s most difficult languages. It includes four types of compound verbs, and words are often composed of two or more roots.
KT – South Korea’s leading mobile operator with more than 22 million subscribers – has improved the smart speaker’s understanding of these words by developing LLMs with around 40 billion parameters. And thanks to the integration with Amazon Alexa, GiGA Genie can also converse with users in English.
“With the transformer-based models, we have achieved significant improvements in the quality of the GiGA Genie smart speaker, as well as our AI Contact Center, or AICC, customer service platform,” said Hwijung Ryu, Head of LLM development team at KT.
AICC is an all-in-one cloud platform that offers AI voice agents and other customer service related applications.
It can take calls and provide requested information or quickly connect customers to human agents for answers to more detailed inquiries. The AICC handles more than 100,000 calls a day across Korea without human intervention, according to Ryu.
“LLMs enable GiGA Genie to gain better language understanding and generate more human-like sentences, and AICC to reduce lookup times by 15 seconds as it summarizes and categorizes query types faster,” he added.
Training large language models
LLM development can be an expensive and time-consuming process that requires deep technical expertise and comprehensive technology investments.
The NVIDIA AI platform has simplified and accelerated this process for KT.
“We trained our LLM models more efficiently thanks to the powerful performance of NVIDIA DGX SuperPOD, as well as the optimized algorithms and 3D parallelism techniques of NeMo Megatron,” Ryu said. “NeMo Megatron is continually adopting new features, which is the biggest advantage we believe it offers to improve the accuracy of our model.”
3D parallelism—a distributed training method in which a very large-scale deep learning model is partitioned across multiple devices—was crucial for training KT’s LLMs. NeMo Megatron made it easy for the team to accomplish this task with the highest throughput, according to Ryu.
“We considered using other platforms, but it was difficult to find an alternative that provided complete environments, from the hardware level to the inference level,” he added. “NVIDIA also provides exceptional expertise from the product, engineering and more teams, so we easily resolved several technical issues.”
By using hyperparameter optimization tools in NeMo Megatron, KT trained its LLMs 2 times faster than with other frameworks, Ryu said. These tools allow users to automatically find the best configurations for LLM training and inference, making the development and deployment process easier and faster.
KT is also considering using the NVIDIA Triton Inference Server to provide an optimized real-time inference service, as well as NVIDIA Basic Command Manager to easily monitor and manage hundreds of nodes in its AI cluster.
“Through LLMs, KT can launch competitive products faster than ever before,” Ryu said. “We also believe that our technology can drive the innovation of other companies because it can be used to improve their value and create innovative products.”
KT plans to release more than 20 natural language understanding and natural language generation APIs for developers in November. Application programming interfaces can be used for tasks such as summarizing and classifying documents, recognizing emotions, and filtering out potentially inappropriate content.
Learn more about game-changing technologies for the age of AI and the Metaverse at NVIDIA GTConline until Thursday, September 22.
Watch NVIDIA Founder and CEO Jensen Huang’s keynote replay below: