Five years ago, the term Large Language Model (or LLM) would probably be something most people — apart from machine learning scientists and software engineers — would have to Google.

But today, LLMs are at the forefront of AI advancements. We’re seeing their use everywhere, powering a wide range of applications from video and content creation to marketing automation to AI agents for customer service. But we rarely see behind the curtain to learn how companies train large language models for specific applications.

In this post, we’re going to look inside the black box . I spoke with Ada's Manager of Security Operations, Evan Harben, to discover how we, at Ada, use LLM capabilities to impact the customer service strategy, and what training processes we employ to ensure their effective and secure deployment.

Large language model 101

What are large language models?

Large Language Models are sophisticated AI systems trained on extensive datasets to understand and generate human language. They are integral to generative AI , which focuses on creating new content by learning patterns from existing data.

LLMs are used in various applications, including AI agents, sentiment analysis, text classification, and authorship attribution and authenticity.

How are large language models trained?

Properly training LLMs is crucial to the quality and accuracy of their output. To learn, LLMs process vast amounts of data, in some cases, ingesting over half a trillion parameters to meet expectations of generating coherent and contextually relevant responses.

With proper training, models can understand language nuances and generate accurate responses to even the most complex inquiries. Conversely, inadequate training can lead to hallucinations , misunderstandings, and even security vulnerabilities.

The training process typically involves three key phases:

1. Pre-training: In this initial phase, LLMs are exposed to a vast corpus of text data from diverse sources. This helps the model learn the general structure and nuances of language, including grammar, facts, and some reasoning abilities.
2. Fine-tuning: After pre-training, LLMs undergo fine-tuning on specific datasets tailored to particular tasks or industries. This step is crucial for adapting the model to understand context-specific language and jargon, enhancing its relevance and accuracy in real-world applications.
3. Evaluation and iteration: Continuous evaluation is essential to ensure the model performs well. This phase involves testing the model on various metrics, including accuracy, coherence, and relevance. Feedback from users and stakeholders is also incorporated to refine the model further.

How can large language models be used in customer service?

LLMs enable AI agents to handle inquiries with remarkable speed and accuracy. These models can understand context, provide relevant responses, and even automate complex processes like troubleshooting and personalized recommendations.

This allows businesses to enhance customer satisfaction and allocate human resources to more strategic tasks .

How do we use LLMs at Ada?

At Ada, LLMs are integral to the capabilities of our AI Agent. We leverage LLMs to deliver exceptional customer service— and not just for the end user.

We enable businesses to incorporate AI into its core business processes; resolving inquiries with unparalleled efficiency and accuracy allows customer service organizations to transition from an agent-first to AI-first model. Even better, our AI Agent learns from interactions to continually improve its performance.

But none of this is possible without properly training large language models, most importantly, to protect people’s data privacy and security.

What LLMs do we use?

Ada uses state-of-the-art models from providers like OpenAI and Anthropic. We actively evaluate new LLMs as they become available on the market to ensure our platform remains at the cutting edge of AI technology. This includes open source LLM and other LLMs available on the AWS or Azure cloud platforms.

What data is used to train Ada’s LLMs?

Generally, there are two types of data that Ada handles:

• End user data and metadata: This encompasses data about the customer and data about the customer’s interaction with the AI Agent, including: personal information, chat transcripts, support phone call recordings, IP address, email address, device information, and files or images shared with the AI Agent.
• Platform data: This is data that the company gives the AI Agent access to so it can do its job, such as: AI Agent answer content, knowledge base content, guidance content, encrypted credentials for integrated business systems, names and emails of AI Managers, and AI Agent customizations.

This data can exist in two different states:

• Unmodified data: Data which has not had any form of anonymization or de-identification applied.
• De-identified data: In certain cases, Ada de-identifies data using a dedicated platform before using or storing it. This process removes any personal identifiers such that data subjects cannot be identified.

For the most part, Ada uses commercially available LLMs from OpenAI and Anthropic — there’s no personal data or Ada proprietary included in their training. Whenever we do need to fine-tune the LLMs using either end user data, end user metadata, or platform data, it’s always de-identified first, and the customer data used to train the generative models will only serve the specific customer.

How is customer data used in training LLMs at Ada?

Data privacy is a cornerstone of Ada's operations. We align our practices with GDPR guidelines, ensuring that personal data is handled with the utmost care. Our de-identification process leverages advanced technologies to remove sensitive information, such as personally identifiable information (PII) and protected health information (PHI).

We continue to prioritize the privacy and security of our clients' data during the LLM training process. Again, while we use commercially available LLMs, no customer data or proprietary Ada data is involved in their initial training. In cases where fine-tuning is necessary, we use de-identified data specific to each customer, ensuring that personal information is never exposed.

Our agreements with third-party providers stipulate that any data sent for response generation is not stored long-term or used for training purposes. We adhere to a zero data retention policy, ensuring that our customers' data remains confidential and secure.

Final thoughts

As the landscape of AI continues to evolve, the importance of training LLMs effectively is coming into focus. With the right training processes in place, businesses can harness the full potential of AI to enhance customer interactions and drive operational efficiency. It’s a no-brainer.

Ada is proud to be at the forefront of this revolution, ensuring that our clients are equipped with the best tools to meet their customers' needs. And as LLMs get smarter, so do we.

As the old adage goes, “It’s not the size of the model, but how you use it.”