Video: Fine-tuning an Intent Detection Model
Evaluation
Mason del Rosario
,
Founding Machine Learning Engineer
July 11, 2024
This short video demo is a companion to my recent blog post on bootstraping and improving LLM fine-tuning.
The video starts with a quick overview of RAG architectures, which we frame as starting with classification followed by retrieval followed by generation. Then, I show how to use Okareo to help evaluate the classification piece of a RAG agent, and I show how Okareo lets you:
Create synthetic fine-tuning scenarios
Visualize classification metrics on model score cards and evaluations
Perform failure row scenario extraction to improve your fine-tuning set
Want to dive deeper with Okareo? Then you can:
Read our documentation to learn more about evaluation and synthetic data concepts.
Sign-up with Okareo for free.
Try the notebooks used in this demo for yourself (Part 1 and Part 2).
This short video demo is a companion to my recent blog post on bootstraping and improving LLM fine-tuning.
The video starts with a quick overview of RAG architectures, which we frame as starting with classification followed by retrieval followed by generation. Then, I show how to use Okareo to help evaluate the classification piece of a RAG agent, and I show how Okareo lets you:
Create synthetic fine-tuning scenarios
Visualize classification metrics on model score cards and evaluations
Perform failure row scenario extraction to improve your fine-tuning set
Want to dive deeper with Okareo? Then you can:
Read our documentation to learn more about evaluation and synthetic data concepts.
Sign-up with Okareo for free.
Try the notebooks used in this demo for yourself (Part 1 and Part 2).
This short video demo is a companion to my recent blog post on bootstraping and improving LLM fine-tuning.
The video starts with a quick overview of RAG architectures, which we frame as starting with classification followed by retrieval followed by generation. Then, I show how to use Okareo to help evaluate the classification piece of a RAG agent, and I show how Okareo lets you:
Create synthetic fine-tuning scenarios
Visualize classification metrics on model score cards and evaluations
Perform failure row scenario extraction to improve your fine-tuning set
Want to dive deeper with Okareo? Then you can:
Read our documentation to learn more about evaluation and synthetic data concepts.
Sign-up with Okareo for free.
Try the notebooks used in this demo for yourself (Part 1 and Part 2).
This short video demo is a companion to my recent blog post on bootstraping and improving LLM fine-tuning.
The video starts with a quick overview of RAG architectures, which we frame as starting with classification followed by retrieval followed by generation. Then, I show how to use Okareo to help evaluate the classification piece of a RAG agent, and I show how Okareo lets you:
Create synthetic fine-tuning scenarios
Visualize classification metrics on model score cards and evaluations
Perform failure row scenario extraction to improve your fine-tuning set
Want to dive deeper with Okareo? Then you can:
Read our documentation to learn more about evaluation and synthetic data concepts.
Sign-up with Okareo for free.
Try the notebooks used in this demo for yourself (Part 1 and Part 2).
