JETSON AI LAB | Self-Learning Llama-3 Voice Agent with Function Calling and Automatic RAG
NVIDIA Developer・2 minutes read
Llama 3 and 53 released new models, with llama 3 trained on more code. Dusty from Nvidia implemented function calling in nanm, added functions like date and time, and enables the bot to provide weather updates and trail recommendations based on user location.
Insights
- Dusty from Nvidia has enabled nanm to call Python or Json functions, insert results into the chat, and continue generating output, showcasing advanced bot capabilities.
- Dusty has integrated functions like date, time, location, and weather using a bot function decorator, enabling personalized responses and real-time weather updates, while also emphasizing future integration with vision language models and vits for enhanced user experience.
Get key ideas from YouTube videos. It’s free
Recent questions
How have llama 3 and 53 improved their models?
llama 3 and 53 have released new major models, with llama 3 being trained on four times more code. These models have been fine-tuned to enhance multimodal vision language capabilities.
What functions has Dusty from Nvidia implemented in nanm?
Dusty has implemented functions like date, time, location, and weather in the backend using a bot function decorator for Rex parsing and Python evaluation in a limited sandbox environment. Additionally, a save function has been added to save extracted text into a database for personalization and memory.
How does the bot from Nvidia invoke weather calls?
The bot successfully invokes weather calls using a simple API like ACU weather, providing real-time weather updates. It can also recommend trails for trail running based on the user's location.
What does Dusty encourage users to experiment with?
Dusty encourages users to experiment with Python functions within the bot and promises further integration with vision language models and vits in the future.
What new features have been added to the bot for personalization?
A save function has been added to save extracted text into a database for personalization and memory, enhancing the user experience with personalized interactions.
Related videos
Andrej Karpathy
[1hr Talk] Intro to Large Language Models
Dwarkesh Patel
Mark Zuckerberg - Llama 3, $10B Models, Caesar Augustus, & 1 GW Datacenters
Lavita AI
Lavita AI AMA
Art of the Problem
ChatGPT: 30 Year History | How AI Learned to Talk
Ticker Symbol: YOU
NVIDIA CEO Jensen Huang Leaves Everyone SPEECHLESS (Supercut)
Summary
00:00
New llama models and Dusty's functions
- New major models have been released by llama 3 and 53, along with fine-tuned multimodal vision language models. llama 3 was trained on four times more code. Dusty from Nvidia has been working on function calling in nanm, enabling the bot to call Python or Json functions, insert results back into the chat, and continue generating output.
- Dusty has implemented functions like date, time, location, and weather in the backend using a bot function decorator for Rex parsing and Python evaluation in a limited sandbox environment. Additionally, a save function has been added to save extracted text into a database for personalization and memory.
- The bot successfully invokes weather calls using a simple API like ACU weather, providing real-time weather updates. It can also recommend trails for trail running based on the user's location. Dusty encourages users to experiment with Python functions and promises further integration with vision language models and vits in the future.
