Hellow guys, Welcome to my website, and you are watching Understanding the Meta Llama 3 Tokenizer | Llama for Developers. and this vIdeo is uploaded by AI at Meta at 2024-06-28T09:01:19-07:00. We are pramote this video only for entertainment and educational perpose only. So, I hop you like our website.
| Name | Understanding the Meta Llama 3 Tokenizer | Llama for Developers |
| Video Uploader | Video From AI at Meta |
| Upload Date | This Video Uploaded At 28-06-2024 16:01:19 |
| Video Discription | Download Meta Llama 3 ➡️ https://go.fb.me/kbpn54 Aston Zhang, research scientist working on Llama at Meta discusses the new tokenizer in Meta Llama 3. He discusses the improvements made to the tokenizer in Meta's latest Llama 3 models. The new tokenizer uses Tiktoken instead of SentencePiece and has a larger vocabulary size of 128k, resulting in better performance on coding, reasoning, and more. The increased vocabulary size allows for more specific and nuanced encoding of inputs, while the higher compression ratio reduces the number of tokens required to represent an input. Additionally, the use of Group Query Attention helps balance out the increased memory and compute needs, resulting in a model that can process larger batches without increasing latency. # Timestamps 00:00 Introduction 00:25 What's new in the Llama 3 tokenizer? 01:58 Vocabulary size and compression ratio 13:01 Performance, efficiency and improving costs 17:46 Recap and resources # Additional Resources • Dive into Deep Learning ebook: https://go.fb.me/ao405f • Getting Started Guide: https://go.fb.me/xucc2m #llama3 #llm #opensource - - - Subscribe: https://www.youtube.com/aiatmeta?sub_confirmation=1 Learn more about our work: https://ai.meta.com # Follow us on social media Follow us on Twitter: https://twitter.com/aiatmeta/ Follow us on LinkedIn: https://www.linkedin.com/showcase/aiatmeta Follow us on Threads: https://threads.net/aiatmeta Follow us on Facebook: https://www.facebook.com/AIatMeta/ Meta AI focuses on bringing the world together by advancing AI, powering meaningful and safe experiences, and conducting open research. |
| Category | Science & Technology |
| Tags | Meta AI | Artificial Intelligence | Meta Llama | Meta Llama 3 | Llama tokenizer | open source AI | LLMs | Large Language Models | Group Query Attention | Tokenizer | Tiktoken | SentencePiece | Vocabulary size | Inference efficiency | State-of-the-art model | Large batch processing | Natural Language Processing (NLP) | Deep Learning | Artificial Intelligence (AI) | Machine Learning | Text Analysis | Language Modeling | AI Latency reduction | AI Compression ratio |
