Hellow guys, Welcome to my website, and you are watching why llama-3-8B is 8 billion parameters instead of 7?. and this vIdeo is uploaded by Chris Hay at 2024-04-21T14:30:33-07:00. We are pramote this video only for entertainment and educational perpose only. So, I hop you like our website.
| Name | why llama-3-8B is 8 billion parameters instead of 7? |
| Video Uploader | Video From Chris Hay |
| Upload Date | This Video Uploaded At 21-04-2024 21:30:33 |
| Video Discription | llama-3 has ditched it's tokenizer and has instead opted to use the same tokenizer as gpt-4 (tiktoken created by openai), it's even using the same first 100K token vocabulary. In this video chris walks through why Meta has switched tokenizer and the implications on the model sizes, embeddings layer and multi-lingual tokenization. he also runs his tokenizer benchmark and show's how it's more efficient in languages such as japanese repos ------ https://github.com/chrishayuk/embeddings https://github.com/chrishayuk/tokenizer-benchmark |
| Category | Science & Technology |
| Tags | chris hay | chrishayuk | gpt-4 | llama-3 | llama-2 | llama | meta ai | meta | meta llama | meta llama 3 | tiktoken | tokenizer | mistral | mistral ai |
