Hellow guys, Welcome to my website, and you are watching Open Pretrained Transformers - Susan Zhang | Stanford MLSys #77. and this vIdeo is uploaded by Stanford MLSys Seminars at 2023-03-01T16:36:38-08:00. We are pramote this video only for entertainment and educational perpose only. So, I hop you like our website.
| Name | Open Pretrained Transformers - Susan Zhang | Stanford MLSys #77 |
| Video Uploader | Video From Stanford MLSys Seminars |
| Upload Date | This Video Uploaded At 02-03-2023 00:36:38 |
| Video Discription | Episode 77 of the Stanford MLSys Seminar “Foundation Models Limited Series”! Speaker: Susan Zhang Talk: Trials of developing OPT-175B Abstract: LLM development at scale is an extraordinarily resource-intensive process, requiring compute resources that many do not have access to. The experimentation process will also appear rather haphazard in comparison, given limited compute-time to fully ablate all architectural / hyperparameter choices. In this talk, we will walk through the development lifecycle of OPT-175B, covering infrastructure and training convergence challenges faced at scale, along with methods of addressing these issues going forward. Bio: Susan Zhang is a research engineer at Meta focused on the development of large-scale language models. Previously, she worked on designing photonic chips at Luminous Computing, scaling reinforcement learning systems at OpenAI, and building large-scale data infrastructure systems at Unity Technologies. Check out our website for the schedule: http://mlsys.stanford.edu Join our mailing list to get weekly updates: https://groups.google.com/forum/#!forum/stanford-mlsys-seminars/join |
| Category | Science & Technology |
| Tags | Science & Technology Download MP4 | Science & Technology Download MP3 | Science & Technology Download MP4 360p | Science & Technology Download MP4 480p | Science & Technology Download MP4 720p | Science & Technology Download MP4 1080p |
