15min History of Reinforcement Learning and Human Feedback

Hellow guys, Welcome to my website, and you are watching 15min History of Reinforcement Learning and Human Feedback. and this vIdeo is uploaded by Nathan Lambert at 2023-12-15T12:16:17-08:00. We are pramote this video only for entertainment and educational perpose only. So, I hop you like our website.

Info About This Video

Name

Video Uploader

Video From Nathan Lambert

Upload Date

This Video Uploaded At 15-12-2023 20:16:17

Video Discription

A talk mirroring a recent paper of mine, The History and Risks of Reinforcement Learning and Human Feedback (https://arxiv.org/abs/2310.13595)

Slides are here: https://docs.google.com/presentation/d/1kBEKvHBugXE5tSnwp2_kQ1VdV5bOP_LveY6gkTe_Vco/edit?usp=sharing

Paper abstract: Reinforcement learning from human feedback (RLHF) has emerged as a powerful technique to make large language models (LLMs) easier to use and more effective. A core piece of the RLHF process is the training and utilization of a model of human preferences that acts as a reward function for optimization. This approach, which operates at the intersection of many stakeholders and academic disciplines, remains poorly understood. RLHF reward models are often cited as being central to achieving performance, yet very few descriptors of capabilities, evaluations, training methods, or open-source models exist. Given this lack of information, further study and transparency is needed for learned RLHF reward models. In this paper, we illustrate the complex history of optimizing preferences, and articulate lines of inquiry to understand the sociotechnical context of reward models. In particular, we highlight the ontological differences between costs, rewards, and preferences at stake in RLHF's foundations, related methodological tensions, and possible research directions to improve general understanding of how reward models function.

Info About This Video

More Videos

Reinforcement Learning through Human Feedback - EXPLAINED! | RLHF

Reinforcement Learning from Human Feedback: From Zero to chatGPT

RLHF+CHATGPT: What you must know

Stanford CS224N | 2023 | Lecture 10 - Prompting, Reinforcement Learning from Human Feedback

New course with Google Cloud: Reinforcement Learning from Human Feedback (RLHF)

Reinforcement Learning from Human Feedback explained with math derivations and the PyTorch code.

Reinforcement Learning from Human Feedback (RLHF)

RLHF - Reinforcement Learning with Human Feedback

Reinforcement Learning: ChatGPT and RLHF

Direct Preference Optimization: Your Language Model is Secretly a Reward Model | DPO paper explained

Reinforcement Learning with Human Feedback - How to train and fine-tune Transformer Models

Reinforcement Learning from Human Feedback (Natural Language Processing at UT Austin)

RLAIF vs. RLHF: the technology behind Anthropic’s Claude (Constitutional AI Explained)

Direct Preference Optimization: Forget RLHF (PPO)

Mastering RLHF with AWS: A Hands-on Workshop on Reinforcement Learning from Human Feedback

Reinforcement Learning with Human Feedback (RLHF)

John Schulman - Reinforcement Learning from Human Feedback: Progress and Challenges

RLHF: How to Learn from Human Feedback with Reinforcement Learning

Reinforcement Learning from Human Feedback Explained (and RLAIF)