Download

Note: please report errata (typos etc.) via e-mail to:  issues ~at~ marl-book ~dot~ com

Download PDF

Follow authors to receive updates: Stefano, Filippos, Lukas

Version history:

• 1 November 2023: final book released. New sections on population-based training, agent modelling in deep MARL, no-regret learning, and many other extensions and improvements.
• 25 August 2023: improved sections in chapter 4, divided old chapter 5 into new chapters 5 & 6, new section 6.3.3 on Bayesian learning, added proofs in section 9.5
• 3 July 2023: fixed typos, added references, detail improvements and clarifications
• 29 May 2023: first release during AAMAS 2023 and ICRA 2023

Citation

Stefano V. Albrecht, Filippos Christianos, and Lukas Schäfer. Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press, 2024.

@book{ marl-book,
  author = {Stefano V. Albrecht and Filippos Christianos and Lukas Sch\"afer},
  title = {Multi-Agent Reinforcement Learning: Foundations and Modern Approaches},
  publisher = {MIT Press},
  year = {2024},
  url = {https://www.marl-book.com}
}

Table of Contents

Preface
Summary of Notation
List of Figures

• Chapter 1: Introduction

Part 1: Foundations of Multi-Agent Reinforcement Learning

• Chapter 2: Reinforcement Learning
• Chapter 3: Games: Models of Multi-Agent Interaction
• Chapter 4: Solution Concepts for Games
• Chapter 5: Multi-Agent Reinforcement Learning in Games: First Steps and Challenges
• Chapter 6: Multi-Agent Reinforcement Learning: Foundational Algorithms

Part 2: Multi-Agent Deep Reinforcement Learning: Algorithms and Practice

• Chapter 7: Deep Learning
• Chapter 8: Deep Reinforcement Learning
• Chapter 9: Multi-Agent Deep Reinforcement Learning
• Chapter 10: Multi-Agent Deep RL in Practice
• Chapter 11: Multi-Agent Environments

Appendix A: Surveys on Multi-Agent Reinforcement Learning

Book Codebase

The book comes with a codebase written in the Python programming language, which contains implementations of several MARL algorithms presented in the book. The primary purpose of the codebase is to provide algorithm code that is self-contained and easy to read.

GitHub repository for the book codebase.

Lecture Slides

Lecture slides for the book will be released in Q1 2024.