Download

Note: this is the pre-print, non-final version of the book. If you spot any issues in the book (typos, errors in definitions, inaccurate descriptions, unclear figures/images, etc.), please report the issue via e-mail to:  issues ~at~ marl-book ~dot~ com

Download pre-print PDF (draft date: 25 August 2023)

To receive updates about new versions, follow us on Twitter: Stefano, Filippos, Lukas

Version history:

• 25 August 2023: improved sections in chapter 4, divided old chapter 5 into new chapters 5 & 6, new section 6.3.3 on Bayesian learning, added proofs in section 9.5
• 3 July 2023: fixed typos, added references, detail improvements and clarifications
• 29 May 2023: first release during AAMAS 2023 and ICRA 2023

Citation

Stefano V. Albrecht, Filippos Christianos, and Lukas Schäfer. Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press, 2023.

@book{ marl-book,
  author = {Stefano V. Albrecht and Filippos Christianos and Lukas Sch\"afer},
  title = {Multi-Agent Reinforcement Learning: Foundations and Modern Approaches},
  publisher = {MIT Press},
  year = {2023},
  url = {https://www.marl-book.com}
}

Table of Contents

Preface
Summary of Notation
List of Figures

• Chapter 1: Introduction

Part 1: Foundations of Multi-Agent Reinforcement Learning

• Chapter 2: Reinforcement Learning
• Chapter 3: Games: Models of Multi-Agent Interaction
• Chapter 4: Solution Concepts for Games
• Chapter 5: Multi-Agent Reinforcement Learning in Games: First Steps and Challenges
• Chapter 6: Multi-Agent Reinforcement Learning: Foundational Algorithms

Part 2: Multi-Agent Deep Reinforcement Learning: Algorithms and Practice

• Chapter 7: Deep Learning
• Chapter 8: Deep Reinforcement Learning
• Chapter 9: Multi-Agent Deep Reinforcement Learning
• Chapter 10: Multi-Agent Deep RL in Practice
• Chapter 11: Multi-Agent Environments

Appendix A: Surveys on Multi-Agent Reinforcement Learning

Book Codebase

The book comes with a codebase written in the Python programming language, which contains implementations of several MARL algorithms presented in the book. The primary purpose of the codebase is to provide algorithm code that is self-contained and easy to read.

GitHub repository for the book codebase.

Lecture Slides

New and updated lecture slides for the book will be released in Q2 2024.

In the meantime, we provide the previous lecture slides used in the Reinforcement Learning course at Edinburgh University, which cover some parts of the book:

• Deep Reinforcement Learning I
• Deep Reinforcement Learning II
• Multi-Agent Reinforcement Learning I
• Multi-Agent Reinforcement Learning II

Please contact the book authors to get access to the Latex source files (uses beamer).