Download

Note: please report errata (typos etc.) via e-mail to:  issues ~at~ marl-book ~dot~ com

Download PDF

Follow authors on X/Twitter to receive updates: Stefano, Filippos, Lukas

Version history and news:

• 3 April 2024: Chinese translated edition of the book will be published by China Machine Press Co., with planned publication in December 2025.
• 29 February 2024: copyedited book. Slight formatting changes, changed from BE to AE spelling, updated survey list in appendix, added index list at end of book.
• 1 November 2023: final pre-print book. New sections on population-based training, agent modelling in deep MARL, no-regret learning, and other extensions and improvements.
• 25 August 2023: improved sections in chapter 4, divided old chapter 5 into new chapters 5 & 6, new section 6.3.3 on Bayesian learning, added proofs in section 9.5.
• 3 July 2023: fixed typos, added references, detail improvements and clarifications.
• 29 May 2023: first book draft release during AAMAS 2023 and ICRA 2023.

Citation

Stefano V. Albrecht, Filippos Christianos, and Lukas Schäfer. Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press, 2024.

@book{ marl-book,
  author = {Stefano V. Albrecht and Filippos Christianos and Lukas Sch\"afer},
  title = {Multi-Agent Reinforcement Learning: Foundations and Modern Approaches},
  publisher = {MIT Press},
  year = {2024},
  url = {https://www.marl-book.com}
}

Table of Contents

Summary of Notation
List of Figures
Preface

• Chapter 1: Introduction

Part 1: Foundations of Multi-Agent Reinforcement Learning

• Chapter 2: Reinforcement Learning
• Chapter 3: Games: Models of Multi-Agent Interaction
• Chapter 4: Solution Concepts for Games
• Chapter 5: Multi-Agent Reinforcement Learning in Games: First Steps and Challenges
• Chapter 6: Multi-Agent Reinforcement Learning: Foundational Algorithms

Part 2: Multi-Agent Deep Reinforcement Learning: Algorithms and Practice

• Chapter 7: Deep Learning
• Chapter 8: Deep Reinforcement Learning
• Chapter 9: Multi-Agent Deep Reinforcement Learning
• Chapter 10: Multi-Agent Deep RL in Practice
• Chapter 11: Multi-Agent Environments

Appendix A: Surveys on Multi-Agent Reinforcement Learning

Book Codebase

The book comes with a codebase written in the Python programming language, which contains implementations of several MARL algorithms presented in the book. The primary purpose of the codebase is to provide algorithm code that is self-contained and easy to read.

GitHub repository for the book codebase.

Lecture Slides

Lecture slides for the book will be released in Q1/2 2024.