Experiment Cookbook
A tutorial on running experiments with oTree
Preface
This book is a work in progress and is being published online as it is written. The current version is a draft.
We welcome feedback and suggestions. You can provide feedback by opening an issue on the GitHub repository.
You can of course browse the pages as the book is being written, or you can also can refer to the slides of the course Ali gave some time ago.
What is this book about?
This book serves as a guide to programming experiments using oTree, a widely-used Python-based framework for behavioral experiments. It originated as lecture notes for an oTree course and has since been updated to reflect the latest version of oTree.
The book is tailored for researchers, especially those with limited programming experience, who want to design and code their own experiments. It follows a step-by-step approach, simplifying potentially challenging concepts for readers new to programming. Additionally, it includes chapters on advanced topics and a collection of recipes (hence the “cookbook” theme 🙂) to support experienced users.
Who are we?
Ali Seyhun Saral is a data and behavioral scientist at the CEB, United Nations. His research focuses on cooperation, group decision-making, and social learning, utilizing methods from microeconomics, game theory, behavioral and experimental economics, agent-based modeling, and social choice theory. He has held positions as a postdoctoral researcher at the Institute for Advanced Study in Toulouse and the University of Bologna, as well as serving as a lab manager at the Max Planck Institute for Research on Collective Goods and the Bilgi Economics Lab of Istanbul.
For more information, visit his personal website.
Austėja Kažemekaitytė is a postdoctoral research fellow at the Department of Economics and Management at the University of Trento. Her research centers on individual decision-making within the framework of behavioral economics, with current work involving psychophysiological measures in consumer choice as part of the COMFOCUS project. Austėja’s research interests also include vulnerability, the psychology of poverty, and behavioral public policy.
For more information, visit her personal website.
How to use this book?
The book is structured to be as self-contained as possible. While it begins with an introduction to oTree, it assumes some prior knowledge of Python. However, for those new to Python or in need of a refresher, We’ve included an extensive crash course in Python as an appendix. You can start there and then circle back to the beginning if necessary.
The book is divided into three parts:
Tutorials: The first part features tutorial chapters with accompanying oTree concepts, indicated by a green icon 📗. These chapters are interconnected, and I recommend that beginners follow them in sequence.
Recipes: The second part, symbolized by an orange icon 📙, serves as the “cookbook.” Here, I provide examples that address common issues or demonstrate frequently used features.
Appendices:The final section, “the appendix,” includes brief crash courses on related subjects, marked with a ledger icon 📒.
What is in the pipeline?
We have plans to expand this book further. Here is a list of topics that have been requested by readers and are currently in development:
- Chat functionality between the experimenter and participants
- Using bots
- Live pages
- Online recruitment platforms
- Dropout management
- Outsourcing coding tasks
- Version management
How to contribute?
This book was initially envisioned as a collaborative project. However, due to our time constraints, it has remained a primarily two-people team effort. Contributions are still warmly welcomed, and suggestions or feedback can be shared by opening an issue or a pull request on the GitHub repository. If you need to know more about how to contribute to this book, you can refer to the CONTRIBUTING file in the repository.
What are some other resources to learn oTree?
(If you have any other resources you’d like to recommend, please let us know!)
Acknowledgements
Creating this book would not have been possible without the encouragement and assistance of numerous individuals. While it is impossible to mention everyone, a few deserve special acknowledgment.
Ali Seyhun Saral is grateful to Christoph Engel for inspiring the teaching of the oTree course at the Max Planck Institute for Research on Collective Goods and a similar course at the IMPRS Summer School in 2023. I thank the course participants for their engagement and feedback.
Special thanks are due to Oliver Kirchkamp, a role model for teaching technical topics to researchers, and to Matteo Ploner, whose courses and discussions on best practices have been invaluable.
Moreover we thank those who provided helpful feedback and suggestions: Sandro Ambuehl, Robert Böhm, Esra Dağlı, Özgür Gürerk, Ingar Haaland, Laura Hoenig, Konstantinos Ioannidis, Nathan Maddix, Stefan Schmidt, Filippo Toscano, and Tobias Werner.
This book was inspired by two fantastic projects: Causal Inference: The Mixtape and The Turing Way. Appreciation is extended to the authors and contributors of these projects for their pioneering work.
Lastly, gratitude is expressed to Chris Wickens, the creator of oTree, for developing an excellent framework and offering continuous support to the community, as well as to the oTree community at large for their ongoing support and feedback.