Description

The past decade has seen an explosion of machine learning research and applications; especially, deep learning methods have enabled key advances in many
application domains, such as computer vision, speech processing, and game playing.
However, the performance of many machine learning methods is very sensitive
to a plethora of design decisions, which constitutes a considerable barrier for
new users. This is particularly true in the booming field of deep learning, where
human engineers need to select the right neural architectures, training procedures,
regularization methods, and hyperparameters of all of these components in order to
make their networks do what they are supposed to do with sufficient performance.
This process has to be repeated for every application. Even experts are often left
with tedious episodes of trial and error until they identify a good set of choices for
a particular dataset.
The field of automated machine learning (AutoML) aims to make these decisions
in a data-driven, objective, and automated way: the user simply provides data,
and the AutoML system automatically determines the approach that performs best
for this particular application. Thereby, AutoML makes state-of-the-art machine
learning approaches accessible to domain scientists who are interested in applying
machine learning but do not have the resources to learn about the technologies
behind it in detail. This can be seen as a democratization of machine learning: with
AutoML, customized state-of-the-art machine learning is at everyone’s fingertips.
As we show in this book, AutoML approaches are already mature enough to
rival and sometimes even outperform human machine learning experts. Put simply,
AutoML can lead to improved performance while saving substantial amounts of
time and money, as machine learning experts are both hard to find and expensive.
As a result, commercial interest in AutoML has grown dramatically in recent years,
and several major tech companies are now developing their own AutoML systems.
We note, though, that the purpose of democratizing machine learning is served much
better by open-source AutoML systems than by proprietary paid black-box services.
This book presents an overview of the fast-moving field of AutoML. Due
to the community’s current focus on deep learning, some researchers nowadays
mistakenly equate AutoML with the topic of neural architecture search (NAS);
ix
x Preface
but of course, if you’re reading this book, you know that – while NAS is an
excellent example of AutoML – there is a lot more to AutoML than NAS. This
book is intended to provide some background and starting points for researchers
interested in developing their own AutoML approaches, highlight available systems
for practitioners who want to apply AutoML to their problems, and provide an
overview of the state of the art to researchers already working in AutoML. The
book is divided into three parts on these different aspects of AutoML