Perspectives on the State and Future of Deep Learning - 2023

Authors: Micah Goldblum, Anima Anandkumar, Richard Baraniuk, Tom Goldstein, Kyunghyun Cho, Zachary C Lipton, Melanie Mitchell, Preetum Nakkiran, Max Welling, Andrew Gordon Wilson

What

This paper presents a collection of opinions from prominent machine learning researchers on the current state and future directions of the field, covering topics like interpretability, benchmarking, the limitations of current paradigms, and the role of academia.

Why

This paper offers valuable insights into the minds of leading experts in machine learning, highlighting key challenges and opportunities that are shaping the field’s trajectory. It provides a glimpse into the future of AI research and its potential impact.

How

The authors conducted a survey, presenting a series of open-ended questions to prominent figures in the machine learning community. The interviewees provided their individual perspectives and insights on each topic.

Result

Some key findings include a consensus that current benchmarking practices are inadequate for capturing complex model behaviors like common sense. There’s also debate on the interpretability of deep learning models, with some believing in its eventual achievement and others expressing skepticism. Additionally, researchers emphasize the need to move beyond scaling existing models and focus on developing new learning paradigms with stronger inductive biases.

LF

The paper acknowledges the limitations of current deep learning approaches, particularly concerning data efficiency and the lack of robust theoretical understanding. It suggests exploring alternative architectures, integrating planning into learning algorithms, and emphasizing multimodal learning as promising future directions.

Abstract

The goal of this series is to chronicle opinions and issues in the field of machine learning as they stand today and as they change over time. The plan is to host this survey periodically until the AI singularity paperclip-frenzy-driven doomsday, keeping an updated list of topical questions and interviewing new community members for each edition. In this issue, we probed people’s opinions on interpretable AI, the value of benchmarking in modern NLP, the state of progress towards understanding deep learning, and the future of academia.