HEAR 2021 NeurIPS Challenge
Holistic Evaluation of Audio Representations

Abstract

What audio embedding approach generalizes best to a wide range of downstream tasks across a variety of everyday domains without fine-tuning? The aim of the HEAR 2021 NeurIPS challenge is to develop a general-purpose audio representation that provides a strong basis for learning in a wide variety of tasks and scenarios. HEAR 2021 evaluates audio representations using a benchmark suite across a variety of domains, including speech, environmental sound, and music. In the spirit of shared exchange, each participant submitted an audio embedding model following a common API that is general-purpose, open-source, and freely available to use. Twenty-nine models by thirteen external teams were evaluated on nineteen diverse downstream tasks derived from sixteen datasets. Open evaluation code, submitted models and datasets are key contributions, enabling comprehensive and reproducible evaluation, as well as previously impossible longitudinal studies. It still remains an open question whether one single general-purpose audio representation can perform as holistically as the human ear.

Summary

Contributions

The main contributions of HEAR 2021 are:

For detailed information about HEAR 2021, please see our upcoming journal article, to appear in the PMLR Issue on NeurIPS 2021 Competitions.

PMLR will also publish a special issue for HEAR 2021. Submissions to the HEAR PMLR special issue are open until 2022-06-30, even if you did not participate in NeurIPS 2021.

Recordings of team presentations at NeurIPS 2021 are available for viewing.

To stay up-to-date with HEAR, please consider following:

Sponsors

We are proud to announce that HEAR 2021 was sponsored by Google and that all competition evaluations were performed on Google Cloud Platform.

NeurIPS Challenge Organizing Team

Joseph Turian and Jordie Shier and Bhiksha Raj and Björn W. Schuller and Christian James Steinmetz and Colin Malloy and George Tzanetakis and Gissel Velarde and Kirk McNally and Max Henry and Nicolas Pinto and Yonatan Bisk and Gyanendra Das and Humair Raj Khan and Camille Noufi and Dorien Herremans and Eduardo Fonseca and Jesse Engel and Justin Salamon and Philippe Esling and Pranay Manocha and Shinji Watanabe and Zeyu Jin

You can learn more about the committee here.

PMLR Journal Editors

Joseph Turian and Björn W. Schuller and Dorien Herremans and Katrin Kirchhoff and Paola Garcia Perera and Philippe Esling.

You can learn more about the journal editors here.