Using deep learning for sound classification in citizen science: a practical approach with soundless

Abstract

The field of Deep Learning has experienced tremendous growth in recent years, sparking interest among users and researchers. However, deploying Deep Learning models in real-world projects presents significant technical challenges. This Master's Thesis provides a practical approach to designing and constructing a custom Deep Learning model for audio classification, intended for use within the Soundless project—a citizen science platform investigating noise pollution and its impact on human health. The primary objective is to construct a custom model deployable within the Android application of the Soundless project. Different model architectures are explored considering the complexity constraints of deploying Deep Learning models on the edge. The model is built using the TensorFlow framework. Evaluated against the ESC-50 benchmark, the model demonstrates prediction accuracies of over 86%. The model is then integrated into an Android app prototype for testing. A custom dataset is constructed, termed NBAC, comprising 780 audio samples covering 13 distinct classes. NBAC is designed to be aligned with the acoustic context of the Soundless project. The model's performance on NBAC achieves over 90% accuracy. Further, this work investigates various implementation alternatives for utilizing and enhancing the model in a production environment. A centralized improvement approach is proposed, which entails locally storing labeled feature representations of audio samples and training a classifier. Alternatively, a decentralized improvement approach is formulated using Federated Learning. Both strategies, leveraging the custom-designed models, yield promising outcomes. They not only preserve the anticipated accuracies but also facilitate the desired enhancements.