AILo Talk - Dr. Marco Zullich, University of Groningen

Title: Model compression: how to make your machine learning model smaller and faster

Abstract:

In the last decade, we have witnessed an explosion in the performance and rise to fame of artificial neural networks (ANNs): from the early success of AlexNet in image classification to the very recent and hotly debated large language models like GPT-4 and ChatGPT from OpenAI, and Google’s PALM. This performance, though, comes at an enormous cost for training and inference, due to the very high number of parameters of these models (more than 1 trillion in some cases). Model compression (MC) gathers a variety of techniques aimed at reducing the number of parameters of machine learning (ML) models, ANNs included, with the aim of either reducing the computational requirements or aiding generalization. In this seminar, we will be covering the topic of MC for various ML models: starting from easier examples connected to linear regression and decision trees, we will shift towards state-of-the-art MC techniques for ANNs, while keeping the level of the discussed topics accessible to attendees with basic knowledge of ML.