MICS Seminar: Jean-Pierre Nadal

Categorization, or classification, is a fundamental cognitive ability. Categorization is also one of the main tasks that machine learning (deep learning), is successfully addressing. A well-known perceptual consequence of categorization in humans and other animals, called "categorical perception", is characterized by intra-categorical compression and inter-categorical separation: two elements, close in stimulus space, are perceived closer if they belong to the same category than if they belong to different categories.
Making use of tools from information theory, I will present a set of results on the modeling of the neural basis of categorical perception. I will then show how they provide hints for the analysis of the geometry of internal representations in shallow and deep artificial neural networks. An important outcome is the analysis of the interplay between geometry and noise in the course of learning. This allows us to propose a coherent view of the efficacy of different heuristic practices in the use of dropout, the most popular regularization technique in machine learning.
This talk is based on joint works with Laurent Bonnasse-Gahot (EHESS, CAMS).