Title: Extrapolating expected accuracies for multi-class classification
Abstract: The difficulty of multi-class classification generally increases with the number of classes.
This raises a natural question: Using data from a subset of the classes, can we predict how well a classifier will scale as the number of classes increases?
In this talk, I will present a framework that allows us to analyze this question. Assuming classes are sampled from a population (and some assumptions about the classifiers), we can identify how expected classification accuracy depends on the number of classes (k) via a specific cumulative distribution function. I will present a non-parametric method for estimating this function, which allows easy extrapolation to K>k. I will show empirical results for face-recognition and character-recognition tasks. Finally, I will discuss why the extrapolation problem may be important for neuroscientists, who are increasingly using mutliclass extrapolation accuracy as a proxy for richness of representation.
This talk is based on joint work with Charles Zheng and Rakesh Achanta.