Advanced methods for prototype-based classification

PhD ceremony: Ms. P. Schneider, 14.45 uur, Academiegebouw, Broerstraat 5, Groningen

Thesis: Advanced methods for prototype-based classification

Promotor(s): prof. M. Biehl, prof. N. Petkov

Faculty: Mathematics and Natural Sciences

 

The thesis presents research on Learning Vector Quantization (LVQ). LVQ is a simple and intuitive classification technique: LVQ algorithms learn prototypical vectors for the classes of the considered data set. The prototypes are used for a distance-based classification, i.e. a patter is assigned to the class represented by the closest prototype with respect to a certain distance measure.

The thesis deals with two issues: a new approach for metric adaptation in LVQ is presented and modifications of one specific learning algorithm, namely Robust Soft LVQ, are introduced.

Metric adaptation techniques allow to learn problem specific distance measures from the training data. Since the classifier's decision depends on distances between prototypes and feature vectors, the selected metric is a key issue for the performance of LVQ. We extend the Euclidean distance by a matrix of adaptive weight values. The diagonal entries of the matrix quantify the importance of single features for classification while off-diagonal elements correspond to the relevance of pairs in the classification scheme. Practical applications show that the advanced metric allows to improve the classification of the employed algorithms significantly. Furthermore, the metric parameters provide insight into the nature of the data. The convergence behaviour of the training algorithms is also analysed theoretically.

The proposed modifications of Robust Soft LVQ concern the treatment of the algorithm's hyperparameter, the decision rule for classification, and we present a generalization of the algorithm with respect to vectorial class labels of the input data. The methods are illustrated by means of practical experiments.