Doctoral Thesis / Dissertation from the year 2008 in the subject Computer Science - Applied, grade: 1, University of Freiburg (Lehrstuhl für Mustererkennung und Bildverarbeitung), language: English, abstract: In this work, we present the problem of automatic appearance-based facial analysis with machine learning techniques and describe common specific sub-problems like face detection, facial feature detection and face recognition which are the crucial parts of many applications in the context of indexation, surveillance, access-control or human-computer interaction. To tackle this problem, we particularly focus on a technique called Convolutional Neural Network (CNN) which is inspired by biological evidence found in the visual cortex of mammalian brains and which has already been applied to many different classi fication problems. Existing CNN-based methods, like the face detection system proposed by Garcia and Delakis, show that this can be a very effective, efficient and robust approach to non-linear image processing tasks. An important step in many automatic facial analysis applications, e.g. face recognition, is face alignment which tries to translate, scale and rotate the face image such that specific facial features are roughly at predefined positions in the image. We propose an efficient approach to this problem using CNNs and experimentally show its very good performance on difficult test images. We further present a CNN-based method for automatic facial feature detection. The proposed system employs a hierarchical procedure which first roughly localizes the eyes, the nose and the mouth and then refines the result by detecting 10 different facial feature points. The detection rate of this method is 96% for the AR database and 87% for the BioID database tolerating an error of 10% of the inter-ocular distance. Finally, we propose a novel face recognition approach based on a specific CNN architecture learning a non-linear mapping of the image space into a lower-dimensional sub-space where the different classes are more easily separable. We applied this method to several public face databases and obtained better recognition rates than with classical face recognition approaches based on PCA or LDA. We also present a CNN-based method for the binary classification problem of gender recognition with face images and achieve a state-of-the-art accuracy. The results presented in this work show that CNNs perform very well on various facial image processing tasks, such as face alignment, facial feature detection and face recognition and clearly demonstrate that the CNN technique is a versatile, efficient and robust approach for facial image analysis.