The performance of current face recognition systems has reached a satisfactory level under controlled conditions. However, when conditions are not controlled, the performance degrades dramatically. This study considers the challenges introduced by variations in expression, pose, and illumination. Existing methods use either spatial or spectral information. In this study, we propose algorithms that make use of spatial and spectral information simultaneously. Spectral features are extracted from hyperspectral images to represent subjects' spectral characteristics. Spatial features are extracted from one or more bands of a hyperspectral image. For expression-invariant recognition, we extract spectral features from three tissue types. We also design a set of 3D Gabor filters to represent spatial and spectral correlations for use as features. We then apply principal component analysis (PCA) to these features to model expression variation. For pose-invariant recognition, we also extract spectral features from three tissue types. 3D face models are learned using correspondences between a generic 3D model and 2D images. We then use the 3D models to synthesize images under novel poses. Next, we design a set of 2D Gabor filters to extract spatial features. We also apply PCA to correspondences to extract features. For illumination-invariant recognition, a basis is learned that is able to represent a variety of illumination conditions. The images are filtered to alleviate shadow effects and a set of 2D Gabor filters is designed to extract phase information. The effectiveness of the algorithms is demonstrated on a database of 200 subjects. We also propose a method to synthesize images with novel illumination conditions. This method can be used to generate images to test the illumination-invariant recognition algorithm. The proposed method first estimates the illumination effects in an image through filtering. Next, an illumination-normalized image is extracted to represent a subject. Lastly, the normalized representation and the estimated illumination effects are combined to synthesize new images of the subject under the estimated illumination conditions.