The main purpose of this thesis focuses on the investigation of the frequency domain analysis and design approaches for nonlinear damping systems. With the development of modern mechanical and civil engineering structures, the vibration control has become a more and more important problem for the structural system protection. As typical energy dissipation equipments for the structural vibration control purpose, damping devices have been designed and fitted in many modern structural systems. Traditional frequency domain design methods for linear damping devices have been widely studied by engineers and applied in engineering practice, where the system output frequency response is equal to the input spectrum multiplied by the system frequency response function. Recently, nonlinear damping devices have received more and more attentions and been applied in practical engineering systems to overcome the limitations of linear damping devices in the system vibration control. The analysis and design of nonlinear systems, however, are far more complicated than the design of linear systems. The frequency domain design methods for linear systems cannot easily be extended to the nonlinear cases. Traditional frequency domain analysis and design methods for nonlinear systems involve complicated computations, and are, consequently, difficult to be applied in practice. Therefore, more effective frequency domain analysis and design approaches should be developed to facilitate the design of nonlinear damping devices and to satisfy the demand for better vibration performance in practical engineering structural systems. Motivated by this requirement, several new frequency domain analysis and design approaches have been proposed for the analysis of the performance and the design of the characteristic parameters of nonlinear viscous damping devices. The main contributions of the research work can be summarized as follows. (1) Based on the Ritz-Galerkin method, a new method for the evaluation of the transmissibility of nonlinear SDOF viscously damped vibration systems under general harmonic excitations is derived. The effects of damping characteristic parameters on the system transmissibility are investigated. The results reveal that properly designed nonlinear fluid viscous dampers can produce more ideal vibration control over a wide frequency range. (2) The Output Frequency Response Function (OFRF) is a concept recently proposed at Sheffield for the analysis and design of nonlinear systems in the frequency domain. Based on the OFRF, a frequency domain analysis and design approach has been developed to study the impact of additional nonlinear viscous damping devices on the vibration isolation behaviours of MDOF viscously damped vibration systems, and to design the characteristic parameters of additional damping devices for a desired system vibration performance. (3) Based on the OFRF, a new concept called Vibration Power Loss Factor (VPLF) is proposed to evaluate the effects of additional fluid viscous dampers on the vibration control of structural systems subjected to general loading excitations. A novel VPLF and OFRF based approach is then proposed for the design of additional fluid viscous dampers to achieve a desired vibration performance when the structural systems are subject to general loading excitations. The advantages of using different types of additional fluid viscous dampers in structural systems for the vibration control purpose are also investigated. (4) Using the Finite Element (FE) model analyses, the effectiveness of the application of the proposed OFRF and VPLF based frequency domain design approaches in the design of additional fluid viscous dampers for the vibration control in more complicated structural systems has been verified. The frequency domain analysis and design approaches proposed in this thesis provide a significant basis and important guidelines for the analysis and design of a wide class of nonlinear viscously damped engineering structural systems. The results reveal the advantages of additional nonlinear viscous damping devices in the system vibration control and have considerable significance for the design of the damping characteristic parameters to achieve a desired system vibration performance.