"Massive multiple-input multiple-output (MIMO), which scales up the number of antennas to the order of tens or even hundreds, promises dramatically improved spectral efficiency and link reliability beyond what can be possibly achieved by conventional MIMO. Recently, in an effort to overcome technical issues such as hardware complexity and power consumption confronted by the conventional fully digital implementation in the large-scale antenna regime, a hybrid RF-baseband precoding/combining architecture has been proposed. By limiting the number of RF chains, a stage of RF analog precoding/combining is introduced in combination with baseband digital precoding/combining. In this research, we propose novel strategies for joint RF-baseband optimization assuming two-timescale channel state information (CSI), which consists of instantaneous effective CSI and statistical CSI. Considering the instantaneous effective CSI-based linear baseband precoding/combining for the point-to-point massive MIMO and nonlinear baseband precoding for the multi-user massive MIMO downlink, RF precoding designs are addressed with respect to the performance metrics of mutual information and mean square error (MSE).We first propose to alleviate the channel estimation overhead in point-to-point massive MIMO backhauling by reducing the high-dimensional channel to a low-dimensional RF beam-space. This is motivated by the observation that the channel power tends to be concentrated in the eigen-domain as a result of limited scattering. Through a joint selection of the constitutive RF transmit and receive beams, the loss of channel power can be decreased, and therefore a near-optimal transmission rate can still be achieved under loose and stringent statistical queueing constraints. When the mechanism of hybrid automatic repeat request (ARQ) is enabled, multiple packet retransmissions are likely to experience independent channel realizations. In addition to the spatial diversity, such time diversity presents another possibility for further performance enhancement in the point-to-point massive MIMO. Building upon the knowledge of the previous failed retransmissions, we sequentially optimize the hybrid precoder and combiner for the current packet retransmission, which leads to improved spectral efficiency over those that are oblivious to the diversity in the time dimension.Without receiver cooperation as in the point-to-point MIMO, linear baseband precoding suffers severe power loss relative to the capacity-achieving dirty-paper coding in a fully loaded homogeneous multi-user environment. This is attributed to the fact that the transmission along the eigen-channel with a vanishingly small channel gain consumes most of the transmit power. By taking advantage of the optimization of the extra degrees of freedom in the form of vector perturbation (VP), the proposed use of minimum MSE (MMSE)-VP in hybrid precoding for single-cell massive MIMO remarkably outperforms the perfect CSI-based fully digital linear counterpart. Moreover, factoring the nonlinear perturbation effect in the RF design delivers a superior error performance to the existing solutions that fail to do so. We then explore coordinated nonlinear hybrid precoding between base stations as a spectrally efficient means to realize single-cell processing with universal frequency reuse. In view of the infeasibility to achieve ICI mitigation between cell-edge users through the spatial correlation-based RF precoding, we resort to the tracking of the high-dimensional CSI, as generated by leveraging the effective CSI together with the channel statistics. Specifically, based on such CSI estimates, the proposed RF block diagonalization effectively limits the adverse impact of ICI, and thus creates multiple single-cell environments in the RF beam domain. The low-dimensional RF beam domain also reduces the exposure of the baseband to the channel estimation errors, and hence lessens the resulting performance degradation." --