Modern statistics often deals with high-dimensional problems that suffer from poor performance guarantees and from the curse of dimensionality. In this thesis, we study how structural assumptions can be used to overcome these difficulties in several estimation problems, spanning three different areas of statistics: shape-constrained estimation, causal discovery, and optimal transport. In the area of shape-constrained estimation, we study the estimation of matrices, first under the assumption of bounded total-variation (TV) and second under the assumption that the underlying matrix is Monge, or supermodular. While the first problem has a long history in image denoising, the latter structure has so far been mainly investigated in the context of computer science and optimization. For TV denoising, we provide fast rates that are adaptive to the underlying edge sparsity of the image, as well as generalizations to other graph structures, including higher-dimensional grid-graphs. For the estimation of Monge matrices, we give near minimax rates for their estimation, including the case where latent permutations act on the rows and columns of the matrix. In the latter case, we also give two computationally efficient and consistent estimators. Moreover, we show how to obtain estimation rates in the related problem of estimating continuous totally positive distributions in 2D. In the area of causal discovery, we investigate a linear cyclic causal model and give an estimator that is near minimax optimal for causal graphs of bounded in-degree. In the area of optimal transport, we introduce the notion of the transport rank of a coupling and provide empirical and theoretical evidence that it can be used to significantly improve rates of estimation of Wasserstein distances and optimal transport plans. Finally, we give near minimax optimal rates for the estimation of smooth optimal transport maps based on a wavelet regularization of the semi-dual objective.