In order to meet the demands of data-hungry applications, modern data storage systems are expected to be increasingly denser. This is a challenging endeavor, and storage engineers are continuously trying to provide novel technologies. However, these new technologies are typically associated with an increase in the number and types of errors, making the goal of securing highly-reliable dense storage devices a tricky challenge. This dissertation focuses on analyzing the errors in addition to providing novel and efficient error correcting coding schemes that are capable of overcoming the aforementioned challenge. In particular, through informed exploitation of the underlying channel characteristics of the storage device being studied, we provide frameworks for systematically generating error correcting codes with mathematical guarantees that offer performance improvements in orders of magnitude relative to the prior state-of-the-art. First, we present a technique to predict the performance of codes given the existence of certain error-prone structures in the graph representation of these codes. Next, we introduce a general framework for the code optimization of non-binary graph-based codes, which works for various interesting channels. Finally, we derive an approach to design high performance spatially-coupled codes particularly for magnetic recording applications. Our frameworks are based on mathematical tools drawn from coding theory and information theory, and rely on advanced mathematical techniques from probability theory, linear algebra, graph theory, combinatorics, and optimization. The proposed frameworks have a vast variety of applications that include both magnetic recording and Flash memory systems. Our frameworks lead to a practical, effective tool for storage engineers to use multi-dimensional storage devices with confidence.