The demand for solid-state memories has been increasing rapidly in recent years thanks to the increasing demand from portable electronic devices like smartphones and tablets. Semiconductor non-volatile memories (NVMs), such as NAND and NOR Flash, is the fastest-growing segment in today's solid-state memories. Looking forward, the further scaling of flash memory devices is becoming more challenging: (1) the high electric fields required for the programming and erase operations; (2) the stringent leakage requirements for long term charge storage. While innovations in cell structure and device materials may help extend Flash memory for another couple of technology nodes, alternative memory solutions must be explored for future non-volatile memory applications. There are varieties of emerging memory technologies being researched as possible candidates for next-generation NVM, such as Phase Change Memory (PCM), Spin Torque Transfer Magnetic Random Access Memory (STT-MRAM), and Resistance Switching Random Access Memory (RRAM), etc. Among these candidates, metal oxide RRAM has attracted plenty of attention in the past a few years. It is one of the most promising candidates for future NVM application for its superior scalability, fast speed, low programming current, long endurance, excellent read immunity, and good retention properties. However, in order to meet the practical application requirements, the RRAMs demonstrated to date still need improvements in the following areas: (1) further scaling down the device size; (2) minimize the switching parameters variations; (3) eliminating the forming process. This thesis aims at addressing and elucidating the above challenges and exploring possible solutions through innovations in device materials and structures, new fabrication techniques, and understanding the device physics through comprehensive device characterizations. While RRAM has the potential as a non-volatile memory technology, another emerging application is the use of RRAM as electronic synapse element for hardware implementation of neuromorphic computing. Due to RRAM's multilevel storage capability and low power consumption, it can behave like an analog memory emulating the function of plastic synapses in a neural network. In this thesis, RRAM devices have been investigated as electronic synapses for demonstrating learning rule. To explore the scaling limit of RRAM cells, carbon nanotube (CNT), which is a naturally single-digit-nm material, is utilized as the memory electrode. We report the first AlOx-based resistive switching memory (RRAM) using carbon nanotubes (CNT) as contact electrodes. CNTs with average diameter of 1.2nm effectively localize the conduction filaments (CFs). The Al/AlOx/CNT device successfully switches over 1E4 cycles with less than 5 [microamperes] programming current. Extreme scaling of the device down to 6nm × 6nm is realized by the CNT/AlOx/CNT cross-point structure and 1E4 switching cycles are achieved. Although CNTs have unique properties such as mechanical stiffness, strength, and high thermal and electrical conductivity compared to other materials, it is very challenging to implement CNTs in mass production for its fabrication difficulties and high production cost. A simple process with electron beam lithography (EBL) was used to fabricate devices with active areas from tens of æm to nm along with atomic-layer deposition (ALD). Scaling trends for forming and switching characteristics are presented. For the smallest device with an active area of a few nm in diameter, AC switching endurance of 1E8 cycles with an over 100× resistance window is demonstrated. In addition, multiple resistance states are shown to be stable after 1E5 read cycles and 1E5 seconds baking at 150 °C. Because EBL is limited by its low throughput and not adequate for large-scale memory manufacturing, low-cost and high-throughput block-copolymer self-assembly lithography serves as a promising extension of optical lithography for technology nodes beyond 10 nm. The fabricated bi-layer TiOx/HfOx devices show excellent performance: low forming voltages (~2.5 V) and low switching voltages (1.5 V); good cycle-to-cycle and device-to-device uniformities, reasonable endurance ( 1E7 cycles) and retention property (> 4E4 s @125 °C). Furthermore, self-assembly patterned single-layer HfOx-based RRAM devices is demonstrated with faster switching speed (~50 ns), multi-level storage (2 bits/cell), longer endurance (> 1E9 cycles), half-selected read immunity (~1E9 cycles), good retention (> 1E5 s @ 125 °C) compared to bi-layer TiOx/HfOx device. Despite the recent advancement on the performance of RRAM devices, however, aiming at mass production, one of the most challenging tasks is to address the concern on the broad dispersion of switching parameters, i.e. cycle-to-cycle uniformity within one device and device-to-device uniformity, which are generally observed in the RRAM cells. HfOx/AlOx bi-layer RRAM devices show a better switching uniformity of the switching voltages and resistances than the single-layer HfOx devices. Despite the improvements on the uniformity, the forming process is still unavoidable. We also explore the use of TiOx/HfOx bi-layer device to achieve forming-free and better uniformity in switching parameters at the same time. Forming-free TiOx/HfOx devices are reported with good cycle-to-cycle uniformity in one device and device-to-device uniformity. Over 1E8 switching cycles is observed. TiOx can be used as an effective buffer layer to improve the uniformity in RRAM device. Finally, AlOx-based resistive switching device (RRAM) with multi-level storage capability was investigated for the potential to serve as an electronic synapse device. The Ti/AlOx/TiN memory stack with memory size 0.48 [micrometers×0.48 [micrometers] was fabricated; the resistive layer AlOx was deposited using ALD method. Multi-level resistance states were obtained by varying the compliance current levels or the applied voltage amplitudes during pulse cycling. These resistance states are thermally stable for over 1E5 s at 125 °C. The memory cell resistance can be continuously increased or decreased from each pulse cycle to pulse cycle. More than 1E5 endurance cycles and reading cycles were demonstrated. We further study the potential using this AlOx-based RRAM as electronic synapse device. Around 1% resistance change per pulse cycling was achieved and a plasticity learning rule pulse scheme was proposed to implement the memory device in large-scale hardware neuromorphic computing system.