Understanding and emulating human-like intelligence has been a long-standing goal of researchers in various domains leading to the emergence of an inter-disciplinary area called Brain-inspired or Neuromorphic Computing. This research area aims to achieve brain- like intelligence and energy efficiency by understanding and emulating its functionality. In the contemporary world of big data-driven analytics that has fueled ever-increasing demands for computing power, combined with the end of Moore's law scaling, the sheer energy cost of providing exascale-compute capability could soon make it economically and ecologically unsustainable. It, therefore, becomes imperative to explore alternate and more energy-efficient computing paradigms and the human brain, with its 20 W operating power budget, provides the ideal inspiration for building these future computing systems. Spiking Neural Networks (SNNs) are a class of biologically-inspired algorithms designed to mimic natural neural networks found in the brain. Besides playing an important role in biological simulations for neuroscience-related studies, SNNs are recently gaining traction as low- power counterparts of high-precision DNNs. However, in order to build systems with brain-like energy efficiency, we need to capture the functionality of billions of neurons and their communication mechanism in hardware, and this requires innovations at the device/circuit, architecture, algorithm and application levels of the computing stack. Further, efficiently utilizing and incorporating the SNN-led temporal computing paradigm in day-to-day tasks on time-dependent data also requires considerable algorithmic and architectural innovations. With these over-arching princi- ples, this dissertation is aimed at addressing the following architectural and algorithmic issues in SNN inference and training: (i) Investigating the design space of scalable, low- power SNNs by taking a holistic approach spanning the device/circuit levels for designing extremely low power spiking neurons and synapses, architectural solutions for efficient scal- ing of these networks, as well as algorithm-level optimizations for improving the accuracy of SNN models. Further, the SNN characteristics are compared against those of deep/analog neural networks (DNN/ANN), the de-facto drivers of modern AI. Based on this study, a low-power SNN, ANN and hybrid SNN-ANN inference architecture is designed using spintronics-based Magnetic Tunnel Junction (MTJ) devices, while also accounting for the deep interactions between the algorithm and the device. (ii) Training an SNN to solve a problem in a user-level application has so far proved to be challenging due to its discrete and temporal nature. SNNs are, therefore, often converted from high-precision ANNs that can be easily trained using gradient descent-based backpropagation. In this chapter, we study the effectiveness of existing ANN-SNN conversion techniques on sparse event-based data emitted by a neuromorphic camera -- several low-power, hardware-friendly techniques are proposed to boost conversion accuracy and their efficacy is evaluated on a gesture recognition task. (iii) Next, we address the computational challenges involved in train- ing a deep SNN using gradient-descent backpropagation, which is the most effective and scalable technique for training DNNs and SNNs from scratch. By reducing the memory footprint and computational overhead of backpropagation through time-based SNN train- ing, we enable the training and exploration of deeper SNNs on resource-limited hardware platforms including edge devices. Techniques such as re-computation, approximation and a combination thereof, are explored in the context of SNN training. In a nutshell, this dissertation identifies the major compute and memory bottlenecks afflicting SNNs today and proposes efficient algorithm-architecture co-design techniques to alleviate them, with the ultimate goal of facilitating the adaption of energy-efficient Neuromorphic Computing in the mainstream computing paradigm.