Renormalization Group Theory, Scaling Laws and Deep Learning

Renormalization Group Theory, Scaling Laws and Deep Learning
Author: Parviz Haggi Mani
Publisher:
Total Pages: 0
Release: 2022
Genre:
ISBN:

Download Renormalization Group Theory, Scaling Laws and Deep Learning Book in PDF, Epub and Kindle

The question of the possibility of intelligent machines is fundamentally intertwined with the machines' ability to reason. Or not. The developments of the recent years point in a completely different direction : What we need is simple, generic but scalable algorithms that can keep learning on their own. This thesis is an attempt to find theoretical explanations to the findings of recent years where empirical evidence has been presented in support of phase transitions in neural networks, power law behavior of various entities, and even evidence of algorithmic universality, all of which are beautifully explained in the context of statistical physics, quantum field theory and statistical field theory but not necessarily in the context of deep learning where no complete theoretical framework is available. Inspired by these developments, and as it turns out, with the overly ambitious goal of providing a solid theoretical explanation of the empirically observed power laws in neu- ral networks, we set out to substantiate the claims that renormalization group theory may be the sought-after theory of deep learning which may explain the above, as well as what we call algorithmic universality.

The Principles of Deep Learning Theory

The Principles of Deep Learning Theory
Author: Daniel A. Roberts
Publisher: Cambridge University Press
Total Pages: 474
Release: 2022-05-26
Genre: Science
ISBN: 1009020927

Download The Principles of Deep Learning Theory Book in PDF, Epub and Kindle

This textbook establishes a theoretical framework for understanding deep learning models of practical relevance. With an approach that borrows from theoretical physics, Roberts and Yaida provide clear and pedagogical explanations of how realistic deep neural networks actually work. To make results from the theoretical forefront accessible, the authors eschew the subject's traditional emphasis on intimidating formality without sacrificing accuracy. Straightforward and approachable, this volume balances detailed first-principle derivations of novel results with insight and intuition for theorists and practitioners alike. This self-contained textbook is ideal for students and researchers interested in artificial intelligence with minimal prerequisites of linear algebra, calculus, and informal probability theory, and it can easily fill a semester-long course on deep learning theory. For the first time, the exciting practical advances in modern artificial intelligence capabilities can be matched with a set of effective principles, providing a timeless blueprint for theoretical research in deep learning.

The Principles of Deep Learning Theory

The Principles of Deep Learning Theory
Author: Daniel A. Roberts
Publisher: Cambridge University Press
Total Pages: 473
Release: 2022-05-26
Genre: Computers
ISBN: 1316519333

Download The Principles of Deep Learning Theory Book in PDF, Epub and Kindle

This volume develops an effective theory approach to understanding deep neural networks of practical relevance.

Machine Learning In Pure Mathematics And Theoretical Physics

Machine Learning In Pure Mathematics And Theoretical Physics
Author: Yang-hui He
Publisher: World Scientific
Total Pages: 418
Release: 2023-06-21
Genre: Computers
ISBN: 1800613717

Download Machine Learning In Pure Mathematics And Theoretical Physics Book in PDF, Epub and Kindle

The juxtaposition of 'machine learning' and 'pure mathematics and theoretical physics' may first appear as contradictory in terms. The rigours of proofs and derivations in the latter seem to reside in a different world from the randomness of data and statistics in the former. Yet, an often under-appreciated component of mathematical discovery, typically not presented in a final draft, is experimentation: both with ideas and with mathematical data. Think of the teenage Gauss, who conjectured the Prime Number Theorem by plotting the prime-counting function, many decades before complex analysis was formalized to offer a proof.Can modern technology in part mimic Gauss's intuition? The past five years saw an explosion of activity in using AI to assist the human mind in uncovering new mathematics: finding patterns, accelerating computations, and raising conjectures via the machine learning of pure, noiseless data. The aim of this book, a first of its kind, is to collect research and survey articles from experts in this emerging dialogue between theoretical mathematics and machine learning. It does not dwell on the well-known multitude of mathematical techniques in deep learning, but focuses on the reverse relationship: how machine learning helps with mathematics. Taking a panoramic approach, the topics range from combinatorics to number theory, and from geometry to quantum field theory and string theory. Aimed at PhD students as well as seasoned researchers, each self-contained chapter offers a glimpse of an exciting future of this symbiosis.

Scaling Laws for Deep Learning

Scaling Laws for Deep Learning
Author: Jonathan Shmuel Rosenfeld
Publisher:
Total Pages: 0
Release: 2021
Genre:
ISBN:

Download Scaling Laws for Deep Learning Book in PDF, Epub and Kindle

Running faster will only get you so far -- it is generally advisable to first understand where the roads lead, then get a car ...

Scaling and Self-Similarity in Physics

Scaling and Self-Similarity in Physics
Author: FRÖHLICH
Publisher: Birkhäuser
Total Pages: 433
Release: 2013-10-24
Genre: Science
ISBN: 9781489967640

Download Scaling and Self-Similarity in Physics Book in PDF, Epub and Kindle

Exact Renormalization Group, The - Proceedings Of The Workshop

Exact Renormalization Group, The - Proceedings Of The Workshop
Author: Alexander Krasnitz
Publisher: World Scientific
Total Pages: 238
Release: 1999-08-13
Genre:
ISBN: 9814543578

Download Exact Renormalization Group, The - Proceedings Of The Workshop Book in PDF, Epub and Kindle

The subject of the exact renormalization group started from pioneering work by Wegner and Houghton in the early seventies and, a decade later, by Polchinski, who formulated the Wilson renormalization group for field theory. In the past decade considerable progress has been made in this field, which includes the development of alternative formulations of the approach and of powerful techniques for solving the exact renormalization group equations, as well as widening of the scope of the exact renormalization group method to include fermions and gauge fields. In particular, two very recent results, namely the manifestly gauge-invariant formulation of the exact renormalization group equation and the proof of the c-theorem in four dimensions, are presented in this volume.

Excluded Volume Effects in Polymer Solutions

Excluded Volume Effects in Polymer Solutions
Author: Lothar Schäfer
Publisher: Springer Science & Business Media
Total Pages: 554
Release: 2012-12-06
Genre: Science
ISBN: 364260093X

Download Excluded Volume Effects in Polymer Solutions Book in PDF, Epub and Kindle

Schäfer gives a concise overview of the static equilibrium properties of polymer solutions. In the first part diagrammatic perturbation theory is derived from scratch. The second part illustrates the basic ideas of the renormalization group (RG). The crucial role of dilation invariance is stressed. The more efficient method of dimensional regularization and minimal subtractions is worked out in part three. The fourth part contains a unified evaluation of the theory to the one loop level. All the important experimental quantities are discussed in detail, and the results are compared extensively to experiment. Empirical methods of data analysis are critically discussed. The final (fifth) part is devoted to extensions of theory. The first three parts of this book may serve as the basis of a course. Parts four and five are hoped to be useful for detailed quantitative evaluations of experiments.

Applied Computational Physics

Applied Computational Physics
Author: Joseph F. Boudreau
Publisher: Oxford University Press
Total Pages: 936
Release: 2018
Genre: Science
ISBN: 0198708637

Download Applied Computational Physics Book in PDF, Epub and Kindle

A textbook that addresses a wide variety of problems in classical and quantum physics. Modern programming techniques are stressed throughout, along with the important topics of encapsulation, polymorphism, and object-oriented design. Scientific problems are physically motivated, solution strategies are developed, and explicit code is presented.

Tensor Network Contractions

Tensor Network Contractions
Author: Shi-Ju Ran
Publisher: Springer Nature
Total Pages: 160
Release: 2020-01-27
Genre: Science
ISBN: 3030344894

Download Tensor Network Contractions Book in PDF, Epub and Kindle

Tensor network is a fundamental mathematical tool with a huge range of applications in physics, such as condensed matter physics, statistic physics, high energy physics, and quantum information sciences. This open access book aims to explain the tensor network contraction approaches in a systematic way, from the basic definitions to the important applications. This book is also useful to those who apply tensor networks in areas beyond physics, such as machine learning and the big-data analysis. Tensor network originates from the numerical renormalization group approach proposed by K. G. Wilson in 1975. Through a rapid development in the last two decades, tensor network has become a powerful numerical tool that can efficiently simulate a wide range of scientific problems, with particular success in quantum many-body physics. Varieties of tensor network algorithms have been proposed for different problems. However, the connections among different algorithms are not well discussed or reviewed. To fill this gap, this book explains the fundamental concepts and basic ideas that connect and/or unify different strategies of the tensor network contraction algorithms. In addition, some of the recent progresses in dealing with tensor decomposition techniques and quantum simulations are also represented in this book to help the readers to better understand tensor network. This open access book is intended for graduated students, but can also be used as a professional book for researchers in the related fields. To understand most of the contents in the book, only basic knowledge of quantum mechanics and linear algebra is required. In order to fully understand some advanced parts, the reader will need to be familiar with notion of condensed matter physics and quantum information, that however are not necessary to understand the main parts of the book. This book is a good source for non-specialists on quantum physics to understand tensor network algorithms and the related mathematics.