I'm so tired but I can't sleep🌊

Standin' on the edge of something much too deep

~ Sarah McLachlan (Lyrics from I Will Remember You)

What is the difference between statistics, machine learning, AI, and data mining?📬

If there are up to 3 variables, it is statistics.If the problem is NP-complete, it is machine learning.If the problem is PSPACE-complete, it is AI.If you don't know what is PSPACE-complete, it is data mining.~ Dan Levin📊

Essentially, all models are wrong, but some are useful

~ George Box

A handful of observations—some erudite, some otherwise—from the trenches of Artificial Intelligence (AI) and Deep Learning👻

I just got kicked out of Barnes and Noble for moving all their classic statistical theory books to the religious section.Do Neural Networks Dream of Strictly Convex Sheep?They call me Dirichlet because all my potential is latent and awaiting allocationBatch algorithms: YOLO*, Online algorithms: Keep Updates and Carry On (*You Only Learn Once)~ Courtesy of @ML_Hipster and @bigdatahipster

Make no mistake about it: Computers process numbers, not symbols. We measure our understanding (and control) by the extent to which we can arithmetize an activity💾

~ Alan J. Perlis

*To get the*lay of the land for the awesome books reviewed in this essay, let's dive right into the pic collage below. It shows the top five deep learning

*foundational*books currently available on the market—and not to worry, yet, about what exactly

*foundational*means in this context. There will be plenty of time to uncover exactly that in this essay, among many other things 😀

In no particular order, then, here are the top five, deep learning foundational books.

The other day, somebody called me something that nobody had

*ever*called me before: an artist. There are, at least, two dimensions to why that praiseful pronouncement startled me—in a very pleasant way of course—and I owe you an explanation for what is turning out to be a rather unusual start to an essay on deep learning. Consider, then, the following dimensions of the artistic startlement:

- "Real Artists Ship": In this memorable phrase from his bestseller entitled
*Linchpin: Are You Indispensable?*(p.101), Seth Godin was reminding us of the importance that shipping a product—be it software, prose, widget, art, or craft—should truly occupy in our collective consciousness 🚢 🚂 🚚 🚅 - We programmers and technologists, too, need inspiration from time to time; inspiration isn't for artists alone. Lest anyone's jaw dropped at that observation—saying in response, hey we're a cerebral and logical bunch—I will to out on a limb and say that as a community we're at
*least*as much artists as we are a bunch of logical, methodical, and goal-driven individuals. To underscore that precise point, I'll remind you that there's an*Art*in Donald Knuth's legendary and eponymous magnum opus entitled*The Art of Computer Programming*(Addison-Wesley), which is actually a book-set composed of four intense volumes 📕 📘 📗 📙

*well*be other dimensions that perhaps registered subliminally—and here I'm reminded of Bollas' haunting phrase when he mentioned about the "unthought known"—but of which I'm not quite aware at the moment. Referring here to what Wallin had in mind when he noted how:

In his final book on attachment, Bowlby quotes Freud who remarked on the characteristic response of the patient who has become aware of something "forgotten": "As a matter of fact I've always known it; only I've never thought of it" (Bowlby, 1988, p. 101). Perhaps Christopher Bollas (1987) who coined the evocative phrase "the unthought known" was reading the same passage from Freud.

How about all that for the beginnings of an essay on...~ David J. Wallin PhD (Attachment in Psychotherapy, Guilford Press)

*Deep Learning*?! Wait a second, lest we get ahead of ourselves 🐎

So I have more than a passing interest in all things deep learning. And casting a glance back at my personal journey, we'll soon dive deep into five deep learning books that have proved immensely helpful to me in grokking this intriguing field at the

*foundational*level. Okay, so what I've got in mind when I use the word

*foundational*is the following composite thinking, all melded together:

With that backgrounder out of the way, let's move into the essay proper—we'll soon be taking an opinionated take at the following deep learningIn the context of deep learning, afoundationalunderstanding is what's gained when you have achieved competency and fluency in the use of the mathematical toolbox that powers thefoundationsof deep learning. Basically, what we're looking at is a conceptual toolbox in which linear algebra is center square—helpful techniques for manipulating groups of numbers simultaneously. The awesomeness of linear algebra has its origins in that it provides structures like vectors and matrices to hold numbers, along with powerful rules to add, subtract, multiply, and divide those numbers. Essentially, it equips you with the wherewithal to slice and dice effortlessly through mountains of data in the quest to identify patterns that interest you ✂ 🌋 🏆

*foundational*books, in turn:

*Linear Algebra: A Modern Introduction*(Brooks Cole) by David Poole.*Good Math: A Geek's Guide to the Beauty of Numbers, Logic, and Computation*(Pragmatic Programmers) by Mark C. Chu-Carroll.*How Mathematicians Think: Using Ambiguity, Contradiction, and Paradox to Create Mathematics*(Princeton University Press) by William Byers.*Matrix Analysis and Applied Linear Algebra*(SIAM: Society for Industrial and Applied Mathematics) by Carl D. Meyer.*Thinking Mathematically 2nd Edition*(Pearson) by J. Mason, L. Burton, K. Stacey.

- Did you find that your experience of reading these foundational books was perhaps
*different*? - Were there
*other*qualities that I did not cover, and which are perhaps the ones that*you*found to be most helpful as you went about learning deep learning at the foundational level? - Did I leave out any of
*your*favorite deep learning foundational books?

**#1**

If you carefully peer into the pic above, I want you to know that it's no accident that the phenomenally good book on deep learning, which was published earlier this year—perhaps

*the*book on the subject, simply entitled

*Deep Learning*(The MIT Press) by Goodfellow, Bengio, and Courville and one that we'll dive into in a

*future*installment in this series of essays—serenely stands guard in the background, while the nuts-and-bolt

*Linear Algebra: A Modern Introduction*(Brooks Cole) by David Poole basks in the limelight with brazen effrontery 🎬

Just to be clear, what I have in mind above in using the all-American phrase (nuts-and-bolt) are precisely those aspects of deep learning that are

*foundational*. And at the core of that foundation—the bedrock if you will—is the subject of linear algebra. At this time, I'm not aware of a book that presents that subject better than

*Linear Algebra: A Modern Introduction*🏆

Frankly, I'm not aware of a kinder, gentler,

Poole shares the gist of this fine book with clarity and enthusiasm in noting that

*and*intelligent approach to introducing yourself to linear algebra than through a study of*Linear Algebra: A Modern Introduction*. This book is all you need to create for yourself a swimmingly good experience, all at your own pace, all on your own time ⏰Poole shares the gist of this fine book with clarity and enthusiasm in noting that

I want students to see linear algebra as an exciting subject and to appreciate its tremendous usefulness. At the same time, I want to help them master the basic concepts and techniques of linear algebra that they will need in other courses, both in mathematics and in other disciplines. I also want students to appreciate the interplay of theoretical, applied, and numerical mathematics that pervades the subject.

To acquaint you better with the tremendously valuable foundational aspects of

*Linear Algebra: A Modern Introduction*, let's have ourselves a peek at its table of contents:Chapter 1. Vectors

Chapter 2. Systems of Linear Equations

Chapter 3. Matrices

Chapter 4. Eigenvalues and Eigenvectors

Chapter 5. Orthogonality

Chapter 6. Vector Spaces

Chapter 7. Distance and Approximation

Appendix AMathematical Notation and Methods of Proof

Appendix BMathematical Induction

Appendix CComplex Numbers

Appendix DPolynomials

Answers to Selected Odd-Numbered Exercises

Look, I'm a sucker for

*adorning*—some might say*embellishing*, though I'll politely beg to differ—my essays with topical quotes, images, and excerpts to create a pleasurable reading experience. So I was pleased and right at home with the format I found in*Linear Algebra: A Modern Introduction*. Here is Poole introducing you to the book in the Preface with this delightful and witty quote:The last thing one knows when writing a book is what to put first 👻

~ Blaise Pascal (Pensées)

And here we have a hilarious quote that adorns the start of the section entitled

*Answers to Selected Odd-Numbered Exercises*, toward the end of*Linear Algebra: A Modern Introduction*Answers are easy. It’s asking the right questions [that’s] hard 😅

~ Doctor Who "The Face of Evil", By Chris Boucher BBC, 1977

Each chapter of

*Linear Algebra: A Modern Introduction*ends with a succinct roundup of key definitions and concepts—here, for example, are the key definitions and concepts that accompany*Chapter 4*. Eigenvalues and Eigenvectors 👣adjoint of a matrix, 276 algebraic multiplicity of an eigenvalue, 294 characteristic equation, 292 characteristic polynomial, 292 cofactor expansion, 266 Cramer’s Rule, 274–275 determinant, 263–265 diagonalizable matrix, 303 eigenvalue, 254 eigenvector, 254 eigenspace, 256 Fundamental Theorem of Invertible Matrices, 296 geometric multiplicity of an eigenvalue, 294 Gerschgorin disk, 319 Gerschgorin’s Disk Theorem, 321 Laplace Expansion Theorem, 266 power method (and its variants), 311–319 properties of determinants, 269–274 similar matrices, 301

Is that good stuff or what, to help you consolidate key concepts? Oh, did I even mention the excellence of the

*applications*of linear algebra you'll find liberally sprinkled across the book? In the words of Poole who is cool and doesn't droolI have not been stingy with the applications: There are many more in the book than can be covered in a single course.However, it is important that students see the impressive range of problems to which linear algebra can be applied.I have included some modern material on finite linear algebra and coding theory that is not normally found in an introductory linear algebra text. There are also several impressive real-world applications of linear algebra and one item of historical, if not practical, interest; these applications are presented as self-contained "vignettes" (italics mine).

I love this book. Don't miss it. It will help build, strengthen,

*and*elevate your foundational skills to pursue deep learning like no other book probably probably could. Again, the uncompromising commitment to help you conceptu aliz e exactly what makes linear algebra tick is evident and writ large throughout the pages of*Linear Algebra: A Modern Introduction.*In the process of reading it, learning ceases to be a chore and instead becomes true pleasure. Yes, it*is*one of those rare books 🏁**#2**

Deep learning or otherwise—actually,

*especially*for deep learning—it will serve us all well to remain mindful of the pervasiveness of math throughout the fabric of technology, science, and especially the math-oriented areas of computer science such as deep learning. So it is no accident that I selected

*Good Math: A Geek's Guide to the Beauty of Numbers, Logic, and Computation*(Pragmatic Programmers) by Mark C. Chu-Carroll to come in at such a high spot on this list 📣

Speaking of the relevance and pervasiveness of math throughout the fabric of science, I can't help but draw parallels to, and resonate with, the marvelously erudite

*and*approachable essay by the Hungarian-American theoretical physicist, engineer, and mathematician Eugene Wigner. It's entitled

*The Unreasonable Effectiveness of Mathematics in the Natural Sciences.*Don't miss it 💎

Meanwhile, reminding ourselves of the adage that there is no royal road to geometry—or to

*deep learning*, for that matter—a sustained study of

*Good Math*will benefit you immensely. Here, then, are some reasons why I'm compelled to pass this judgment on

*Good Math*, starting with the fact that

*Good Math*is accessible to anyone with a basic high school background in math 🎃- To get the most out of it, you need nothing except curiosity 🎈
- It doesn't have to be read cover-to-cover since each chapter is pretty much standalone 🎯
*Good Math*systematically builds your enthusiasm for math instead of boring you to tears 🎓- Trust me on the point above, especially, since you'll need to ramp up your prowess in the area of handling mathematical machinery 📐
- In turn, to the point above, in cascading fashion with the point that preceded
*it*, the barriers to entry are low when it comes to deep learning because you do not—I repeat, you do*not*—need any esoteric math beyond good old linear algebra and optimization techniques to conquer deep learning 🌋 *Good Math*conveys the joy of the mathematical landscape to the inquisitive mind 🏄

*Machine Learning*(ML) course taught by Andrew Ng, then with Stanford University—which will make you do

*deep*dives (pun was totally unintentional) through the ocean of linear algebra, so I have a

*pretty*good idea of what I'm talking about in the bulleted list above 🐘

And while we're talking about Coursera, I'll mention in passing that they've got some of the most well-thought and well-designed courses available online; it was actually my passion for the Scala programming language which, by the way, brought me to Coursera in the first place. And that's how I discovered their offering on ML and other cool stuff. Other courses I took—and for which I also earned a certificate each—include the following, which I can highly recommend, should anyone have an interest in this sort of thing:

*Functional Programming Principles in Scala**Functional Program Design in Scala**Parallel Programming*

*Scala for Machine Learning*(Packt Publishing) by Patrick R. Nicolas. I've made at least six passes through the heft of

*Scala for Machine Learning*—all 400-plus pages in their glory with a highlighter in hand and a trusty Macbook at my side—and

*still*keep finding new nuggets each time I revisit its pages. Go figure 👣

My favorite chapters in

*Good Math*happen to be consecutively located, and they are:

17. Axiomatic Set Theory: Keep the Good, Dump the Bad

17.1 The Axioms of ZFC Set Theory

17.2 The Insanity of Choice

17.3 Why?

18. Models: Using Sets as the LEGOs of the Math World

18.1 Building Natural Numbers

18.2 Models from Models: From Naturals to Integers and Beyond!

19. Transfinite Numbers: Counting and Ordering Infinite SetsLast, but not the least, the honest and intelligent humor that's liberally sprinkled throughout

19.1 Introducing the Transfinite Cardinals

19.2 The Continuum Hypothesis

19.3 Where in Infinity?

*Good Math*is best revealed, I think, right at the outset of the book, in the

*Dedication*section in fact, where we hear Chu-Carroll tell the reader that

Dig in, and you'll see what I'm talking about. This is a book for someone who is looking for inspiration and gentle guidance on mastering mathematics—and don't weThis book is dedicated to the memory of my father, Irving Carroll (zt"l). He set me on the road to becoming a math geek, which is why this book exists. More importantly, he showed me, by example, how to be a mensch: by living honestly, with compassion, humor, integrity, and hard work.

*all*? If that's you, don't miss

*Good Math*🍒

**#3**

You won't find a shred of linear algebra in the next book here, which is a superb one, by the way, and entitled

To drive home the point—though at the risk of sounding like a broken record—I should divulge that I've always been endearingly impressed by the onion metaphor of conceptualizing, the constant revisiting of basics for increasingly profounder conceptualizations. I trace this to my undergrad years when I spent countless hours poring over the mesmerizing pages of the classic MIT textbook entitled

So it's no surprise that a distinguished mathematician and progressive thinker—David Tall, coauthor of

While the whole book is marvelous, I found myself especially reveling in the awesomeness of two standout chapters:

There are so many lovely passages I wished to share from the pages of

*How Mathematicians Think: Using Ambiguity, Contradiction, and Paradox to Create Mathematics*(Princeton University Press) by William Byers. You will, however, find*plenty*in its pages to get you thinking mathematically; that, I think you'll agree, counts for a lot if you wish to equip yourself for tackling—in terms of conceptualizing—the mathematical machinery that underpins linear algebra. Again, absolutely*nothing*to dread: No rocket science going on here 🚀 What you have here is simply stellar guidance on learning how to build beautiful precept upon precept upon precept, till you've built for yourself a grand conceptual edifice of magnificence and wonder ⛺ 🏡 🏠 🏢 🏰To drive home the point—though at the risk of sounding like a broken record—I should divulge that I've always been endearingly impressed by the onion metaphor of conceptualizing, the constant revisiting of basics for increasingly profounder conceptualizations. I trace this to my undergrad years when I spent countless hours poring over the mesmerizing pages of the classic MIT textbook entitled

*Circuits, Signals, and Systems*by William M. Siebert 🚂*Algebraic Number Theory*and*Fermat's Last Theorem*—had this to say about*How Mathematicians Think*. In a rave review of the Byers book we're looking at here, here was Tall's call:This is an important book, one that should cause an epoch-making change in the way we think about mathematics.To help you get situated with the marvelously relevant scope of the contents ofWhile mathematics is often presented as an immutable, absolute science in which theorems can be proved for all time in a platonic sense, here we see the creative, human aspect of mathematics and its paradoxes and conflicts. This has all the hallmarks of a must-read book (italics mine).

*How Mathematicians Think*for foundational mathematics, let's take a peek at its table of contents:INTRODUCTIONI've read

Turning on the Light

SECTION I—THE LIGHT OF AMBIGUITY

CHAPTER 1:Ambiguity in Mathematics

CHAPTER 2:The Contradictory in Mathematics

CHAPTER 3:Paradoxes and Mathematics: Infinity and the Real Numbers

CHAPTER 4:More Paradoxes of Infinity: Geometry, Cardinality, and Beyond

SECTION II—THE LIGHT AS IDEA

CHAPTER 5:The Idea as an Organizing Principle

CHAPTER 6:Ideas, Logic, and Paradox

CHAPTER 7:Great Ideas

SECTION III—THE LIGHT AND THE EYE OF THE BEHOLDER

CHAPTER 8:The Truth of Mathematics

CHAPTER 9:Conclusion: Is Mathematics Algorithmic or Creative?

Notes

Bibliography

Index

*How Mathematicians Think*at least four times since acquiring it a few years ago; it's given me the joy of tackling the heart of conceptual mathematical machinery in a way*no*other book has, imbuing me with renewed vigor to fearlessly pursue the guts of linear algebra—I thank Byers for giving us this gem 🎁While the whole book is marvelous, I found myself especially reveling in the awesomeness of two standout chapters:

- CHAPTER 5:
*The Idea as an Organizing Principle*) 🚥 - CHAPTER 9 (
*Conclusion: Is Mathematics Algorithmic or Creative?*) 🚦

*Deep Thinking: What Mathematics Can Teach Us About the Mind*—is also just plain awesome. I*nearly*included it in this essay as well. Alas, I simply didn't have enough time to do justice to its content, and had to reluctantly jettison it off for now; perhaps it'll have to wait for a future installment in this series of essays on deep learning 🐳There are so many lovely passages I wished to share from the pages of

*How Mathematicians Think*that I had a hard time selecting one. Plus we would've digressed; then again, I remind myself of the name of this blog,*Programming Digressions*😂 So let's look at just this one brief passage in which Byers is proposing to the reader an intriguing question—at the outset of CHAPTER 9 (*Conclusion: Is Mathematics Algorithmic or Creative?*)*to be precise—in a thoughtful style that's the hallmark of this book:*And speaking of the sentence above in italics, that's where you—yes, you, future deep learning expert—come in. If I were you, I wouldn't missI am asking whether it is conceivable that at some time in the future computers could completely take over the show, whether a machine could be programmed to "do" mathematics from start to finish. This would involve (among other activities) examining mathematical situations or situations that potentially could be mathematized, producing data about these situations, generating conjectures, and demonstrating the validity or invalidity of these conjectures.Put in this way, the answer to the question of whether a computer could ever do mathematics is clearly "No!"(The discussion about whether a computer can do mathematics is usually restricted to the last of these activities, namely, demonstrating the validity of certain mathematical statements.) (italics mine)

*How Mathematicians Think*for the life of me. Enough said 🐬**#4**

You have a true workhorse of a book in this one which is entitled

*Matrix Analysis and Applied Linear Algebra*(SIAM: Society for Industrial and Applied Mathematics) by Carl D. Meyer. The guts of linear algebra are laid bare in its pages unlike in any other book on the subject. Yes, I still stand by my earlier assessment that the book by Poole (

*Linear Algebra: A Modern Introduction*) is matchless when it comes to helping you build, strengthen,

*and*elevate your foundational skills to pursue deep learning as far as you wish to—which is why it came in at the top spot in our list—the undeniable fact is that you'll also benefit from being exposed to yet another perspective, or two 🗿

And

*Matrix Analysis and Applied Linear Algebra*is simply stellar in its clarity, rigor,

*and*friendly style of explaining everything. It's got tons and tons of nicely worked-out examples of linear algebra problems. Okay, rest assured, this book is

*not*written in the drive-by style of something like a book in the Schaum's outline series, decent though the books in that series are, filling a useful gap in disseminating knowledge at a mechanical level.

So if your learning style leads you in the direction of seeking an examples-driven approach, you're going to love

*Matrix Analysis and Applied Linear Algebra*💝

As a long-time subscriber to the inimitably brilliant magazine

*MIT Technology Review*(affectionately know in tech circles simply as

*MIT TR*), I had cited a brief excerpt from one of its issues elsewhere, noting how, when you program in Clojure—as you would with any Lisp dialect—you get to define your own domain-specific languages (DSLs) to suit your needs. I had also talked about how there's no conceptual burden whatsoever in that context, because you become a language

*designer*, making your own DSLs through which you encode exactly how your specific business use cases work. But we digress 👻

What I

*did*wish to cite from*MIT TR*now, though, is an incisive take by Nanette Byrnes on the pervasive applications of deep learning to seemingly limitless areas of business. Byrnes, who is*MIT TR*'s senior editor for business, reflects on the state of the union in the AI universe in general, and the deep learning world in particular—in her editorial comments in a fairly recent issue of*MIT TR*(July/August 2017 Vol. 120, No. 4)—noting cogently how 🚀Artificial intelligence, a technology that we believe will drive much of the economic growth over the next few years, is sprinkled throughout the list. The company at the top, Nvidia, has gained expertise in AI and used it to transform itself. Once known as a maker of chips for gaming, it is now a leading player in deep learning and autonomous vehicles. Amazon, No. 3, is on the list again for its ambitions to build an AI-powered store and place the technology at the heart of the home of the future.

And while we're still on the subject of the undeniably brilliant coverage by

*MIT TR*of all things related to the future of technology, basically where technology will take us next—trust me, they are definitely*not*making idle boasts when they claim that the mission of*MIT TR*is to equip its audiences with the intelligence to understand a world shaped by technology—let's listen to an even*more*intriguing pointer on a topic that's decidedly germane to our deep dive into, um, the ocean of deep learning 🐳
So I already hear a few of you groaning—at least those of you who are familiar with my propensity for digressing at the drop of a hat to the exploration of related themes—and muttering under your

breath, though hopefully not taking your blog writer's name in vain, with something like, "Akram, oh Akram, stop digressing

*already*! 😱 😭
And godspeed, invoking a phrase (i.e. "make it snappy") that I recently used elsewhere, I offer to you these prescient words of Tom Simonite as he shares an intriguing pointer to unraveling the significance of deep learning to transforming our evolving worldview of business. So let's now hear Simonite as he opines on how Google thinks it can wrest the cloud computing market away from Amazon by helping companies make use of machine learning. Thus, in a nice wrap-up of how AI and deep learning are propelling Google into yet higher stratospheres of technological prowess, Simonite reminds us in a fairly recent issue of

*MIT TR*(July/August 2017 Vol. 120, No. 4) how 🏄Early in 2015, artificial-intelligence researchers at Google created an obscure piece of software calledTensorFlow. Two years later the tool, which is used in building machine-learning software, underpins many future ambitions of Google and its parent company, Alphabet.TensorFlowmakes it much easier for the company’s engineers to translate new approaches to artificial intelligence into practical code, improving services such as search and the accuracy of speech recognition. But just months afterTensorFlowwas released to Google’s army of coders, the company also began offering it to the world for free.

Is that intriguing or what? If you want to have a piece of the pie, the advice I have for you is to get started with busting your linear algebra chops, which, in turn, will take you as far as you wish to go in the

*land*—wait, I may have got my metaphors messed up a bit here—the*ocean*of deep learning 🌊
Sorry, I had got distracted a bit here while scribbling down these thoughts to share with you, listening as I simultaneously was, to a duet by the singers Lata Mangeshkar and Kishore Kumar, the latter having a notable mention in an essay elsewhere—and who happens to be one of my all-time favorite singers—not to mention a brief segue into the land of Scala and OCaml with a friend 🐫

But I've returned from those digressions. So I was saying... 🏀

*Linear Algebra: A Modern Introduction.*Meyer nicely sums up the essence of his fine book in an introductory section entitled

*Purpose, Gap, and Challenge*by reminding us that

The purpose of this text is to present the contemporary theory and applications of linear algebra to university students studying mathematics, engineering, or applied science at the post-calculus level. Because linear algebra is usually encountered between basic problem solving courses such as calculus or differential equations and more advanced courses that require students to cope with mathematical rigors,the challenge in teaching applied linear algebra is to expose some of the scaffolding while conditioning students to appreciate the utility and beauty of the subject. Effectively meeting this challenge and bridging the inherent gaps between basic and more advanced mathematics are primary goals of this book (italics mine).

To help you get situated with the scope of the contents of

*Matrix Analysis and Applied Linear Algebra*and their relevance to foundational mathematics, let's take a peek at its table of contents 🎯Chapter 1:Linear Equations

Chapter 2:Matrix Algebra

Chapter 3:Vector Spaces

Chapter 4:Norms, Inner Products, and Orthogonality

Chapter 5:Determinants

Chapter 6:Eigenvalues and Eigenvectors

Chapter 7:Perron–Frobenius Theory

While the whole book is rock solid, I found myself especially resonating with the content of the following standout chapter:

5.Norms, Inner Products, and Orthogonality

5.1 Vector Norms

5.2 Matrix Norms

5.3 Inner-Product Spaces

5.4 Orthogonal Vectors

5.5 Gram–Schmidt Procedure

5.6 Unitaryand Orthogonal Matrices

5.7 Orthogonal Reduction

5.8 Discrete Fourier Transform

5.9 Complementary Subspaces

5.10 Range-Nullspace Decomposition

5.11 Orthogonal Decomposition

5.12 Singular Value Decomposition

5.13 Orthogonal Projection

5.14 WhyLeast Squares?

5.15 Angles between Subspaces

Chapter 5 epitomized clarity and grace. However, if the stuff above sounds a bit Greek now, take heart 💔 Remember,

*all*experts were beginners once 👶 👦 👨 👴 💀
Finally, let's wrap up our

*dive*into this rock solid book—wait, wait, I think I got my metaphors messed up big time here—because a dive into a book that's*rock*solid might, um, be rather painful 😂 So let's rewind a bit and rephrase all that and instead say something like: Let's wrap up our*drilling*into this rock solid book—now*that*sounds better, doesn't it?—by checking out what Meyer has to say in another introductory section entitled*Comprehensiveness and Flexibility*. There, the author of*Matrix Analysis and Applied Linear Algebra*reminds us thatA rather comprehensive treatment of linear algebra and its applications is presented and, consequently, the book is not meant to be devoured cover-to-cover in a typical one-semester course. However, the presentation is structured to provide flexibility in topic selection so that the text can be easily adapted to meet the demands of different course outlines without suffering breaks in continuity.Each section contains basic material paired with straightforward explanations, examples, and exercises. But every section also contains a degree of depth coupled with thought-provoking examples and exercises that can take interested students to a higher level.The exercises are formulated not only to make a student think about material from a current section, but they are designed also to pave the way for ideas in future sections in a smooth and often transparent manner(italics mine).

*Matrix Analysis and Applied Linear Algebra*a true workhorse of a book 🐎

**#5**

Last, but certainly not the least, is another remarkable book that will boost your mastery of mathematical thinking in an effortless and delightful way:

*Thinking Mathematically 2nd Edition*(Pearson) by J. Mason, L. Burton, and K. Stacey. I'm not aware of a kinder, gentler approach to introducing yourself to a study of the foundational aspects of mathematics that will, in turn, prepare you to take on linear algebra in earnest—let's also remind ourselves that linear algebra,

*in turn*, is your ticket to the exciting world of deep learning. And I'm doing my bit here to lower the barriers to entry 🚧 🚥 🚦

Let's put it this way:

*Thinking Mathematically*is probably what David Perkins has in mind when, in his indispensable and highly readable book entitled

*Making Education Whole*(Jossey-Bass) he talks about the concept of "learning by wholes". That's the central theme of

*that*book where Perkins tells us in his engaging style that

...this is what learning by wholes is all about. Learning by wholes aims squarely at learning from the lively now.Its goal is to build learning out of endeavors experienced as immediately meaningful and worthwhile—junior versions of the whole game that build toward more sophisticated versions.Its commitment is to leverage features of good naturalistic learning, whether we are talking about Bierstadt, baseball, or Barcelona. Its method is to systematize important features of such learning through the seven principles. Its credo says that good learning is learning from a richly experienced today with tomorrow in view (italics mine).

My academic degrees are from the Massachusetts Institute of Technology (MIT). I was a mathematics major. After I finished the undergraduate work, I continued into a doctoral program, developing an interest in mathematical approaches to artificial intelligence. Artificial intelligence is the study of how to get computers to undertake intelligent activities, such as playing chess or proving mathematical theorems or controlling a robot to do interesting and challenging things.Having made the case, I suppose, for the rightful place of a holistic approach—one marked by profound respect for beauty in all its shapes and forms—I'll add that there is perhaps no better way to introducing the essence ofMy work on artificial intelligence stimulated my interest in thinking and learning in human beings.After finishing my degree, I slid over into the world of cognitive psychology and education, but the why of that is another story. Right now, you can picture me in the foothills of the dissertation range, thinking about what kind of research on artificial intelligence to attempt (italics mine).

*Thinking Mathematically*than by way of the following brief section from the book's

*Introduction*where the authors tells us that

Thinking Mathematicallyis about mathematical processes, and not about any particular branch of mathematics. Our aim is to show how to make a start on any question, how to attack it effectively and how to learn from the experience. Time and effort spent studying these processes of enquiry are wisely invested because doing so can bring you closer to realizing your full potential for mathematical thinking.

Experience in working with students of all ages has convinced us that mathematical thinking can be improved by

- tackling questions conscientiously;
- reflecting on this experience;
- linking feelings with action;
- studying the process of resolving problems; and
- noticing how what you learn fits in with your own experience.

Then, farther along in the book, in a subsection entitled

Don't miss this book! I'll add only this much that if you read

*How to use this book effectively!*to be precise, the authors pointedly tell the reader thatDare I add that, after reading the brief excerpt above, we were all vigorously reassuring ourselves, saying things like the following to ourselves, "Sure,Thinking Mathematicallyis a book to beusedrather than read, so its value depends on how energetically the reader works through the questions posed throughout the text...Probably the single most important lesson to be learned is that being stuck is an honourable state and an essential part of improving thinking.However, to get the most out of being stuck, it is not enough to think for a few minutes and then read on (italics mine, and BTW, "honourable" isnotmisspelled; all I can say as an American 🏈 is that it's just our Aussie friends 🏉 getting carried away 🏃 with spelling things in their own wayward way 😉).

*other*people may get stuck while learning. But*us*technologists—*we*uber hacker types who chew and spit functors, monads, CQRS and stuff as if they were watermelon seeds—*we*never get stuck, now do we ever? YMMV 😇Don't miss this book! I'll add only this much that if you read

*Thinking Mathematically*carefully, you can't help but see the intellectual imprint of the legendary George Pólya—the Stanford University mathematician who enlightened the world with the then-groundbreaking concept of*heuristics*—writ large in its pages. In fact, the authors of this book graciously acknowledge as much, noting in an introductory section entitled*The Power of an Experiential Approach*howThe original book was conceived as an exposition of our own experience as mathematical thinkers, profoundly influenced by the work of George Pólya.

Until we meet next time, in

*our*next essay—hey, I hasten to add that I'm positively not using the royal "our" there and instead simply indicating that this is*our*blog, and not mine alone—I'll slip in this meme edgewise: About the pic above, the one that adorns this book's review, my copy of*Thinking Mathematically*, sleek black color and all, finds itself valiantly propped up against the computer monitor standing across from my Mac keyboard 🔭
So here's the deal: My, um, overly-well-worn paperback copy of

*Thinking Mathematically*literally fell apart at the seams, shall we say, from extended use; the careful reader will note in the pic above how my copy of the book spiral-bound. As they say, some books fall into*disuse*; others into*use*; and yet others into*overuse*🎻In the end, and much as I hinted at, at the very outset, I invite your comments—Having now read the brief take each on the books above 💤

- Do you find that your experience of reading any of these books was different? 🐢
- Did I perhaps not cover some qualities, and which are the ones that you actually found the most helpful as you learned, um, deep learning and its ecosystem? 🌎
- Did I leave out any of your favorite deep learning foundational books? 🚛

This essay somehow got me resonating with two songs—both by Bryan Adams—so I've got lyrics from those songs as well to accompany the pic collage below,

*each*of the two dedicated to my better half, with love 💝

She's not into deep learning. In fact, she is blissfully oblivious to the world of programming; she cringes anytime she sees me sitting in front of computer monitors filled to overflowing with seemingly indecipherable code, symbolic math, consoles with stack traces moving across the monitors in eery unison—look, I'm into Java and Scala after all. My better half—a big fan of Mother Teresa and doing charitable work—is into

*much*deeper things. Among other things, she has leavened my pragmatism by imbuing me with the primacy of humbleness, and helping everyone around us grow and blossom. Darling, this essay is for you 🌿And to you, as you embark on the journey of a lifetime, setting sail on the sea of deep learning, I say with much warmth and friendship,

*Bon voyage*🐬

I'd like to see you, thought I'd let you know

I wanna be with you everyday

Cause I've got a feeling that's beginning to grow

And there's only one thing I can say

I'm ready - to love you

I'm ready - to hold you

I'm ready - I'm ready

Ready as I'm gonna be

She left me a long note when she left me here

Told me that love was hard to find

But baby it's easy and I'll make it clear

That there's only one thing on my mind

I'm ready - to love you

I'm ready - to hold you

I'm ready - I'm ready

Ready as I'm gonna be

~ Bryan Adams (Lyrics fromI'm Ready) 🏃

The biggest lie you ever told - your deepest fear 'bout growin' old

The longest night you ever spent - the angriest letter you never sent

The boy you swore you'd never leave - the one you kissed on new years's eve

The sweetest dream you had last night -- your darkest hour, your hardest fight

I wanna know you - like I know myself

I'm waitin' for you - there ain't no one else

Talk to me baby - scream and shout

I want to know you - inside out

I wanna dig down deep - I wanna lose some sleep

I wanna scream and shout - I wanna know you inside out

I wanna take my time - I wanna know your mind

Ya know there ain't no doubt - I wanna know you inside out

The saddest song you ever heard - the most you said with just one word

The loneliest prayer you ever prayed - the truest vow you ever made

What makes you laugh, what makes you cry what makes you mad, what gets you by

You highest hight, your lowest low - these things I want to know

I wanna know you - like I know myself

I'm waitin' for you - there ain't no one else

Talk to me baby - scream and shout

I want to know you - inside out

I wanna dig down deep - I wanna lose some sleep

I wanna scream and shout - I wanna know you inside out

I wanna take my time - I wanna know your mind

Ya know there ain't no doubt - I wanna know you inside out

I wanna know your soul - I wanna lose control

C'mon n' let it out - I wanna know you inside out

Ya gotta dig down deep - I wanna lose some sleep

I wanna scream and shout - I wanna know you inside out

Tell me everything..

~ Bryan Adams (Lyrics fromInside Out) 🎧