Saturday, September 16, 2017

Best Deep Learning Books (Foundational)

I'm so tired but I can't sleep
Standin' on the edge of something much too deep
~ Sarah McLachlan (Lyrics from I Will Remember You)

What is the difference between statistics, machine learning, AI, and data mining? ๐Ÿ“ฌ
  • If there are up to 3 variables, it is statistics.
  • If the problem is NP-complete, it is machine learning.
  • If the problem is PSPACE-complete, it is AI.
  • If you don't know what is PSPACE-complete, it is data mining.
~ Dan Levin

Essentially, all models are wrong, but some are useful
~ George Box
A handful of observations—some erudite, some otherwise—from the trenches of Artificial Intelligence (AI) and Deep Learning ๐Ÿ‘ป
  • I just got kicked out of Barnes and Noble for moving all their classic statistical theory books to the religious section. 
  • Do Neural Networks Dream of Strictly Convex Sheep? 
  • They call me Dirichlet because all my potential is latent and awaiting allocation 
  • Batch algorithms: YOLO*, Online algorithms: Keep Updates and Carry On (*You Only Learn Once) 
~ Courtesy of @ML_Hipster and @bigdatahipster 

Make no mistake about it: Computers process numbers, not symbols. We measure our understanding (and control) by the extent to which we can arithmetize an activity ๐Ÿ’พ
~ Alan J. Perlis
To get the lay of the land for the awesome books reviewed in this essay, let's dive right into the pic collage below. It shows the top five deep learning foundational books currently available on the market—and not to worry, yet, about what exactly foundational means in this context. There will be plenty of time to uncover exactly that in this essay, among many other things ๐Ÿ˜€

In no particular order, then, here are the top five, deep learning foundational books.

The other day, somebody called me something that nobody had ever called me before: an artist. There are, at least, two dimensions to why that praiseful pronouncement startled me—in a very pleasant way of course—and I owe you an explanation for what is turning out to be a rather unusual start to an essay on deep learning. Consider, then, the following dimensions of the artistic startlement:
  • "Real Artists Ship": In this memorable phrase from his bestseller entitled Linchpin: Are You Indispensable? (p.101), Seth Godin was reminding us of the importance that shipping a product—be it software, prose, widget, art, or craft—should truly occupy in our collective consciousness ๐Ÿšข ๐Ÿš‚ ๐Ÿšš ๐Ÿš…
  • We programmers and technologists, too, need inspiration from time to time; inspiration isn't for artists alone. Lest anyone's jaw dropped at that observation—saying in response, hey we're a cerebral and logical bunch—I will to out on a limb and say that as a community we're at least as much artists as we are a bunch of logical, methodical, and goal-driven individuals. To underscore that precise point, I'll remind you that there's an Art in Donald Knuth's legendary and eponymous magnum opus entitled The Art of Computer Programming (Addison-Wesley), which is actually a book-set composed of four intense volumes ๐Ÿ“• ๐Ÿ“˜ ๐Ÿ“— ๐Ÿ“™
And to bring closure to that artistic startlement, I'll add that there may well be other dimensions that perhaps registered subliminally—and here I'm reminded of Bollas' haunting phrase when he mentioned about the "unthought known"—but of which I'm not quite aware at the moment. Referring here to what Wallin had in mind when he noted how:
In his final book on attachment, Bowlby quotes Freud who remarked on the characteristic response of the patient who has become aware of something "forgotten": "As a matter of fact I've always known it; only I've never thought of it" (Bowlby, 1988, p. 101). Perhaps Christopher Bollas (1987) who coined the evocative phrase "the unthought known" was reading the same passage from Freud. 
~ David J. Wallin PhD (Attachment in Psychotherapy, Guilford Press)
How about all that for the beginnings of an essay on... Deep Learning?! Wait a second, lest we get ahead of ourselves ๐ŸŽ

So I have more than a passing interest in all things deep learning. And casting a glance back at my personal journey, we'll soon dive deep into five deep learning books that have proved immensely helpful to me in grokking this intriguing field at the foundational level. Okay, so what I've got in mind when I use the word foundational is the following composite thinking, all melded together: 
In the context of deep learning, a foundational understanding is what's gained when you have achieved competency and fluency in the use of the mathematical toolbox that powers the foundations of deep learning. Basically, what we're looking at is a conceptual toolbox in which linear algebra is center square—helpful techniques for manipulating groups of numbers simultaneously. The awesomeness of linear algebra has its origins in that it provides structures like vectors and matrices to hold numbers, along with powerful rules to add, subtract, multiply, and divide those numbers. Essentially, it equips you with the wherewithal to slice and dice effortlessly through mountains of data in the quest to identify patterns that interest you ✂ ๐ŸŒ‹ ๐Ÿ†
With that backgrounder out of the way, let's move into the essay proper—we'll soon be taking an opinionated take at the following deep learning foundational books, in turn:
  1. Linear Algebra: A Modern Introduction (Brooks Cole) by David Poole. 
  2. Good Math: A Geek's Guide to the Beauty of Numbers, Logic, and Computation (Pragmatic Programmers) by Mark C. Chu-Carroll. 
  3. How Mathematicians Think: Using Ambiguity, Contradiction, and Paradox to Create Mathematics (Princeton University Press) by William Byers. 
  4. Matrix Analysis and Applied Linear Algebra (SIAM: Society for Industrial and Applied Mathematics) by Carl D. Meyer. 
  5. Thinking Mathematically 2nd Edition (Pearson) by J. Mason, L. Burton, K. Stacey.
But first, I invite you to start thinking about the comments you may wish to share with other readers: Once you've looked at the brief take each on these books, I invite you to share with other readers your answers to the following three questions:
  • Did you find that your experience of reading these foundational books was perhaps different
  • Were there other qualities that I did not cover, and which are perhaps the ones that you found to be most helpful as you went about learning deep learning at the foundational level?
  • Did I leave out any of your favorite deep learning foundational books?

If you carefully peer into the pic above, I want you to know that it's no accident that the phenomenally good book on deep learning, which was published earlier this year—perhaps the book on the subject, simply entitled Deep Learning (The MIT Press) by Goodfellow,  Bengio, and Courville and one that we'll dive into in a future installment in this series of essays—serenely stands guard in the background, while the nuts-and-bolt Linear Algebra: A Modern Introduction (Brooks Cole) by David Poole basks in the limelight with brazen effrontery ๐ŸŽฌ

Just to be clear, what I have in mind above in using the all-American phrase (nuts-and-bolt) are precisely those aspects of deep learning that are foundational. And at the core of that foundation—the bedrock if you will—is the subject of linear algebra. At this time, I'm not aware of a book that presents that subject better than Linear Algebra: A Modern Introduction ๐Ÿ†

Frankly, I'm not aware of a kinder, gentler, and intelligent approach to introducing yourself to linear algebra than through a study of Linear Algebra: A Modern Introduction. This book is all you need to create for yourself a swimmingly good experience, all at your own pace, all on your own time ⏰

Poole shares the gist of this fine book with clarity and enthusiasm in noting that
I want students to see linear algebra as an exciting subject and to appreciate its tremendous usefulness. At the same time, I want to help them master the basic concepts and techniques of linear algebra that they will need in other courses, both in mathematics and in other disciplines. I also want students to appreciate the interplay of theoretical, applied, and numerical mathematics that pervades the subject.
To acquaint you better with the tremendously valuable foundational aspects of Linear Algebra: A Modern Introduction, let's have ourselves a peek at its table of contents:
Chapter 1. Vectors
Chapter 2. Systems of Linear Equations
Chapter 3. Matrices
Chapter 4. Eigenvalues and Eigenvectors
Chapter 5. Orthogonality
Chapter 6. Vector Spaces
Chapter 7. Distance and Approximation
Appendix A Mathematical Notation and Methods of Proof
Appendix B Mathematical Induction
Appendix C Complex Numbers
Appendix D Polynomials
Answers to Selected Odd-Numbered Exercises
Look, I'm a sucker for adorning—some might say embellishing, though I'll politely beg to differ—my essays with topical quotes, images, and excerpts to create a pleasurable reading experience. So I was pleased and right at home with the format I found in Linear Algebra: A Modern Introduction. Here is Poole introducing you to the book in the Preface with this delightful and witty quote:
The last thing one knows when writing a book is what to put first ๐Ÿ‘ป
~ Blaise Pascal (Pensรฉes)
And here we have a hilarious quote that adorns the start of the section entitled Answers to Selected Odd-Numbered Exercises, toward the end of Linear Algebra: A Modern Introduction
Answers are easy. It’s asking the right questions [that’s] hard ๐Ÿ˜…
~ Doctor Who "The Face of Evil", By Chris Boucher BBC, 1977
Each chapter of Linear Algebra: A Modern Introduction ends with a succinct roundup of key definitions and concepts—here, for example, are the key definitions and concepts that accompany Chapter 4. Eigenvalues and Eigenvectors ๐Ÿ‘ฃ
adjoint of a matrix, 276 algebraic multiplicity of an eigenvalue, 294 characteristic equation, 292 characteristic polynomial, 292 cofactor expansion, 266 Cramer’s Rule, 274–275 determinant, 263–265 diagonalizable matrix, 303 eigenvalue, 254 eigenvector, 254 eigenspace, 256 Fundamental Theorem of Invertible Matrices, 296 geometric multiplicity of an eigenvalue, 294 Gerschgorin disk, 319 Gerschgorin’s Disk Theorem, 321 Laplace Expansion Theorem, 266 power method (and its variants), 311–319 properties of determinants, 269–274 similar matrices, 301
Is that good stuff or what, to help you consolidate key concepts? Oh, did I even mention the excellence of the applications of linear algebra you'll find liberally sprinkled across the book? In the words of Poole who is cool and doesn't drool
I have not been stingy with the applications: There are many more in the book than can be covered in a single course. However, it is important that students see the impressive range of problems to which linear algebra can be applied. I have included some modern material on finite linear algebra and coding theory that is not normally found in an introductory linear algebra text. There are also several impressive real-world applications of linear algebra and one item of historical, if not practical, interest; these applications are presented as self-contained "vignettes" (italics mine).
I love this book. Don't miss it. It will help build, strengthen, and elevate your foundational skills to pursue deep learning like no other book probably probably could. Again, the uncompromising commitment to help you conceptu aliz e exactly what makes linear algebra tick is evident and writ large throughout the pages of Linear Algebra: A Modern Introduction. In the process of reading it, learning ceases to be a chore and instead becomes true pleasure. Yes, it is one of those rare books ๐Ÿ

Deep learning or otherwise—actually, especially for deep learning—it will serve us all well to remain mindful of the pervasiveness of math throughout the fabric of technology, science, and especially the math-oriented areas of computer science such as deep learning. So it is no accident that I selected Good Math: A Geek's Guide to the Beauty of Numbers, Logic, and Computation (Pragmatic Programmers) by Mark C. Chu-Carroll to come in at such a high spot on this list ๐Ÿ“ฃ

Speaking of the relevance and pervasiveness of math throughout the fabric of science, I can't help but draw parallels to, and resonate with, the marvelously erudite and approachable essay by the Hungarian-American theoretical physicist, engineer, and mathematician Eugene Wigner. It's entitled The Unreasonable Effectiveness of Mathematics in the Natural Sciences. Don't miss it ๐Ÿ’Ž

Meanwhile, reminding ourselves of the adage that there is no royal road to geometry—or to deep learning, for that matter—a sustained study of Good Math will benefit you immensely. Here, then, are some reasons why I'm compelled to pass this judgment on Good Math, starting with the fact that
  • Good Math is accessible to anyone with a basic high school background in math ๐ŸŽƒ
  • To get the most out of it, you need nothing except curiosity ๐ŸŽˆ
  • It doesn't have to be read cover-to-cover since each chapter is pretty much standalone ๐ŸŽฏ
  • Good Math systematically builds your enthusiasm for math instead of boring you to tears ๐ŸŽ“
  • Trust me on the point above, especially, since you'll need to ramp up your prowess in the area of handling mathematical machinery ๐Ÿ“
  • In turn, to the point above, in cascading fashion with the point that preceded it, the barriers to entry are low when it comes to deep learning because you do not—I repeat, you do not—need any esoteric math beyond good old linear algebra and optimization techniques to conquer deep learning ๐ŸŒ‹
  • Good Math conveys the joy of the mathematical landscape to the inquisitive mind ๐Ÿ„
To put my thoughts above in context, let's segue a bit. So I earned a certificate online last year from Coursera—specifically a certificate for the Machine Learning (ML) course taught by Andrew Ng, then with Stanford University—which will make you do deep dives (pun was totally unintentional) through the ocean of linear algebra, so I have a pretty good idea of what I'm talking about in the bulleted list above ๐Ÿ˜

And while we're talking about Coursera, I'll mention in passing that they've got some of the most well-thought and well-designed courses available online; it was actually my passion for the Scala programming language which, by the way, brought me to Coursera in the first place. And that's how I discovered their offering on ML and other cool stuff. Other courses I took—and for which I also earned a certificate each—include the following, which I can highly recommend, should anyone have an interest in this sort of thing:
  • Functional Programming Principles in Scala 
  • Functional Program Design in Scala 
  • Parallel Programming
Scala, by the way, is incredibly well-suited for doing ML, but don't get me even started there; suffice it to say that the definitive book on the confluence of Scala and ML is IMHO the one entitled Scala for Machine Learning (Packt Publishing) by Patrick R. Nicolas. I've made at least six passes through the heft of Scala for Machine Learning—all 400-plus pages in their glory with a highlighter in hand and a trusty Macbook at my side—and still keep finding new nuggets each time I revisit its pages. Go figure ๐Ÿ‘ฃ

My favorite chapters in Good Math happen to be consecutively located, and they are:
17. Axiomatic Set Theory: Keep the Good, Dump the Bad
17.1 The Axioms of ZFC Set Theory
17.2 The Insanity of Choice
17.3 Why? 
18. Models: Using Sets as the LEGOs of the Math World
18.1 Building Natural Numbers
18.2 Models from Models: From Naturals to Integers and Beyond! 
19. Transfinite Numbers: Counting and Ordering Infinite Sets
19.1 Introducing the Transfinite Cardinals
19.2 The Continuum Hypothesis
19.3 Where in Infinity?
Last, but not the least, the honest and intelligent humor that's liberally sprinkled throughout Good Math is best revealed, I think, right at the outset of the book, in the Dedication section in fact, where we hear Chu-Carroll tell the reader that
This book is dedicated to the memory of my father, Irving Carroll (zt"l). He set me on the road to becoming a math geek, which is why this book exists. More importantly, he showed me, by example, how to be a mensch: by living honestly, with compassion, humor, integrity, and hard work.
Dig in, and you'll see what I'm talking about. This is a book for someone who is looking for inspiration and gentle guidance on mastering mathematics—and don't we all? If that's you, don't miss Good Math ๐Ÿ’

You won't find a shred of linear algebra in the next book here, which is a superb one, by the way, and entitled How Mathematicians Think: Using Ambiguity, Contradiction, and Paradox to Create Mathematics (Princeton University Press) by William Byers. You will, however, find plenty in its pages to get you thinking mathematically; that, I think you'll agree, counts for a lot if you wish to equip yourself for tackling—in terms of conceptualizing—the mathematical machinery that underpins linear algebra. Again, absolutely nothing to dread: No rocket science going on here ๐Ÿš€ What you have here is simply stellar guidance on learning how to build beautiful precept upon precept upon precept, till you've built for yourself a grand conceptual edifice of magnificence and wonder ⛺ ๐Ÿก ๐Ÿ   ๐Ÿข ๐Ÿฐ

To drive home the point—though at the risk of sounding like a broken record—I should divulge that I've always been endearingly impressed by the onion metaphor of conceptualizing, the constant revisiting of basics for increasingly profounder conceptualizations. I trace this to my undergrad years when I spent countless hours poring over the mesmerizing pages of the classic MIT textbook entitled Circuits, Signals, and Systems by William M. Siebert ๐Ÿš‚

So it's no surprise that a distinguished mathematician and progressive thinker—David Tall, coauthor of Algebraic Number Theory and Fermat's Last Theorem—had this to say about How Mathematicians Think. In a rave review of the Byers book we're looking at here, here was Tall's call:
This is an important book, one that should cause an epoch-making change in the way we think about mathematics. While mathematics is often presented as an immutable, absolute science in which theorems can be proved for all time in a platonic sense, here we see the creative, human aspect of mathematics and its paradoxes and conflicts. This has all the hallmarks of a must-read book (italics mine).
To help you get situated with the marvelously relevant scope of the contents of How Mathematicians Think for foundational mathematics, let's take a peek at its table of contents:
Turning on the Light
CHAPTER 1: Ambiguity in Mathematics
CHAPTER 2: The Contradictory in Mathematics
CHAPTER 3: Paradoxes and Mathematics: Infinity and the Real Numbers
CHAPTER 4: More Paradoxes of Infinity: Geometry, Cardinality, and Beyond
CHAPTER 5: The Idea as an Organizing Principle
CHAPTER 6: Ideas, Logic, and Paradox
CHAPTER 7: Great Ideas
CHAPTER 8: The Truth of Mathematics
CHAPTER 9: Conclusion: Is Mathematics Algorithmic or Creative?
I've read How Mathematicians Think at least four times since acquiring it a few years ago; it's given me the joy of tackling the heart of conceptual mathematical machinery in a way no other book has, imbuing me with renewed vigor to fearlessly pursue the guts of linear algebra—I thank Byers for giving us this gem ๐ŸŽ

While the whole book is marvelous, I found myself especially reveling in the awesomeness of two standout chapters:
  • CHAPTER 5: The Idea as an Organizing Principle) ๐Ÿšฅ
  • CHAPTER 9 (Conclusion: Is Mathematics Algorithmic or Creative?) ๐Ÿšฆ
So I've also got two other books by Byers as well, and one of them—Deep Thinking: What Mathematics Can Teach Us About the Mind—is also just plain awesome. I nearly included it in this essay as well. Alas, I simply didn't have enough time to do justice to its content, and had to reluctantly jettison it off for now; perhaps it'll have to wait for a future installment in this series of essays on deep learning ๐Ÿณ

There are so many lovely passages I wished to share from the pages of How Mathematicians Think that I had a hard time selecting one. Plus we would've digressed; then again, I remind myself of the name of this blog, Programming Digressions ๐Ÿ˜‚ So let's look at just this one brief passage in which Byers is proposing to the reader an intriguing question—at the outset of CHAPTER 9 (Conclusion: Is Mathematics Algorithmic or Creative?) to be precise—in a thoughtful style that's the hallmark of this book:
I am asking whether it is conceivable that at some time in the future computers could completely take over the show, whether a machine could be programmed to "do" mathematics from start to finish. This would involve (among other activities) examining mathematical situations or situations that potentially could be mathematized, producing data about these situations, generating conjectures, and demonstrating the validity or invalidity of these conjectures. Put in this way, the answer to the question of whether a computer could ever do mathematics is clearly "No!" (The discussion about whether a computer can do mathematics is usually restricted to the last of these activities, namely, demonstrating the validity of certain mathematical statements.) (italics mine)
And speaking of the sentence above in italics, that's where you—yes, you, future deep learning expert—come in. If I were you, I wouldn't miss How Mathematicians Think for the life of me. Enough said ๐Ÿฌ

You have a true workhorse of a book in this one which is entitled Matrix Analysis and Applied Linear Algebra (SIAM: Society for Industrial and Applied Mathematics) by Carl D. Meyer. The guts of linear algebra are laid bare in its pages unlike in any other book on the subject. Yes, I still stand by my earlier assessment that the book by Poole (Linear Algebra: A Modern Introduction) is matchless when it comes to helping you build, strengthen, and elevate your foundational skills to pursue deep learning as far as you wish to—which is why it came in at the top spot in our list—the undeniable fact is that you'll also benefit from being exposed to yet another perspective, or two ๐Ÿ—ฟ

And Matrix Analysis and Applied Linear Algebra is simply stellar in its clarity, rigor, and friendly style of explaining everything. It's got tons and tons of nicely worked-out examples of linear algebra problems. Okay, rest assured, this book is not written in the drive-by style of something like a book in the Schaum's outline series, decent though the books in that series are, filling a useful gap in disseminating knowledge at a mechanical level.

So if your learning style leads you in the direction of seeking an examples-driven approach, you're going to love Matrix Analysis and Applied Linear Algebra ๐Ÿ’

As a long-time subscriber to the inimitably brilliant magazine MIT Technology Review (affectionately know in tech circles simply as MIT TR), I had cited a brief excerpt from one of its issues elsewhere, noting how, when you program in Clojure—as you would with any Lisp dialect—you get to define your own domain-specific languages (DSLs) to suit your needs. I had also talked about how there's no conceptual burden whatsoever in that context, because you become a language designer, making your own DSLs through which you encode exactly how your specific business use cases work. But we digress ๐Ÿ‘ป

What I did wish to cite from MIT TR now, though, is an incisive take by Nanette Byrnes on the pervasive applications of deep learning to seemingly limitless areas of business. Byrnes, who is MIT TR's senior editor for business, reflects on the state of the union in the AI universe in general, and the deep learning world in particular—in her editorial comments in a fairly recent issue of MIT TR (July/August 2017 Vol. 120, No. 4)—noting cogently how ๐Ÿš€
Artificial intelligence, a technology that we believe will drive much of the economic growth over the next few years, is sprinkled throughout the list. The company at the top, Nvidia, has gained expertise in AI and used it to transform itself. Once known as a maker of chips for gaming, it is now a leading player in deep learning and autonomous vehicles. Amazon, No. 3, is on the list again for its ambitions to build an AI-powered store and place the technology at the heart of the home of the future.
And while we're still on the subject of the undeniably brilliant coverage by MIT TR of all things related to the future of technology, basically where technology will take us next—trust me, they are definitely not making idle boasts when they claim that the mission of MIT TR is to equip its audiences with the intelligence to understand a world shaped by technology—let's listen to an even more intriguing pointer on a topic that's decidedly germane to our deep dive into, um, the ocean of deep learning ๐Ÿณ

So I already hear a few of you groaning—at least those of you who are familiar with my propensity for digressing at the drop of a hat to the exploration of related themes—and muttering under your 
breath, though hopefully not taking your blog writer's name in vain, with something like, "Akram, oh Akram, stop digressing already! ๐Ÿ˜ฑ ๐Ÿ˜ญ

And godspeed, invoking a phrase (i.e. "make it snappy") that I recently used elsewhere, I offer to you these prescient words of Tom Simonite as he shares an intriguing pointer to unraveling the significance of deep learning to transforming our evolving worldview of business. So let's now hear Simonite as he opines on how Google thinks it can wrest the cloud computing market away from Amazon by helping companies make use of machine learning. Thus, in a nice wrap-up of how AI and deep learning are propelling Google into yet higher stratospheres of technological prowess, Simonite reminds us in a fairly recent issue of MIT TR (July/August 2017 Vol. 120, No. 4) how ๐Ÿ„
Early in 2015, artificial-intelligence researchers at Google created an obscure piece of software called TensorFlow. Two years later the tool, which is used in building machine-learning software, underpins many future ambitions of Google and its parent company, Alphabet. TensorFlow makes it much easier for the company’s engineers to translate new approaches to artificial intelligence into practical code, improving services such as search and the accuracy of speech recognition. But just months after TensorFlow was released to Google’s army of coders, the company also began offering it to the world for free.
Is that intriguing or what? If you want to have a piece of the pie, the advice I have for you is to get started with busting your linear algebra chops, which, in turn, will take you as far as you wish to go in the land—wait, I may have got my metaphors messed up a bit here—the ocean of deep learning ๐ŸŒŠ

Sorry, I had got distracted a bit here while scribbling down these thoughts to share with you, listening as I simultaneously was, to a duet by the singers Lata Mangeshkar and Kishore Kumar, the latter having a notable mention in an essay elsewhere—and who happens to be one of my all-time favorite singers—not to mention a brief segue into the land of Scala and OCaml with a friend ๐Ÿซ

But I've returned from those digressions. So I was saying... ๐Ÿ€

Again, the uncompromising commitment to help the reader conceptualize what exactly makes linear algebra tick is evident and writ large throughout the pages of Linear Algebra: A Modern Introduction. Meyer nicely sums up the essence of his fine book in an introductory section entitled Purpose, Gap, and Challenge by reminding us that
The purpose of this text is to present the contemporary theory and applications of linear algebra to university students studying mathematics, engineering, or applied science at the post-calculus level. Because linear algebra is usually encountered between basic problem solving courses such as calculus or differential equations and more advanced courses that require students to cope with mathematical rigors, the challenge in teaching applied linear algebra is to expose some of the scaffolding while conditioning students to appreciate the utility and beauty of the subject. Effectively meeting this challenge and bridging the inherent gaps between basic and more advanced mathematics are primary goals of this book (italics mine).
To help you get situated with the scope of the contents of Matrix Analysis and Applied Linear Algebra and their relevance to foundational mathematics, let's take a peek at its table of contents ๐ŸŽฏ
Chapter 1: Linear Equations
Chapter 2: Matrix Algebra
Chapter 3: Vector Spaces
Chapter 4: Norms, Inner Products, and Orthogonality
Chapter 5: Determinants
Chapter 6: Eigenvalues and Eigenvectors
Chapter 7: Perron–Frobenius Theory
While the whole book is rock solid, I found myself especially resonating with the content of the following standout chapter:
5. Norms, Inner Products, and Orthogonality
5.1 Vector Norms
5.2 Matrix Norms
5.3 Inner-Product Spaces
5.4 Orthogonal Vectors
5.5 Gram–Schmidt Procedure
5.6 Unitaryand Orthogonal Matrices
5.7 Orthogonal Reduction
5.8 Discrete Fourier Transform
5.9 Complementary Subspaces
5.10 Range-Nullspace Decomposition
5.11 Orthogonal Decomposition
5.12 Singular Value Decomposition
5.13 Orthogonal Projection
5.14 WhyLeast Squares?
5.15 Angles between Subspaces
Chapter 5 epitomized clarity and grace. However, if the stuff above sounds a bit Greek now, take heart ๐Ÿ’” Remember, all experts were beginners once ๐Ÿ‘ถ ๐Ÿ‘ฆ ๐Ÿ‘จ ๐Ÿ‘ด ๐Ÿ’€

Finally, let's wrap up our dive into this rock solid book—wait, wait, I think I got my metaphors messed up big time here—because a dive into a book that's rock solid might, um, be rather painful ๐Ÿ˜‚ So let's rewind a bit and rephrase all that and instead say something like: Let's wrap up our drilling into this rock solid book—now that sounds better, doesn't it?—by checking out what Meyer has to say in another introductory section entitled Comprehensiveness and Flexibility. There, the author of Matrix Analysis and Applied Linear Algebra reminds us that
A rather comprehensive treatment of linear algebra and its applications is presented and, consequently, the book is not meant to be devoured cover-to-cover in a typical one-semester course. However, the presentation is structured to provide flexibility in topic selection so that the text can be easily adapted to meet the demands of different course outlines without suffering breaks in continuity. Each section contains basic material paired with straightforward explanations, examples, and exercises. But every section also contains a degree of depth coupled with thought-provoking examples and exercises that can take interested students to a higher level. The exercises are formulated not only to make a student think about material from a current section, but they are designed also to pave the way for ideas in future sections in a smooth and often transparent manner (italics mine).
And there you have it—much as I said at the outset, you have in Matrix Analysis and Applied Linear Algebra a true workhorse of a book ๐ŸŽ

Last, but certainly not the least, is another remarkable book that will boost your mastery of mathematical thinking in an effortless and delightful way: Thinking Mathematically 2nd Edition (Pearson) by J. Mason, L. Burton, and K. Stacey. I'm not aware of a kinder, gentler approach to introducing yourself to a study of the foundational aspects of mathematics that will, in turn, prepare you to take on linear algebra in earnest—let's also remind ourselves that linear algebra, in turn, is your ticket to the exciting world of deep learning. And I'm doing my bit here to lower the barriers to entry ๐Ÿšง ๐Ÿšฅ ๐Ÿšฆ

Let's put it this way: Thinking Mathematically is probably what David Perkins has in mind when, in his indispensable and highly readable book entitled Making Education Whole (Jossey-Bass) he talks about the concept of "learning by wholes". That's the central theme of that book where Perkins tells us in his engaging style that
...this is what learning by wholes is all about. Learning by wholes aims squarely at learning from the lively now. Its goal is to build learning out of endeavors experienced as immediately meaningful and worthwhile—junior versions of the whole game that build toward more sophisticated versions. Its commitment is to leverage features of good naturalistic learning, whether we are talking about Bierstadt, baseball, or Barcelona. Its method is to systematize important features of such learning through the seven principles. Its credo says that good learning is learning from a richly experienced today with tomorrow in view (italics mine).
If you buy that philosophy—and frankly, it's an eminently sensible one—notwithstanding that Perkins, the author of the aforesaid philosophy, has impeccable credentials. In fact, in his own words, Perkins shares this intriguing insight by way of background, telling the reader how
My academic degrees are from the Massachusetts Institute of Technology (MIT). I was a mathematics major. After I finished the undergraduate work, I continued into a doctoral program, developing an interest in mathematical approaches to artificial intelligence. Artificial intelligence is the study of how to get computers to undertake intelligent activities, such as playing chess or proving mathematical theorems or controlling a robot to do interesting and challenging things. My work on artificial intelligence stimulated my interest in thinking and learning in human beings. After finishing my degree, I slid over into the world of cognitive psychology and education, but the why of that is another story. Right now, you can picture me in the foothills of the dissertation range, thinking about what kind of research on artificial intelligence to attempt (italics mine).
Having made the case, I suppose, for the rightful place of a holistic approach—one marked by profound respect for beauty in all its shapes and forms—I'll add that there is perhaps no better way to introducing the essence of Thinking Mathematically than by way of the following brief section from the book's Introduction where the authors tells us that
Thinking Mathematically is about mathematical processes, and not about any particular branch of mathematics. Our aim is to show how to make a start on any question, how to attack it effectively and how to learn from the experience. Time and effort spent studying these processes of enquiry are wisely invested because doing so can bring you closer to realizing your full potential for mathematical thinking.  
Experience in working with students of all ages has convinced us that mathematical thinking can be improved by
  • tackling questions conscientiously;
  • reflecting on this experience;
  • linking feelings with action;
  • studying the process of resolving problems; and
  • noticing how what you learn fits in with your own experience.
Then, farther along in the book, in a subsection entitled How to use this book effectively! to be precise, the authors pointedly tell the reader that
Thinking Mathematically is a book to be used rather than read, so its value depends on how energetically the reader works through the questions posed throughout the text... Probably the single most important lesson to be learned is that being stuck is an honourable state and an essential part of improving thinking. However, to get the most out of being stuck, it is not enough to think for a few minutes and then read on (italics mine, and BTW, "honourable" is not misspelled; all I can say as an American ๐Ÿˆ  is that it's just our Aussie friends ๐Ÿ‰ getting carried away ๐Ÿƒ with spelling things in their own wayward way ๐Ÿ˜‰).
Dare I add that, after reading the brief excerpt above, we were all vigorously reassuring ourselves, saying things like the following to ourselves, "Sure, other people may get stuck while learning. But us technologists—we uber hacker types who chew and spit functors, monads, CQRS and stuff as if they were watermelon seedswe never get stuck, now do we ever? YMMV ๐Ÿ˜‡

Don't miss this book! I'll add only this much that if you read Thinking Mathematically carefully, you can't help but see the intellectual imprint of the legendary George Pรณlya—the Stanford University mathematician who enlightened the world with the then-groundbreaking concept of heuristics—writ large in its pages. In fact, the authors of this book graciously acknowledge as much, noting in an introductory section entitled The Power of an Experiential Approach how
The original book was conceived as an exposition of our own experience as mathematical thinkers, profoundly influenced by the work of George Pรณlya.
Until we meet next time, in our next essay—hey, I hasten to add that I'm positively not using the royal "our" there and instead simply indicating that this is our blog, and not mine alone—I'll slip in this meme edgewise: About the pic above, the one that adorns this book's review, my copy of Thinking Mathematically, sleek black color and all, finds itself valiantly propped up against the computer monitor standing across from my Mac keyboard ๐Ÿ”ญ

So here's the deal: My, um, overly-well-worn paperback copy of Thinking Mathematically literally fell apart at the seams, shall we say, from extended use; the careful reader will note in the pic above how my copy of the book spiral-bound. As they say, some books fall into disuse; others into use; and yet others into overuse ๐ŸŽป 

In the end, and much as I hinted at, at the very outset, I invite your comments—Having now read the brief take each on the books above ๐Ÿ’ค
  • Do you find that your experience of reading any of these books was different? ๐Ÿข
  • Did I perhaps not cover some qualities, and which are the ones that you actually found the most helpful as you learned, um, deep learning and its ecosystem? ๐ŸŒŽ
  • Did I leave out any of your favorite deep learning foundational books? ๐Ÿš›
My hope is that these brief vignettes will help you in your journey to grokking deep learning—I leave you with a fleetingly brief pic collage below this time; a more elaborate pic collage awaits you in a future essay just around the corner ๐Ÿ’

This essay somehow got me resonating with two songs—both by Bryan Adams—so I've got lyrics from those songs as well to accompany the pic collage below, each of the two dedicated to my better half, with love ๐Ÿ’ 

She's not into deep learning. In fact, she is blissfully oblivious to the world of programming; she cringes anytime she sees me sitting in front of computer monitors filled to overflowing with seemingly indecipherable code, symbolic math, consoles with stack traces moving across the monitors in eery unison—look, I'm into Java and Scala after all. My better half—a big fan of Mother Teresa and doing charitable work—is into much deeper things. Among other things, she has leavened my pragmatism by imbuing me with the primacy of humbleness, and helping everyone around us grow and blossom. Darling, this essay is for you ๐ŸŒฟ

And to you, as you embark on the journey of a lifetime, setting sail on the sea of deep learning, I say with much warmth and friendship, Bon voyage ๐Ÿฌ
I'd like to see you, thought I'd let you know
I wanna be with you everyday
Cause I've got a feeling that's beginning to grow
And there's only one thing I can say 
I'm ready - to love you
I'm ready - to hold you
I'm ready - I'm ready
Ready as I'm gonna be 
She left me a long note when she left me here
Told me that love was hard to find
But baby it's easy and I'll make it clear
That there's only one thing on my mind 
I'm ready - to love you
I'm ready - to hold you
I'm ready - I'm ready
Ready as I'm gonna be
~ Bryan Adams (Lyrics from I'm Ready) ๐Ÿƒ
The biggest lie you ever told - your deepest fear 'bout growin' old
The longest night you ever spent - the angriest letter you never sent
The boy you swore you'd never leave - the one you kissed on new years's eve
The sweetest dream you had last night -- your darkest hour, your hardest fight 
I wanna know you - like I know myself
I'm waitin' for you - there ain't no one else
Talk to me baby - scream and shout
I want to know you - inside out
I wanna dig down deep - I wanna lose some sleep
I wanna scream and shout - I wanna know you inside out
I wanna take my time - I wanna know your mind
Ya know there ain't no doubt - I wanna know you inside out 
The saddest song you ever heard - the most you said with just one word
The loneliest prayer you ever prayed - the truest vow you ever made
What makes you laugh, what makes you cry what makes you mad, what gets you by
You highest hight, your lowest low - these things I want to know 
I wanna know you - like I know myself
I'm waitin' for you - there ain't no one else
Talk to me baby - scream and shout
I want to know you - inside out
I wanna dig down deep - I wanna lose some sleep
I wanna scream and shout - I wanna know you inside out
I wanna take my time - I wanna know your mind
Ya know there ain't no doubt - I wanna know you inside out 
I wanna know your soul - I wanna lose control
C'mon n' let it out - I wanna know you inside out
Ya gotta dig down deep - I wanna lose some sleep
I wanna scream and shout - I wanna know you inside out
Tell me everything..
~ Bryan Adams (Lyrics from Inside Out) ๐ŸŽง

Monday, September 11, 2017

Best Algorithms Books (Part 2)

He had read much, if one considers his long life; but his contemplation was much more than his reading. He was wont to say that if he had read as much as other men he should have known no more than other men (italics mine)
~ Isaac Asimov (my late Dad was a Sci Fi buff, and Asimov was one of his fav authors)
Today is September 11, and as Americans, what better way to put our patriotism into practice than by ๐Ÿ‡บ๐Ÿ‡ธ Helping those affected by Hurricane Irma (Florida) and ๐Ÿ‡บ๐Ÿ‡ธ continue helping those affected by Hurricane Harvey (Texas)
This was the first graph problem I ever posed myself and solved. The amazing thing was that I didn't publish it. It was not amazing at the time. At the time, algorithms were hardly considered a scientific topic.
~ Edsger W. Dijkstra (
as quoted by Dennis Shasha and Cathy Lazere in Out of their Minds: The Lives and Discoveries of 15 Great Computer Scientists (Copernicus))  
You’ve Come a Long Way, Baby (Or Have You?)
~ Ruth Rosen 
A lot of careers are based on a perceived need and people find ways to solve critical problemsmedical careers, for examplewhile while other careers like computer science are chosen because of the kind of mind structure that you develop as a child.
~ Donald Knuth
I refuse to accept the view that mankind is so tragically bound to the starless midnight of racism and war that the bright daybreak of peace and brotherhood can never become a reality... I believe that unarmed truth and unconditional love will have the final word.
~ Martin Luther King, Jr.
Sometimes I will stay up in my room for a day trying to get two sentences that will flow, that will just seem as if they were always there.
~ Maya Angelou, 
in Conversations with Maya Angelou, ed. Jeffrey M. Elliot (Jackson, MS: University Press of Mississippi, 1989), p.59 
First, thank you all so much for the warm reception that you've given to the first installment in this series of reviews of algorithms books, by way of an earlier essay—post really, though I've become enamored of calling them essays, after Paul Graham's trendy essays—which was entitled Best Algorithms Books (Part 1) ๐ŸŽ‰

In this essay, which has that trailing "Part 2" in its name, I'm making good on my promise—one that I made toward the end of the previous installment in this seriesof a second installment. In a tangential sort of way, the numbering I've used here for the five books (reflecting their relative ranking), and for which we'll soon be doing a deep dive, is inspired by legendary computer scientist Donald Knuth's Potrzebie system of weights and measures ๐Ÿ˜†

Look, I might be recalcitrant at times, when it comes to digressing in my essays, but I'm not incorrigible, at least I hope not lol; I trust that my occasional obstinacy in digressing does not come across as bordering on impudence. Rest assured that, this time around anywayunlike the previous installment where digressions were center squaredigressions will appear only after the essay proper ๐Ÿ˜‰

So here we go, starting with a ranking of the second half of the top ten algorithms books of all time—the top five, of course, were covered in the prior essay, entitled Best Algorithms Books (Part 1). Here, then, are the next five stellar books on the enduringly evergreen subject of algorithms ๐ŸŒณ
6.  Algorithm Design and Applications (Wiley) by Michael T. Goodrich and Roberto Tamassia ๐ŸŒฟ
7.  Algorithmics: The Spirit of Computing 3rd Edition (Springer) by David Harel and Yishai Feldman ๐Ÿ€
8.  Pearls of Functional Algorithm Design (Cambridge University Press) by Richard Bird ๐ŸŒต
9.  How to Think About Algorithms (Cambridge University Press) by Jeff Edmonds ๐ŸŒด
10. Algorithms Unplugged (Springer) by Berthold Vรถcking et al ๐ŸŒฒ
Since writing the prior essay on algorithms, I'm even more convinced that there is no subject in computer science—at least from a practitioner's perspective—more central to its vitality than that of algorithms. In a nod to an observation I had made elsewhere, I'm eagerly thinking here to how my fascination with algorithms, and in particular their rich applicability to distributed programming, would have us travel back through the mists of time; OK, just kidding, so I'm not that old ๐Ÿ™‰
If you squint hard at the pic above—I surreptitiously dragged it in since we got to digressing, even if ever so briefly, about traveling back through the mists of time—you'll note the shadowy presence of your blog writer in an act of taking a virtuously recursive pic ๐ŸŽƒ Oh, and speaking of art, we should sometime chat about the time when my childhood buddy (who also happens to be my nephew) and I took an awesome tour of the mind-blowing Solomon R. Guggenheim Museum in New York City, nearly two decades ago๐ŸŽข 

๐Ÿ”ฆ Mild alert regarding strongly opinionated and iconoclastic views ahead—though I hasten to assure you that it's all polite discourse throughout—and where I, gasp, bypass CLRS yet again in my take on  the pursuit of algorithmic excellence ๐Ÿ”ฆ

So, following on the heels of the first installment—Best Algorithms Books (Part 1)—in this series, I  now present an equally opinionated look at another set of outstanding books on algorithms, those in the list above. I will, though, be casting a glance back at my constantly-in-progress, and happily interminable, trek through the fascinating land of algorithms. As I do so, I'll be taking an opinionated look at the following books, in turn. With that, I present to you the second set of books that have proved incredibly helpful to me in grokking algorithms ⚽ ⚾ ๐Ÿ€ ๐Ÿˆ ๐ŸŽพ ๐ŸŠ ๐Ÿƒ ๐Ÿ†

I came upon this remarkable book by sheer accident, while doing an online search for background material in connection with the forerunner of this book—Data Structures and Algorithms in Java 6th Edition (Wiley), one of whose three authors (Roberto Tamassia) also happens to be the coauthor of this fine book: Algorithm Design and Applications (Wiley) by Michael T. Goodrich and Roberto Tamassia ๐Ÿ†

Capturing the essence of Algorithm Design and Applications is made difficult by the fact that there is so much that is so good about it. But let's try. So the best analogy I can draw is by way of saying that if you took Data Structures and Algorithms in Java 6th Edition—a book that has an unwavering, laser-sharp focus on sticking to just the basic facts (Pink Floyd, anyone?)—and had it do a deep dive into the ocean that lurks beneath the surface to truly revel in the wonders of the teeming depths of uncompromising rigor, then you would get this amazing book, Algorithm Design and Applications.
I'll need some information first
Just the basic facts

~ Roger Waters (lyrics from the Pink Floyd song Comfortably Numb)
If you're looking to read only one algorithms book (to pick from this second set of books), look no farther than this extraordinarily well done volume. In its pages, you can unhurried stroll through the vistas of algorithm design, with a resoundingly emphatic nod to, and emphasis on design. If you're wanting to get to know the nuts and bolts of algorithms in an intelligent and pragmatic way—you know, roll up our sleeves and learn sufficiently and solidly enough to get the job done way?—then this book's precursor of sorts (Data Structures and Algorithms in Java 6th Edition) would be a great choice.

However, should you wish to go deeper, much deeper, and with decidedly greater nuance and sophistication, then this book—Algorithm Design and Applications—may well be the book for you. It's seemingly simple presentation style belies incredible depth, which simply couldn't have just happened by accident; that depth is hard to convey. You really do have to get your hands on this gem to see for yourself its sparkling offerings on shedding a flood of light to reveal the vast vistas that inhere in the land of algorithms. So kudos to the authors—Professor Goodrich (UC Irvine) and Professor Tamassia (Brown University)—for distilling their design wisdom into the pages of this gem.

I say: High commendations from a fellow craftsman who plies the serene—and at times turbulently creative—seas of computer science in the unrelenting quest to elevate his programming skills to the next level, in his own wayward ways (oxymoron alert) ๐Ÿ„ So awesome is Algorithm Design and Applications that I was compelled to look for anything else published by its authors...

And guess what? Yep, I did find, and buy, yet another fine book by the same authors, yay ๐ŸŽ“ That one is entitled Handbook of Graph Drawing and Visualization: Discrete Mathematics and Its Applications by Roberto Tamassia and it's looking really good. Now if I can only make time to read it in any depth; till then, I'll try to save it from languishing in my library-inbox of sorts ๐Ÿ“ฌ

My formula there, in selecting books for study, is blindingly simple in that all I do is remind myself: Akram, you found something good, now you stick to it! That's it ๐Ÿ™‹ That's why I have a handful of authors whose work I like so much that I'll simply buy and read anything they write. Period. Should you be interested in their work, I invite you to check out an essay I wrote a while ago, called Top Thought Leaders to Follow. In there, you'll meet not only familiar luminaries of our field—Martin Fowler and Robert “Uncle Bob” Martin, just to name two—but also other thought leaders whose names may be less familiar: Allow me to introduce to you, for example, Elizabeth Grace Saunders, Salman Khan, and Cal Newport. Their trailblazing work is not to be missed ๐Ÿ™‡

So I was saying... Having discovered—in recent years, actually—in Algorithm Design and Applications a treasure trove of design wizardry, I'm still hacking away at the nuggets that I keep uncovering in its glorious pages, as and when I can make time. Ah time, most precious of commodities that thou art, if only we could find ways to manage thee, and manage thee well ⏳⏰⌛

Algorithm Design and Applications is replete with tasteful illustrations, which are neither overdone nor cheesy. It's got tons of crystal-clear presentations of theorems, along with accompanying proofs that are, IMHO, marvels of elegance and beauty. Truth be told, had I given more thought to it earlier—given the premium that I, as a software craftsman, place on pursuing the nexus between beautiful code and beautiful proseAlgorithm Design and Applications may well have landed in the top spot of my best algorithms books of all time ๐Ÿ’ฏ

While a paperback edition is not available at this time, I can only recommend plunking down the extra money to get a hardcover copy of Algorithm Design and Applications. Trust me, I don't have a cent to make off of the sales of this book ๐Ÿ’ฐ ๐Ÿ’ฒ ๐Ÿ’ต ๐Ÿ˜ฑ But as a writer who's committed to serving his readers, I feel compelled to share with you the fruits of my research labors: Share with you all that I've found—and continue to find—about what works, and what doesn't, what's truly useful, and what isn't ๐Ÿ’ช

Though I have the ebook version of this book as well, I find it far more effective to study the physical book—highlighter in hand, plus a stash of tape flags with which to mark up the pages close at hand, too. Surely there is a method to the madness, something like a visual variant of coding and cataloging, inspired by the venerable Dewey Decimal system that permeates our libraries, or at least it used to, last time I checked, many moons ago, as a graduate student at Texas A&M University ๐ŸŒ›

Speaking of a method to the madness, I may well be in good company because I find myself reminded of a marvelous passage that I savored a few years ago, in the wise and scintillating pages of the book The Distraction Addiction: Getting the Information You Need and the Communication You Want, Without Enraging Your Family, Annoying Your Colleagues, and Destroying Your Soul

The author of the preceding book (Alex Soojung-Kim Pang) unabashedly went for it when selecting that whale-like title for his book lol, didn't he? ๐Ÿณ Anyhow, he notes with stellar prescience how
As Elizabeth Dunn, an anthropologist at the University of Colorado–Boulder, put it, "If I can skim it, I read it on Kindle, but if I really have to know it, I need a printed page. Work I have to really concentrate on and know, anything I have to annotate, and poetry are all impossible to read on the Kindle. I cannot retain enough of what I read on Kindle to use it for things I will have to work to master." Stephen Herrod, the chief technology officer of VMWare, concurs. He’ll take his Kindle on the road, but "I tend to print out deeper articles from the web if I know I’ll need to think more about them." 
Books and print are valuable when you need to read intensively and with few distractions. What’s interesting is that reading books is often a very physical kind of reading. People talk about having to underline, annotate, and move between several books at once in order to engage in a kind of nonlinear, cross-textual reading. Novelist Nancy Etchemendy said, "If I’m going to use a book for reference or for mastering a difficult body of material, I find it helpful to annotate and bookmark.  
It is still impractical to do those things on e-readers." Microsoft Research scientist Ho John Lee says, "If I receive something important online, I print it out first, so I can spread it out, mark it up, and make notes. I never do any important sustained reading online." The stability of physical books is also useful for people with strong visual memories. Dunn remembers ideas by visualizing their physical location on the page. If she reads a book on a Kindle, she “can’t remember a thing about a book beyond its general argument.”  
Consequently, "I do almost all of my labor and thought-intensive reading in print form." For me, serious reading involves marking up, underlining, and annotating books; it’s a martial art, and it requires the material engagement and support that paper can provide and that screens conspicuously lack. Fifty years ago, MIT professor and hypertext pioneer Vannevar Bush imagined we’d do this kind of intensive, interactive, relational reading on the memex, an electronic system he proposed in 1945. Today, people who really have to know their stuff still choose paper.
So I was saying... No doubt we've all grown accustomed—in any given technical book that's worth its salt—to the exercises which routinely appear at the end of any given chapter. Well, be prepared to be wowed by what Algorithm Design and Applications does in this department: While this book does have all that, and then some, what truly sets it apart is the way those end-of-chapter exercises are thematically organized. Here's the deal: To guide the reader in maximizing her reading experience, the authors have arranged the exercises in thematic chunks, labeled Reinforcement, Creativity, and Applications, respectively.

Here is an example each of those thematic chunks—Reinforcement, Creativity, and Applications—that will hopefully give you a flavor of what to expect in the thematically arranged end-of-chapter exercises:
Example—Show that the best-case running time of quick-sort on a sequence of size n with distinct elements is O(n log n). 
Example—Let A be a collection of objects. Describe an efficient method for converting A into a set. That is, remove all duplicates from A. What is the running time of this method? 
Example—Bob has a set, A, of n nuts and a set, B, of n bolts, such that each nut has a unique matching bolt. Unfortunately, the nuts in A all look the same, and the bolts in B all look the same as well. The only comparison that Bob can make is to take a nut-bolt pair (a, b), such that a ∈ A and b ∈ B, and test if the threads of a are larger, smaller, or a perfect match with the threads of b. Describe an efficient algorithm for Bob to match up all of his nuts and bolts. What is the running time of this algorithm?
Only one other algorithms book comes to mind that comes somewhat close to this one in the department of high caliber end-of-chapter exercises: Algorithm Design (Pearson) by Jon Kleinberg and ร‰va Tardos.

Oh, and as if that weren't enough, the authors of Algorithm Design and Applications round out each chapter with a cogent set of notes that include tips on how best to pursue a given topic in even greater depth—all I can say is, "Guys, can you please have your marketing department do a better job of making this gem of a beautiful book more widely known?!"

So here's a question: Why does CLRS get all the limelight when most all of us in the programming community are practitioners, and not researchers. And yes, most of us aren't into bondage-and-discipline languages either, as hilariously defined in the The New Hacker's Dictionary, Third Edition. We prefer our algorithmic resources to be written for normal people; I hasten to point out that The MIT Press has other books which are consistently at the right level of difficulty and comprehension and are, frankly, plain awesome!

And then we had this aberration in the form of CLRS, alas. Cryptic notations that I have neither the time nor the inclination to master in order to wade through the morass of equally enigmatic coverage. I concede that CLRS is peerless when it comes to the matter of a single resource for finding references to the sprawling literature on the genteel subject of algorithms—well, it used to be a courtly subject until folks started putting the substantial algorithmic literature to their remarkable advantage and in the process propelling their businesses into the upper stratosphere of profitability ๐Ÿš€

As a sucker for adorning—some might say embellishing, though I'll politely beg to differ—essays with topical quotes, images, and excerpts to create a pleasurable reading experience, I was pleased by the way each chapter in Algorithm Design and Applications starts out with an endearing photo each that is refreshingly black-and-white and nicely complements the theme of any given chapter. For example, the theme of Chapter 16—Network Flow and Matching—is pleasingly accentuated by an endearing photo of Yellowstone Falls (1941) by Ansel Adams ๐Ÿ‘ ๐Ÿข ๐Ÿ ๐Ÿž ๐ŸŒ

I love this book. Don't miss it. It will help you elevate your programming skills, plus—and more importantly so, your design skills—to the next level. Again, the uncompromising commitment to help the reader become the best designer she can be is evident and writ large throughout the pages of Algorithm Design and Applications. In the process of reading it, learning ceases to be a chore and instead becomes true pleasure. Yes, it is one of those rare books ๐Ÿ

Before moving on to the next algorithms book in this opinionated essay, I confess that I would have—if only I could turn back the hands of time—put this book in the pantheon of my top five algorithms books, which you hopefully enjoyed finding more about in the previous installment of this series of deep-dive reviews into the algorithms genre. So I ruefully ask of you to please keep this in mind as you continue to revisit this series of deep-dive reviews.


Here's a confession that I feel compelled to make: I've never been terribly impressed by the publishing quality of the books published by Springer—with some notable exceptions—and while this book is an improvement, it sure could have used a better format. Look, every book simply cannot hope to have a premier quality format like that of Algorithms 4th Edition (Addison-Wesley) by Robert Sedgewick and Kevin Wayne, which, as many of you will recall, got the top spot in Best Algorithms Books (Part 1).

There should IMHO, however, be a modicum of quality to enhance the reading experience ☕ And here I bring closure to the thought on a happy note  Springer has redeemed itself by publishing another book of excellent quality, reaffirming my faith in what their publication quality can be. Plus you don't have to go far to get the scoop on that book—Algorithms Unplugged (Springer) by Berthold Vรถcking et al—since it happens to be in the tenth spot in our list of the top 10 algorithms books (Should I do a third installment of this series of essays in the future, then we would naturally get the top 15 algorithms books; that may or may not happen, depending on reader interest, which BTW has been tremendous so far woohoo). Anyhow, we'll get to Algorithms Unplugged in short order. Stay tuned ๐ŸŽฌ ๐ŸŽ ๐ŸŽฏ

Meanwhile, we return to the seventh book in our list. So the second book on our list has content that is nearly unmatched in the length and breadth of the computer science and programming literature. Ladies and gentlemen, rest assured that the stellar content of this book—and allow me to introduce Algorithmics: The Spirit of Computing 3rd Edition (Springer) by David Harel and Yishai Feldman—more than makes up for the, um, less than stellar book format, which is not too shabby either. By the way, there are no problems whatsoever with the ebook, of which I also own a copy; ah yes, the digital media remains unencumbered by the frailties that sometimes attend the physical media ⛳

The reason Algorithmics got such a high spot on the list is its stellar content, the likes of which I have yet to see anywhere else! Dr Stan Scott, with Queen's University (Belfast, Ireland), probably put it the best when he noted in The Times Higher Education Supplement how this book
Through the use of tantalizing questions and aptly chosen and often amusing examples, the author transmits to the reader the excitement and intellectual satisfaction of computer science research. Without the use of formal mathematics and without any sacrifice of intellectual integrity, he conveys to the general reader the profound principles on which computer science is founded and which hitherto were only accessible in abstruse and esoteric textbooks and papers. 
This is scientific writing at its best.
Oh goodness, it sure is scientific writing at its best: The charter of this blog—to host essays at the intersection of culture, software, technology, and science—is proud to share a book that is the culmination, and then some, of the ineffable confluence of software, technology, and science ๐Ÿ’ป ๐Ÿ“ฆ ๐Ÿ“ก

You should know upfront that Algorithmics has a story to tell. This ain't no cobweb-laced, put-you-to-sleep boat anchor that would sink you ⚓ Think of it as an adventure through the guts of computer science, with a strong emphasis, of course, on the concepts, ideas, methods and results that are fundamental to computer science. Also know that you won't get bogged down with formalisms, though they (i.e. formalisms) do have a rightful place; take this from someone smitten by the lambda calculus ๐Ÿ˜Ž

The subtitle—The Spirit of Computing—of Algorithmics is spot on because once you make your way through the pages of this remarkable book, you won't look at computers in quite the same way. You'll come away with kinship for the true spirit of computers: In other words, exactly what it is that make them tick (hint, hint: their algorithms) ⏰⌚

For those unfamiliar with the name David Harel, suffice it to say that he is a hugely influential individual in the firmament of theoretical computer science, being one of the leading researchers. He also happens to be the chairman of the Applied Mathematics and Computer Science Department at the renowned Weizmann Institute of Science ๐Ÿฐ

Think of Algorithmics as an endearing tour of theoretical computer science at the proverbial elevation of 50,000 feet, or is it 100,000 feet, or, heavens be praised, 10,000 feet, though that seems a bit too close to the ground for comfort? ๐Ÿš Anyhow, I think you got the idea, right? ๐Ÿ˜‰

Since we're talking about having a head for numbers, you might as well know that I'm incorrigibly bad at arithmetic, though pretty darned good at the logical layering and analysis—and synthesis, to be sure—which I use on a daily basis while going about my software design and development activities. Go figure!

The writing style of Algorithmics is truly exceptional; we have in Harel a master who is surveying his area of expertise with a magisterial command that reflects both his perspicacity and perspicuity. And believe it or not, it's written in a clear style that makes it suitable even for the layman—and laywoman, to be sure! I hastened to add the latter since I support egalitarian feminism, which you may have gathered from the quote atop this essay ๐Ÿ‘ฐ
You’ve Come a Long Way, Baby (Or Have You?) 
~ Ruth Rosen
The Rosen quote above, by the way, was aimed squarely at accentuating and amplifying the prior quote—in the context in which it had appeared, juxtaposed as it was with the following quote, atop this essay—where we heard a luminary of our field reminiscing about how
This was the first graph problem I ever posed myself and solved. The amazing thing was that I didn't publish it. It was not amazing at the time. At the time, algorithms were hardly considered a scientific topic (bold is mine) 
~ Edsger W. Dijkstra (as quoted by Dennis Shasha and Cathy Lazere in Out of their Minds: The Lives and Discoveries of 15 Great Computer Scientists (Copernicus))
To dig deeper still, I'd recommend a math textbook or two to accompany your study of Algorithmics—in and of itself, though, it's plenty good as a superb starting point ๐Ÿš€

And listen, to those who have objected to Algorithmics, saying that they returned this masterpiece because they demurred to the Biblical quotes that are tastefully sprinkled across its pages, all I have to say is this: Guys and gals, let's all of us please grow up ๐ŸŒ ๐ŸŒŽ ๐ŸŒ We're one global family, which is a point that brings me to the fourth of the four quotes that I had selected, after much deliberation, to adorn the outset of this essay:
I refuse to accept the view that mankind is so tragically bound to the starless midnight of racism and war that the bright daybreak of peace and brotherhood can never become a reality... I believe that unarmed truth and unconditional love will have the final word. 
~ Martin Luther King, Jr.
In sum, and in the words of Harel—the author of Algorithmics—from his preface to the book, we need to listen to an important story, and the fundamental premise that
This book tells a story. The story concerns the concepts, ideas, methods, and results
fundamental to computer science. It is not specifically about computer technology,
nor is it about computer programming, though obviously it is heavily influenced by
Many advanced textbooks do treat the fundamentals, but by their very nature they concentrate on specific topics, and do so at an advanced technical level that is usually unsuitable for the general reader. Even professional programmers and systems analysts might lack the background or motivation required to get through books aimed at full-time computer science students.  
Curiously, there appears to be very little written material devoted to the science of computing and aimed at the technically-oriented general reader as well as the computer professional. This fact is doubly curious in view of the abundance of precisely this kind of literature in most other scientific areas, such as physics, biology, chemistry, and mathematics, not to mention humanities and the arts. There appears to be an acute need for a technically-detailed, expository account of the fundamentals of computer science; one that suffers as little as possible from the bit/byte or semicolon syndromes and their derivatives, one that transcends the technological and linguistic whirlpool of specifics, and one that is useful both to a sophisticated layperson and to a computer expert. It seems that we have all been too busy with the revolution to be bothered with satisfying such a need.
If you come to Algorithmics with the following sort of idea, I can nearly guarantee that you won't be disappointed. Period.
Computer Science is no more about computers than astronomy is about telescopes.
Next up, we'll segue into an algorithmic twist on the beguiling world of functional programming, an area which—as long-time readers of my blog will attest to—was, and remains, near and dear to my heart ๐Ÿ’•

Two things to keep in mind as you read what I have to say here about Pearls of Functional Algorithm Design (Cambridge University Press) by Richard Bird. As a writer who's committed to serving his readers, I want to get my biases in front of you to let you make better and more informed choices as you go about your journeys of learning. In no particular order, please know that
  • I am intrigued by the promise of blending object-orientation (OOP) with functional programming (FP), as I've tried to articulate elsewhere ๐ŸŒน
  • I firmly believe that a programming language like Scala is well-positioned to serve as a bridge between the object-orientation of Java and the functional programming languages, as I've tried to demonstrate elsewhere ๐Ÿš 
  • I am in full agreement with Eric Raymond's assessment that the Lisp programming language—think Clojure, the Lisp-for-the-JVM programming language—is worth learning for the profound enlightenment experience you will have when you finally get it, as I've tried to explain elsewhere ๐Ÿ’ก
My biases in this area—and I think I got them all in front of you—inform and permeate my thinking and, in turn, will permeate the assessment of another gem of a book: Pearls of Functional Algorithm Design ☄

Seldom have I come across a book that was more appropriately named than this one; the reference to pearls is spot on, because this book is a gem in the rough. I seldom hear it mentioned in the programming community, which is a pity, because it has a ton to offer.

But first things first: All examples in Pearls of Functional Algorithm Design are presented in the Haskell programming language, which, by all admissions, is a pure functional programming language. If there is a programming language purer than Haskell, then I'm not aware of it. Then again, I'm decidedly not a language researcher or purist; for that sort of thing, you'll want to turn to someone like the esteemed Mark C. Chu-Carroll, PhD, author of a delightful book called Good Math (The Pragmatic Bookshelf). Chu-Carroll is—by any objective standard—an inveterate collector of programming  languages ๐Ÿš ๐Ÿš ๐Ÿš By his own delightfully candid admission in Good Math ๐Ÿ‘ป
In real life, I’m not a mathematician; I’m a computer scientist. I’m still a math geek, mind you, but what I really do is very much in the realm of applied math, working on building systems to help people build programs. One of my pathological obsessions is programming languages. Since I first got exposed to TRS-80 Model 1 BASIC back in middle school, I’ve been absolutely nuts about programming languages. Last time I counted, I’d learned about 150 different languages, and I’ve picked up more since then. I’ve written programs in most of them. Like I said, I’m nuts (italics mine).
As for me, my tastes are far more pragmatic, guided by my trusty compass that consists of the twin forces of taste and beauty. Yes Sir—and Gentle Lady—I'm a pragmatist to my core. Here's a telltale clue that will surely give everything away in this area: I think that the Scala programming language is the next best thing since sliced bread ๐Ÿž ๐Ÿž ๐Ÿž Yes, there is another JVM language in the shape of the lovely Lisp dialect that we know as the Clojure programming language; plus I've done plenty of dabbling in Clojure over the years. On top of that, we got lambdas added to the supple workhorse of a language that is Java when Java 8 appeared, yay! But there are only so many hours in a day to keep up with all the cool happenings in the JVM ecosystem, and elsewhere ⏳⏰⌛ Enough said.

Me, I'm more like Indiana Jones, if anyone remembers the intrepid archaeologist and adventurer from the classic movie Raiders of the Lost Ark—in which our nearly-fearless hero is hired by the U.S. government to find the Ark of the Covenant before the enemy does—constantly trying to unearth better ways of going about improving the art of programming, always on the lookout to uncover more intelligent approaches to crafting software, and perhaps trying to decipher the torrential fugues of design-related tactical paradigms in motion that swirl around in seemingly indecipherable patterns, unnervingly so at times. By the way, and lest there be any confusion, I hasten to point out that the preceding musings about software craftsmanship—and software craftswomanship, to be sure—which followed the trailing em-dash above was me speaking, not Professor Jones. Last time I checked, Professor Jones wasn't that much into software ๐Ÿ˜‰

Okay, so I'm not going to stretch that analogy too much, because it may not hold up for all I know lol. I will, though draw your attention briefly to an absolutely fantastic book—Surfaces and Essences: Analogy as the Fuel and Fire of Thinking (Basic Books) by Douglas Hofstadter and Emmanuel Sander—should you wish to digress and find out just how far analogies can take you on your intellectual journeys. Here, then, is a typical passage from the magisterial Surfaces and Essences in which the authors are telling a marvelous tale by retelling how
If we examine the pathway that Einstein took to reach the extended principle of equivalence (which applies to all phenomena of physics, not just to those of mechanics), we see that he re-exploited, in a new context, an analogy that he had already exploited once. More specifically, he carried out the same conceptual extension in two different contexts, each time starting with the concept of mechanical experiment and ending up with the more abstract concept of physical experiment... 
And Einstein used this analogical leap in two situations that were themselves analogous (first in extending the principle of Galilean relativity to yield special relativity, and later in extending the restricted principle of equivalence to yield the founding ideas of general relativity). 
Thus, the most advanced breakthrough of Einstein’s life came out of an analogical leap that was analogous to another analogical leap—thus an analogy between analogies, or, if you will, a meta-analogy. 
This recalls a remark once made by the Polish mathematician Stefan Banach, as recounted by his friend Stanislaw Ulam: "Good mathematicians see analogies between theorems or theories, but the very best ones see analogies between analogies." And along the same lines, Einstein’s Scottish predecessor James Clerk Maxwell once observed that rather than being attracted by parallels between different principles of physics, he was attracted by parallels between parallels, which would certainly seem to be the quintessence of abstraction (italics mine).
And trust me, there's no castle-building going on here; just plain old-fashioned abstract thinking, carved along the contours of what the amazing and legendary physicist Richard Feynman symbolized where his thinking style is on display—as memorably captured in the book entitled Smarter Than You Think: How Technology is Changing Our Minds for the Better (Penguin Press) by Clive Thompson ๐Ÿš‚

When historian Charles Weiner looked over a pile of Richard Feynman’s notebooks, he called them a wonderful 'record of his day-to-day work' to which Feynman demurred
"No, no!", Feynman objected strongly.
"They aren’t a record of my thinking process. They are my thinking process. I actually did the work on the paper."
"Well," Weiner said, "The work was done in your head, but the record of it is still here."
"No, it’s not a record, not really. It’s working. You have to work on paper and this is the paper. Okay?", Feynman explained.
Let's compile our computer programs in our head, shall we? Ready, set, go! It's Turing machines and lambda calculus all the way down—I kid you not ๐Ÿ˜‰

Feynman the Nobel laureate—he basically transformed the Physics department at Caltech into the powerhouse that it has remained ever since—truly understood that writing his equations and ideas on paper was crucial to his thought.

Let's now circle back to Pearls of Functional Algorithm Design and hear what Bird has to say about his motivation for putting together this pearly and lustrous book. In Bird's words
The idea they had in mind was to emulate the very successful series of essays that Jon Bentley had written in the 1980s under the title “Programming Pearls” in the Communications of the ACM. Bentley wrote about his pearls: 
Just as natural pearls grow from grains of sand that have irritated oysters, these programming pearls have grown from real problems that have irritated programmers. The programs are fun, and they teach important programming techniques and fundamental design principles. 
Equational reasoning dominates everything in this book. 
My interest has always been in algorithms and their design. Hence the title of this book is Pearls of Functional Algorithm Design rather than the more general Functional Pearls. Many, though by no means all, of the pearls start with a specification in Haskell and then go on to calculate a more efficient version. 
My aim in writing these particular pearls is to see to what extent algorithm design can be cast in a familiar mathematical tradition of calculating a result by using well-established mathematical principles, theorems and laws. While it is generally true in mathematics that calculations are designed to simplify complicated things, in algorithm design it is usually the other way around: simple but inefficient programs are transformed into more efficient versions that can be completely opaque. It is not the final program that is the pearl; rather it is the calculation that yields it. 
Other pearls, some of which contain very few calculations, are devoted to trying to give simple explanations of some interesting and subtle algorithms. Explaining the ideas behind an algorithm is so much easier in a functional style than in a procedural one: the constituent functions can be more easily separated, they are brief and they capture powerful patterns of computation.
To give you a tantalizing view of the pearls that await you in this oyster, consider its delectable offering by way of the table of content:
Preface page ix
1 The smallest free number 1
2 A surpassing problem 7
3 Improving on saddleback search 12
4 A selection problem 21
5 Sorting pairwise sums 27
6 Making a century 33
7 Building a tree with minimum height 41
8 Unravelling greedy algorithms 50
9 Finding celebrities 56
10 Removing duplicates 64
11 Not the maximum segment sum 73
12 Ranking suffixes 79
13 The Burrows–Wheeler transform 91
14 The last tail 102
15 All the common prefixes 112
16 The Boyer–Moore algorithm 117
17 The Knuth–Morris–Pratt algorithm 127
18 Planning solves the Rush Hour problem 136
19 A simple Sudoku solver 147
20 The Countdown problem 156
21 Hylomorphisms and nexuses 168
22 Three ways of computing determinants 180
23 Inside the convex hull 188
24 Rational arithmetic coding 198
25 Integer arithmetic coding 208
26 The Schorr–Waite algorithm 221
27 Orderly insertion 231
28 Loopless functional algorithms 242
29 The Johnson–Trotter algorithm 251
30 Spider spinning for dummies
Is that an awesome menu, um, table of contents I meant to say, or what? ๐Ÿช ๐Ÿฉ ๐Ÿฎ ๐Ÿ‘œ ๐Ÿ‘ ⛱

Look, ssshhhh, let's digress just a tad, and nobody will be the wiser ๐Ÿ˜ด๐Ÿ˜ฒ

And it's not that egregious of a digression anyway, since I happened to have noted above that all examples in Pearls of Functional Algorithm Design are presented in the Haskell programming language—so let's listen to what Guy Steele has to say about Haskell. As longtime readers of my blog know, Steele is one of my programming heroes; he is widely regarded as the father of Common Lisp; and he happens to be the lead author of the Java Language Specification. This is what Steele had to say in one of Seibel's interviews in the sparkling-with-insights book entitled Coders at Work in connection with Haskell:
Haskell is a beautiful language. I love Haskell. I don't use it that much... On the other hand, now Haskell has discovered monads and they have dragged in the I/O monad and now the transactional-memory monad. There's a theory that it's functional and maybe that does give you a leg up. 
On the other hand it's feeling more and more imperative. And I can't resist thinking of the White Knight in Through the Looking Glass—"I was thinking of a plan to dye one's whiskers green, and always use so large a fan that they could not be seen." And in some ways, monads strike me as that fan, where you're dragging in the I/O and trying to hide it again—are the side effects really there, or are they really not? ☕ ๐Ÿฑ ☕
Just remember: This digression will be our secret, okay, and we'll be all good? ๐Ÿ˜‡

Now, to illustrate the flow of the narratives in Pearls of Functional Algorithm Design, let's take, as an example, the delightful third chapter, which is prosaically entitled Improving on Saddleback Search. The setting is a classroom tutorial on functional algorithm design. There are four students: Anne, Jack, Mary and Theo and we proceed like so, Socratic dialog and all, with the Teacher going first ๐ŸŽ“
Teacher: Good morning class. Today I would like you to design a function invert that takes two arguments: a function f from pairs of natural numbers to natural numbers, and a natural number z. The value invert f z is a list of all pairs (x , y) satisfying f (x , y) = z . You can assume that f is strictly increasing in each argument, but nothing else. 
Jack: That seems an easy problem. Since f is a function on naturals and is increasing in each argument, we know that f (x , y) = z implies x ≤ z and y ≤ z . Hence we can define invert by a simple search of all possible pairs of values: 
invert f z = [(x , y) | x ←[0 .. z ], y ←[0 .. z ], f (x , y) z ] 
Doesn’t this solve the problem? 
Teacher: Yes it does, but your solution involves (z + 1)2 evaluations of f . Since f may be very expensive to compute, I would like a solution with as few evaluations of f as possible. 
Theo: Well, it is easy to halve the evaluations. Since f (x , y) ≥ x +y if f is increasing, the search can be confined to values on or below the diagonal of the square: 
invert f z = [(x , y) | x ←[0 .. z ], y ←[0 .. z − x ], f (x , y) z ] 
Come to think of it, you can replace the two upper bounds by z − f (0, 0) and z − x − f (0, 0). Then if z < f (0, 0) the search terminates at once.
Ah yes, charming Socratic dialog and all, with some glimmers, dare I say, of the scintillating magnum opus of a landmark book that many of us affectionately call GEB—for the initials in Gรถdel, Escher, Bach: An Eternal Golden Braid—by the Pulitzer Prize-winning author Douglas Hofstadter. Remember how we were regaled in GEB with mind-bending dialogs between crabs and ants and stuff? ๐ŸŒ ๐Ÿœ ๐ŸŠ ๐Ÿ ๐Ÿš

Well, back to Pearls of Functional Algorithm Design, allow me to ask: Were you starting to think thoughts such as the following? Groan, a tome with strong functional programming undercurrents—surely we need to brace ourselves for a deadly boring treatise? So let me comfort you by saying, Trust me on this one; not a trace of boredom anywhere in sight with this gem. What you'll be in for is a delightfully pleasant surprise of discovering algorithmic excellence from the functional point of view ๐ŸŽˆ

This book will, as it were, grab your imagination and whisk you away to Neverland, in the fabulous tradition of Peter Pan ๐ŸŒˆ Here are other candid thoughts that I jotted down earlier during recent transcontinental transit—I wrote the bulk of this essay while flying from Austin to San Diego. Hmm... That might explain the the flight of fantasy with which we had a brief skirmish above. Some other thoughts are as follows ๐ŸŒน

You will see the elegant mathematical machinery neatly unpacked and put together again—something that Humpty Dumpty failed at after he had had a great fall from the wall—to see exactly what it is that makes an intricate (functional) algorithm  tick. Remember, too, how four-score men and four-score more, could not make Humpty Dumpty where he was before, alas ๐Ÿ“ But hey, that was then, and this is now ๐ŸŽ‰

Fear not, though, intrepid reader! The mathematical machinery—which ably accompanies the narratives in Pearls of Functional Algorithm Design to marvelous effect—is both packed and neatly unpacked without a hitch, whew. And yes, Bird pulls no punches; yet he neither talks down condescendingly nor resorts to hand waving nor dumbs things down. This book hit the sweet spot for me ๐Ÿฏ It's simply marvelous in a delicious way.

The presentation is pleasing, broken down into digestible chunks: aptly annotated code snippets, helpful tables, elegant mathematical underpinnings that have neatly been drawn into a cohesive narrative. Pearls of Functional Algorithm Design is, alas, sorely lacking in the illustrations department. This was a bit of a letdown for visually oriented designers and programmers like me—should you wish to see everything that a book can be, firing on all cylinders, as it were, look no further than the tour de force entitled The Nature of Computation (Oxford University Press) by Cristopher Moore and Stephan Mertens, on which I have lavished much attention elsewhere, in the first installment of this series of essays, in fact ๐ŸŒฟ

While The Nature of Computation did not come in at the first spot—I valiantly resisted putting it in the first spot, sorely tempted though I was—simply because I would be doing a disservice to my readers by pointing them in the direction of a book that, awesome as it is, is not an introductory one. Think classical utilitarianism ala Jeremy Bentham and John Stuart Mill who held that we ought to maximize the good, that is, bring about "the greatest amount of good for the greatest number".

The pearls in this book—the brute force algorithms—are progressively rewritten into their increasingly efficient counterpart algorithms. The way this is all wrought, the manner in which the intermediate steps are crafted, is something to behold, and I really shouldn't take away your excitement prematurely, oh no ๐Ÿ”ฆ

Dig in, and you'll see what I'm talking about. As a prelude, you may also wish to look at precisely this sort of transformational craftsmanship on glorious display—albeit from the distinctly different yet equally venerable object-oriented perspective—in the astonishingly good book Refactoring to Patterns by Joshua Kerievsky (Addison-Wesley Professional) as I've tried to capture in an essay elsewhere ๐ŸŽ ๐Ÿฐ

This is a book for someone who likes to think. If that's you, don't miss Pearls of Functional Algorithm Design ๐Ÿ’

Next up, we have a book that's drenched in a remarkably creative approach to teaching algorithms: How to Think About Algorithms (Cambridge University Press) by Jeff Edmonds. I have not seen a book quite like this—on the subject of algorithms or otherwise—anywhere else!

People that I've talked to have referred to How to Think About Algorithm as a life changer. What many have—myself prominently included—appreciated a lot about this book's approach is its uncompromising devotion to studying a number of well-known algorithms that solve important problems, presenting the associated pitfalls, going over several ways to solve the problem at hand, even when intuition contradicts reality, and finally explaining the correct solution ๐Ÿ’ฏ

The author, Edmonds, capture the gestalt of this book rather nicely in the highly detailed preface to the book by breaking up his recommendations to studying How to Think About Algorithm into easily digestible chunks, grouped under headings like the following, beginning with
Meta-Algorithms: Students must learn so many algorithms that they are sometimes overwhelmed. In order to facilitate their understanding, most textbooks cover the standard themes of iterative algorithms, recursion, greedy algorithms, and dynamic programming. Generally, however, when it comes to presenting the algorithms themselves and their proofs of correctness, the concepts are hidden within optimized code and slick proofs. One goal of this book is to present a uniform and clean way of thinking about algorithms. We do this by focusing on the structure and proof of correctness of iterative and recursive meta-algorithms, and within these the greedy and dynamic programming meta-algorithms. By learning these and their proofs of correctness, most actual algorithms can be easily understood. The challenge is that thinking about meta-algorithms requires a great deal of abstract thinking. 
Abstract Thinking: Students are very good at learning how to apply a concrete code to a concrete input instance. They tend, however, to find it difficult to think abstractly about the algorithms. I maintain that the more abstractions a person has from which to view the problem, the deeper his understanding of it will be, the more tools he will have at his disposal, and the better prepared he will be to design his own innovative ways to solve new problems. Hence, I present a number of different notations, analogies, and paradigms within which to develop and to think about algorithms. 
Dreaming: I would like to emphasis the importance of thinking, even daydreaming, about the material. This can be done while going through your day – while swimming, showering, cooking, or lying in bed. Ask questions. Why is it done this way and not that way? Invent other algorithms for solving a problem. Then look for input instances for which your algorithm gives the wrong answer. Mathematics is not all linear thinking. If the essence of the material, what the questions are really asking, is allowed to seep down into your subconscious then with time little thoughts will begin to percolate up. Pursue these ideas. Sometimes even flashes of inspiration appear.
Did anyone catch that, Dreaming? Here's the deal: Another book—The Nature of Computation—is precisely what I have in mind here. In fact, I had elaborated a bit on this theme in an essay elsewhere, where I had shared with you the following thought, noting how
Algorithms, more than anything else in the entire fabric of computer science, IMHO, evolve and thrive in the hotbed of creativity, and who better to illustrate this with than the true legend of Kekulรฉ, who saw in a dream a snake eating its own tail, and which led Kekule—upon awaking from his dream—to the monumental discovery of the benzene molecule's structure ๐Ÿ
Fancy that, I'm caught quoting myself in a virtual act of self-referential circularity; goodness, maybe next we'll be contemplating Salvador Dali's melting watches and stuff ⏰⏳⌛๐ŸŽ

This book is a creative whack on the side of one's head. To illustrate this point, lest you think that I'm engaging in psychobabble, consider this typical piece of advice from the pages of How to Think About Algorithm—and this fine book is liberally sprinkled with such similar nuggets of wisdom—on how to use the metaphor of analogy as a guide. Thus, Edmonds cogently advises us to
Worry about one step at a time. Don’t get pulled into the strong desire to understand the entire computation at once. Generally, this only brings fear and unhappiness. I repeat the wisdom taught by both the Buddhists and the twelve-step programs: Today you may feel like like you were dropped off in a strange city without knowing how you got there. Do not worry about the past or the future. Be reassured that you are somewhere along the correct road. Your goal is only to take one step so that you make progress and stay on the road. Another analogy is to imagine you are part of a relay race. A teammate hands you the baton. Your job is only to carry it once around the track and hand it to the next teammate (italics mine).
Did that grab your interest? It sure grabbed mine when I first read How to Think About Algorithms, ages ago, since which time I've revisited its pages many times over ๐ŸŽป I believe that How to Think About Algorithm will make an awesome second book on algorithms; it may be a bit much to digest as your very first introduction to the exciting world of algorithms.


Last, but certainly not the least, is another remarkable book on algorithms: Algorithms Unplugged (Springer) by Berthold Vรถcking et al. Don't overlook this cool book or let its seeming simplicity put you off; true, a non-computer science individual could, with some effort, pick it up and run with it, without running into too much difficulty in understanding how algorithms work ๐ŸŽฏ

I'm not aware of a kinder, gentler approach to introducing yourself to the study of algorithms. For example, consider this typical piece of guidance, all of it gently carved into easily digestible, yet solidly intelligent chunks—Here, Heribert Vollmer and Dorothea Wagner (both are among the contributors to this nicely eclectic book) are giving an overview of Optimization in Part IV of the book. They remind us how
For many other important optimization problems no efficient solution algorithm is known to this day. The only way to find the optimal solution is to compare all possible solutions. The time requirement for this simple procedure is of course dependent on the number of solutions and hence in general it is very large. For example, in Chap. 39 a knapsack has to be packed optimally for a hike, but there are many different ways to use its capacity. Also the above problem to determine the shortest round-trip through a number of cities is one of the hard problems for which we do not know how to find an optimal solution. But in Chap. 40 we will see how a so-called approximation algorithm finds a tour that is maybe not the shortest one but one whose length usually is quite close to the optimum; in the worst case it is twice as long. The final chapter of this book, Chap. 41, introduces simulated annealing, an algorithmic method that produces approximate solutions for a number of optimization problems with certain mathematical properties. The name of this magical method is due to an analogy with an industrial technique that involves the heating and controlled cooling "annealing") of a material to improve its stability (italics mine).
Finally, in an act of virtual self-recursion—with a vigorous nod to the adage that imitation is the sincerest form of flattery—I am pilfering a paragraph from my first installment of this essay and reminding us all that ☕
There is a lot going on in this excellent book, far more than I can do justice to at this juncture. Suffice it to say that I'm becoming increasingly and acutely aware of a startling phrase—at least that's how the phrase grabbed me on my first encounter with it—that "Real Artists Ship". There, in his bestseller entitled Linchpin: Are You Indispensable? (p.101), Seth Godin was reminding us of the importance that shipping a product must occupy in our collective consciousness ๐Ÿšš
So it is that I will now exit, but not before an admittedly flamboyant flourish that harkens back to the 1937 Disney classic, Snow White and the Seven Dwarfs, in which the Evil Queen utters the famous line that I'm here taking the liberty of wildly paraphrasing—with a nod to Godin's moniker that "Real Artists Ship"—by intoning
Magic mirror, on the wall—which one is the fastest shipping method of them all?
๐Ÿš‚ ๐Ÿšƒ ๐Ÿšš ๐Ÿšข ๐Ÿš…
In the end—and much as I hinted at, at the outset—I invite your comments. Equally importantly, please, do please consider helping our fellow Americans who are bracing themselves for the fury of Hurricane Irma.

I'll mention this as a friendly reminder—your comments are welcomed, as always—that having read my brief take each on a handful of the finest algorithms books...
  • Did you find that your experience of reading these books was perhaps different? 
  • Were there other qualities, perhaps, that I did not cover, and which happen to be the ones that you found the more helpful—in these or other books—as you mastered algorithms? 
  • Did I leave out any of your favorite algorithms books?! 
  • I've covered only a tiny fraction of the algorithms books that I've read, necessarily limited by the time available—stay tuned for future installments of this essay 
I trust that these brief vignettes will help you in some small way along your journey to grokking algorithms ๐Ÿšฃ And yay, we made it through a whole essay without my digressing much—now fancy that. And now that we're done with the, ahem, essay proper, however, we surely can digress a tad, can't we? ๐Ÿ˜‚

Woohoo, here we go ⛷

See, didn't I tell you that while I may be recalcitrant at times, when it comes to digressingI try to do so with often a method to my madnessI'm hopefully not that incorrigible, and people who know me hardly think of me as a curmudgeon ๐Ÿ˜‡

With that, I rest my case; now you tell me if I made good on my promise that —unlike the previous installment where digressions were center squaredigressions were going to appear only after the essay proper ๐Ÿ

In the end, and in no particular order, here then is a fine set of yet other books on algorithmsand data structures, to be sure—which I had in mind when I sat down to scribble some musings on the subject of algorithms, a subject that ever remains near and dear to my heart ๐Ÿ’
What is this thing that builds our dreams, yet slips away from us. 
Forever is our today, 
Who wants to live forever, 
Who wants to live forever, 
Forever is our today, 
Who waits forever anyway? 
~ Queen (Lyrics from Who Wants To Live Forever)
In a properly automated and educated world, then, machines may prove to be the true humanizing influence. It may be that machines will do the work that makes life possible and that human beings will do all the other things that make life pleasant and worthwhile. 
~ Isaac Asimov, Robot Visions
For God’s sake, let us be men, not monkeys minding machines, or sitting with our tails curled while the machine amuses us, the radio or film or gramophone. Monkeys with a bland grin on our faces.
~ D.H. Lawrence, Selected Letters
If you plan on being anything less than you are capable of being, you will probably be unhappy all the days of your life.
~ Abraham Maslow
The art of the bird is to conceal its nest both as to position and as to material, but now and then it is betrayed into weaving into its structure showy and bizarre bits of this or that, which give its secret away and which seem to violate all the traditions of its kind. 
~ John Burroughs
I've got the key to my castle in the air, but whether I can unlock the door remains to be seen. 
~ Louisa May Alcott, Little Women
Humanity has the stars in its future, and that future is too important to be lost under the burden of juvenile folly and ignorant superstition. 
~ Isaac Asimov
And then there was the classic among classics: Tracy Kidder's unprecedented tour de force 
The Soul of a New Machine that was quite the publishing event, and which won a well-deserved Pulitzer Prize, notwithstanding the oh-so-endearing malapropism it engendered ๐Ÿฃ