Saturday, September 30, 2017

Best Deep Learning Books (Popular)

The Quotes ๐Ÿ“Ÿ

Explaining Metaphysics to the nation–
I wish he would explain his Explanation
๐Ÿ˜‰
~ Lord Byron (Don Juan: Dedication)
Understanding, n. A cerebral secretion that enables one having it to know a house from a horse by the roof on the house. Its nature and laws have been exhaustively expounded by Locke, who rode a house, and Kant, who lived in a horse ๐Ÿ‘ป
~ Ambrose Bierce (The Devil's Dictionary) 
It requires a very unusual mind to undertake the analysis of the obvious ๐Ÿ”ฌ
~ Alfred North Whitehead (English mathematician and philosopher) 


Deep Learning (Popular) Books: The List ๐Ÿ“ฃ ๐Ÿ“ข

And here we have the two stellar books at which we'll soon be taking an opinionated look, in turn:

Extra, extra! ๐Ÿ“ฃ With the pic above as our guide, hold on to the thought that we'll later be taking an exclusive, behind-the-scenes tour of how your blogger went about improvising and capturing the photos—those which adorn the books being reviewed—for this essay. But all that will have to wait till the end of this essay ⏰

Here's the deal: I've received copious feedback from readers to cut to the chase when beginning any given essay. And I am dead serious about honoring the wishes of my readers. So before we get carried away with the inevitable excitement you would expect to accompany an exclusive behind-the-scenes tour, let's settle for a quick metaphor: Think Monty Python's Flying Circus, complete with how a prop—yep, the one bolstered precariously on the revolving chair in the pic above—painfully fell on your blogger's foot. You'll be relieved to know, or maybe not, that your blogger mostly escaped the accident unscathed, though he did cry a bit ๐Ÿ˜ญ Enough said on the whole matter. Let's dive right into some deep learning goodness ๐Ÿ‘ฃ

Preamble ⛱


I'm acutely aware of the gnawing sensation that the purists in the field of deep learning would have me drawn-and-quartered for selecting the two books for this essay: While the phrase "deep learning" does make a handful of appearances in one of the books—in The Master Algorithm to be precise—that phrase doesn't so much as make a single appearance in the other book (Deep Work) ๐Ÿ˜ฑ

So what's up with that? Here's what: Let's remind ourselves that the field of deep learning
  • ...does not exist in a vacuum
  • ...has deep roots in machine learning (ML)
  • ...is not a fad
  • ...is not hype
  • ...has become inextricably enmeshed with every discipline under the sun
  • ...requires a different level of abstraction to comprehend it meaningfully enough
  • ...needs a different mindset in order to run with it and to achieve focused success
I hope the list above suffices to convince you all to rise in unison and thwart all would-be attempts—potentially from purists who would violently demur at my selection of books for this essay—at pillorying your blog's author ๐ŸŽฑ But then again, I only need remind us all that some people can get oh-so-emotional ๐Ÿ˜น

A long time ago, in a galaxy far away, your blogger used to assiduously write programs in the C++ programming language. But that was before he discovered Java, Scala, Python, and indeed the whole world of distributed computing. And yet there come to occsionally haunt him—especially right now since we happen to be talking about the admittedly macabre themes of "violently demurring purists" and of "getting drawn-and-quartered"—remembrance of things past, for example this remembrance of his education while working in the C++ trenches:
As Chair of the working group for the standard C++ library, Mike Vilot was told, "If there isn't a standard string type, there will be blood in the streets!" (Some people get so emotional.) Calm yourself and put away those hatchets and truncheons—the standard C++ library has strings.
~ Scott Meyers in More Effective C++: 35 New Ways to Improve Your Programs and Designs (Addison-Wesley Professional).
Lest we digress too much, I hasten to add that, after bidding farewell to the C++ programming community a long time ago, I did make peace with my fellow, hard-core C++ programmers. That was elsewhere in the context of musing on how all languages inevitably carry some baggage—it's the relative degree of the conceptual burden that a programmer has to bear, when using the model of a given language, and which sets a language apart from others ๐Ÿ‘’

Okay, let's take a deep breath. We're already done with our first and last digression in this essay—most of the rest of digressions will be waiting for you once we get to the end of reviewing two sterling books here ๐Ÿ˜Ž

Introduction ๐Ÿ”Ž


I had initially toyed with the idea of calling this essay "Best Deep Learning Books (Thematic)". But the more I reflected on (the parenthetical use of the) word "thematic", the more I realized that we would be entering the realm of "diplomacy". That realization alone made me cringe and I rushed to replace "thematic" by "popular".

So while this essay is a variation on the theme of deep learning—hence my toying with the title "Best Deep Learning Books (Thematic)"—I feel that the title "Best Deep Learning Books (Popular)" does a better job of reminding us of what exactly we're talking about. So there you have it ๐Ÿ’ญ And that's what you get this time: "Best Deep Learning Books (Popular)" ๐Ÿ

Oh, lest you were confused by this somewhat nitpicky decision—whether to use "thematic" or "popular" in the parenthetical rejoinder of the essay—allow me to explain ever-so-briefly what I had in mind in mentioning the "diplomacy" realization above. So it's been said that
A diplomat is a person who can talk for an hour without saying anything ๐Ÿ˜‰
Enough said ๐Ÿ’‚ That, of course, is decidely not my style; I'm the opposite, in that I do my level best, research, get to the heart of the matter, and formulate it to lay the subject matter bare for you—I assume, and hope, that you probably don't want your blogger to write any other way either ⛳

Refresher ๐ŸŒน


Recap of Past Two Essays (In this Series)

We all—myself prominently incuded—need refreshers every now and then to load the intellectual goods into our working memory ๐Ÿ˜

Since we went over the background for deep learning in detail in the first two installments to this series of essays—this essay of course being the third installment—we won't rehash that context here. At the same time, I cordially invite you to look up those (two) previous essays to refresh you memory on the
  1. Foundational aspects of Deep Learning
  2. Pragmatic aspects of Deep Learning
And should anyone need proof of the bias for action, and the solid pragmatism—never a purist I was, and never I will be—that I bring to my daily work, allow me to point out merely that I've been spending most of my waking hours designing and crafting distributed systems (using Java and Scala). At the same time, though, the fields of Artificial Intelligence, Machine Learning, and of course Deep Learning never wander far from my mind ๐Ÿ’

Now this will be a first! I've never quoted Quora before; notwithstanding the pseudo-alliteration (neologism alert here, having just blithely used the phrase "quoted Quora"!) it's appearing more and more regularly on my radar now. In fact, as a long-time (once-passive) subscriber to Quora, I continue to remain impressed by the high quality of their targeted updates (by way of the intriguing emails that continue to land in my inbox, with pointers to Quora questions-and-answers) ๐Ÿ“ฌ

Something of Quora

With that preamble, I invite you to have a look at an intriguing question which was posted fairly recently to Quora and ably answered by Andrew Ng (co-founder of Coursera and adjunct Professor of Stanford University). Here was the question ๐Ÿ‘ถ
I have a toddler. How should she prepare herself for the job market 15 years from now in the world of AI? (italics mine)
That such questions are even being asked is, to say the least, incredibly encouraging in that this phenomenon underscores the degree to which the quest for harnessing the power of Artificial Intelligence (AI) has permeated the public's consciousness. In turn, as a clearinghouse of sorts for ideas—which the blog you're reading now is all about—I recommend that you look up the full answer via the link that precedes the quote above. At any rate, here is part of the answer (by Andrew Ng) to that question:
Yes, do teach her to code. More importantly, cultivate in her the ability to keep learning
In the CS world, all of us are used to needing to jump every ~5 years onto new technologies and paradigms of thinking (internet -> cloud -> mobile -> AI/machine learning), because new technologies get invented at that pace that obsolete parts of what we were previously doing. So CS people are used to learning new things all the time. 
The thing that’s now changed is that CS has infected pretty much every other industry. So now it’s not just the CS world that has to change every few years. It’s that everyone now needs to change. That’s why being able to keep learning will be the most important career skill you can teach your daughter (italics mine).
Frankly, I was impressed by the delightfully articulate answer by Andrew Ng; again, I encourage to look up the full answer at Quora—it resonates with the theme of this essay in that paradigms don't exist in a vacuum; ideas, and indeed paradigms themselves, evolve ever-so-organically ๐Ÿ˜Ž
Any little song that you want to sing
Little songs that you want to sing
Any song will do
Any little song that you want to sing
Little songs that you want to sing
It's up to you
Little songs that you want to sing
A little song that you want to sing
You're blue ๐ŸŽต
~ Robert Plant (Lyrics from In The Mood)

1. The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World (Basic Books) by Pedro Domingos. ๐Ÿณ


One of the Most Fun Books Ever!

This book is proof positive that it's possible to be fun and substantial at the same time. Before we dive into the goodies you can expect to find in the pages of The Master Algorithm, let's first hear from Kirkus Reviews, which probably summed it up the best in noting how this book is
[An] enthusiastic but not dumbed-down introduction to machine learning... lucid and consistently informative... With wit, vision, and scholarship, Domingos describes how these scientists are creating programs that allow a computer to teach itself. Readers...will discover fascinating insights.
So let's please have you squint at the pic above ๐Ÿ” You'll notice that The Master Algorithm is sandwiched between two decidedly popular books. Flanking it on the left-hand side and right-hand side, respectively, are:
  • Seeing What Others Don't: The Remarkable Ways We Gain Insights (PublicAffairs) by Gary Klein
  • Only Humans Need Apply: Winners and Losers in the Age of Smart Machines (HarperBusiness) by Thomas Davenport and Julia Kirby.
I'm resisting the sorely tempting urge to digress into a word or two on the two intriguing books I mentioned above in passing ✊ But this time we're determined to proceed with discipline, with nary a lapse into those digressions. At any rate, if you agree with the basic premise of Carly Fiorina—former president, and chair of Hewlett-Packard—in her observation that "The goal is to turn data into information, and information into insight", then you may find yourself agreeing with my assessment that the themes which these books explore are actually intertwined, and deeply so. Those themes include, but are not limited to
  • Seeing patterns where others don't ๐Ÿ‘€
  • Handling the tension between fostering human creativity and the relentless march of automation ๐Ÿ’ช
  • Figuring out how to best manage the very fabric of business—at least as we know it today—which is undergoing tumultous upheaval ๐Ÿš›
  • Detecting patterns, at both the human and machine level ⛷
  • Dealing with the catalyst that Machine Learning (ML) is, and which is truly fueling the abovementioned upheaval ๐Ÿš€
Allow me to  remind us all—even as I ask that you keep the themes above in mind—that what you want to look for in The Master Algorithm is its unerring focus on teasing out and elaborating how these themes are intertwined.

Plus, having read and re-read The Master Algorithm several times over, I truly felt that its author (Pedro Domingos) did a great job of clearly articulating the book's themes. His narrative style was methodical and discriminating (in the good sense of the word "discriminating", I hasted to add lol, in that the narrative nicely delineates and discriminates between the ambiguities that inevitably lurk at the boundaries of related ideas). In the process, I also felt that The Master Algorithm nicely sidesteps the following devilish defintions that conceivably might be applied to books of a lesser meter. I didn't really say it; just sayin' ๐Ÿ˜‰
Discriminate, v.i. To note the particulars in which one person or thing is, if possible, more objectionable than another.
Discussion, n. A method of confirming others in their errors.
~ Ambrose Bierce (The Devil's Dictionary) 
So that's another cool thing going for The Master Algorithm ๐Ÿ˜Ž

The Joy of Reading this Book

In my mind, The Master Algorithm is to the world of machine learning—and to deep learning in turn—what The Joy of Clojure is to the world of programming, and what Joy of Cooking is (was?!) to the culinary world (I can't definitively judge in the latter matter since my culinary skills barely hover above zero) ๐Ÿ”

Yes, the former is that good, and let me tell you why. For starters, The Master Algorithm blends an infectiously engaging style with the impeccable credentials of a leading researcher in the field of machine learning and AI: Domingos is a professor of computer science at the University of Washington. He is a winner of the SIGKDD Innovation Award, the highest honor in data science.

A "Map" of the Book

Domingos nicely lays out the map of The Master Algorithm in an introductory section (Prologue) of the book by pointing out how
I had a number of different but overlapping audiences in mind when writing this book. If you’re curious what all the hubbub surrounding big data and machine learning is about and suspect that there’s something deeper going on than what you see in the papers, you’re right! This book is your guide to the revolution (italics mine).
You really should look up The Master Algorithm for the details that Domingos immediately goes on to add to the point above—which was to describe one kind of its intended audience—by adding how readers of other diverse backgrounds will find much of value in its pages. In his words, you, too, are the intended audience of The Master Algorithm
  • If your main interest is in the business uses of machine learning, ...
  • If you’re a citizen or policy maker concerned with the social and political issues raised by big data and machine learning, ...
  • If you’re a scientist or engineer, ...
  • If you’re a machine-learning expert, ...
  • If you’re a student of any age—a high schooler wondering what to major in, a college undergraduate deciding whether to go into research, or a seasoned professional considering a career change—, ...
Again, I'm resisting the sorely tempting urge to digress into a word or two on my use of the word "Map"—as in "A Map of the Cat?", and as used by one of my heroes in science, the legendary physicist Richard Feynman—in the heading above ๐Ÿฑ Your blogger's will power is being put to a severe test, you all! ๐ŸŽง


Let's Get Ourselves Acquainted

To acquaint you better with the especially valuable aspects of The Master Algorithm, let's have ourselves a peek at its table of contents ๐Ÿฎ
Prologue
1 The Machine-Learning Revolution
2 The Master Algorithm
3 Hume’s Problem of Induction
4 How Does Your Brain Learn?
5 Evolution: Nature’s Learning Algorithm
6 In the Church of the Reverend Bayes
7 You Are What You Resemble
8 Learning Without a Teacher
9 The Pieces of the Puzzle Fall into Place
10 This Is the World on Machine Learning
Epilogue
Acknowledgments
Further Readings
Index
The Master Algorithm covers a lot of territory, and does it really well!

Get to Know the Larger Context in Which Deep Learning Operates

First let's listen to what one of the world's most distinguished computer scientists has to say about Domingos' The Master Algorithm ๐ŸŒŽ
This book is a sheer pleasure, mixed with education. I am recommending it to all my students, those who studied machine learning, those who are about to do it and those who are about to teach it (italics mine).
~ Judea Pearl, Professor of Computer Science, UCLA and winner of the A. M. Turing Award
Allow me to add this much: The undeniably profound impact of Judea Pearl on our field is writ large in our field's literature. Here, I will refer you to merely two resources: (1) This first resource is a bit diffuse and is on the algorithms side of our field, while (2) this second resource is much more pointed in that a classic in our field—Koller and Friedman's stellar book entitled Probabilistic Graphical Models—has the following glowing words for the inspiring influence of Pearl's work: In particular, to locate the aforementioned classic, head over to the subsection entitled "The Book to Read After Getting Comfortable with Linear Algebra" under our chat about yet another classic—the modern classic on deep learning of course—the inimitable book entitled simply Deep Learning (The MIT Press) by Goodfellow, Bengio and Courville.

But first, we have here Koller and Friedman acknowledging and paying tribute to Pearl's inspiring
work ๐ŸŽ“
Much of our core views on probabilistic models have been influenced by Judea Pearl. Judea through his persuasive writing and vivid presentations inspired us, and many other researchers of our generation, to plunge into research in this field.
~ Daphne Koller and Nir Friedman, in the Acknowledgments section of Probabilistic Graphical Models: Principles and Techniques (The MIT Press)
To paraphrase what one of my programming heroes—the late John Vlissides, who was the "V" in the "GHJV" which was the Gang of Four (GoF)—had to say about legendary programmer Robert “Uncle Bob” Martin's book Agile Software Development, Principles, Patterns, and Practices (Pearson), "When Pearl speaks, better listen up" ๐Ÿ“ฃ 

Having underscored the point that Domingos' The Master Algorithm is "all stuff and no fluff"—otherwise, you wouldn't have a distinguished computer scientist like Judea Pearl profusely praising it—let's return to a more detailed coverage of what The Master Algorithm has to offer you. For one thing, it's replete with engaging, relevant, and simply marvelous anecdotes that inexorably pull the reader deeper into the vortex of the field. Take this endearing anecdote for example in which Domingos tells us about
[Geoff] Hinton, a psychologist turned computer scientist and great-great-grandson of George Boole, the inventor of the logical calculus used in all digital computers, is the world’s leading connectionist. He has tried longer and harder to understand how the brain works than anyone else. He tells of coming home from work one day in a state of great excitement, exclaiming "I did it! I've figured out how the brain works!" His daughter replied, "Oh, Dad, not again!" Hinton’s latest passion is deep learning, which we’ll meet later in this chapter. He was also involved in the development of backpropagation, an even better algorithm than Boltzmann machines for solving the credit-assignment problem that we’ll look at next (italics mine).
I don't know about you, but I re-read that passage several times over to savor the delightful father-daughter moment that must've been ๐Ÿ‘ฑ ๐Ÿ‘ง

Another thing I liked a lot about The Master Algorithm is that Domingos has clearly taken great care to ensure that the narrative flows smoothly. And take it from me, someone who has been known to read a book or two, and even made an international reputation for himself by thinking once or twice a week—much as I noted in response to a reader comment elsewhere—the narrative of The Master Algorithm flows seamlessly, like honey. Not mollasses, but honey, to be sure; not viscous, but easygoing and light ๐Ÿฏ

Engaging Style, Comprehensive Coverage, and a Model of Clarity

I also liked a lot that Domingos liberally sprinkled The Master Algorithm with copious signposts to help the reader navigate the narrative with the greatest of ease; I would be pressed hard to think of another book that has accomplished this with greater finesse! Here, as an example, and much earlier in the book, we have Domingos telling the reader how 
The second goal of this book is thus to enable you to invent the Master Algorithm. You’d think this would require heavy-duty mathematics and severe theoretical work. On the contrary, what it requires is stepping back from the mathematical arcana to see the overarching pattern of learning phenomena; and for this the layman, approaching the forest from a distance, is in some ways better placed than the specialist, already deeply immersed in the study of particular trees. Once we have the conceptual solution, we can fill in the mathematical details; but that is not for this book, and not the most important part (italics mine).
Another terrific feature of The Master Algorithm is that it provides great context for each and every topic as the narrative unfolds over the span of the book. Here, for example, Domingos is enlightening the reader on the delicate connection between the "volume of data available for analysis" and the "corresponding number of hypotheses that survive" ๐Ÿ”Ž
Bottom line: learning is a race between the amount of data you have and the number of hypotheses you consider. More data exponentially reduces the number of hypotheses that survive, but if you start with a lot of them, you may still have some bad ones left at the end... You can even figure out in advance how many examples you’ll need to be pretty sure that the learner’s chosen hypothesis is very close to the true one, provided it fits all the data; in other words, for the hypothesis to be probably approximately correct. Harvard’s Leslie Valiant received the Turing Award, the Nobel Prize of computer science, for inventing this type of analysis, which he describes in his book entitled, appropriately enough, Probably Approximately Correct.
Is that good or what? And oh goodness, I'm yet again resisting the temptation to digress a tad into the rip-roaring fun book that Probably Approximately Correct is—yep, your blogger's will power is being put to the test, yet again, and this is all for you ๐Ÿ‘บ

Great Sense of Humor

While I haven't met Domingos in person, yet, I can clearly tell that he's got a great sense of humor! In a section in The Master Algorithm, where he is regaling us with the crucial role of analogy in machine learning, he begins that section by posing a question to the reader:
Is there anything analogy can’t do? Not according to Douglas Hofstadter, cognitive scientist and author of Gรถdel, Escher, Bach: An Eternal Golden Braid. Hofstadter, who looks a bit like the Grinch’s good twin, is probably the world’s best-known analogizer. In their book Surfaces and Essences: Analogy as the Fuel and Fire of Thinking, Hofstadter and his collaborator Emmanuel Sander argue passionately that all intelligent behavior reduces to analogy. Everything we learn or discover, from the meaning of everyday words like mother and play to the brilliant insights of geniuses like Albert Einstein and ร‰variste Galois, is the result of analogy in action.
My hardcover copy of The Master Algorithm is lit up with highlighter marks; I might as well have dipped the whole book in flourescent highlighter ink ๐Ÿ˜‰ This book has just a ton of great things going for it!

Rejoice

It's all in here, folks. Rejoice ๐ŸŽถ Your search for the resource from which to gently learn all about the larger context for deep learning is now officially over. All we to do now is set up camp and start reading ⛺ And hey, you do know what I mean, don't you? I'm not offering any learning camps—at least not at this time anyway—just mentioning the camping metaphor FWIW ๐ŸŽช

Judgment

I love this delightful book ๐Ÿ’• Don't miss it. It will help you gently learn—at your own pace, at your own time—all about the larger context in which the specialized subject of deep learning operates. It's hard to imagine anyone other than Domingos having the guts to take on such an ambitious goal (i.e. covering the whole expanse of the machine learning field in a comprehensive and engaging style) and then pulling it off as successfully as Domingos clearly has ๐Ÿ†

2. Deep Work: Rules for Focused Success in a Distracted World (Grand Central Publishing) by Cal Newport ๐Ÿ‹


Appreciating the Guts of Learning (and Working) Deeply

Pardon my admittedly visceral choice of words ("guts" and all) in the subheading above; it was just that I wanted to drive the point home ๐Ÿš— (This was visceral alert #1). With that, we now come to the second of the two books in this essay. And much as I said at the outset, I'm acutely aware of the gnawing sensation that purists in the field of deep learning would have me drawn-and-quartered for selecting the books for this essay: While the phrase "deep learning" makes a handful of appearances in of these two books—in The Master Algorithm to be precise—that phrase doesn't so much as make a single appearance in the other book ๐Ÿ˜ฑ

And thus it is that we now come to this next, remarkable book that's entitled Deep Work: Rules for Focused Success in a Distracted World (Grand Central Publishing) by Cal Newport. This would be a great time to remind ourselves that the field of deep learning
  • ...does not exist in a vacuum
  • ...has deep roots in machine learning (ML)
  • ...is not a fad
  • ...is not hype
  • ...has become inextricably enmeshed with every discipline under the sun
Other than the tangential observation that "hope springs eternal in the heart" of your blogger, I trust that the list above suffices to convince you all to rise in unison and thwart all would-be attempts—potentially from purists who would violently demur at my selection of books for this essay—at pillorying your blog's author for not being a purist, I also hope that you're coming to realize the even larger, global context within which human learning itself operates. We truly need to get a good grip on the ideals we want to strive for, or else risk ending up somewhere else, somewhere on a trajectory where we really don't want to be ๐ŸŒ

My choice for selecting Deep Work for this essay, I believe, is justified by the remarkable effectiveness with which it tackles the abovementioned ideas and ideals ⛵ As participants in the community of readers—with me serving as your host on this blog—you are surely not intimidated by taking on some bold and novel explorations, are you? True to the title of Deep Work, to its subtitle, to be precise—Rules for Focused Success in a Distracted World—this remarkable book takes on the ambitious goal of laying bare the gory details of how you, too, can achieve focused success in a distracted world ๐Ÿฉ

I might as well add: Pardon my admittedly visceral choice of words ("gory" and all) above; it was just that I wished, once again, to drive the point home ๐Ÿš™ (And yes, this was visceral alert #2) ๐Ÿ˜‰


Riveting Style of Presentation

Deep Work is written in an engaging style and is highy readable. On top of that, it's eminently substantial. Now how about that for a powerful combination? While I haven't met Cal in person, yet, I've followed his work closely for many years. In fact, it's not a coincidence that—and this was many, many moons ago—I had selected Cal for the top spot on my list of top thought leaders. You can read those details elsewhere, but here in a nutshell is how I had introduced Cal:
At the top of my list is Cal Newport, who is just about the most clear-eyed thinker I know of. Cal teaches at Georgetown University in Washington, D.C, where he is an Assistant Professor of Computer Science. What's unique about Cal are his uniquely original insights, which he shares with the world through his Study Hacks Blog: Decoding Patterns of Success. As the name of his blog signals, his posts seek to capture the essence of achieving meaningful success through wide-ranging, engagingly written, and eminently thought-provoking discussions. 
The reason I now mention his academic credentials—he earned his Ph.D. from MIT, and graduated from Dartmouth College—is that some of the most elegantly stellar thinking I've ever come across is regularly on display in his blogs (italics mine—or as the author of the preceding words, did I even need to point that out? Meta-recursion alert, folks!) ๐Ÿ‘ป
At first blush, the connection of Deep Work to the field of deep learning—in as it relates to helping you master deep learning—may strike the casual observer as even more tenuous than the connection which the other book (i.e. The Master Algorithm) has to the same goal. So let's "unwrap" this a bit, as we would unwrap a birthday present ๐ŸŽ

Unwrapping and Unpacking the Idea

And don't worry about getting tangled up in this whole "unwrapping" and "unpacking" business: I'll digress ever so briefly here, and that purely to illustrate a point, hopefully making the point all the more vivid in your mind and hence memorable. So here's the deal—much as we explored in an essay elsewhere—we're going to look at a related metaphor from the Clojure programming language (which just happens to be a Lisp dialect, and I happen to have more than a passing interest in all things Lisp) ๐ŸŽƒ

Remember how sequential "destructuring" represents a sequential data structure as a Clojure vector within a let binding? Or how built-in operations support the deconstruction of a syntax object, the composition of new syntax objects from old ones?

Wait, wait, oh wait! I need to get you a more apt metaphor that truly vivifies the notion I have in mind for the "unpacking" business I mentioned above. So let's rewind a bit and instead refer ourselves to an answer I had offered in response to a reader comment elsewhere ๐Ÿ˜Ž

Likely the greatest hurdle to a deep understanding of what it takes to achieve focused success in a distracted world is that we're actually swimming in that world. We miss, therefore, the proverbial forest for the trees ๐ŸŒฒ๐ŸŒณ๐ŸŒด๐ŸŒฒ๐ŸŒด๐ŸŒณ

Unwrapping and Unpacking the Idea Some More

We need to think at a different level of abstraction, and this is not terribly different from what I think Peter Seibel had in mind when he was explaining in his stellar book entitled Practical Common Lisp (Apress) that grokking macros is impeded, ironically enough, because that they’re so well integrated into the language. Yep, exactly, and so we miss yet again the proverbial forest for the trees ๐ŸŒฒ๐ŸŒณ๐ŸŒด๐ŸŒฒ๐ŸŒด๐ŸŒณ

And rather than leave it dangling, it's worthwhile to bring the preceding thought to closure—not Clojure, mind you (pun intended, for a change)—so let's hear Seibel himself telling the reader in Practical Common Lisp about the notion of how ๐Ÿ˜†
Perhaps the biggest barrier to a proper understanding of macros is, ironically, that they’re so well integrated into the language. In many ways they seem like just a funny kind of function— they’re written in Lisp, they take arguments and return results, and they allow you to abstract away distracting details. Yet despite these many similarities, macros operate at a different level than functions and create a totally different kind of abstraction (italics mine).
And that's precisely where Deep Work comes in, offering in its pages the ideas—or "rules", to use his vernacular—that are at just the right level of abstraction. That, IMHO, is what makes the book click, YMMV ๐Ÿš™

True to its title, to its subtitle, to be precise—Rules for Focused Success in a Distracted WorldDeep Work takes on the ambitious goal of laying bare how you, too, can achieve focused success in a distracted world

Let's Get Ourselves Acquainted

To acquaint you better with the solidly practical aspects of Deep Work, let's take a peek at the table of contents:
Welcome
Introduction
PART 1: The Idea
Chapter 1: Deep Work Is Valuable
Chapter 2: Deep Work Is Rare
Chapter 3: Deep Work Is Meaningful
PART 2: The Rules
Rule #1: Work Deeply
Rule #2: Embrace Boredom
Rule #3: Quit Social Media
Rule #4: Drain the Shallows
Conclusion
Did you see Rule #3—Quit Social Media—and found yourself groaning, and muttering under your breath some select words for your blogger, saying something like, "Akram, you're nothing but trouble. Your digressions notwithstanding, you're now asking us to contemplate (gasp) a life without social media?!" Sorry that I burst the bubble so unceremoniously:  Okay, so I didn't say this was going to be easy ๐Ÿ‘ป

All this after having mentioned elsewhere—in an exploration of algorithms, of all the places—how it is algorithms that power, among many other worldwide software offerings of course, Facebook's News Feed. And how, as much as we may be loathe to admit it, the Facebook News Feed is where many of us love to waste our time ⛱

But guess what? Yep, and while you don't have to go cold turkey, we've got to brace ourselves for cultivating some discipline in there ⛩

Oh, did you also notice Rule #4—Drain the Shallows—and found yourself cringing and reflexively think whether this has got anything to do with "draining the swamp"? Let me assure you, it does not. Again, as a loyal American, I respect the views of readers from all political persuasions; I was simply using poetic license to express mine, and that too in passing ⛷

A Definition and a Hypothesis

You will find in the introductory chapter of Deep Work a definition first ๐ŸŽน
Deep Work: Professional activities performed in a state of distraction-free concentration that push your cognitive capabilities to their limit. These efforts create new value, improve your skill, and are hard to replicate (italics mine).
...and then the hypothesis itself ๐ŸŽถ
The Deep Work Hypothesis: The ability to perform deep work is becoming increasingly rare at exactly the same time it is becoming increasingly valuable in our economy. As a consequence, the few who cultivate this skill, and then make it the core of their working life, will thrive (italics mine).
Finally, Cal points out with gravitas, and very correctly so—methinks that the author of Deep Work is on to something really, really important—what the implications of deep work are, beginning with these stark words, reminding the reader that ๐ŸŠ
Deep work is not, in other words, an old-fashioned skill falling into irrelevance. It’s instead a crucial ability for anyone looking to move ahead in a globally competitive information economy that tends to chew up and spit out those who aren’t earning their keep. The real rewards are reserved not for those who are comfortable using Facebook (a shallow task, easily replicated), but instead for those who are comfortable building the innovative distributed systems that run the service (a decidedly deep task, hard to replicate). Deep work is so important that we might consider it, to use the phrasing of business writer Eric Barker, "the superpower of the 21st century." (italics mine)
It just so happens that the italicized phrase above refers to something I do for a living. And so it is that I can attest to the verity of Cal's assessment.

Much like the other book that we explored earlier—The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World—Cal's book is filled to overflowing with engaging, relevant, and simply delightful anecdotes that inexorably pull the reader deeper into the vortex of the business of learning how to focus meaningfully.

    Standout Feature

    The number one standout feature of Deep Work is that its author speaks from deep experience. I didn't pass this judgment lightly, either; as I said at the outset, I've followed Cal's work closely for many years, which has given me ample time to form my impressions and opinions. I would go so far as to say that Deep Work is his best work yet ๐Ÿ’ฐ

    Another standout feature of Deep Work is its uncompromising commitment to doggedly pursuing the ins and outs of how you, too, can achieve focused success in a distracted world ๐ŸŽช

    It's this uncompromising commitment to help the reader that won me over, many times over!

    You have to check out Deep Work for yourself. Here, I'll merely mention in passing some fascinating anecdotes that can light up your experience of reading it and make the book so engaging:
    • How Cal—a theoretical computer scientist who performed his doctoral training in MIT’s Theory of Computation group—talks about spending a decade cultivating the ability to concentrate on hard things. And his intriguing telling of a MacArthur "genius grant" winner who is not on Twitter and who published sixteen papers last year.
    • The "grand gesture strategy" in which J.K. Rowling took extreme steps to complete The Deathly Hallows, the final book in her Harry Potter series. "As I was finishing Deathly Hallows there came a day where the window cleaner came, the kids were at home, the dogs were barking," Rowling recalled in an interview. And the rest is, as they say, history.
    • Recollections by Cal of his seven years at MIT, when he worked on the site of the institute’s famed Building 20 and how, in MIT lore, "it's generally believed that this haphazard combination of different disciplines, thrown together in a large reconfigurable building, led to chance encounters and a spirit of inventiveness that generated breakthroughs at a fast pace..."

    Judgment

    Dig in to the pages of Deep Work and you'll get a pretty good idea of why I'm babbling excitedly about it! This is a book for someone who is looking for inspiring, practical, gentle—and frankly, sometimes rather stark, bracing, yet salutory—guidance on the crucial theme of mastering the core practices that will enable you to achieve focused success in a distracted world. Don't miss this stellar book ๐Ÿ„


    Connoisseur, n. A specialist who knows everything about something and nothing about anything else ๐ŸŽ“
    ~ Ambrose Bierce (The Devil's Dictionary) 
    I would be most content if my children grew up to be the kind of people who think decorating consists mostly of building enough bookshelves ๐Ÿ“˜
    ~ Anna Quindlen 
    Philosophy, in one of its functions, is the critic of cosmologies. It is its function to harmonise, refashion, and justify divergent intuitions as to the nature of things. It has to insist on the scrutiny of the ultimate ideas, and on the retention of the whole of the evidence in shaping our cosmological scheme. Its business is to render explicit, and—so far as may be—efficient, a process which otherwise is unconsciously performed without rational tests ๐Ÿ”ญ
    ~ Alfred North Whitehead (English mathematician and philosopher)

    An Invitation ๐Ÿ“ฃ

    In the end, I invite your comments—Having now read the brief take each on the books above ๐Ÿ’ค
    • Do you find that your experience of reading any of these popular books was different? ๐Ÿข
    • Did I perhaps not cover some qualities, and which are the ones that you actually found the most helpful as you learned the larger milieu within which (deep) learning operates? ๐ŸŒŽ
    • Did I leave out any of your favorite deep learning popular books? ๐Ÿš›
    My hope is that the two brief vignettes will help you in your journey to grokking (deep) learning!

    Aha, there it is, flanking the left-hand side of the motley crew (or is it crue?!) of a ragtag smattering of books in the pic above—The Master Algorithm—bedecked with a carefully chosen, color-coded army of tape flags ๐ŸŽ‰ Come to look at it a bit more, the neighboring books of The Master Algorithm, too, are lit up with those trusty tape flags ๐Ÿ™‹

    You can read up more on "the method to the madness" elsewhere—specifically in the context of when we had dived into the pages of Algorithm Design and Applications—where I had quoted Elizabeth Dunn (an anthropologist at the University of Colorado–Boulder) who, in turn, had been quoted in the wise and scintillating pages of the book The Distraction Addiction: Getting the Information You Need and the Communication You Want, Without Enraging Your Family, Annoying Your Colleagues, and Destroying Your Soul how
    Consequently, "I do almost all of my labor and thought-intensive reading in print form." For me, serious reading involves marking up, underlining, and annotating books; it’s a martial art, and it requires the material engagement and support that paper can provide and that screens conspicuously lack. Fifty years ago, MIT professor and hypertext pioneer Vannevar Bush imagined we’d do this kind of intensive, interactive, relational reading on the memex, an electronic system he proposed in 1945. Today, people who really have to know their stuff still choose paper (italics mine).
    Oh, I nearly forgot to mention, this, too, in connection with the (four) books in the pic above. Did any of those books grab your attention? If so, do please consider posting your comments to let other readers—and me—know about which one of these gems you would like to hear more about, won't you? At this time, we've already covered one of them (The Master Algorithm) in some depth.

    But how about the rest of those books, the three that remain untouched in their pristine beauty ? Do please keep in mind that we all stand to benefit from ongoing dialogs by way of comments on this blog ๐Ÿ’


    My Passion for Sharing ๐Ÿ’

    Thought I'd mention: I don't make a single cent out of the links such as those that I provide in this essay and elsewhere. For exampe, the two links atop this essay, incidentally, will take you to their respective Amazon.com page each. Similar links, in here and in the other essays elsewhere on this blog, might take you, for example, to The MIT Press, and so on and so forth ๐Ÿ‘ž ๐Ÿ‘ก ๐Ÿ‘Ÿ ๐Ÿ‘ข ๐Ÿ‘ฃ 

    I write purely from my passion for sharing with you what I have learned, and what I continue to learn, about the ever-evolving paradigms in our software development industry. There is, I hasten to add for those so inclined, absolutely nothing wrong with monetizing a blog—I covered this and other related points in some depth in an essay elsewhere on technical blogging—after all, we've all heard countless times how
    The business of America is business๐Ÿ’ฐ
    When it comes to blogging, I'm squarely an explorer of ideas; I share with you what's on my mind at any given point in time; I relish doing so, and want to serve you well ๐ŸŽ

    Remember, too, the wisdom in these words:
    Of making many books there is no end, and much study is a weariness of the flesh
    What I will ask of you is to please consider posting your comments to let other readers—and me—know your thoughts, impressions, observations, anything really: Relax ⛱ We're in the comfortable, unencumbered orbit of programming digressions central ⛳

    After all, and much as I said above, do please keep in mind that we all stand to benefit from ongoing dialogs by way of comments on this blog ๐Ÿš€


    A Digression on the Postage Stamps Theme...

    Hey, did anyone notice the postage stamp theme in the previous installment to this series of deep learning essays? Hmm... Perhaps you're scratching your head, and quizzically wondering aloud, "What's that you just said, Akram?". That's fine, go ahead and please get your coffee—I'll wait for you. Once our caffiene has kicked in, we'll take in this memorable quote ☕
    Neither snow nor rain nor heat nor gloom of night stays these couriers from the swift completion of their appointed rounds.
    Unofficial motto of the of the United States Postal Service (USPS) that's long been associated with the American postman
    So I was once a budding philatelist—that's the million dollar word for a stamp collector—and I paid a tribute of sorts to the theme of stamp collecting by embedding all the pics (which adorned the previous deep learning essay) with postage stamp borders. Woohoo, I sure got a kick out of doing that! ⛹

    Speaking of stamp collecting, I was once also an eager numismatist—now that's the million dollar word for a coin collector—so the theme of coin collecting might crop up as the underlying theme in a future essay. A word to the wise ๐Ÿช Extra brownie points for the first few readers who spot the coin collecting theme in a future essay, and report their observation by way of a comment or two on this blog ๐Ÿ’ฏ

    A Behind-the-scenes Exclusive


    We First Set the Scene ⛑

    Finally, we have here an exclusive behind-the-scenes tour of the props for this essay ๐ŸŽฌ Much as I mentioned at the outset, where I had invited you to settle for a quick metaphor: Think Monty Python's Flying Circus, complete with how a prop—yep, the prop perched perilously on the prodigious revolving chair in the pic above—painfully fell on your blogger's foot, though he escaped unscathed, mostly ๐Ÿ‘ป

    I mean, what's the big deal with a Bandaid here or there, to heal an injury here or there, right? Oh, if only you knew half the things your blogger does to make you—yes, the thousand and thousands of you who come here every month—to keep coming back for even more ⛺

    Well, here we are, having re-emerged from the dark depths of the deep learning ocean, following our dive to retrieve all the deep learning oysters and pearls we could get our hands on, both past and present. Remember how you had been promised at the very outset—and here I nudge your attention to the kludgy props and stuff in the pic above—a sneak peek into how your blogger went about improvising and capturing the various pics? ⛄

    For one thing, the whole prop setup made me think recursion; yeah, leave it to me to cook up the most interesting connections between a variety of seemingly unrelated ideas and themes. The theme of recursion, in turn, got me thinking to a hilarious and inimitable episode from the Monty Python's Flying Circus series. That's where we go next ⛷

    And Then We Dive Right In ๐ŸŠ

    So let's have a look at the hilarious meta-recursion going on in the Monty Python's Flying Circus side-splitting funny episode entitled The Lost World of Roiurama—in particular the segment "Who's filming us?"—from which I'll share the following, ever-so-brief dialog in which our four intrepid explorers find themselves hopelessly lost in a jungle. Here, then, are some of the plaintive and frantic thoughts of those explorers ๐ŸŒณ๐ŸŒณ๐ŸŒณ๐ŸŒณ
    FIRST EXPLORER: Wait a moment!
    FOURTH EXPLORER: What is it?
    FIRST EXPLORER: If we're on film, there must be someone filming us.
    SECOND EXPLORER: My God, Betty, you're right!
    Oh goodness, you really should look that all up for yourself in Monty Python's Flying Circus: Complete and Annotated...All the Bits (Black Dog & Leventhal; Complete and annotated ed edition) by Luke Dempsey or the video itself on YouTube.

    And should you be moved to inquire into the origins of how the name "Python" came to be chosen by Guido van Rossum—the creator of the incredibly popular Python programming language and known to the Python community as its "Benevolent Dictator For Life" (BDFL)—I invite you to check this out as well in the context of whether Python is suitable for Machine Learning, and indeed Deep Learning itself...

    Till we meet next time, you have a great, productive week ๐ŸŽฏ

    Collage of Pics and Lyrics ๐ŸŽธ

    I'm in the mood for a melody
    I'm in the mood for a melody
    I'm in the mood 
    I can make you dance - I can make you sing
    I can make you dance - I can make you sing
    If you want me to 
    Any little song that you want to sing
    Little songs that you want to sing
    Any song will do 
    Any little song that you want to sing
    Little songs that you want to sing
    It's up to you 
    Little songs that you want to sing
    A little song that you want to sing
    You're blue 
    I'm in the mood, I'm in the mood, I'm in the mood
    I can write it on the door - I can put it on the floor
    I can do anything that you want me for
    If you want me to 
    I can do it right - I can do it wrong
    'Cause a matter of fact it'll turn out to be strong
    If you want me to
    ~ Robert Plant (Lyrics from In The Mood)

    Sunday, September 24, 2017

    Best Deep Learning Books (Pragmatic)

    The Quotes ๐Ÿ“Œ

    How deep is your love,
    How deep is your love
    I really mean to learn
    ๐ŸŒŠ
    ~ Bee Gees (Lyrics from "How Deep Is Your Love")
    The field of AI is broad and has been around for a long time. Deep learning ๐Ÿ‘ถ  is a subset of the field of machine learning ๐Ÿ‘ฆ  which is a subfield of AI ๐Ÿ‘จ
    ~ Josh Patterson and Adam Gibson—Deep Learning: A Practitioner's Approach (O'Reilly Media)

    The computing field is always in need of new cliches: Banality soothes our nerves ๐ŸŸ
    ~ Alan Perlis

    Ironically, abstract and formal tasks that are among the most difficult mental undertakings for a human being are among the easiest for a computer... A person’s everyday life requires an immense amount of knowledge about the world... Computers need to capture this same knowledge in order to behave in an intelligent way. One of the key challenges in artificial intelligence is how to get this informal knowledge into a computer ๐Ÿ‘ฃ
    ~ Ian Goodfellow, Yoshua Bengio and Aaron Courville—Deep Learning (The MIT Press)

    A year spent in artificial intelligence is enough to make one believe in God ๐Ÿ˜‡
    ~ Alan Perlis
    All I know is the way I feel
    When it's real, I keep it alive
    The road is long, there are mountains in our way
    But we climb a step every day 

    ~ Joe Cocker (Lyrics from "Up Where We Belong")

    Deep Learning (Pragmatic) Books: The List ๐Ÿ‘• ๐Ÿ‘”

    And here we have the two stellar books at which we'll soon be taking an opinionated look, in turn:
    1. Deep Learning (The MIT Press) by Ian Goodfellow, Yoshua Bengio and Aaron Courville ๐Ÿณ
    2. Deep Learning: A Practitioner's Approach (O'Reilly Media) by Josh Patterson and Adam Gibson ๐Ÿ‹


    Preamble ⛱


    While we won't be exploring either one of the two fantastic books in the pic above—or the inimitably brilliant magazine MIT Technology Review—in this essay, it may be good to know that both are classics in the field of AI and you can't go wrong with either one of those two books! Coincidentally, Ian Goodfellow is featured as one of MIT TR's 35 innovators under 35 (in the MIT TR issue that's on the right-hand side in the pic above). And you can read details in there about how "Goodfellow, now a sta‘ff research scientist with Google Brain, wondered if two neural networks could work in tandem.... Goodfellow coded the first example of what he named a generative adversarial network, or GAN. The dueling-neural-network approach has vastly improved learning from unlabeled data." Great stuff ๐Ÿ„

    Introduction ๐Ÿ”Ž


    The Nuts and Bolts of Deep Learning: A First Definition

    Much as I mentioned in the first installment in this series of essays on deep learning, I have more than a passing interest in deep learning. Whereas the first installment focused on the foundational aspect of deep learning, the focus of the second installment is squarely on its pragmatic aspects ๐ŸŽ

    To set the stage, let's first hear what Goodfellow, Bengio and Courville have to say—in Deep Learning (The MIT Press), which is perhaps the definitive book on the subject—by way of background to the crucial relevance of deep learning to solving pressing problems in machine learning (ML). Thus, in the introductory section of Deep Learning, the authors remind us how ๐ŸŽฏ
    Many artificial intelligence tasks can be solved by designing the right set of features to extract for that task, then providing these features to a simple machine learning algorithm... For many tasks, however, it is difficult to know what features should be extracted...  
    One solution to this problem is to use machine learning [via an] approach [that] is known as representation learning. A major source of difficulty in many real-world artificial intelligence applications is that many of the factors of variation influence every single piece of data we are able to observe. 
    Deep learning solves this central problem in representation learning by introducing representations that are expressed in terms of other, simpler representations. Deep learning enables the computer to build complex concepts out of simpler concepts. The quintessential example of a deep learning model is the feedforward deep network, or multilayer perceptron (MLP)...
    Was the quote above a model of clarity or what? ๐Ÿ˜Ž And so it should not surprise anyone that I selected the stellar book Deep Learning (The MIT Press) for the top spot ๐Ÿ† Much more on it follows soon...

    The Nuts and Bolts of Deep Learning: A Second Definition

    Meanwhile, it just so happens that the book that comes in at the second spot has also got sparkling wisdom to shed on the essence of how the deep learning paradigm came to be. Thus, in a marvelously apt section in that book—and here we're talking about Deep Learning: A Practitioner's Approach (O'Reilly Media) by Patterson and Gibson—that goes under the heading What Is Deep Learning?, the authors clue us in to why ๐ŸŽณ
    Deep learning has been a challenge to define for many because it has changed forms slowly over the past decade. One useful definition specifies that deep learning deals with a “neural network with more than two layers.” The problematic aspect to this definition is that it makes deep learning sound as if it has been around since the 1980s. We feel that neural networks had to transcend architecturally from the earlier network styles (in conjunction with a lot more processing power)   before showing the spectacular results seen in more recent years. Following are some of the facets in this
    evolution of neural networks:
    • More neurons than previous networks
    • More complex ways of connecting layers/ neurons in NNs
    • Explosion in the amount of computing power available to train Automatic feature extraction 
    Patterson and Gibson go on to add that ๐Ÿ™
    For the purposes of this book, we’ll define deep learning as neural networks with a large number of parameters and layers in one of four fundamental network architectures:
    • Unsupervised pre-trained networks
    • Convolutional neural networks
    • Recurrent neural networks
    • Recursive neural networks
    With the definitional machinery in place, let's get our hands on the fun stuff, shall we? ๐ŸŸ ๐Ÿฉ ๐Ÿญ

    What Does "Pragmatic" Mean, Exactly?

    Casting a glance back at my personal journey, we'll soon dive deep into two deep learning books that have proved immensely helpful to me in grokking this intriguing field at the pragmatic level. Okay, so what I've got in mind when I use the word pragmatic is the following composite thinking, all globbed together after I put some deliberate thought into it ๐Ÿ”
    In the context of deep learning, a pragmatic understanding is what you've gained after achieving competency and fluency in the use of the programmatic toolbox that powers the practice of deep learning. Whereas what we looked at previously (in the first installment of this deep learning series) were the foundational aspects of deep learning—think linear algebra and all that other fun mathematical jazz—the focus of this second installment is squarely on its pragmatic aspects: I'm assuming that you're approaching deep learning as a practitioner—you may be an engineer, statistician, analyst, artist—entering or perhaps already practicing the art and science of machine learning (ML) on a daily basis and now want to find out the best resources available on this planet to get your hands dirty with the practice of deep learning.
    You are, in other words, a denizen of the land of technology and the allied arts. The awesomeness of the pragmatic aspects of deep learning lies, at least partially, in its ever-evolving nature; the world of machine learning (ML) has become inextricably enmeshed in pretty much every discipline under the sun. And as those other disciplines evolve, so, too, does deep learning. Talk about exciting times ☕

    Personal Background

    Many of you are longtime readers of this blog, and therefore already familiar with the background—professional, intellectual, technical, and artistic—which I bring to this blog. And hey you, yes, you the know-it-all right there in the back row who said that they're familiar with my, um, unprofessional background, heh, I need to have a word with you in private ๐Ÿ˜‰

    Seriously, though, for new readers of this blog, it's only fair that I share with you an ever-so-brief backgrounder. That way, you'll know my strengths, weaknesses, propensities—and oh yes—my biases as well ๐Ÿ˜‡

    We'll soon dive deep into the ocean of deep learning, but first, keeping in mind what I said above, here then is a backgrounder by way of a handful of pointers. I present them in no particular order, so please don't ascribe any significance to how high—or how low, for that matter—any pointer may appear in the following list, starting with the fact that
    • Directly relevant to this essay, I should point out that while I've read large swathes of the two books reviewed in this essay, I haven't yet read the two in their entirety. I'm getting there, slowly, but surely, one neural network at a time ⏳
    • At the moment, the fields of deep learning, AI, and machine learning (ML)—AI of course being the proud parent of ML—continue to intrigue me, as they have for many years ๐Ÿ‘’
    • In fact, the dissertation I did for my Masters was on the then-esoteric-sounding topic of Pattern Recognition Methods using Neural Network Algorithms. So for example, I know the back-propagation algorithm like, um, the back of my hand ๐ŸŒฟ
    • Speaking of that evocative phrase—back of my hand—look for the hauntingly beautiful lyrics ("I knew the pathway like the back of my hand") later, from the Lily Allen song Somewhere Only We Know ๐Ÿ‡ ๐Ÿป
    • Please know that the pointer immediately above was not random; it was my attempt at subtly preparing you for what my regular readers know all too well as, ahem, "digressions"—the exploration of themes related to the one under discussion๐Ÿšถ
    • Please know, too, that my propensity for digressing at the drop of a hat is not quite random either; I've gone to some lengths to explain exactly that, elsewhere ๐Ÿšฉ
    • My full time professional focus currently is on designing and crafting scalable enterprise software though the fields of deep learning, ML, and AI continue to intrigue me ๐Ÿš€
    • Finally, to get a flavor for your blogger's pursuits in the areas—as are not related, at least not directly, to AI, ML, and deep learning—I invite you to check out some fun stuff by way of the handful of links that follow now, closing out this randomly-ordered list of pointers ⛵
    • My irreverent take on the role of eventual consistency in distributed systems ๐Ÿง
    • A decidedly feisty dive into the best that there is in the universe of algorithms ⛷
    • And ah yes, I took the reactive programming bull by its horns, and not too long ago, either ๐Ÿš…
    • In rounding out these revelations, I might as well mention the fun my readers and I had with a digression through a seemingly staid subject—a deep dive into blending object orientation (OOP) and functional programming (FP) ๐Ÿ”ง
    Okay, so now we're standing on the edge of the diving-board that's perched at some height above the swimming pool of deep learning knowledge, ready to take a dive into the placid—and sometime not-too-placid—waters of deep learning ๐Ÿ’ง ๐ŸŠ ๐ŸŒŠ 

    And since we've already had a quick, drive-by look at the background (professional, intellectual, technical, and artistic) that I bring to this blog, let's bring closure to that thought by noting that—and I went into some detail on precisely this point in my response to a recent reader comment—with thousands of readers coming to this blog every month to read these essays, the responsibility of honoring the time of so many readers has irrevocably altered my view of how an essay ought to be written. I really can't go into more detail than that here; I invite anyone interested, though, to check out, by all means, musings on exactly this theme in the aforementioned response ๐Ÿ”ฆ

      1. Deep Learning (The MIT Press) by Goodfellow, Bengio and Courville ๐Ÿณ


      A Phenomenally Readable Introductory Book

      Let's please have you squint at the pic above ๐Ÿ” You'll notice some decidedly pragmatic books flanking this modern classic on each side: (1) On its left-hand side is another classic, this one by Martin Fowler, entitled Refactoring: Improving the Design of Existing Code (Addison-Wesley Professional) and (2) on its right-hand side is the chock-full-of-practicalities book entitled Scalable Internet Architectures (Sams Publishing) by Theo Schlossnagle—I mean, books don't get any more practical than that, unless I've been living in a hole ๐Ÿšง

      And so it is that this modern classic (Deep Learning) is right at home with its pragmatic brethren—and sisters, to be sure ร  la maniรจre de Rosie the Riveter—on the bookshelf rack in the pic above.

      In my mind, Deep Learning is to the world of deep learning what the Gang Of Four (GoF) book is to the world of software design patterns. Yes, the former is that good, and let me tell you why. For starters, Deep Learning is an incredibly polished work, much in the tradition of the highly refined GoF book; I'll boldly venture out to say that Deep Learning has set an even higher standard for what a technical book—or non-technical book for that matter—can offer to enhance the reading experience in an endearing and pleasing way. In a similar vein, you may wish to also take a peek at another awesome and incredibly well done book, another fav of mine, entitled Refactoring to Patterns (Addison-Wesley) by Joshua Kerievsky ๐Ÿ€

      The Book to Read After Getting Comfortable with Linear Algebra

      Some of you may recall what had I said, in connection with Deep Learning, in the first installment of this series of deep learning essays—that it serenely stands guard in the background, while the nuts-and-bolt Linear Algebra: A Modern Introduction (Brooks Cole) by David Poole basks in the limelight with brazen effrontery ๐ŸŽฌ

      Okay, so once you're ready to take on the world of deep learning, you simply can't go wrong with this modern classic: Deep Learning (The MIT Press) by Ian Goodfellow, Yoshua Bengio and Aaron Courville. For one thing, its editing has clearly been finessed to near-perfection. Look, I'm a big fan of books from The MIT Press—a notable exception being Introduction to Algorithms 3rd Edition (MIT Press) by Cormen, Leiserson, Rivest, and Stein—and Deep Learning did not disappoint, not one bit. The long wait for its publication was well worth it!

      Oh, and lest anyone cry heresy regarding my judgement of CLRS (those being the initials of that other book's authors: Cormen, Leiserson, Rivest, and Stein), please know that CLRS does make for a good research book, for finding citations and that sort of thing—for that it's probably unbeatable—since it happens to be replete, to overflowing, with copious cross-references to the vast literature on algorithms. Given its intimidating style, though, I'm not still not convinced that it's a good introductory book on the subject. I also suggest that you look up a much more detailed dsicussion, should you be interested, in yet another irreverent essay elsewhere on the joy of algorithms ⛑

      Oh goodness, don't anyone even get me started on my high esteem of books from The MIT Press; that could easily take up the remainder of this essay. Here, I'll mention in passing merely two of my favs from those esteemed publishers:
      As for Deep Learning, it simply can't be beat for comprehensiveness, engaging style of presentation, and clarity. At this time, I'm not aware of any other book that introduces and covers this fascinating field better than Deep Learning ๐Ÿ†

      Rejoice

      It's all in here, folks. Rejoice ๐ŸŽถ Your search for the resource from which to best learn deeply about deep learning is now officially over. All we to do now is set up camp and start reading ⛺ And hey, you do know what I mean, don't you? I'm not offering any learning camps—at least not at this time anyway—just mentioning the camping metaphor FWIW ๐ŸŽช

      Let's Get Ourselves Acquainted

      To acquaint you better with the profoundly valuable pragmatic aspects of Deep Learning, let's have ourselves a peek at its table of contents ๐Ÿ“‚
       I Introduction
      1. Linear Algebra
      2. Applied Math and Machine Learning Basics
      3. Probability and Information Theory
      4. Numerical Computation
      5. Machine Learning Basics 
      II Deep Networks: Modern Practices
      6. Deep Feedforward Networks
      7. Regularization for Deep Learning
      8. Optimization for Training Deep Models
      9. Convolutional Networks
      10. Sequence Modeling: Recurrent and Recursive Nets
      11. Practical Methodology
      12. Applications 
      III Deep Learning Research
      13. Linear Factor Models
      14. Autoencoders
      15. Representation Learning
      16. Structured Probabilistic Models for Deep Learning
      17. Monte Carlo Methods
      18. Confronting the Partition Function
      19. Approximate Inference
      20. Deep Generative Models 
      Bibliography
      Index

      A Learning Experience Imbued With Joy

      Frankly, I'm not aware of a kinder, gentler, and intelligent approach to introducing yourself to the field than through a study of Deep Learning. This book is pretty much all you need to create for yourself a swimmingly good experience, all at your own pace, all on your own time ⏰

      By the way, the other deep learning (pragmatic) book that we'll be talking about later in this essay—Deep Learning: A Practitioner's Approach (O'Reilly Media)—would make a terrific complement to your reading of Deep Learning (The MIT Press), but more on that in just a bit ๐Ÿ’

      The book's three co-authors (Goodfellow, Bengio and Courville)—maybe one day they'll come to be known as the "three amigos" of the deep learning world, much like Rumbaugh, Booch, and Jacobson came to be affectionately known as the "three amigos" of the object-oriented world—share some terrific points to help you get situated. To help you decide whether this is indeed the book for you, they've included a useful introductory section (Section 1.1) entitled Who Should Read This Book? where they tell the reader how ๐ŸŽ“ ๐Ÿ„
      This book can be useful for a variety of readers, but we wrote it with two target audiences in mind. One of these target audiences is university students (undergraduate or graduate) learning about machine learning, including those who are beginning a career in deep learning and artificial intelligence research. The other target audience is software engineers who do not have a machine learning or statistics background but want to rapidly acquire one and begin using deep learning in their product or platform. Deep learning has already proved useful in many software disciplines, including computer vision, speech and audio processing, natural language processing, robotics, bioinformatics and chemistry, video games, search engines, online advertising and finance (italics mine).

      Engaging Style, Comprehensive Coverage, and a Model of Clarity

      Let's try to get a gist of the comprehensiveness, engaging style, and clarity which this fine book can bring to your study of deep learning by looking up, as an example, their take on that sturdy stalwart of this territory, eigendecomposition, as they make mention (in Section 2.7 to be precise) as follows in describing eigendecomposition by remarking on how ๐ŸŽณ
      Many mathematical objects can be understood better by breaking them into constituent parts, or finding some properties of them that are universal, not caused by the way we choose to represent them... 
      Much as we can discover something about the true nature of an integer by decomposing it into prime factors, we can also decompose matrices in ways that show us information about their functional properties that is not obvious from the representation of the matrix as an array of elements. One of the most widely used kinds of matrix decomposition is called eigendecomposition, in which we decompose a matrix into a set of eigenvectors and eigenvalues (italics mine).
      I thought that the passage above beautifully explains the guts of a typical deep learning topic—and that topic just happened to be eigendecomposition—as simply as it can be done, not any simpler, and without in the least diluting the substance and rigor of the topic. Nobody wants that, right? To that I'll simply add that pretty much the entire book carries through with it the same clarity and rigor ✂

      And let's be fair, lest the authors' take above struck you as a tad too conceptual: The field of deep learning is intrinsically conceptual to its core, deeply interwoven as it is with the sublime beauty of math. And I hasten to add, too, lest your blogger's claim—that this essay is all about the pragmatic aspects of deep learning—struck you as a tad too bold. Allow me to quote Paul Graham's sparkling-with-wit opinion on this precise topic when he noted, in his stellar book entitled Hackers & Painters: Big Ideas from the Computer Age (O'Reilly Media), how the landmark programming language Lisp embodies mathematical ideas to perfection in that ๐ŸŽป
      Suddenly, in a matter of weeks, McCarthy found his theoretical exercise transformed into an actual programming language—and a more powerful one than he had intended. So the short explanation of why this 1950s language is not obsolete is that it was not technology but math, and math doesn't get stale (italics mine).

      Peering Into the Future

      I owe you all—perhaps in a future essay—an exploration of some of the best that the glory of math has to offer to practitioners like us. But that will have to wait for another time; should I forget, please remind me, won't you? ⏰

      Meanwhile, and albeit on a different topic, though one that's imbued—perhaps as equally and as deeply as the field of deep learningwith the timeless strands of mathematical rigor and determinism, I recommend that you look up what math has been doing to the world of programming lately ๐Ÿš€


      Crystal-clear Descriptions of Algorithms

      Nearly every chapter of Deep Learning is replete with succinct and crystal-clear descriptions of algorithms. It's hard to convey the remarkable effectiveness and finesse with which the "three amigos" of the deep learning world—the book's three co-authors (Goodfellow, Bengio and Courville)—capture the essence of a ton of crucial deep learning algorithms. Ah, good old algorithms, a subject that remains close to the heart of this computer scientist and software practitioner ๐Ÿ’

      But hard as it surely is to convey that remarkable effectiveness, let's give it a try, shall we? ๐Ÿˆ

      For example, we have in Chapter 18—a chapter with the rather bracing and evocative title of Confronting the Partition Function—an algorithm for those trusty Monte Carlo Markov chain (MCMC) methods gaping at the reader in its full glory. I refer you to (Algorithm 18.1)—"A naive MCMC algorithm for maximizing the log-likelihood with an intractable partition function using gradient ascent".

      Earlier on in the book, you'll find the goods on the early stopping meta-algorithm: (Algorithm 7.1)—"The early stopping meta-algorithm for determining the best amount of time to train. This meta-algorithm is a general strategy that works well with a variety of training algorithms and ways of quantifying error on the validation set".

      And earlier still in Deep Learning, yet, you will run into something of the back-propagation algorithm: (Algorithm 6.5)—"The outermost skeleton of the back-propagation algorithm. This portion does simple setup and cleanup work. Most of the important work happens in the build_grad subroutine of algorithm 6.6".

      Now if only that other book from The MIT Press, the one which I mentioned earlier, the looks-fashionable-to-have-a-copy-of CLRS—those being the initials of the authors of that other book, Introduction to AlgorithmsCormen, Leiserson, Rivest, and Stein—had taken a page from this book (Deep Learning), also of course from The MIT Press, I would have gladly covered it in the two installments (Part 1 and Part2) thus far, in the algorithms series of essays elsewhere on this blog site. But, um, tough luck so far ๐ŸŽฑ

      And frankly, with the publication of Deep Learning, the "three amigos" of the deep learning world have proved that there's absolutely no need whatsoever to resort to a cryptic style of presenting algorithms. Indeed, as the "three amigos" have demonstrated, all that can be accomplished with natural language, and with great clarity. Period ๐Ÿ€

      Oh, and did I mention that Deep Learning is lavishly illustrated, with many of those illustrations in the splendor of full color ๐ŸŽƒ

      Judgment

      I love this book ๐Ÿ’• Don't miss it. It will help build, strengthen, and elevate your pragmatic deep learning skills from the ground up. It's hard to imagine anyone else having the guts to take on such an ambitious goal—cover the field of deep learning in its entirety with comprehensiveness, engaging style, and clarity—and then pulling it off as successfully as have the three co-authors (Goodfellow, Bengio and Courville) of Deep Learning, the "three amigos" of the deep learning world. For its brilliance and polish, I stand by my verdict that Deep Learning is to the world of deep learning what the Gang Of Four (GoF) book is to the world of software design patterns ๐Ÿ†

      2. Deep Learning: A Practitioner's Approach (O'Reilly Media) by Josh Patterson and Adam Gibson ๐Ÿ‹

      Appreciating the Pragmatics of Deep Learning

      In the second spot we have a near-namesake of the book that landed in the first spot, and which was of course simply entitled Deep Learning. So the book landing in the second spot has a subtitle, and is entitled Deep Learning: A Practitioner's Approach. In several ways, Deep Learning: A Practitioner's Approach goes one step further than its near-namesake. For one thing, it is closer to the metal of the application space and goes into the guts of high-quality accompanying implementations—in the Java programming language—of many deep learning algorithms ๐ŸŠ ๐ŸŽฏ

      Should the acronym DL4J already means anything to you, please know that one of the book's two co-authors (Josh Patterson) is active in the open source community, contributing to projects such as DL4J, Apache Mahout, and IterativeReduce. So the acronym DL4J is shorthand for Deeplearning4j ๐ŸŽป Be prepared to get up close and personal with DL4J over the course of poring over the pages of this amazing book ๐ŸŒน

      An Invitation

      But first, and much as I had invited you earlier in this essay—while exploring the book in the top spot—I invite you again to please peer into the pic above ๐Ÿ” Did you notice some decidedly pragmatic books flanking Deep Learning: A Practitioner's Approach on each side? Among other books on the shelf in the pic, (1) on its left-hand side is the classic entitled Refactoring: Improving the Design of Existing Code (Addison-Wesley Professional) by Martin Fowler,  and (2) on its right-hand side is Scalable Internet Architectures (Sams Publishing) by Theo Schlossnagle—books don't get any more pragmatic than that, do they really?

      And so it is that this aptly titled book is right at home with its pragmatic brethren—and sisters, to be sure ร  la maniรจre de Rosie the Riveter—on the bookshelf rack in the pic above.

      Let's Get Ourselves Acquainted

      To acquaint you better with the solidly practical aspects of Deep Learning: A Practitioner's Approach, let's take a peek at the table of contents ๐Ÿ“–
      1. A Review of Machine Learning
      2. Foundations of Neural Networks and Deep Learning
      3. Fundamentals of Deep Networks
      4. Major Architectures of Deep Networks
      5. Building Deep Networks
      6. Tuning Deep Networks
      7. Tuning Specific Deep Network Architectures
      8. Vectorization
      9. Using Deep Learning and DL4J on Spark 
      Appendix A. What Is Artificial Intelligence?
      Appendix B. RL4J and Reinforcement Learning
      Appendix C. Numbers Everyone Should Know
      Appendix D. Neural Networks and Backpropagation: A Mathematical Approach
      Appendix E. Using the ND4J API
      Appendix F. Using DataVec
      Appendix G. Working with DL4J from Source
      Appendix H. Setting Up DL4J Projects
      Appendix I. Setting Up GPUs for DL4J Projects
      Appendix J. Troubleshooting DL4J Installations 
      Index
      In a helpful introductory section entitled What’s in This Book?, the authors give a nice roundup of what to expect going forward, starting with the point that
      The first four chapters of this book are focused on enough theory and fundamentals to give you, the practitioner, a working foundation for the rest of the book. The last five chapters then work from these concepts to lead you through a series of practical paths in deep learning using DL4J:
      • Building deep networks
      • Advanced tuning techniques
      • Vectorization for different data types 
      • Running deep learning workflows on Spark
      My impressions of this fabulously clean and remarkably effective division between theory and practice are something like the following: Between the ๐ŸŽง first four chapters (which reveal the conceptual machinery of deep learning) and the even more valuable (IMHO) ๐ŸŽพ last five chapters (which then go on to divulge the equally crucial practical aspects of deep learning) ๐ŸŽง ๐ŸŽพ

      No Royal Road to the Practice of Deep Learning

      Reminding ourselves of the adage that there is no royal road to geometry—or to deep learning, for that matter—a sustained study of Deep Learning: A Practitioner's Approach will help you immensely in the long run. The authors share the essence of this book's gravitas by telling the reader how
      We designed the book in this manner because we felt there was a need for a book covering "enough theory" while being practical enough to build production-class deep learning workflows. We feel that this hybrid approach to the book's coverage fits this space well (italics mine).
      Okay, so here's the deal—anytime someone make mention of that million dollar word above, "hybrid", I reflexively start thinking of the closely allied notion of "blended" ๐Ÿ“ฌ I strongly feel that conceptual "blending" is an often-overlooked notion that has much to offer to us practitioners. And the reason for that is simple: We need powerful tools in our toolbox if we are to have a good shot at taming complexity in software, and elsewhere. There's actually a ton that I've already written, elsewhere, on precisely that notion through what I recall was a deliberate—and hopefully scientific—analysis of blending object orientation (OOP) ๐ŸŽ with functional programming (FP) ⛩

      Should the reader's interest be piqued, I'll mention in passing that the tour de force book entitled The Way We Think: Conceptual Blending And The Mind's Hidden Complexities by Gilles Fauconnier and Mark Turner (Basic Books) is likely the final word on (conceptual) blending, taken to its logical conclusion. Yes, we need all the tools—conceptual, algorithmic, infrastructural, thematic, AI, probabilistic, you name it—that we can get our hands on ๐Ÿด

      A Bit on Coursera for Context

      To put my thoughts above in context, let's segue a bit. So I earned a certificate online last year from Coursera—specifically a certificate for the Machine Learning (ML) course taught by Andrew Ng, then with Stanford University—which will make you do deep dives (pun was totally unintentional) through the ocean of linear algebra, so I have a pretty good idea of what I'm talking about in the preceding section ๐Ÿ˜

      For the large part, I happily pursue the craft of software development these days, primarily using the Java and Scala programming languages. At the same time, I've remained intrigued enough over the years by ML and deep learning to pick up the Octave programming language for scientific computing; learning the Octave programming language, in fact, came about especially while I was taking the Andrew Ng Machine Learning (ML) course I mentioned above.

      And while we're talking about Coursera, I'll mention in passing that they've got some of the most well-thought and well-designed courses available online; it was actually my passion for the Scala programming language which, by the way, brought me to Coursera in the first place. And that's how I discovered their offering on ML and other cool stuff. Other courses I took—and for which I also earned a certificate each—include the following, which I can highly recommend, should anyone have an interest in this sort of thing ๐ŸŽ“
      • Functional Programming Principles in Scala 
      • Functional Program Design in Scala 
      • Parallel Programming

      Standout Feature

      Okay, so we won't be talking about those kinds of features—such as what you might have had in mind, along the lines of automatic feature extraction—here, since do happen to be deep into deep learning territory ๐Ÿ˜‰ The feature I have in mind here is the copious use of super-helpful "fact boxes" throughout the pages of Deep Learning: A Practitioner's Approach. Let's look at an example to get a better sense for what I'm talking about.
      Machine Learning Versus Data Mining Data
      Data mining has been around for many decades, and like many terms in machine learning, it is misunderstood or used poorly. For the context of this book, we consider the practice of "data mining" to be "extracting information from data." Machine learning differs in that it refers to the algorithms used during data mining for acquiring the structural descriptions from the raw data. Here’s a simple way to think of data mining:  
      To learn concepts
          we need examples of raw data
      Examples are made of rows or instances of the data
          Which show specific patterns in the data
      The machine learns concepts from these patterns in the data
          Through algorithms in machine learning 
      Overall, this process can be considered "data mining."
      Much like the one above, similarly helpful "fact boxes" are liberally sprinkled throughout the book, pixie dust style. I found them immensely helpful, and I think that you will, too. Ah yes, since we began this section with a preamble on the concept of "features", I'm happy to report that Deep Learning: A Practitioner's Approach even has a "fact box" entitled What Is a Feature? ๐Ÿ’ช

      The Joy of Learning

      Look, I'm a sucker for adorning my essays—some might say embellishing, though I'll politely beg to differ—with topical quotes, images, and excerpts to create a pleasurable reading experience. So I was pleased and right at home with the format I found in Deep Learning: A Practitioner's Approach. Here are the authors, introducing the reader to Chapter 4 (Major Architectures of Deep Networks) with a quote that's as profound as it is thought provoking:
      The mother art is architecture. Without an architecture of our own we have no soul of our own civilization ๐Ÿฐ
      ~ Frank Lloyd Wright (legendary American architect, designer of the masterpiece Fallingwater)
      Okay, let's take a peek at just one more example to get a richer flavor of just how marvelously well wrought is the authors' use of lead-in quotes to every single chapter of the book—here, they're introducing Chapter 8 (Vectorization) with a quote that's as apt as it is witty for the subject matter being introduced ๐Ÿš ๐Ÿ˜‰
      New York City, Old St. Joe, Albuquerque, New Mexico
      This old rig is humming and rolling and she’s doing fine
      If somebody wants to know what’s become of this so and so
      Tell em’ I’m somewhere looking for the end of that long white line
      ~ Sturgill Simpson, Long White Line
      Maybe it's just me... But as I had emphatically pointed out in an essay elsewhere that, along with learning and stuff, "I mean, we got to have ourselves some fun along the ride, don't we?". And oh yes, a sucker for meaningfully adorning essay I'll ever remain ๐ŸŽˆ 

      At the same time, the responsibility that comes with honoring the precious time of so many readers has irrevocably altered my view of how an essay ought to be written. It has not, and nothing ever will, alter my writing style, as I've tried to explain in response to a reader comment in an essay elsewhere ๐Ÿ˜Ž


      Judgment

      Dig in to the pages of Deep Learning: A Practitioner's Approach and you'll see what I'm talking about. This is a book for someone who is looking for gentle, inspiring, and eminently practical guidance on mastering the pragmatics of deep learning—and don't we all? If that's you, don't miss this stellar book ๐ŸŽช

      An Invitation ๐Ÿ“ฃ


      In the end, I invite your comments—Having now read the brief take each on the books above ๐Ÿ’ค
      • Do you find that your experience of reading any of these pragmatic books was different? ๐Ÿข
      • Did I perhaps not cover some qualities, and which are the ones that you actually found the most helpful as you learned the practicalities of deep learning? ๐ŸŒŽ
      • Did I leave out any of your favorite deep learning pragmatic books? ๐Ÿš›
      My hope is that the brief vignettes will help you in your journey to grokking deep learning pragmatics ๐Ÿ€ As promised in the first installment in this series on deep learning, I leave you with a more elaborate pic collage this time. Do please post your comments to let other readers—and me—know how you like the pic collage, won't you? ๐Ÿ’


      Bon Voyage ๐Ÿšข 

      And to you, as you embark on the journey of a lifetime, setting sail on the sea of deep learning, I say with much warmth and friendship, Bon voyage ☕

      Did any of the books in the pic above—they are among my favs—grab your attention? If so, do please post your comments to let other readers—and me—know about which one of these gems you would like to hear more about, won't you? We all stand to benefit from ongoing dialogs ๐Ÿ’

      AI ๐Ÿ‘จ is the Proud Parent of Machine Learning ๐Ÿ‘ฆ Which is, in turn, the Proud Parent of Deep Learning ๐Ÿ‘ถ

      Just to drive home an important point—at the risk of sounding like a broken record—let's not forget the helpful relationship that is indicated in the heading above! ๐Ÿ“€

      Collage of Pics and Lyrics ๐ŸŽธ

      I walked across an empty land
      I knew the pathway like the back of my hand
      I felt the earth beneath my feet
      Sat by the river and it made me complete 
      Oh simple thing, where have you gone?
      I'm getting tired and I need someone to rely on 
      I came across a fallen tree
      I felt the branches of it looking at me
      Is this the place we used to love?
      Is this the place that I've been dreaming of? 
      Oh simple thing, where have you gone?
      I'm getting old and I need something to rely on 
      And if you have a minute why don't we go
      Talk about it somewhere only we know?
      This could be the end of everything,
      So why don't we go
      Somewhere only we know?
      Somewhere only we know? 
      Oh simple thing, where have you gone?
      I'm getting old and I need someone to rely on
      So tell me when you're gonna let me in
      I'm getting tired and I need somewhere to begin 
      And if you have a minute why don't we go
      Talk about it somewhere only we know?
      'Cause this could be the end of everything,
      So why don't we go
      Somewhere only we know?
      Somewhere only we know?
      ~ Lily Allen (Lyrics from Somewhere Only We Know) ๐Ÿ‡ ๐Ÿป
       
      Now why would the emerald-green and haunting Tetris-syle pic above, tied as it is to that awesome movie The Matrix—yup, that's Neo Anderson peering back at us—turn up in an essay on deep learning?! Anyone? Hint, hint: Much as we talked about in the first installment in this series on deep learning, linear algebra remains the bedrock on which the edifice of deep learning is built. And to get anywhere in linear algebra, make matrices your best friends forever ๐Ÿ‘ฑ๐Ÿ‘ฐ

      Ah yes, so here's what I was going to say about the pic above (of a tiny corner of one of my bookshelves), which has at its center square the gem of a book entitled If A, Then B: How the World Discovered Logic (Columbia University Press) by Michael Shenefelt and Heidi White ๐Ÿ’Ž

      So it goes something like the following—relying as it does on the machinery of standard categorical syllogisms—where you can't help but notice the airtight logic of my argument ๐Ÿ‘ป

      I love my wife.
      My wife loves logic.
      Therefore, I love logic

      I noticed some of you cringing at potential leaks in the, um, airtight logic of the argument I present above ๐Ÿ˜ฑ so let's quickly wrap this all up with some endearing lyrics that were sung inimitably and mellifluously by the one-and-only Rocket Man when he crooned
      I hope you don’t mind that I put down in words
      How wonderful life is while you’re in the world.
      ~ Elton John (Lyrics from Your Song) ๐ŸŽง
      And what an awesome movie Dune was, that mind-blowing, off the wall classic! Imagine, if you will, the grains of sand in the desert as the data points that your enterprise needs you to get a grip on ๐Ÿซ Your mission, should you choose to accept it, is to find interesting patterns in those data points ๐Ÿ“ถ Ladies and gentlemen! Dare I say, with our confidence bolstered by this newfound deep learning savvy, that we're ready to take it on? Yay, glad to hear that—So bring it on! ๐Ÿˆ

      Feeling giddy as we surely are at this point in time, our spirits buoyed by what the craft of software can help humanity accomplish, let's not forget the momentous and gloriously successful mission that Cassini has recently accomplished ๐Ÿš€ Time to celebrate ๐ŸŽ‰ ๐Ÿ’
      A lovestruck Romeo sings a streetsuss a serenade
      Laying everybody low with a love song that he made
      Finds a streetlight, steps out of the shade
      Says something like, You and me babe—how 'bout it?"
      ... 
      Juliet, the dice was loaded from the start
      And I bet, then you exploded in my heart
      And I forget, I forget the movie song
      When you gonna realize it was just that the time was wrong, Juliet?
      ... 
      I can't do the talks like they talk on the TV
      And I can't do a love song like the way it's meant to be
      I can't do everything but I'll do anything for you
      I can't do anything except be in love with you
      ~ Dire Straits (Lyrics from Romeo And Juliet)