Saturday, August 9, 2014

Taming Complexity in Software, and Elsewhere

Complexity kills. Complexity sucks the life out of users, developers and IT. Complexity makes products difficult to plan, build, test and use... And as time goes on, and as software products mature—even with the best of intent—complexity is inescapable.
 ~ Ray Ozzie, former chief software architect at Microsoft.
There are subjects...that can be appreciated only if you make an effort proportional to their innate complexity.  To hearken back to something like the language of courtly love in medieval France, there are certain objects of our affection that reveal their beauty and charm only when we make a chivalrous but determined assault on their defenses; they remain impregnable if we don't lay siege to the fortress of their inherent complexity.
~ Christian Queinnec, in Lisp In Small Pieces (Cambridge University Press, 2003), p. xvi

The first of the two quotes atop this essay (from Ray Ozzie) clearly reflects the angst that accompanies the predicament of having to deal with accidental complexity—everything that does not directly relate to the solution of a given problem, but that we have to deal with nonetheless—while the second quote (from Christian Queinnec's acrostically-titled Lisp book) reflects the gist of essential complexity, which is the core problem that we have to solve, usually consisting of the legitimately difficult problems to unravel (in software development as well as in other fields).

I will present some ideas from thoughts leaders in diverse fields on taming complexity in the fields of software, and beyond; OK, I confess to being a die-hard programmer, but nobody has accused me of being parochial, at least not yet. So think of this essay as a survey of sorts, rather than a precise study or, heaven forbid, a thesis thereof ;-)
Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it. 
 ~ Alan Perlis (Epigram # 58), in Epigrams In Programming
As Russ Olsen observes in his fine book Eloquent Ruby (Addison-Wesley Professional, 2011)—incidentally, along with David Flanagan's The Ruby Programming Language (O'Reilly, 2008), it is, in my opinion, the finest book ever written on the Ruby programming language—it takes a lot of self-discipline to deal with either accidental complexity or to engage in people management (a role that Olsen notes that he took on for a period of time) and which is the prime motivation for his pithy observation that follows. It is, in fact, a preamble to his thoughts on the subject, and where I elide to leave the rest to your imagination. Olsen notes: Instead of rolling into work every morning, eager to do battle with the essential complexity of the universe...

I find this notion—different strokes for different folks, or the seemingly endless variation in individuals as to how we get a kick out of our particularly chosen field(s) of pursuit—an endless source of ever-changing, ever-refining, kaleidoscope-like fascination. Psychologist and philosopher William James' The Varieties of Religious Experience anyone? Speaking for myself, I love working as a software developer, but I also digress from the subject at hand, so I'll refrain from tangentially veering off any further, save the following lovely quote that I recently came across ;-)
Enthusiasm is excitement with inspiration, motivation, and a pinch of creativity. Enthusiasm is not the same as just being excited. One gets excited about going on a roller coaster. One becomes enthusiastic about creating and building a roller coaster. ~ Bo Bennett
While writing about the subject of anticipatory thinking, Gary Klein observes in his remarkable book Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making (A Bradford Book, The MIT Press, 2011) that
I think that in complex and ambiguous settings, people should actively speculate, instead of passively absorbing data... Experts distinguish themselves by their ability to anticipate what might happen next... By forming sharp expectancies, experts can notice surprises more readily. 
We often have to make sense of highly uncertain situations, but uncertainty is isn't always reduced by gathering more and more information. Sense-making is not just a matter of connecting the dots; sense-making determines what counts as a dot!
Leslie Valiant—the T. Jefferson Coolidge Professor of Computer Science and Applied Mathematics at Harvard University—shares some delightful insights that vivify and underscore the nature of essential complexity in the computational enterprise when he notes in the chapter entitled Machines as Ecorithms: Why is artificial intelligence difficult to achieve? in his fine book Probably Approximately Correct that
Turing discussed the dilemma between having machines programmed and having them learn... Why exactly are the results of human learning so difficult to replicate otherwise? ... A programmer has to go much further in adapting the system being programmed than a teacher has to the learner.... Learning can be accomplished without needing anyone to explicitly understand either the state of the learner or the complexities of the world. A programmer would need to understand both to be successful
~ Leslie Valiant, in Probably Approximately Correct: Nature’s Algorithms for Learning and Prospering in a Complex World (Basic Books, 2013), pp. 164-65
Neal Ford, an application architect at ThoughtWorks, has a keenly discerning section in his excellent book The Productive Programmer (O'Reilly, 2008) that's dedicated to bringing out the distinction between exactly the two types of complexities that we're talking about here. He notes
No one sets out to spend more time dealing with accidental complexity than essential complexity. But lots of organizations end up doing little more than servicing the accidental complexity layers that accrue over time. Much of the current fad for SOA (service-oriented architecture) derives from companies trying to mitigate the amount of accidental complexity that has built up over time. SOA is an architectural style that binds disparate applications together so that they can communicate. Very rarely would you ever define that problem as the driving force of your business. It is, however, exactly what you’d do if you had lots of uncommunicative applications around that needed to share information. Sounds like accidental complexity to me. The vendorization of the SOA architectural style is the Enterprise Service Bus (ESB), whose chief selling point seems to be that the solution to the problem of middleware is more middleware. Does adding complexity reduce complexity? Hardly...
Identification is the first step in ridding yourself of accidental complexity. Think about the processes, policies, and technical mazes with which you currently deal. Noticing what is essential may lead you toward abandoning something whose contribution to the real problem outweighs the amount of complexity it introduces to the overall problem. For example, you may think you need a data warehouse, but the amount of complexity it adds to the overall problem isn’t worth the benefits it might provide. You can never kill all accidental complexity with software, but you can continually try to minimize it.
To my mind—and as it would apply to effectively maintaining existing (software legacy) code bases—Ford's preceding remark about continually try[ing] to minimize it (viz. complexity) sounds like a call to continually refactor software!  Martin Fowler has literally written the landmark, canonical book on refactoring—the process of restructuring existing computer code without changing its external behavior—and the details are well worth looking up by starting here...

In rounding out the discussion (as it applies to the two types of complexities that we're talking about here), Ford notes that
Java and C# in particular were designed to make C++ easier and less error prone, so the developers built-in some fairly serious restrictions in the interest of keeping average developers out of trouble... 
Because these languages are general-purpose languages, you can get pretty much anything done…with enough effort. Java kept bumping into stuff that would be nice to do but was way too much work, so frameworks were built. And built. And built. Aspects were added. More frameworks were built...
~ Neal Ford, The Productive Programmer (O'Reilly, 2008) 
At this point, I'm going to throw this out there: Anyone even want to remember what it was like dealing with Enterprise JavaBeans (EJB), which is a managed, server-side component architecture for modular construction of enterprise applications? Dare I say that they can admirably serve as the poster child of accidental complexity? In all fairness, the EJB specification has of late undergone an extreme makeover
Gradually an industry consensus emerged that the original EJB specification's primary virtue—enabling transactional integrity over distributed applications—was of limited use to most enterprise applications, and the functionality delivered by simpler frameworks like Spring and Hibernate was more useful. Accordingly, the EJB 3.0 specification (JSR 220) was a radical departure from its predecessors, following this new paradigm... 
Accordingly, in practical terms, EJB 3.0 is much more lightweight and nearly a completely new API, bearing little resemblance to the previous EJB specifications.
Funny, isn't it, that we keep coming back to the bare simplicity of the basics, that we almost sense the comfort of having arrived when we find ourselves dealing with the good type of complexity? Yes, the good type of complexity—essential complexity, which consists of the legitimately difficult problems to solve, as well as hard material to master, in software development as well as in other fields—is always there to be mined, but to reach out to it, we have to rise above the currents swirling around in the sea of accidental complexity so we can focus on the essential complexity, and thereby make headway and progress toward our goals.

Since the previous two examples relate to accidental complexity, let's look at one final example, this one rooted in essential complexity: Consider a comprehensive account of the semantics and the implementation of the whole Lisp family of languages, namely Lisp, Scheme and related dialects. All of that in the guise of detailed descriptions of 11 interpreters and two compilers, including very recent techniques of interpretation and compilation. Roll that up into a book and you have Christian Queinnec's Lisp In Small Pieces (Cambridge University Press, 2003), which is mentioned atop this essay, in the second of the two quotes. I've read only portions of that book; if it isn't essential complexity that Queinnec is dissecting in exquisite details, then, frankly, I don't know what essential complexity is. Period. But I have a hunch—given the extremely high Signal-to-Noise Ratio (SNR) of ideas-to-pages, to the point where the defined features are reduced to a simple lambda-calculus, along with the introduction of denotational semantics—that Queinnec is indeed talking about the good type of complexity...

I will leave you with two interesting books to look up. It would be unfair to draw comparisons between those two books, though I'll say this much that the second of the two is quite a bit more readable, while the first is more theoretical. Note, too, that the publication of these two books is spaced apart by a decade, so perhaps the intervening period of time led to a refinement in thinking about how better to tackle complexity in the heart of software...
  • Domain-Driven Design: Tackling Complexity in the Heart of Software
     ~ Eric Evans (Addison-Wesley Professional, 2003)
  • Implementing Domain-Driven Design
     ~ Vaughn Vernon (Addison-Wesley Professional, 2013)
...or at least that's what I think at this point in time, because a decade is an eternity for any fieldthe software development field conspicuously includedthat lives at the intersection of technology and creativity ;-)
It is hard enough to remember my opinions, without also remembering my reasons for them!
 ~ Friedrich Nietzsche, as quoted in Cristopher Moore's and Stephan Mertens' The Nature of Computation (Oxford University Press, 2011)

Sunday, July 20, 2014

Time Management for Software Developers

And how is clarity to be achieved? Mainly by taking trouble; and by writing to serve people rather than to impress them.
~ F.L. Lucas 
A writer's problem does not change. He himself changes, and the world he lives in changes, but his problem remains the same. It is always how to write truly, and having found what is true, to project it in such a way that it becomes part of the experience of the person who reads it.
 ~ Ernest Hemingway

It truly wasn't until I had read Elizabeth Grace Saunders' The 3 Secrets to Effective Time Investment: How to Achieve More Success with Less Stress that my faith in the time management advice genre was restored. If you're like most people, you've probably found yourself cringing at the mere mention of time management; I used to. But this gem of a book, with its brilliant take on time investment, has changed all that. Far from being a run-of-the-mill time management title, it goes right to the heart of what works, what doesn't and, perhaps most importantly, helping you understand why certain time investment strategies work, while others don't.

I've found both quotes—which appear at the top of this essay—faithfully personified by the author of The 3 Secrets to Effective Time Investment, and embodied in the pages of this book. Jenny Blake (author of Life After College), in praising The 3 Secrets to Effective Time Investment, perfectly captures the book's central message when she notes that Elizabeth "...eloquently moves us past traditional and outdated concepts around time management and instead lays a compelling foundation for time investment—so that we can all focus on what matters most to our individual lives and careers...."

I'd say that The 3 Secrets to Effective Time Investment has clearly over-achieved its claim, which I see prominently emblazoned across the dust jacket's back of my hard-cover copy of the book: Time management skills that work! Reboot your entire life in three simple steps. Fellow software developers, take note of the rebooting metaphor in the preceding claim :-) While this book is definitely not limited to us software developers, I have yet to see another book that comes even remotely close to what we need to stay on top of our dizzying project schedules and deadlines...
In mathematics, you don't understand things. You just get used to them.
~ John von Neumann
Elizabeth's book is the one you want to read, should you read only a single book this year! At long last, there is absolutely no need either to cringe on hearing the phrase time management, or to philosophically resign yourself to half-heartedly (i.e. mechanically) getting used to the advice offered by other time management books—as for the quote above, I imagine that the legendary John von Neumann was being facetious. So read up on the strategies in this book, and put them into practice to reap the benefits. Thank you, Elizabeth!

And I have Cal Newport to thank for bringing this book to my attention, by way of his foreword to the book. His written work (both his blog and books) has also had a profound impact on me over the years, and I own a copy of every book that he has written so far—While Cal's book How to Win at College: Surprising Secrets for Success from the Country's Top Students is ostensibly aimed at college students, a lot of it is every bit as relevant to those of us in the workplace; it has amazingly helpful and original strategies for mastering hard, technical material. And that's all stuff which is highly relevant to what we do daily as computer programmers.

To give you a flavor of the topics covered, here is a handful of my favorite chapters from Cal's book:
  • Write as if Going for a Pulitzer
  • Laugh Every Day
  • Don't Undersleep, Don't Oversleep
  • Stay in Touch
  • “Don't Have No Regrets”
Lest I forget to mention, an excellent newsletter serves to supplement the book The 3 Secrets to Effective Time Investment. And finally, to give you a flavor of some the topics covered in the book, here is a handful of my favorite sections from Elizabeth's book:
  • Strengthen Simple Routines
  • Increase Your Hope: Optimism 
  • Increase Your Peace: Mind Time
  • Priority-Based Decision Making
  • How to Pacify Your Inner Routine Rebel
Last, but by no means the least, the written works by Elizabeth and Cal reflect good taste in that they both treat the reader with great respect, and that shines through the pages of their books, newsletters, blogs, etc.!
I would be most content if my children grew up to be the kind of people who think decorating consists mostly of building enough bookshelves.
~ Anna Quindlen

Saturday, July 19, 2014

On Paul Graham's Essays, and of Y Combinator

In literature, the ambition of the novice is to acquire the literary language; the struggle of the adept is to get rid of it
~ George Bernard Shaw

Some of the finest prose I've ever come across has been in the pages written by Paul Graham. In an essay that I wrote earlier (entitled On Lisp and Paul Graham)—and where I was referring to the two fine books Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp by Peter Norvig (Morgan Kaufmann, 1992) and On Lisp: Advanced Techniques for Common Lisp by Paul Graham (Prentice Hall, 1994)—I had noted that, "While both are replete with good advice, it is Paul Graham's On Lisp that also has the distinction of having been written spectacularly well; it has to be read to be believed!"

And it is precisely Graham's book On Lisp that I had in mind, as I chose the title of this essay that you're reading. Speaking of essay titles, and going about it a bit recursively, here's what Graham observed about the title of his book On Lisp
The title is intended to stress the importance of bottom-up programming in Lisp. Instead of just writing your program in Lisp, you can write your own language on Lisp, and write your program in that... Lisp was meant from the start to be an extensible language. The language itself is mostly a collection of Lisp functions, no different from the ones you define yourself...
So this is an acknowledgment of, and appreciation for, the fine essays—and books, to be sure—that Graham has written. This essay that you're reading is partly entitled On Paul Graham's Essays because I've gained so much from reading them over the years—Each writer has to find his or her own voice, though I confess that I've been drawn toward emulating his writing style. In particular, his book Hackers & Painters: Big Ideas from the Computer Age (O'Reilly) is a joy to read. Do check it out to see the care that he has lavished in crafting the narrative...
Sometimes I will stay up in my room for a day trying to get two sentences that will flow, that will just seem as if they were always there.
~ Maya Angelou, in Conversations with Maya Angelou, ed. Jeffrey M. Elliot (Jackson, MS: University Press of Mississippi, 1989), p.59
I'll leave it to you to look up his fine essays and books. Meanwhile, if only to convey a flavor of his stellar writing, here is an excerpt from On Lisp. In particular, look for the sublime metaphors that Graham uses in the second of the following two paragraphs with the cue sentence Language and program evolve together:
...The traditional approach is called top-down design: you say “the purpose of the program is to do these seven things, so I divide it into seven major subroutines. The first subroutine has to do these four things, so it in turn will have four of its own subroutines,” and so on. This process continues until the whole program has the right level of granularity—each part large enough to do something substantial, but small enough to be understood as a single unit. 
Experienced Lisp programmers divide up their programs differently. As well as top-down design, they follow a principle which could be called bottom-up design—changing the language to suit the problem. In Lisp, you don’t just write your program down toward the language, you also build the language up toward your program. As you’re writing a program you may think “I wish Lisp had such-and-such an operator.” So you go and write it. Afterward you realize that using the new operator would simplify the design of another part of the program, and so on. Language and program evolve together (italics are mine). Like the border between two warring states, the boundary between language and program is drawn and redrawn, until eventually it comes to rest along the mountains and rivers, the natural frontiers of your problem. In the end your program will look as if the language had been designed for it. And when language and program fit one another well, you end up with code which is clear, small, and efficient.
And this essay that you're reading is partly entitled Of Y Combinator if only to underscore that pursuing computer science, programming, and mathematics is eminently pragmatic, rather than a path that meanders around dainty ivory towers :-) My apologies to the purists who read this and cringe, though I hasten to add that the pursuit of pure mathematics and theoretical computer science is a worthy one, and has a lot of merit—Do check out The Nature of Computation (by Cristopher Moore and Stephan Mertens, Oxford University Press, 2011) for a lovely exposition of these subjects. That book is a work of tasteful art, if you will, and Marc Mézard at the Université de Paris probably captured its spirit the best when he wrote of it as follows
If you want to learn about complexity classes, scaling laws in computation, undecidability, randomized algorithms, how to prepare a dinner with Pommard, Quail and Roquefort, or the new ideas that quantum theory brings to computation, this is the right book. It offers a wonderful tour through many facets of computer science. It is precise and gets into details when necessary, but the main thread is always at hand, and entertaining anecdotes help to keep the pace.
~ Marc Mézard, Université de Paris Sud, Orsay
Randall Stross notes in his engrossing book The Launch Pad: Inside Y Combinator (Portfolio Trade, 2013) that Graham gave his company (Y Combinator) "...the curious name, borrowed from computer science... It is a mischievously obscure choice. It refers to a particular kind of function, for receiving data, performing a calculation, and sending a result back, that most programmers have never used or even understand."

If you look up Y Combinator in Wikipedia, you'll immediately get a disambiguation page notifying you that Y Combinator commonly refers either to
How about that—a bit of esoteric math jostling with the very best of American entrepreneurship—for highlighting the applicability of computer science, programming, and mathematics to solving real world problems? Another blog that may interest you is the one by Mark Chu-Carroll, and which goes by the name Good Math :-)

I sense that this essay is getting more and more into the territory inspired by Douglas R. Hofstadter's magnum opus Gödel, Escher, Bach: An Eternal Golden Braid as I weave more strands into the discussion thread, so I'm going to stop here, but not before leaving you with an intriguing quote from Orwell...
Writing a book is a horrible, exhausting struggle, like a long bout of some painful illness. One would never undertake such a thing if one were not driven on by some demon whom one can neither resist nor understand. For all one knows that demon is simply the same instinct that makes a baby squall for attention. And yet it is also true that one can write nothing readable unless one constantly struggles to efface one's own personality. Good prose is like a windowpane.
~ George Orwell, Why I Write (England Your England and Other Essays)
...and an equally intriguing one from Gloria Steinem
For me, writing is the only thing that passes the three tests of metier: (1) when I'm doing it, I don't feel that I should be doing something else instead; (2) it produces a sense of accomplishment and, once in a while, pride; and (3) it's frightening.
~ Gloria Steinem

Saturday, July 12, 2014

Kafka, Voldemort, oh my!

The quintessential American writer Mark Twain observed, and he may well have been thinking of coincidences when he did so, that
Truth is stranger than fiction, but it is because Fiction is obliged to stick to possibilities; Truth isn't.

And so it was that this past Friday that an esteemed coworker referred in passing to a recurring theme from the works of Franz Kafka. Then, later the same day, over the lunch hour to be precise, as I checked my personal email, I read a new email from a former coworker and friend, telling me about his use of Kafka in his latest software project... No, not that Kafka :-)

I'm not referring to the author Kafka, whom several of you will remember, as I did, from undergraduate English Literature classes as the author of that eerily surreal story, entitled Metamorphosis, in which the protagonist wakes up one morning to discover that he has, overnight, metamorphosed into a giant cockroach; good grief, some authors do get carried away with their imagination, at times, don't they!

The Kafka I have in mind, and which my friend is currently using in his software project is Apache Kafka, a high-throughput, distributed messaging system (Apache Kafka Coding Guide)
Kafka is system software, and certain things are appropriate in system software that are not appropriate elsewhere. Sockets, bytes, concurrency, and distribution are our core competency which means we will have a more "from scratch" implementation of some of these things then would be appropriate for software elsewhere in the stack. This is because we need to be exceptionally good at these things. This does not excuse fiddly low-level code, but it does excuse spending a little extra time to make sure that our file-system structures, networking code, threading model, are all done perfectly right for our application rather than just trying to glue together ill-fitting off-the-shelf pieces (well-fitting off-the-shelf pieces are great though)...
OK, referring one more time to the title of this essay (Kafka, Voldemort, oh my!), it's not that Voldemort either :-)

I do confess, though, that the Harry Potter movies (whose characters include that devious and dastardly Voldemort, aka he-who-must-not-be-named) are aesthetically and technologically among the best-made in cinematic history, almost right up there with Blade Runner, the classic, futuristic movie starring Harrison Ford, based on the book Do Androids Dream of Electric Sheep?).

The Voldemort project which I have in mind here is a software project that provides a distributed key-value storage system. Data is automatically replicated over multiple servers, and each node is independent of other nodes, with no central point of failure or coordination. Concurrency in Voldemort, as I understand it from having briefly glanced at its API, makes use of the usual advisory annotations for thread safety.

At any rate, having laid out very briefly the concurrency aspect each of Kafka and Voldemort, the case can be made, as I hear it, that using the Scala programming language would make a given code-base less cluttered, and hence that much easier to grok than, say, the equivalent version written in Java. But neither the shepherds nor the users of Java have been sitting idly, and we fortunately have first-class resources on which to rely.

And speaking of concurrency, consider this essay as a more specialized follow-up to the fuller, albeit general, discussion of concurrent programming (for the JVM) in my previous essay, and which is entitled Java Concurrency in Retrospect. Heading the list of first-class resources on which we Java programmers can rely on is of course the amazingly rich and highly regarded book Java Concurrency in Practice, by Brian Goetz, et al (Addison-Wesley). Among the other, excellent resources, I would recommend these without any reservation:
  • Concurrent Programming in Java: Design Principles and Pattern, 2nd Edition, by Doug Lea (Addison-Wesley)
  • Growing Object-Oriented Software, Guided by Tests by Steve Freeman and Nat Pryce (Addison-Wesley)
  • Programming Concurrency on the JVM: Mastering Synchronization, STM, and Actors, by Venkat Subramaniam (Pragmatic Bookshelf)
Of these, Growing Object-Oriented Software, Guided by Tests is a gem of a book. The authors get to the heart of the matter by noting that
There’s no getting away from it: concurrency complicates matters. It is a challenge when doing test-driven development. Unit tests cannot give you as much confidence in system quality because concurrency and synchronization are system-wide concerns. When writing tests, you have to worry about getting the synchronization right within the system and between the test and the system. Test failures are harder to diagnose because exceptions may be swallowed by background threads or tests may just time out with no clear explanation. 
Meanwhile, our conceptualization of concurrency (i.e. how we see concurrency) will surely continue to evolve, accelerated and refined by the emergence of the polyglot languages targeting the JVM...
We do not see things as they are, we see them as we are.
~ Anais Nin, in Cities of the Interior

Java Concurrency in Retrospect

The title of this essay is a bit tongue-in-cheek because, no matter how promising the new stalwart JVM languages (And here I'm thinking of the most promising of those languages, viz. Scala and Clojure) are with helping us manage concurrency, the fact is that Java is going to be around in a big way (for a long time, and that's my two cents' worth). In fact, the title of this essay (i.e. Java Concurrency in Retrospect) is in part a tribute to, and a spin on, the title of the amazingly rich and highly regarded book Java Concurrency in Practice, by Brian Goetz, et al.

The book Java Concurrency in Practice is so well-established as staple reading material that it's commonly and affectionately referred to simply as JCIP, somewhat akin to how the Gang of Four (the authors of the seminal work ...) are endearingly called the GoF (sans the late John Vlissides, one of the most terrific writers to ever have graced our software industry's literature, thinking here especially to Design Patterns:Elements of Reusable Object-Oriented Software as well as to Pattern Hatching: Design Patterns Applied). Adding that JCIP is evidently also esteemed in academia; a friend of mine, who is an algorithms aficionado as well as a professor of computer science at The University of Texas at Austin, thinks highly of it. Another Austin-based professional that I know, a web applications architect who helps start-ups and early stage companies implement high performance, fault tolerant platforms, told me that he recommends to everyone that they read JCIP in its entirely, every six months. These are anecdotes, but I think that you get the idea :-)

Bruce Tate, the author of Beyond Java, has probably the most germane and succinct round-up of exactly what it is that makes JCIP so compelling. Tate had noted the following thoughts upon its publication, and his words ring true today:
I've focused a career on simplifying simple problems, but this book ambitiously and effectively works to simplify a complex but critical subject: concurrency. Java Concurrency in Practice is revolutionary in its approach, smooth and easy in style, and timely in its delivery—it's destined to be a very important book.
And there is no hand-waving going on in the pages of JCIP. No fluff, all stuff, as they say! There are tons of Java code listings peppered throughout the book, along with clear explanations. Here's a small sample of the types of listings you can expect to find in it:
1.1 Non-thread-safe sequence generator
1.2 Thread-safe sequence generator
2.1 A stateless servlet
2.2 Servlet that counts requests without the necessary synchronization. Don't do this.
2.3 Race condition in lazy initialization. Don't do this.
2.4 Servlet that counts requests using AtomicLong
2.5 Servlet that attempts to cache its last result without adequate atomicity. Don't do this.
2.6 Servlet that caches last result, but with unnacceptably poor concurrency. Don't do this.
2.7 Code that would deadlock if intrinsic locks were not re-entrant
2.8 Servlet that caches its last request and result
3.1 Sharing variables without synchronization. Don't do this.
3.2 Non-thread-safe mutable integer holder
3.3 Thread-safe mutable integer holder
3.4 Counting sheep
3.5 Publishing an object
Now, the polyglot JVM languages (Again, I'm thinking here of the eminently promising languages Scala and Clojure) offer compelling solutions to solving concurrency concerns in corporate software development. In referring to Paul Graham’s essay Beating the Averages, which is an interesting look at the inner workings of his company Viaweb during the years prior to being bought by Yahoo! Inc. in 1998, authors Michael Fogus and Chris Houser note in their fantastic book The Joy of Clojure: Thinking the Clojure Way (Manning Publications) that:
Clojure is a dialect of Lisp directly supporting concurrent software development using functional programming techniques, and like the Lisp described in Beating the Averages, provides an environment conducive to agility. Clojure fosters agility in ways that many popular programming languages can’t. Many programming languages are bewitched with most or all of the following:
  • Verbosity 
  • Unavoidable boilerplate 
  • A long thought-code-feedback loop 
  • Incidental complexity 
  • Difficulties in extension 
  • Deficiencies in supporting crucial programming paradigms
The author Nilanjan Raychaudhuri adds some highly relevant observations of his own, in his book Scala in Action (Manning Publications), when he notes that:
  • Only a handful of programmers know how to write a correct, concurrent application or program. The correctness of the program is important. 
  • Debugging multithreaded programs is difficult. The same program that causes deadlock in production might not have any locking issues when debugging locally. Sometimes threading issues show up after years of running in production. 
  • Threading encourages shared state concurrency, and it’s hard to make programs run in parallel because of locks, semaphores, and dependencies between threads.
In particular, Clojure provides the Software Transactional Memory model to tame concurrency concerns. Again, this is Fogus and Houser from their previously cited work:
On one side of the equation (italics are mine), Clojure utilizes Software Transactional Memory (STM), agents, a clear distinction between identity and value types, arbitrary polymorphism, and functional programming to provide an environment conducive to making sense of state in general, and especially in the face of concurrency. On the other side, Clojure shares a symbiotic relationship with the Java Virtual Machine, thus allowing prospective developers to avoid the costs of maintaining yet another infrastructure while leveraging existing libraries.
Raychaudhuri notes further in his excellent book Scala in Action (Manning Publications) the solution that Scala brings to the table when he notes that:
...The current concurrency model is too hard for developers to grok, and we need a better concurrent programming model that will help developers easily write and maintain concurrent programs... Scala takes a totally different approach to concurrency: the Actor model. An actor is a mathematical model of concurrent computation that encapsulates data, code, and its own thread of control and communicates asynchronously using immutable (no side effects) message-passing techniques...
Compelling as the solutions are for the JVM, the one from Clojure and the one from Scala, both compelling in their own right and each worthy of investigation, we meanwhile have massive Java code-bases already in existence, so JCIP will continue to light the way in helping us developers tame the beast that is concurrency.

During my daily programming, although I may not necessarily be thinking explicitly in terms of concurrency concerns all the time, the theme of handling concurrency, nonetheless, forms a fairly consistent backdrop. In conclusion, I should point out yet another terrific online resource, not necessarily concurrency-focused, by the lead author of JCIP, Brian Goetz.

That link will take you to the excellent Java theory and practice Technical Library hosted under the auspices of IBM developerworks. The articles by Goetz are of the highest quality; I'm talking about as polished as the finest book you'll find, right up there with Design Patterns:Elements of Reusable Object-Oriented Software, or Effective Java by Joshua Bloch. Actually, I had been searching  during this past year for helpful material on grokking proxies and their use in decorating messaging in framework code, and that's how I came across the Java theory and practice Technical Library. And I sure am glad I did :-) Here's a small sample of the kind of articles you can expect to find there:
  • To mutate or not to mutate?
  • Decorating with dynamic proxies
  • State replication in the Web tier
  • Are all stateful Web applications broken?
  • Introduction to non-blocking algorithms
  • Instrumenting applications with JMX
Here's to continually improving our software code-bases through refactoring. Did someone say refactoring? Now that sound like the topic for another essay...

Sunday, July 6, 2014

On Lisp and Paul Graham

The overwhelming impression that Peter Seibel's book Practical Common Lisp left on my mind was that it had been written by someone who cares as much about the art of programming as he does for the craft of writing well. In fact, there is an intriguing self-admission by Seibel in that book's About the Author section where he notes
Peter Seibel is either a writer-turned-programmer or a programmer-turned-writer. After picking up an undergraduate degree in English and working briefly as a journalist, he was seduced by the Web...
And wow, does his command of both Lisp and writing shine throughout the book! Finally, in the Where to Go Next section of Practical Common Lisp, Seibel points out that
Two books that cover general Common Lisp technique are Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp by Peter Norvig (Morgan Kaufmann, 1992) and On Lisp: Advanced Techniques for Common Lisp by Paul Graham (Prentice Hall, 1994). The former provides a solid introduction to artificial intelligence techniques while teaching quite a bit about how to write good Common Lisp code, and the latter is especially good in its treatment of macros.
I have copies of both of these preceding excellent books. While both are replete with good advice on programming, it is Paul Graham's On Lisp that also has the distinction of having been written spectacularly well; it has to be read to be believed! Noting briefly that the two other books by Graham (I own and have read the copy each of those first-class books) are:
  • ASNI Common Lisp (Prentice Hall)
  • Hackers & Painters: Big Ideas from the Computer Age (O'Reilly)

When Graham's Hackers & Painters came out, many years ago (it predates my reading any of his works on Lisp), I was so impressed by its quality that I wrote to him, telling him how highly I thought of his book, plus recommending to him the best book available on this planet on writing well, namely Writing with Style: Conversations on the Art of Writing by John R. Trimble. Graham graciously replied to me, saying that he in fact had a copy of Writing with Style in his bookshelf. I was pleasantly surprised, because while most everyone has heard of The Elements of Style (by Strunk and White), hardly anyone is even aware of John R. Trimble's stellar gem of a book. In my mind at least, and with all due respect to the former book (because I have read and appreciated its advice), Writing with Style is the book that The Elements of Style wants to be when it grows up...

I wish to to add here that Graham's Hackers & Painters uses the word hacking in the fine and true connotations it is meant to carry, when applied to the art of computer programming:
At any rate, I'll leave to you the reading of the fine books that I mention above. Meanwhile, if only to convey a flavor of the stellar book that On Lisp is, here is an excerpt from that book. In particular, do look for the sublime metaphors that Graham uses in the second of the following two paragraphs with the cue sentence Language and program evolve together:
...The traditional approach is called top-down design: you say “the purpose of the program is to do these seven things, so I divide it into seven major subroutines. The first subroutine has to do these four things, so it in turn will have four of its own subroutines,” and so on. This process continues until the whole program has the right level of granularity—each part large enough to do something substantial, but small enough to be understood as a single unit. 
Experienced Lisp programmers divide up their programs differently. As well as top-down design, they follow a principle which could be called bottom-up design—changing the language to suit the problem. In Lisp, you don’t just write your program down toward the language, you also build the language up toward your program. As you’re writing a program you may think “I wish Lisp had such-and-such an operator.” So you go and write it. Afterward you realize that using the new operator would simplify the design of another part of the program, and so on. Language and program evolve together (italics are mine). Like the border between two warring states, the boundary between language and program is drawn and redrawn, until eventually it comes to rest along the mountains and rivers, the natural frontiers of your problem. In the end your program will look as if the language had been designed for it. And when language and program fit one another well, you end up with code which is clear, small, and efficient.

Musings on Programming in the large, and on Functional Programming

I'm having a lot of fun these days with the enterprise Java software project at my workplace (General Motors) here in Austin (Texas), and have had scarce little time for doing functional programming lately. In this essay, I'm doing something of a context switch: By the way, my reference to the term context switch (i.e. the process of storing and restoring the state of a process or thread so that execution can be resumed from the same point at a later time) will likely reveal my roots in embedded systems programming, which is where I got my start as an application programmer, with Schlumberger Corporation in South Carolina, circa 1995.

With that, my hope with this essay—and those to follow—is to share some musings on programming in the large. I continue to get a kick out of crafting computer programs to solve hard problems! And this is the venue where I will share my ever-evolving understanding (grokking, if you will) of crafting software-intensive systems... Since we're on the subject of functional programming (FP), I would like to put on the virtual stack the context of my most recent forays into FP, which I undertook during my spare time (evenings and weekends) over the past two years or so, coinciding with my career at eBay and PayPal.

Scala and Clojure were the two primary languages that served as the vehicles for doing FP. And of all the FP books that I’ve read, here are my top picks:
  • The Joy of Clojure, by Michael Fogus (Manning)
    Brilliant and inspiring. Having read a fair amount of his writings, I can see without hesitation that pretty much anything by Fogus is good! Check out his pithy writings and Github contributions (I’ve lost track of the projects on which Fogus is a committer). Also, check out his other book, Functional JavaScript: Introducing Functional Programming with Underscore.js.
  • Scala Cookbook: Recipes for Object-Oriented and Functional Programming, by Alvin Alexander (O'Reilly)
    This is a really nice collection of recipes to experiment with Scala on your REPL... I couldn’t resist the temptation to add that Alexander is a fellow Texas A&M alum; needless to say, he’s got to be good, right? Universty of Texas' Longhorns, please take note ;-)
  • Types and Programming Languages, by Benjamin C. Pierce
    This book from The MIT Press is a bit tough-going in places, but well worth the effort if you can stomach the mathematical machinery (lambda to its core) presented in the book, and which serves as the vehicle for quintessentially delivering the ideas. I've read portions of this book so far...
  • Practical Common Lisp, by Peter Seibel
    My route, a bit meandering, I confess, through the FP terrain took me from my first stop—Clojure and, BTW, look no further than the CounterClockwise plugin for Eclipse for hacking Clojure—to the second stop, i.e. Common Lisp, then a bit of Scheme thrown in, and finally arriving at Scala, which is a language showing every sign of surviving—and thriving—as a viable FP language that’s eminently suitable for corporate development. And IntelliJ IDEA 13 is my IDE of choice. A message here to the JetBrains’ team, if somehow we can funnel this sentiment to them: Why are you not providing the same level of support for Clojure, the loveliest Lisp around, a gift from Rich Hickey, and a language that meshes seamlessly with the JVM ecosystem, and then some?), whereas other Lisps—notwithstanding Common Lisp and Scheme, of course—have fallen by the wayside, like the husks of computer science projects. Anyhow, of the books on this list, this is one of the two that is both brilliant and inspiring!
  • Scala in Depth, by Joshua D. Suereth (Manning)
    This is Scala straight from the horse’s mouth. Suereth is a virtuoso of type theory, the Scala tool chain and, come to think of it, everything Scala… I can highly recommend Scala in Action, by Nilanjan Raychaudhuri (Manning) as an excellent first book leading up to Scala in Depth.
  • Programming in Scala: A Comprehensive Step-by-Step Guide, 2nd Edition, by Martin Odersky
    A bit of a boat anchor, weighing in at 850 page-plus, ahem (and evidently the motivation and rationale for the emergence of another concise book that’s titled, rather tongue-in-cheek, I would say, Scala for the Impatient, by Cay S. Horstmann), but straight from the horse’s mouth. This one’s a keeper, too.
  • Clojure Programming, by Chas Emerick, Brian Carper, Christophe Grand (O'Reilly)
    This is an eminently readable, instructive, and wide-ranging coverage of all things Clojure.
  • Functional Programming Patterns in Scala and Clojure: Write Lean Programs for the JVM, by Michael Bevilacqua-Linn (The Pragmatic Programmers)
    This one has turned out to be a gem of a book, replete with advice on using Scala and Clojure to solve in-depth problems, and seeing how familiar object-oriented patterns can become more concise with functional programming and patterns—This one has a very high Signal-to-Noise Ratio (SNR) of ideas-to-pages :-)