Saturday, September 28, 2013

The Cathedral and the Bazaar - Notes to Self..

Given below are the notes from reading the essay "The Cathedral and the Bazaar" by Eric S Raymond (ER)

  • Cathedral and Bazaar refers to the two differing ways of developing  open source software.
  • ER had been preaching small tools, rapid prototyping and evolutionary programming for years.  And he believed that a centralized, a priori approach was required above a certain critical complexity. 
  • Important software systems (e.g. OS and Emacs) needed to be built like cathedrals, carefully crafted by individual wizards or small set of people working in isolation, with no beta to be released before it's time (Cathedral Approach).
  • ER goes on to claim that Linux overturned much of what he knew. Linux's way of working - release early and often ( even two releases in a day), delegate everything you can, be open to the point of promiscuity - surprised him.
  • For ER, Linux community seemed to resemble a great babbling bazaar of differing agendas and approaches, out of which the emergence of a coherent and stable system seemed like a miracle to him.
  • ER was also shocked by the fact the bazaar style of development worked and worked well too and didn't fall apart as Linux became more and more complex.
  • This book is an experiment in which he implements the linux way of development, shares his lessons learnt.   The project is about building a POP protocol compatible mail client (fetchmail). He didn't build the software from ground up, but improvised on a mail client that already existed.

Why it is important:The essay helped convince most existing open source and free software projects to adopt Bazaar-style open development models, fully or partially—including GNU Emacs and GCC.  It also made Netscape Communications Corporation to release the source code for Netscape Communicator (year 1998) and start the Mozilla project.

Note:I am not surprised at The Mythical Man-Month(TMMM) being quoted in so many places. I am of the humble opinion that the book hasn't been completely understood by people in the software industry.

Given below are the lessons learnt, with anecdotal accounts and ER's comments, as applicable.

#1. Every good work of software starts by scratching a developer's personal itch.  "An important trait of the great ones is constructive laziness. They know that you get an A not for effort but for results, and that it's almost always easier to start from a good partial solution than from nothing at all."

#2. Good programmers know what to write. Great ones know what to rewrite (and reuse). Linus Torvalds didn't actually try to write Linux from scratch. Instead, he started by reusing code and ideas from Minix, a tiny Unix-like operating system for PC clones. Eventually all the Minix code went away or was completely rewrittenóbut while it was there, it provided scaffolding for the infant that would eventually become Linux.

#3. "Plan to throw one away; you will, anyhow." (learnt from The Mythical Man-Month, Chapter 11) - To put it another way, you often don't really understand the problem until after the first time you implement a solution. The second time, maybe you know enough to do it right. So if you want to get it right, be ready to start over at least once

#4. If you have the right attitude, interesting problems will find you.  ER says that in a software culture that encourages code-sharing, interesting problems finding the right person is a natural way for a project to evolve. 

#5. When you lose interest in a program, your last duty to it is to hand it off to a competent successor. This is what the original author of the 'popclient' program did. He lost interest in the program and handed over the complete responsibility to ER. 

#6. Treating your users as co-developers is your least-hassle route to rapid code improvement and effective debuggingER thinks that Linus' cleverest and most consequential hack was not the construction of the Linux kernel itself, but rather his invention of the Linux development model. 

#7. Release early. Release often (RERO). And listen to your customers. The rationale for this is given below in ER's comment section.

#8. Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone. "Given enough eyeballs, all bugs are shallow.'' 

------------ER's Comment Starts--------------
"In Linus's Law, I think, lies the core difference underlying the cathedral-builder and bazaar styles. In the cathedral-builder view of programming, bugs and development problems are tricky, insidious, deep phenomena. It takes months of scrutiny by a dedicated few to develop confidence that you've winkled them all out. Thus the long release intervals, and the inevitable disappointment when long-awaited releases are not perfect."

"In the bazaar view, on the other hand, you assume that bugs are generally shallow phenomenon, at least, that they turn shallow pretty quickly when exposed to a thousand eager co-developers pounding on every single new release. Accordingly you release often in order to get more corrections, and as a beneficial side effect you have less to lose if an occasional botch gets out the door."

Brooks in TMMM even made an off-hand observation related to this: ``The total cost of maintaining a widely used program is typically 40 percent or more of the cost of developing it. Surprisingly this cost is strongly affected by the number of users. More users find more bugs.'' 

------------ER's Comment Ends--------------

#9. Smart data structures and dumb code works a lot better than the other way around. ER adds that Brooks mentioned in Chapter 9 of TMMM "Show me your flowchart and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchart; it'll be obvious.'' And allowing for thirty years of terminological/cultural shift, it's the same point.

#10. If you treat your beta-testers as if they're your most valuable resource, they will respond by becoming your most valuable resource. From the beginning of the project, ER got bug reports of a very high quality, often with good fixes attached. He also got thoughtful criticism, fan mail, and intelligent feature suggestions. 

#11. The next best thing to having good ideas is recognizing good ideas from your users. Sometimes the later is better.

#12. Often, the most striking and innovative solutions come from realizing that your concept of the problem was wrong.

#13. "Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away.'' Here ER quotes Antoine de Saint-ExupÈry (who was an aviator and aircraft designer when he wasn't authoring classic children's books)

#14. Any tool should be useful in the expected way, but a truly great tool lends itself to uses you never expected.

#15. When writing gateway software of any kind, take pains to disturb the data stream as little as possible and never throw away information unless the recipient forces you to!

#16. When your language is nowhere near Turing-complete, syntactic sugar can be your friend.

#17. A security system is only as secure as its secret. Beware of pseudo-secrets.

#18. To solve an interesting problem, start by finding a problem that is interesting to you. The best hacks start out as personal solutions to the author's everyday problems, and spread because the problem turns out to be typical for a large class of users. 

#19: Provided the development coordinator has a communications medium at least as good as the Internet, and knows how to lead without coercion, many heads are inevitably better than one.

------------ER's Comment Starts--------------
Necessary Preconditions for the Bazaar Style: When you start community-building, what you need to be able to present is a plausible promise. Your program doesn't have to work particularly well. It can be crude, buggy, incomplete, and poorly documented. What it must not fail to do is (a) run, and (b) convince potential co-developers that it can be evolved into something really neat in the foreseeable future.

In order to build a development community, you need to attract people, interest them in what you're doing, and keep them happy about the amount of work they're doing. Technical sizzle will go a long way towards accomplishing this, but it's far from the whole story. The personality you project matters, too.

I think the future of open-source software will increasingly belong to people who know how to play Linus's game, people who leave behind the cathedral and embrace the bazaar. This is not to say that individual vision and brilliance will no longer matter; rather, I think that the cutting edge of open-source software will belong to people who start from individual vision and brilliance, then amplify it through the effective construction of voluntary communities of interest.
------------ER's Comment Ends--------------

No comments:

Post a Comment