Metaprogramming is hard
Monday, December 20, 2004
In Ian Bickings blog, he has a posting on "The Challenge of Metaprogramming" where he argues that Python was right to resist metaprogramming because it is "hard" to do right:
"Lisp deserves its place as the first high-level programming language. There were other languages of the time which let programmers work at a higher level than they had before, but they did it by putting higher level concepts into the language. Lisp was the first that made it possible to add higher level concepts. It placed power in the hands of the programmer, instead of leaving it in the hands of the language designer.That's an interesting concept - reduce the power of your programming language because some people will find macros too hard! Since the precept is that "Metaprogramming is powerful, and was central to Viaweb" but that "for many (most?) people macros are going to be really really hard." I would have thought that the correct conclusion would be to allow only your smart people to do metaprogramming. Just as with database applications (where you normally have a DBA who manages your data model), why not just limit the metaprogramming in your application to a select few? Macros aren't hard to use, they're just hard to write. As I posted previously, the "Lisp Difference" is that it allows you to create whatever domain-specific language you need for the problem space that you're working in. By "dumbing down" your programming language, you just place an artificial limit on what you can potentially do with that programming language.
So if it was so hot, why aren't we all coding Lisp now? Those were days when resources were much more limited, but that didn't stop Lisp, and it still should have been ready when the computing resources became available. Lisp should have been poised to take over the world (and a lot of people thought it would). There's lots of different theories about why this didn't happen, but I'll offer one more.
Lisp makes good programmers really productive, more than they could be in another language. Paul Graham talks about this in Beating the Averages. He made great software and sold it for a bundle to Yahoo. But now it's been reimplemented in C++_. Why, oh why?
It's easy to blame stupid people for this sort of thing, except that it keeps happening over and over. Metaprogramming is powerful, and was central to Viaweb (20-25% of the code, according to Graham). I think this is an example of Common Lisp's fatal flaw (and since Common Lisp is the standard bearer for all Lisps, it is Lisp's fatal flaw).
Lisp metaprogramming is hard. All metaprogramming is hard. But if you can make due with, say, 75% less code due to metaprogramming, and the code is only 100% harder to understand, the net result is that the code is half as hard to understand in its entirety.
Except Lisp metaprogramming scales wrong. For a really good Lisp programmer, the result may be a lot better than 0.25 x 2.00 (i.e., 25% of the code, 200% as hard to understand, end result is 0.50 times as difficult). But for someone who doesn't know Lisp it's like 0.25 x 10.00 (25% of the code, 1000% as hard to understand = 2.5 times as hard, net; and often it's worse than that), because even after they learn Lisp, they have to learn the macros, and for many (most?) people macros are going to be really really hard."
But, I guess I'm just preaching to the converted here. ;-)
