Lisp is for Entrepreneurs
Wednesday, January 18, 2006
Paul Graham is famous for having written a web store application called Viaweb in Lisp and subsequently selling it to Yahoo (where it now operates under the name of Yahoo Store). Paul's reason for using Lisp was:
"Our hypothesis was that if we wrote our software in Lisp, we'd be able to get features done faster than our competitors, and also to do things in our software that they couldn't do. And because Lisp was so high-level, we wouldn't need a big development team, so our costs would be lower. If this were so, we could offer a better product for less money, and still make a profit. We would end up getting all the users, and our competitors would get none, and eventually go out of business."Yahoo's rationale for the acquisition:
""Viaweb has set the standard for fast, reliable and secure tools enabling Web merchants to build and operate online stores. Integrating this into Yahoo!'s extensive base of Web merchants and its vast audience of shoppers is a great combination.""However, after acquiring Viaweb, Yahoo subsequently re-wrote the application in C++ and Perl. Paul described the reasons for Yahoo doing this:
"(a) The reason they rewrote it was entirely that the current engineers didn't understand Lisp and were too afraid to learn it.Now, a similar situation has happened with the PS2 game developer Naughty Dog. They developed a custom DSL using Lisp to create some of their most famous games. The company's co-founder, Andy Gavin, "used Allegro CL to create a programming language called "GOAL" (Game Oriented Assembly Lisp), which he designed specifically for the development of Naughty Dog's games. Using this custom dialect whose compiler and development environment was written in Allegro CL, he and his team were able to produce hundreds of different game objects with sophisticated real-time behavior and animation, and more realistic graphics. "The behaviors are faster to develop, and more compact than an equivalent C program, allowing for rapid prototyping and experimentation.""
(b) The resulting program is a new world's record case of Greenspun's Tenth Rule. The Yahoo Store Editor called compile at runtime on s-expressions made on the fly. To translate this into C++ they literally had to write a Lisp interpreter.
(c) Even then, they had to drop some features (involving advanced uses of closures)."
In 2001, Naughty Dog was bought out by Sony. The rationale for the acquisition:
"After it became clear that Naughty Dog was one of the few companies that were pushing the envelope on PS2 from a technical standpoint, Sony finally purchased Naughty Dog in January 2001 to strengthen first-party development."However, after Sony acquired Naughty Dog, they decided to stop using Lisp. The lead programmer describes the reasons for this:
"In all honesty, the biggest reason we're not using GOAL for next-gen development is because we're now part of Sony. I can only imagine Sony's shock when they purchased Naughty Dog a few years back, hoping to be able to leverage some of our technology across other Sony studios, and then realized that there was no way anyone else would be able to use any of our codebase. :)So, Yahoo acquires Viaweb and rewrites it. The end result is inferior to the original Lisp-based product. Sony acquires Naughty Dog and decides to eliminate the Lisp-based development infrastructure. The end result is an inferior game development environment. Sure, there were probably a lot of reasons for these decisions by Yahoo and Sony (probably none of which had anything to do with productivity or technical excellence - large companies have other drivers which have more importance than these two); however, the end result for both companies has been something inferior to what they originally acquired.
Sony wants us to be able to share code with other studios, and this works both ways - both other studios using our code and vice versa. Add this to the difficulty curve of learning a new language for new hires, lack of support from external development tools (we had our own compiler, linker, and debugger, and pretty much had to use Emacs as our IDE), etc, means that there are clearly a lot of other factors involved. Note, however, that these issues aren't really technical problems, they're social ones.
I can definitively say that the investment in GOAL was worth it for our PS2 titles, despite the initial setup time and maintenance. Our productivity gains were huge, and were more than worth the time investment. This time around, however, the circumstances aren't quite the same. If we were still an independent studio, I'm almost positive we'd be extending GOAL for the next-generation of development. As it is, we are looking into alternative approaches (custom preprocessors, assemblers, linkers, you name it) - but all of these approaches fall short in many ways of the unified language and environment we had with GOAL.
That said, if there was a serious effort on the part of the game development community to develop and standardize a language for game development, everyone could gain the benefits without suffering the drawbacks (lack of code-sharing, learning curve, etc). And if there's enough community support, it would only be a matter of time before some really high-quality commercial tools came out to work with the language."
But, for both Paul Graham and Naughty Dog, the use of Lisp allowed them to develop products that pushed them to the front of the pack. They were subsequently bought out; however, the fact that their Lisp tools were subsequently discarded does not throw a negative shadow on Lisp. Lisp got them to where they needed to be to succeed - definitely a quality that entrepreneurs want in a programming language!
