Dynamically changing running Lisp code
Monday, February 23, 2004
New Lisp users often have trouble grasping the dynamic nature of Lisp programming. They read something (this particular quote is from Paul Graham's description of how he used Lisp's dynamic nature in ViaWeb) like the following:
"Lisp's interactive toplevel is a great help in developing software rapidly. But the biggest advantage for us was probably in finding bugs. As I mentioned before, with Web-based applications you have the users' data on your servers and can usually reproduce bugs.After reading comments like this, the newbie is usually impressed but often confused as to the actual "mechanics" of how this was done in Lisp. Edi Weitz recently provided a good explanation (and example) in a couple of posts on c.l.l. in response to a question from Artem Baguinski:
When one of the customer support people came to me with a report of a bug in the editor, I would load the code into the Lisp interpreter and log into the users' account. If I was able to reproduce the bug I'd get an actual break loop, telling me exactly what was going wrong. Often I could fix the code and release a fix right away. And when I say right away, I mean while the user was still on the phone. "
"> then i've found that useful-function is buggy, what do i do now?
> change the definition in repl window? or in a text window and then
> eval-last-defun and try that out?
A common way would be to change the function's definition directly in the source code and then from there eval-last-defun so the new function is immediately available in the REPL (together with everything else that you've loaded and evaluated) so you can check if it works now.
> i guess what i'm looking for is a description of other peoples
> everyday lisp programming routines... so i can incorporate bits of
> your experience in my programming practices ;-)
Well, maybe not directly related to your question but at least to "whole language always available:" I've programmed a web-based soccer sweepstakes. It was originally written in PHP (yuck!) but soon thereafter converted to Common Lisp (sitting behind an Apache using Marc Battyani's mod_lisp). I mostly play with a couple of friends but I've also sold it to a commercial customer who used it successfully for about 500 of his clients.
Anyway, the game grew more complicated every year and we're always trying to tune the rules to make it more fun. During this season's winter break (December 2003) I had to implement some new rules which took effect in January. After trying out these new rules on my laptop I connected to the server using Dan Barlow's detachtty and was in my REPL. There I loaded the new functions, compiled them, and that was it. It's still the same application that was started last June, all the objects (players, matches, bets) are still identical to what they were half a year ago. I think this is quite cool and virtually impossible with most other languages.
> are the changes you've made persistent? like if the server has to
> reboot are they somewhere in files?
Others have already commented on this but I'll try to summarize the options that you have:"
- You just change things interactively in the REPL. That way the changes definitely aren't permanent and are lost if you reboot the machine or restart the Lisp image.
- You write your changes into patch files and load them (optionally compiling them first). You have to load these patch files in the right order if you start anew but the benefit is that you can use them as a kind of diary of what you did to your app. You can also use this method to patch code you've already delivered to customers - just send them a FASL file (containing compiled Lisp code) that they can LOAD into their app. (This of course assumes that you've provided them with some means to use LOAD.)
- You use something like MK:DEFSYSTEM or ASDF (similar to make) which keeps track of dependencies and file access times. Now you can make changes directly in your source files and ask the system definition utility to re-compile and re-load what's necessary.
- From time to time, i.e. after you've made significant changes, you save the whole Lisp image. This isn't part of the ANSI standard but virtually every Lisp supports this. If you have to start anew you start from the most recently saved image instead of the default image and you're back in business.
