Clementson's Blog

Hints to Macro Writers

Wednesday, August 10, 2005

There's a thread (actually a series of threads) running over on the lightweight languages mailing list that talks about module and package systems and hygienic macros vs non-hygienic macros (among a lot of other things). In it, Lenny Foner makes an intelligent post about types of macros and why/where you would use them. I've reproduced his post below; however, one thing that you don't want to miss is the link to "Hints to Macro Writers" (a section from the Symbolics LispM documentation that he's converted into HTML) that contains a lot of useful information about writing macros.

Here are Lenny's comments:

"I'd like to nail down what's going on when people who've used Common Lisp complain about the macro system. Those of you who've complained--- can you make some comments about the notes below?

Issue zero: Have you had something like Meta-. available, so your editing tools can instantly find the definition of any macro you don't know about? Without that, dealing with someone else's macro code would be unbearable---but then again, I'd find even dealing with someone else's -functions- unbearable... [And did you use some sort of system-definition tool, in case you had enough macros to put in a file that wanted to be defined before anything else?]

Issue one: I've taken a bit of time and converted a section from the Symbolics documentation, called "Hints for Macro Writers", from the Lispm's Concordia format into HTML (using lots of Emacs macros, but let's not go -there- when talking about macros... :) It's in http://foner.www.media.mit.edu/people/foner/Miscellaneous/macros.html [Any odd formatting or typos are probably from the conversion.] I actually used (parts of) this exact document about 10-12 years ago at the Media Lab when teaching a couple of seminars on how to use Lisp effectively to some of the grad students there; I wish I knew who in particular to thank for creating it.

That document lays out a whole bunch of useful strategies for using macros. I found it very helpful to read when I was a fledgling CL programmer, when I was first using Lispms as an undergraduate and before going to Symbolics. (It's been around at least a couple of decades, I think.) Are these strategies that you used? If not, would reading a document such as this have helped you?

A lot of it talks about hygiene (though not by that name, of course, since it's old), and it also talks about modularity, debugging, and issues like order of evaluation. Reading through it, one might be struck that there seem to be many pitfalls, but they're also really easy to sidestep, and I'd think that learning the special language that hygenic macros seem to demand wouldn't be all that much easier.

Issue two: What sort of macros did you try to write, and how big were they?

Let me be more specific. When I write macros, I tend to write two types almost exclusively (with examples below):

• Type 1: Macros whose whole purpose is side-effects: they expand into code that defines things in the toplevel environment, or update global variables (same thing, basically).
• Type 2: Macros which surround code and which try not to capture anything inadvertently.

Typical "type 1" macros in CL are things like defparameter, defconst, defgeneric, etc etc. I tend to write a lot of macros that get called like:

  (define-my-widget FOO :color 'red)

and which typically expand into something that side-effects a hashtable somewhere, e.g.,

  (defmacro DEFINE-MY-WIDGET name &rest parameters
     `(setf (gethash ,name *SOME-TABLE*) (list ,parameters)))

or whatever. (Note that I haven't typed any of this into a listener and so it's completely untested, etc etc.)

Typically, when I'm making a a "type 1" macro, it's a macro and not a function just so I don't have stick lots of quotes in, e.g.,

  (define-my-widget 'foo :color 'red)

or because I'm doing something with syntax that would be clumsy to do with a function, or perhaps because I want something that expands to -nothing- with no runtime decision penalty if not debugging, etc etc.

(I've often used such a thing to make enumerated types when talking to other languages (e.g., C) or device drivers or whatever, where I have to define slots and I'd rather each definition was in its own toplevel form so I can find it with the editor. Note that such a macro -cannot- do something like

  (incf *SLOT*)

because then reevaluating the macro would doubly-increment the slot, so typically such enumeration macros either need the slot number to be explicit, or they have to be designed to take lots of slots at once and define the entire enumeration at once---I mention this just because I wonder whether things like this are why people might get screwed up with macros.)

Type 2 macros, on the other hand, are of the venerable with- form, e.g., (with-open-file <foo>) and very, very often tend to be things like

  (with-froboz-held
    (blah)
    (biff))

and are defined like

  (defmacro with-froboz-held &body body
     `(unwind-protect
          (progn ,@body)
       (do-some-cleanup)))
  

and these are the kinds where, of course, one winds up doing things like

  (defmacro blarg &body body
     (let ((my-doohickey (gensym)))
       `(let ((,my-doohickey (whatever)))
           ,@body)))
  

to avoid name conflicts and which hygenic macros would solve.

Sure, writing macro-defining-macros with triply-nested combinations of backquotes, commas, and at-signs can get tricky (I've certainly written that sort of thing, but they're annoying to debug and often unnecessary). Instead, 90% of what -I- write are really pretty simple. So to those who've complained about macros in general or CL macros in particular---what were you trying to do, and what were the failure modes?"