Bill Clementson's Blog

Bits and pieces (mostly Lisp-related) that I collect from the ether.

October 2006
Sun Mon Tue Wed Thu Fri Sat
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31
Sep  Nov

VM and Gmail Setup for Dummies

Monday, October 23, 2006

Ok, let me say right up front that I know NOTHING about setting up mail on a Unix-like OS (specifically, Mac OS X). So, all of you Unix mail hackers can quietly snicker in your beer and go read some other article. All of my life, I've had my email handled by either a corporate server or by an Internet email provider, so I've never had to learn anything about email setup (well, there was this one time when I was using Win32 and Emacs and foolishly thought I could switch over to using an Emacs mail client, but my psychiatrist has advised me to try to forget that experience! ;-) ).

So, you might ask, why would I want to compromise my intellectual virginity by attempting to do something now? Well, the problem is that I'm really tired of the Gmail UI. I've tried to get into this Web 2.0 thing and use only a browser client for email, but it's not really working for me. The interface seems pretty sluggish at times (not sure what this might be due to) and I use a number of 3rd party Firefox extensions (namely, Hit-a-Hint, surfkeys, and keyconfig which allow me to use the keyboard a lot more instead of the mouse when I'm working in Firefox) and their bindings sometimes conflict with Gmail's bindings (yeah, yeah, I know about the basic HTML Gmail client, but c'mon!). So, I decided to make a stab at getting an Emacs-based mail client working with Gmail.

My first choice was Gnus. I already use Gnus for reading news and thought that I would like the Gnus Way for mail too. I decided to first google around to see whether anyone had done this before. Unfortunately, the first link I hit was the GnusGmail page. This seemed to imply that it wasn't possible to use Gnus with Gmail without a lot of non-standard "tweaks". (not that I'm opposed to non-standard tweaks; however, I prefer to be able to do my "tweaking" in a step-wise manner and not a "big bang" approach where I won't know if something works or not until I've got everything installed). Also, since I'm using Mac OS X and not Linux, I didn't know whether the suggested approach would work or not. Also, the other google hits on Gnus+Gmail were mostly "horror stories" of different problems people had trying to get Gnus and Gmail working together. So, I decided to send a message to the Mac OS X Emacs Mailing List and the Aquamacs Emacs Developers List to see what had worked for others. I wasn't exactly flooded with replies (actually, later on, Chris Dean sent me a couple of emails with some details about how he had setup Gnus with Gmail; however, I had already started down the VM road at that stage). Then Terry Jones sent me an email asking if I had tried using VM. I hadn't (and VM was my second choice anyhow), so I sent another email to the lists asking if anyone had been successful with VM and Gmail. This resulted in a more positive response, so I decided to have a go at getting VM setup. After a bit of hair-pulling, knashing-of-teeth, and choice epithets (and a bit of advice and config file help from José M Figueroa-O'Farrill and Terry Jones), I got everything going! Now I am able to read and send Gmail-based mail from within Emacs using VM! Halleluja! Glory be!

For the benefit of anyone who might have to do the same (and, just in case my backup strategy isn't quite perfect ;-) ), I'll outline what I had to do here. There may also be some better/alternative settings that should be used. If anyone sees anything that could be improved, please let me know and I'll update this blog entry.

First of all, here is what Gmail requires:
Incoming Mail (POP3) Server: pop.gmail.com
Use SSL: Yes
Port: 995
Outgoing Mail (SMTP) Server: smtp.gmail.com (use authentication)
Use Authentication: Yes
Use STARTTLS: Yes (some clients call this SSL)
Port: 465 or 587
Account Name: your Gmail username (including @gmail.com)
Email Address: your full Gmail email address (username@gmail.com)
Password: your Gmail password

So, to setup Emacs to use VM to access Gmail emails:

  1. Enable POP in Gmail
  2. Download VM and build/install it as per the VM README file.
  3. Download and install w3m and emacs-w3m (for reading HTML pages) and vm-w3m.el (for reading HTML pages in VM) so that you can read HTML mail with VM (there are alternatives for reading HTML pages in Emacs; however, I prefer w3m). Alternatively, some people prefer to just delete any HTML mail they get ;-).
  4. Download and install stunnel to allow POP within SSL. There are various alternative ways you can do this - I used Fink.
  5. Setup your .emacs file:
    ;;__________________________________________________________________________
    ;;;;    VM - Mail
    ;; Some file locations are relative to my HOME or BIN directories (defvar home-dir) (setq home-dir (concat (expand-file-name "~") "/")) (defvar bin-dir (concat home-dir "bin/"))
    ;; Basic VM setup (push (concat bin-dir "emacs/site-lisp/vm") load-path) (autoload 'vm "vm" "Start VM on your primary inbox." t) (autoload 'vm-other-frame "vm" "Like `vm' but starts in another frame." t) (autoload 'vm-visit-folder "vm" "Start VM on an arbitrary folder." t) (autoload 'vm-visit-virtual-folder "vm" "Visit a VM virtual folder." t) (autoload 'vm-mode "vm" "Run VM major mode on a buffer" t) (autoload 'vm-mail "vm" "Send a mail message using VM." t) (autoload 'vm-submit-bug-report "vm" "Send a bug report about VM." t) (setq vm-toolbar-pixmap-directory (concat (expand-file-name "~") "/bin/emacs/site-lisp/vm/pixmaps")) (setq vm-image-directory (concat (expand-file-name "~") "/bin/emacs/site-lisp/vm/pixmaps")) (setenv "PATH" (concat (concat (expand-file-name "~") "/bin/emacs/site-lisp/vm/bin") ":" (getenv "PATH"))) (setq send-mail-function 'sendmail-send-it) (setq user-full-name "User Name") (setq user-mail-address "username@gmail.com") (setq mail-archive-file-name "~/Mail/SENT")
    ;; Configure inbound mail (POP) (setq vm-spool-files '(("~/INBOX" "pop-ssl:pop.gmail.com:995:pass:username:*" "~/INBOX.CRASH")))
    ;; Use W3M to read HTML email (require 'w3m-load) (setq vm-mime-use-w3-for-text/html nil) (setq vm-url-browser 'w3m) (load "vm-w3m") (setq w3m-input-coding-system 'utf-8 w3m-output-coding-system 'utf-8)
    ;; Configure outbound mail (SMTP) (setq smtpmail-starttls-credentials '(("smtp.gmail.com" 587 nil nil)) smtpmail-smtp-server "smtp.gmail.com" smtpmail-default-smtp-server "smtp.gmail.com" send-mail-function 'smtpmail-send-it message-send-mail-function 'smtpmail-send-it smtpmail-smtp-service 587 smtpmail-auth-credentials '(("smtp.gmail.com" 587 "username@gmail.com" nil)))
    The first section of the above is normal VM setup stuff. The key line is for vm-spool-files. The relevant documentation is under the Spool Files and Pop Spool Files sections of the VM manual. The lines after the vm-spool-files line are for setting up w3m to display HTML emails and the lines after that are for setting up outbound mail (using smtpmail). (Note: I've substituted "username" for my own Gmail username is this example code).
  6. At this point, you should be able to launch VM inside Emacs (using "M-x vm") and get any emails that haven't been read in Gmail from your Gmail Inbox and send new messages and replies to existing emails.
Now, I am able to read and send emails from within Emacs! At last, freedom from that Web2.0 Gmail client!

Update-2006-10-24: Maybe I posted too quickly - for some unknown reason, I can now no longer send emails from VM. My emails all bounce with the following error message:
: host gmail-smtp.l.google.com[64.233.183.109] said: 530 5.7.0
   Must issue a STARTTLS command first y23sm1067533nfb (in reply to MAIL FROM
   command)
Obviously, my postfix setup still needs a bit of work. Can anyone suggest:
  1. Why my setup was working for a few hours yesterday?
  2. What I might need to do to get it working again?
Darn, I guess this had gone a bit too smoothly! :-(

Update-2006-10-25: Thanks to Erik Hetzner and Pedro Kröger who both suggested that I use smtpmail instead of postfix for sending email through Gmail. Thanks to them, I was able to greatly reduce the number of cofinguration steps and avoid the hassles of trying to get a postfix configuration setup properly. I have eliminated the postfix steps that were in the original post.

Update-2009-01-26: Thomas Danckaert sent in the following email with a fix for xemacs:
"I read your article on how to set up vm for gmail. However, after making all the configurations you mentioned, I still got messages 'Must issue starttls command first'. I've noticed quite a few people have this problem, so you might help a lot of people by adding the following solution to your article:

I was only able to solve the problem after several hours of googling. The cause is that I use xemacs (I suppose you use GNU emacs?) and that smtpmail.el uses a function "with-no-warnings", which does not exist in xemacs. This can then be amended by adding the following code to site-start.el:
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defun with-no-warnings (&rest body)
 (car (last body)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
I'm very new to emacs, so if what I wrote didn't make much sense, you might want to read the following discussion by people who actually know what they're talking about :)

http://groups.google.com/group/gnu.emacs.bug/browse_thread/thread/be1ad3235b22585f?q=vm+starttls#28770327fcb6df42 "

emacs Copyright © 2009 by Bill Clementson