CL to Java
Sunday, June 27, 2004
Ok, let's look at what would appear to be a fairly common occurrence for many CL programmers today: you would like to use Common Lisp but need to either use Java or deal with Java code. There can be a variety of different reasons for this including:
- Better Suited: Your company's standard is Java but the particular problem you need to write code for is either much easier to program in Lisp or there is already some Lisp code that you can use as a base. Ideally, you would like to write the code in Lisp but have it either translated to Java or directly compiled to Java byte code.
- Interoperabiltiy: You need to interoperate with other code that is written in Java. You can write your code in CL but need some mechanism to call out to the Java code.
- Libraries: There are specific packages that are written in Java that you would like to use (and there are no CL equivalents). This is similar to #2, but, in this case, you need to be able to use the packages and the class definitions in the packages in the same way that a Java programmer would use them.
- Translate: Write in a Lisp-like language and convert it to Java code. This approach would probably be best suited for #1 above.
- Java-based Lisp: Use a CL implementation that is either written in Java or compiles to Java byte code. This approach would be appropriate for #1-#3 above
- Foreign Language Interface: Use some mechanism to communicate between CL code and Java code. This could be done using JNI or some other mechanism. This approach is probably most appropriate for #2 and #3 above but may (in some cases) also be acceptable for #1.
- Remote Procedure Calls: Use an RPC protocol to communicate (either on the same computer or different computers) between the CL code and Java code. Although some of the specific alternatives in the other approaches can also be used if the programs are on different machines, the RPC options provide more explicit support for remote calls. As with the FLI approach, this approach is probably most appropriate for #2 and #3 above but may (in some cases) also be acceptable for #1.
- Translators:
- Linj: Linj is an acronym for "Lisp in Java" or, in the tradition of the recursive acronyms, "Linj is not Java." Linj is a new language in the Lisp tradition and was invented to allow Lisp programmers to quickly develop and extend Java programs without writing a single line of Java. The Linj compiler translates Linj programs into human-readable Java programs, serving two different purposes: (1) it allows a very smooth integration with the Java libraries, and (2) it allows the generated code to be read, used and modified by Java programmers. Currently available for ACL, SBCL (on Linux and Mac OS X) and CMUCL (on Linux).
- JiL - Java in Lisp: Java in Lisp (jil) is a language for writing programs to run on the Java Virtual Machine (jvm). It uses a syntax familiar to Lisp programmers. jil programs can be generated by Lisp programs. This allows a Lisp program to migrate functionality over to a jvm. JiL is developed by Franz and available only for Franz's Allegro CL.
- STELLA: STELLA is a strongly typed, object-oriented, Lisp-like language specifically geared to support artificial intelligence applications. STELLA programs are first translated into a target language such as Common Lisp, C++ or Java, and then compiled with the native target language compiler to generate executable code. The language constructs of STELLA are restricted to those that can be translated fairly directly into native constructs of the intended target languages. This allows STELLA to be translated into efficient, conventional and readable Lisp, C++ and Java code. The resulting native code can be understood and to some extent even maintained by programmers who don't know STELLA, and it can easily be interfaced with other programs not written in STELLA. STELLA is known to work on ACL and CMUCL.
- Java-based Lisps:
- ABCL: Armed Bear Common Lisp is a Common Lisp implementation that runs on a Java Virtual Machine. It includes a compiler to Java bytecodes, and allows access to Java libraries from Common Lisp. Armed Bear Lisp is distributed under the GNU GPL as a component of the Armed Bear J editor, where it serves as an extension language (rather like Emacs Lisp for Emacs). However, you can also run ABCL standalone, without the J editor.
- Jatha: Jatha is a Java library that implements a fairly large subset of Common LISP, including most of the datatypes (e.g. packages, bignums) and their associated methods. The API allows programmatic access to LISP from Java, either using an eval() method or using Java methods
- Java-based Schemes: Although I'm trying to focus on Java-based CL alternatives, it is also worth mentioning a few of the Scheme implementations in Java that are available. SISC is a full R5RS Scheme interpreter implementation in Java. Kawa is a Scheme environment, written in Java, that compiles Scheme code into Java byte-codes. The Bigloo JVM code generator delivers highly portable JVM class files (or jar files) that can be run on any JVM runtime environment. JScheme is Peter Norvig's implementation of Scheme in Java and is a compact, R4RS implementation. Depending on what your requirements are, one of these Scheme-in-Java implementations might be suitable.
- FLI's:
- jLinker: jLinker allows dynamic, unpremeditated access to the public methods, constructors, and members of Java classes from the Lisp runtime environment. The end result is that the Lisp application may call Java methods as if they were Lisp functions. The documentation of the Java class is all that the Lisp programmer needs to know to use the Java library effectively. Franz has accomplished this goal with the use of remote interfaces supported by a proprietary socket connection. Only Franz's Allegro CL is supported.
- JACOL: JACOL is a framework for allowing Java and Common Lisp to interoperate. Its intended use is to enable Common Lisp to function as an extension language for Java. But, it can also be used to expose Java APIs (such as Swing and DOM) to Lisp applications. JACOL requires JDK 1.2 or higher and GNU CLISP. Thanks to the flexibility provided by Java reflection, JACOL was able to be implemented using a surprisingly simple design oriented around Java and Lisp "object servers", transparently using sockets for communication. JACOL allows existing Java applications to be manipulated through Lisp without requiring any code changes or recompilation. The socket-based approach has some additional advantages by allowing remote Java and Lisp processes to communicate, and by providing a modular foundation that can easily be used to allow other programming languages to interoperate with Java and/or Lisp.
- LIJOS: The LIJOS system is a Lisp Implementation of the Java ObjectStream protocol. That is, it is an implementation, in Lisp, of the Java serializing ObjectStream protocol that supports the (de)serialization of Java objects into and from appropriately mapped Lisp objects. LIJOS runs in ACL 5.0 but is "intended to be very portable". (Note: the web site for LIJOS seems to be down at the moment.)
- jfli: Although jfli is still vaporware (there's supposed to be a beta available in about a month), it is interesting enough to warrant mentioning. It was announced recently in this posting (look for the messages by Rich Hickey) on the LispWorks mailing list. It is supposed to be a fairly extensive interface to Java from LispWorks, which will be released as open source under the Common Public License. The approach taken was to embed a JVM instance in the Lisp process using JNI using LispWorks' own FLI and no C (or Java!). The objective is to provide comprehensive, safe, dynamic and Lisp-y access to Java and Java libraries as if they were Lisp libraries, for use in Lisp programs, i.e. an emphasis on working in Lisp rather than in Java. This one sounds really promising and I'll definitely be trying it out when it becomes available!
- RPC's:
- CORBA: Both ACL and Lispworks have a CORBA implementation in their commercial CL products. There are also some free CORBA packages available for a number of the open source CL's.
- XML-RPC: There are versions available for ACL, CLISP/CMUCL, LispWorks/OpenMCL/SBCL and Corman CL.
- SOAP: At present, there is only Franz's ACL SOAP implementation available. SOAP offers both RPC-style and Document-style communication options (XML-RPC only offers RPC), so it is a good choice for either synchronous or asynchronous communication between programs.
I found jLinker to be an excellent product. Franz ships a number of jLinker demo programs with ACL that really show off jLinker's capabilities nicely. You can either call out to Java from CL or use jLinker to instantiate Java classes inside of CL and work with them much the same as you would work with CLOS objects (e.g. - you don't need a Java program to talk to, you can just utilize existing Java package libraries).
ABCL is still a work in progress but is showing a lot of promise (if you want to try it out, be sure to use the CVS code as the last official release was in 2003 and the CVS code has quite a few enhancements). ABCL handles some CL code really well but is incomplete in a number of areas. There is the beginning of support for ABCL in SLIME and the cl-bench benchmark tests are starting to include tests for ABCL as well.
Both SISC and Kawa appeared to be good Scheme-in-Java options but I have not really done any more than some superficial testing with them. Chris Double, on the other hand, has done a lot of work with SISC and Java-based web servers. Have a look through some of his weblog postings if you're interested in this approach.
I have done some XML-RPC and SOAP testing with a number of different CL packages. XML-RPC is very easy to use and (if your needs are relatively straight-forward) is a nice option for distributed computing across different language implementations. SOAP is more full-featured and complicated; however, it is where most of the corporate world is heading and is the more likely option that you will have available to you if you are writing software that must interoperate with Java code in an enterprise (Note: I recently posted about a meeting of the Denver Area Lisp User Group that discussed using SOAP with CL).
So, there are a number of different CL<->Java alternatives available now and some interesting ones on the horizon. I'd like to hear about other people's experiences, so please let me know if you've tried out some of these (or other) alternatives.
