Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Use config packages. *protoc* and *protoc-gen-lisp* move from ASDF to

  PROTOBUF-CONFIG.
README: Mention Lispworks support.  Re-fill paragraphs.
  • Loading branch information...
commit 819368cfc1c55999b1292df07a5b6bb18e0650c8 1 parent da903f7
Robert Brown authored
64 README
View
@@ -1,30 +1,30 @@
-Protobuf is an implementation of Google's protocol buffers for Common Lisp.
+Protobuf is a Common Lisp implementation of Google's protocol buffers.
-The protobuf repository contains a compiler from .proto files to Lisp and
-the run time support needed by the generated Lisp code. Not all protocol
-buffer features are implemented but the code works well enough that simple
-serialization tests pass.
+The protobuf repository contains a compiler from .proto files to Lisp and the
+run time support needed by the generated Lisp code. Not all protocol buffer
+features are implemented, but the code works well enough that serialization
+tests pass.
-The protobuf code has been tested with ABCL, CCL, CLISP, and SBCL. In the
-past it has also worked with Allegro Common Lisp, but I have not tested it
-recently.
+The protobuf code has been tested with ABCL, CCL, CLISP, Lispworks, and SBCL.
+In the past it has also worked with Allegro Common Lisp, but I have not tested
+it recently.
INSTALLATION
============
-0. Install the software needed to clone a git repository and compile C++
-code. You'll need at least git, G++, and libtool, which on a Linux system
-can often be installed with a package manager:
+0. Install the software needed to clone a git repository and compile C++ code.
+You'll need at least git, G++, and libtool, which on a Linux system can often
+be installed with a package manager:
- apt-get install git g++ libtool
+ sudo apt-get install git g++ libtool
1. The Common Lisp protocol buffer compiler is implemented as a plugin to
-Google's protocol buffer compiler, so you must first build and install
-Google's compiler, which is called protoc. The required code and
-documentation are available here:
+Google's protocol buffer compiler, so you must first build and install Google's
+compiler, which is called protoc. The required code and documentation are
+available here:
http://code.google.com/p/protobuf/
@@ -53,15 +53,15 @@ become:
make install
-4. Download and install other Common Lisp packages that protobuf
-depends on. You'll need ASDF, but it comes pre-installed in most
-Common Lisp distributions. You'll also need com.google.base, which
-you can download by cloning its git repository:
+4. Download and install other Common Lisp packages that protobuf depends on.
+You'll need ASDF, but it comes pre-installed in most Common Lisp distributions.
+You'll also need com.google.base, which you can download by cloning its git
+repository:
git clone git://github.com/brown/base.git
-If you're not using Allegro, CLisp, or SBCL, you may need
-trivial-utf8, which is available here:
+If you're not using Allegro, CLisp, or SBCL, you may need trivial-utf8, which
+is available here:
http://common-lisp.net/project/trivial-utf-8/
@@ -72,17 +72,19 @@ dependencies:
hu.dwim.asdf
alexandria
+The trivial-utf-8 and hu.dwim.stefil packages and their dependencies are
+available via Quicklisp: http://www.quicklisp.org/
-5. Make com.google.base.asd and protobuf.asd available to ASDF. There
-are several ways to do this and you should consult the ASDF
-documentation to determine what will work best for you. ASDF and its
-manual are available here:
+
+5. Make com.google.base.asd and protobuf.asd available to ASDF. There are
+several ways to do this and you should consult the ASDF documentation to
+determine what will work best for you. ASDF and its manual are available here:
http://common-lisp.net/project/asdf/
-On my system, I tell ASDF the locations of the com.google.base and
-protobuf systems by creating a file called source-registry.conf in
-directory /home/brown/.config/common-lisp/ with the following contents:
+On my system, I tell ASDF where to find the com.google.base and protobuf system
+files by creating a file called source-registry.conf in directory
+/home/brown/.config/common-lisp/ with the following contents:
(:source-registry
(:tree "/home/brown/src/base/")
@@ -92,9 +94,9 @@ directory /home/brown/.config/common-lisp/ with the following contents:
6. Tell ASDF where to find Google's protocol buffer compiler and the Common
Lisp plugin. The file paths to use depend on where you have installed each.
-You can either edit the pathnames used to initialize ASDF::*PROTOC* and
-ASDF:*PROTOC-GEN-LISP* in protobuf.asd or you can set these variables in a
-Lisp initialization file before protobuf.asd is loaded by ASDF.
+You can either edit the pathnames used to initialize PROTOBUF-CONFIG:*PROTOC*
+and PROTOBUF-CONFIG:*PROTOC-GEN-LISP* in protobuf.asd or you can set these
+variables in a Lisp initialization file before protobuf.asd is loaded by ASDF.
7. Compile and load all the protobuf code:
11 message-test.lisp
View
@@ -36,7 +36,8 @@
(:documentation "Tests for protocol buffer messages.")
(:use #:common-lisp
#:com.google.base
- #:hu.dwim.stefil)
+ #:hu.dwim.stefil
+ #:protobuf-test-config)
(:export #:test-message))
(in-package #:message-test)
@@ -47,15 +48,11 @@
(in-suite test-message)
-(defconst +pwd+ #.(make-pathname
- :directory (pathname-directory
- (or *compile-file-truename* *load-truename*))))
-
(defconst +golden-file-name+
- (merge-pathnames "google/protobuf/testdata/golden_message" +pwd+))
+ (merge-pathnames "google/protobuf/testdata/golden_message" *base-directory*))
(defconst +golden-packed-file-name+
- (merge-pathnames "google/protobuf/testdata/golden_packed_fields_message" +pwd+))
+ (merge-pathnames "google/protobuf/testdata/golden_packed_fields_message" *base-directory*))
(defconst +optional-field-info+
;; field name, default value, value set by tests
11 proto-lisp-test.lisp
View
@@ -37,7 +37,8 @@
(:documentation "Test the Lisp implementation of protocol buffers.")
(:use #:common-lisp
#:com.google.base
- #:hu.dwim.stefil)
+ #:hu.dwim.stefil
+ #:protobuf-test-config)
(:export #:test-proto-lisp))
(in-package #:proto-lisp-test)
@@ -48,14 +49,10 @@
(in-suite test-proto-lisp)
-(defconst +pwd+ #.(make-pathname
- :directory (pathname-directory
- (or *compile-file-truename* *load-truename*))))
-
-(defconst +golden-file-name+ (merge-pathnames "golden" +pwd+)
+(defconst +golden-file-name+ (merge-pathnames "golden" *base-directory*)
"Pathname of a file containing correct protocol buffer data.")
-(defconst +test-file-name+ (merge-pathnames "test-output-file" +pwd+)
+(defconst +test-file-name+ (merge-pathnames "test-output-file" *base-directory*)
"Pathname of file to which we write protocol buffer data.")
(defmacro assert-string-equal ((field protobuf) string)
77 protobuf-test.asd
View
@@ -1,36 +1,34 @@
+;;;; Copyright 2011 Google Inc. All Rights Reserved
-;;;; protobuf-test.asd
+;;;; Redistribution and use in source and binary forms, with or without
+;;;; modification, are permitted provided that the following conditions are
+;;;; met:
+;;;; * Redistributions of source code must retain the above copyright
+;;;; notice, this list of conditions and the following disclaimer.
+;;;; * Redistributions in binary form must reproduce the above
+;;;; copyright notice, this list of conditions and the following disclaimer
+;;;; in the documentation and/or other materials provided with the
+;;;; distribution.
+;;;; * Neither the name of Google Inc. nor the names of its
+;;;; contributors may be used to endorse or promote products derived from
+;;;; this software without specific prior written permission.
-;; Copyright 2010, Google Inc. All rights reserved.
+;;;; THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+;;;; "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+;;;; LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+;;;; A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+;;;; OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+;;;; SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+;;;; LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+;;;; DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+;;;; THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+;;;; (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+;;;; OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-;; Redistribution and use in source and binary forms, with or without
-;; modification, are permitted provided that the following conditions are
-;; met:
+;;;; Author: brown@google.com (Robert Brown)
-;; * Redistributions of source code must retain the above copyright
-;; notice, this list of conditions and the following disclaimer.
-;; * Redistributions in binary form must reproduce the above
-;; copyright notice, this list of conditions and the following disclaimer
-;; in the documentation and/or other materials provided with the
-;; distribution.
-;; * Neither the name of Google Inc. nor the names of its
-;; contributors may be used to endorse or promote products derived from
-;; this software without specific prior written permission.
-
-;; THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-;; "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-;; LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-;; A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-;; OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-;; SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-;; LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-;; DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-;; THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-;; (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-;; OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-(cl:in-package #:common-lisp-user)
+(in-package #:common-lisp-user)
(defpackage #:protobuf-test-system
(:documentation "System definitions for protocol buffer test code.")
@@ -42,7 +40,7 @@
:name "Protocol Buffer Test"
:description "Protocol buffer test code"
:long-description "Code to test the protocol buffer compiler and support libraries."
- :version "0.5.1"
+ :version "0.5.2"
:author "Robert Brown"
:license "See file COPYING and the copyright messages in individual files."
:defsystem-depends-on (#:protobuf)
@@ -51,18 +49,14 @@
((:static-file "golden")
(:file "message-test" :depends-on ("unittest"))
(:file "wire-format-test")
-
- ;; Old protocol buffer tests
- ;; XXXX: Delete these when the new proto2 tests cover all the functionality.
-
+ ;; Old protocol buffer tests.
+ ;; TODO(brown): Delete when the new proto2 tests cover all the functionality.
(:file "proto-lisp-test" :depends-on ("testproto1" "testproto2"))
;; Two protocol buffers used by the old tests.
+ ;; TODO(brown): Delete when the new proto2 tests cover all the functionality.
(:protobuf-source-file "testproto1")
(:protobuf-source-file "testproto2")
-
- ;; Test protocol buffers and protobuf definitions used by the proto2
- ;; compiler.
-
+ ;; Test protocol buffers and protobuf definitions used by the proto2 compiler.
(:protobuf-source-file "descriptor" :proto-pathname "google/protobuf/descriptor")
(:protobuf-source-file "unittest_import" :proto-pathname "google/protobuf/unittest_import")
(:protobuf-source-file "unittest"
@@ -74,3 +68,12 @@
(funcall (read-from-string "message-test:test-message"))
(funcall (read-from-string "proto-lisp-test:test-proto-lisp"))
(funcall (read-from-string "wire-format-test:test-wire-format")))
+
+(defpackage #:protobuf-test-config
+ (:documentation "Configuration information for PROTOBUF-TEST.")
+ (:use #:common-lisp)
+ (:export *base-directory*))
+
+(in-package #:protobuf-test-config)
+
+(defparameter *base-directory* (make-pathname :name nil :type nil :defaults *load-truename*))
105 protobuf.asd
View
@@ -28,26 +28,37 @@
;;;; Author: brown@google.com (Robert Brown)
-(cl:in-package #:asdf)
+;;;; Protobuf configuration
-;; Pathname of Google's protocol buffer compiler. You must replace this
-;; pathname with the appropriate one for your system.
+(in-package #:common-lisp-user)
+
+(defpackage #:protobuf-config
+ (:documentation "Configuration information for PROTOBUF.")
+ (:use #:common-lisp)
+ (:export *protoc*
+ *protoc-gen-lisp*
+ *protoc-relative-path*))
+
+(in-package #:protobuf-config)
+
+;; Pathname of Google's protocol buffer compiler. You must replace this pathname with the
+;; appropriate one for your system or set its value before this file is loaded.
(defvar *protoc* #p"/local/software/package/google-protobuf/bin/protoc"
"Pathname of Google's protocol buffer compiler.")
-(export '*protoc*)
-;; Pathname of the Lisp protocol buffer compiler backend. You must replace
-;; this pathname with the appropriate one for your system.
+;; Pathname of the Lisp protocol buffer compiler backend. You must replace this pathname with the
+;; appropriate one for your system or set its value before this file is loaded.
(defvar *protoc-gen-lisp* #p"/local/software/package/protoc-gen-lisp/bin/protoc-gen-lisp"
"Pathname of the Lisp protocol buffer compiler backend, protoc-gen-lisp.")
-(export '*protoc-gen-lisp*)
(defvar *protoc-relative-path* nil
"Proto file arguments to the protobuf compiler are relative paths?")
-(export '*protoc-relative-path*)
+;;;; ASDF package changes
+
+(in-package #:asdf)
(defclass protobuf-source-file (cl-source-file)
((relative-proto-pathname
@@ -59,40 +70,41 @@
:initform ()
:initarg :proto-search-path
:reader search-path
- :documentation "List containing directories where the protocol buffer
-compiler should search for imported protobuf files. Non-absolute pathnames
-are treated as relative to the directory containing the DEFSYSTEM form in
-which they appear."))
+ :documentation "List containing directories where the protocol buffer compiler should search
+for imported protobuf files. Non-absolute pathnames are treated as relative to the directory
+containing the DEFSYSTEM form in which they appear."))
(:documentation "A protocol buffer definition file."))
(export '(protobuf-source-file proto-pathname search-path))
+;;;; ASDF system definition
(in-package #:common-lisp-user)
(defpackage #:protobuf-system
(:documentation "System definitions for protocol buffer code.")
- (:use #:common-lisp #:asdf))
+ (:use #:common-lisp
+ #:asdf
+ #:protobuf-config))
(in-package #:protobuf-system)
-
(defclass proto-to-lisp (operation)
()
- (:documentation "An ASDF operation that compiles a .proto file containing
-protocol buffer definitions into a Lisp source file."))
+ (:documentation "An ASDF operation that compiles a .proto file containing protocol buffer
+definitions into a Lisp source file."))
(defmethod component-depends-on ((operation compile-op) (component protobuf-source-file))
- "Compiling a protocol buffer file depends on generating Lisp source code
-for the protobuf, but also on loading package definitions and in-line
-function definitions that the machine-generated protobuf Lisp code uses."
+ "Compiling a protocol buffer file depends on generating Lisp source code for the protobuf, but
+also on loading package definitions and in-line function definitions that the machine-generated
+protobuf Lisp code uses."
`((proto-to-lisp ,(component-name component))
,@(call-next-method)))
(defmethod component-depends-on ((operation load-op) (component protobuf-source-file))
- "Loading a protocol buffer file depends on generating Lisp source code
-for the protobuf, but also on loading package definitions and in-line
-function definitions that the machine-generated protobuf Lisp code uses."
+ "Loading a protocol buffer file depends on generating Lisp source code for the protobuf, but also
+on loading package definitions and in-line function definitions that the machine-generated protobuf
+Lisp code uses."
`((proto-to-lisp ,(component-name component))
,@(call-next-method)))
@@ -114,14 +126,13 @@ translated into Lisp source code for this PROTO-FILE component."
(list *protoc* *protoc-gen-lisp* (proto-input component)))
(defmethod output-files ((operation proto-to-lisp) (component protobuf-source-file))
- "Arrange for the Lisp output file of PROTO-TO-LISP operations to be stored
-where fasl files are located."
+ "Arrange for the Lisp output file of PROTO-TO-LISP operations to be stored where fasl files are
+located."
(values (list (component-pathname component))
nil)) ; allow around methods to translate
(defun resolve-relative-pathname (path parent-path)
- "When PATH doesn't have an absolute directory component, treat it as
-relative to PARENT-PATH."
+ "When PATH doesn't have an absolute directory component, treat it as relative to PARENT-PATH."
(let* ((pathname (pathname path))
(directory (pathname-directory pathname)))
(if (and (list directory) (eq (car directory) :absolute))
@@ -139,8 +150,8 @@ relative to PARENT-PATH."
(resolve-relative-pathname path parent-path))
search-path))))
-;; XXXX: This before method would not be needed if PROTO-TO-LISP were a
-;; subclass of COMPILE-OP. Should we make that change?
+;; TODO(brown): This before method would not be needed if PROTO-TO-LISP were a subclass of
+;; COMPILE-OP. Should we make that change?
(defmethod perform :before ((operation proto-to-lisp) (component protobuf-source-file))
(map nil #'ensure-directories-exist (output-files operation component)))
@@ -150,9 +161,8 @@ relative to PARENT-PATH."
(source-file-argument (if *protoc-relative-path*
(file-namestring source-file)
(namestring source-file)))
- ;; Around methods on output-file may globally redirect output
- ;; products, so we must call that method instead of executing
- ;; (component-pathname component).
+ ;; Around methods on output-file may globally redirect output products, so we must call
+ ;; that method instead of executing (component-pathname component).
(output-file (first (output-files operation component)))
(search-path (cons (directory-namestring source-file) (resolve-search-path component)))
(status (run-shell-command "~A --plugin=~A --proto_path=~{~A~^:~} --lisp_out=~A ~A"
@@ -165,18 +175,17 @@ relative to PARENT-PATH."
(error 'compile-failed :component component :operation operation))))
(defmethod asdf::component-self-dependencies :around ((op load-op) (c protobuf-source-file))
- "Remove PROTO-TO-LISP operations from self dependencies. Otherwise, the
-Lisp output files of PROTO-TO-LISP are considered to be input files for
-LOAD-OP, which means ASDF loads both the .lisp file and the .fasl file."
+ "Remove PROTO-TO-LISP operations from self dependencies. Otherwise, the Lisp output files of
+PROTO-TO-LISP are considered to be input files for LOAD-OP, which means ASDF loads both the .lisp
+file and the .fasl file."
(remove-if (lambda (x)
(eq (car x) 'proto-to-lisp))
(call-next-method)))
-;; The following code was copied from asdf.lisp and modified slightly to set
-;; SOURCE-FILE to a pathname in the directory where fasl files are stored.
-;; The PERFORM method defined in asdf.lisp for instances of CL-SOURCE-FILE
-;; computes SOURCE-FILE by calling COMPONENT-PATHNAME instead of by calling
-;; the INPUT-FILES generic function. I think this is a misfeature of ASDF.
+;; The following code was copied from asdf.lisp and modified slightly to set SOURCE-FILE to a
+;; pathname in the directory where fasl files are stored. The PERFORM method defined in asdf.lisp
+;; for instances of CL-SOURCE-FILE computes SOURCE-FILE by calling COMPONENT-PATHNAME instead of by
+;; calling the INPUT-FILES generic function. I think this is a misfeature of ASDF.
(defmethod perform ((operation compile-op) (c protobuf-source-file))
(let ((source-file (make-pathname :name (pathname-name (component-pathname c))
@@ -190,30 +199,22 @@ LOAD-OP, which means ASDF loads both the .lisp file and the .fasl file."
(asdf::compile-op-flags operation))
(when warnings-p
(case (operation-on-warnings operation)
- (:warn (warn
- "~@<COMPILE-FILE warned while performing ~A on ~A.~@:>"
- operation c))
+ (:warn (warn "~@<COMPILE-FILE warned while performing ~A on ~A.~@:>" operation c))
(:error (error 'compile-warned :component c :operation operation))
(:ignore nil)))
(when failure-p
(case (operation-on-failure operation)
- (:warn (warn
- "~@<COMPILE-FILE failed while performing ~A on ~A.~@:>"
- operation c))
+ (:warn (warn "~@<COMPILE-FILE failed while performing ~A on ~A.~@:>" operation c))
(:error (error 'compile-failed :component c :operation operation))
(:ignore nil)))
(unless output
(error 'compile-error :component c :operation operation)))))
-
-;;; Protocol buffer support code.
-
-
(defsystem protobuf
:name "Protocol Buffer"
:description "Protocol buffer code"
:long-description "A Common Lisp implementation of Google's protocol buffer support libraries."
- :version "0.7.2"
+ :version "0.7.3"
:author "Robert Brown"
:license "See file COPYING and the copyright messages in individual files."
;; After loading the system, announce its availability.
@@ -231,6 +232,10 @@ LOAD-OP, which means ASDF loads both the .lisp file and the .fasl file."
(:file "package")
(:file "protocol-buffer" :depends-on ("package"))
#-(or abcl allegro cmu sbcl) (:file "portable-float" :depends-on ("package"))
+ ;; TODO(brown): The file lispworks-float.lisp implementations the same interface as
+ ;; portable-float.lisp. Currently, some functions have a Lispworks-specific implementation,
+ ;; while others invoke the portable functions. All implemented using portable functions should
+ ;; be rewritten.
#+lispworks (:file "lispworks-float" :depends-on ("package"))
(:file "wire-format"
:depends-on ("package"
2  varint/varint-test.asd
View
@@ -42,4 +42,4 @@
((:file "varint_test")))
(defmethod perform ((operation test-op) (component (eql (find-system :varint-test))))
- (funcall (intern (string '#:test-varint) '#:varint-test)))
+ (funcall (read-from-string "varint-test:test-varint")))
Please sign in to comment.
Something went wrong with that request. Please try again.