Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Dependency support #16

Merged
merged 3 commits into from

3 participants

@lpetre

I'm trying to setup a custom buildpack that can provide libzmq if necessary. The issue I ran into is that libzmq requires uuid-dev, which is in utils-linux. I can use vulcan to build utils-linux, but then was faced with the question: how do I get that version of utils-linux onto the machine that is building libzmq.

Rather than setup a custom buildpack for vulcan that included utils-linux, I added an option to vulcan build that lets me include any number of vulcan built dependencies. These dependencies get pulled down via http and expanded into a deps folder, which is then added to the LDFLAGS/CFLAGS/etc environment variables.

This also includes my other pull request, to fix the cradle errors.

lpetre added some commits
@lpetre lpetre Fixing cradle api fab44be
@lpetre lpetre Fixing dependency versions 24c1a17
@lpetre lpetre adding support for vulcan build -d <dep1>,...,<depN>
 dependencies are other vulcan built libraries which are required for the current build
24bf5ba
@ddollar ddollar merged commit b39363c into from
@tjwebb

When I try to use this, I simply get this output:

$ vulcan build -v -d "https://s3.amazonaws.com/gisio-geoserver/tools/fuse-2.9.vulcan.tgz"
Packaging local directory... done
Uploading source package... Internal Server Error!! Unknown error, no build output given
@tjwebb

Wow finally figured this out. This works:

vulcan build -v --deps '["https://s3.amazonaws.com/gisio-geoserver/tools/fuse-2.9.vulcan.tgz"]'

This is documented precisely nowhere. But although it downloads my dependency, it doesn't seem to work. That is, I don't think it's being properly extracted and installed. Can someone comment on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 22, 2012
  1. @lpetre

    Fixing cradle api

    lpetre authored
  2. @lpetre

    Fixing dependency versions

    lpetre authored
  3. @lpetre

    adding support for vulcan build -d <dep1>,...,<depN>

    lpetre authored
     dependencies are other vulcan built libraries which are required for the current build
This page is out of date. Refresh to see the latest.
View
5 lib/vulcan/cli.rb
@@ -25,6 +25,7 @@ class Vulcan::CLI < Thor
method_option :output, :aliases => "-o", :desc => "output build artifacts to this file"
method_option :prefix, :aliases => "-p", :desc => "the build/install --prefix of the software"
method_option :source, :aliases => "-s", :desc => "the source directory to build from"
+ method_option :deps, :aliases => "-d", :desc => "other vulcan compiled libraries to build with"
method_option :verbose, :aliases => "-v", :desc => "show the full build output", :type => :boolean
def build
@@ -35,6 +36,7 @@ def build
output = options[:output] || "/tmp/#{name}.tgz"
prefix = options[:prefix] || "/app/vendor/#{name}"
command = options[:command] || "./configure --prefix #{prefix} && make install"
+ deps = options[:deps] || ""
server = URI.parse(ENV["MAKE_SERVER"] || "http://#{app}.herokuapp.com")
Dir.mktmpdir do |dir|
@@ -47,7 +49,8 @@ def build
"code" => UploadIO.new(input, "application/octet-stream", "input.tgz"),
"command" => command,
"prefix" => prefix,
- "secret" => config[:secret]
+ "secret" => config[:secret],
+ "deps" => deps
puts ">> Building with: #{command}"
response = Net::HTTP.start(server.host, server.port) do |http|
View
34 server/bin/make
@@ -10,7 +10,8 @@ id = ARGV.shift
require "couchrest"
require "fileutils"
require "tmpdir"
-
+require "rest_client"
+
db = CouchRest.database!(ENV["CLOUDANT_URL"] + "/make")
Dir.mktmpdir do |dir|
@@ -19,11 +20,42 @@ Dir.mktmpdir do |dir|
command = doc["command"]
prefix = doc["prefix"]
+ deps = doc["deps"] || ""
File.open("input.tgz", "wb") do |file|
file.print doc.fetch_attachment("input")
end
+ deps.split(",").each_with_index do |dep, index|
+ puts "downloading #{dep}"
+ File.open("dep_#{index}.tgz", "wb") do |file|
+ begin
+ file.print RestClient.get(dep)
+ rescue Exception => ex
+ puts ex.inspect
+ end
+ end
+ end
+
+ FileUtils.mkdir_p "deps"
+
+ Dir.chdir("deps") do
+ deps.split(",").each_with_index do |dep, index|
+ %x{ tar xzf ../dep_#{index}.tgz 2>&1 }
+ end
+ end
+
+ unless deps.empty?
+ ENV["LDFLAGS"] ||= ""
+ ENV["LDFLAGS"] += " -L%s/deps/lib" % dir
+
+ ENV["CFLAGS"] ||= ""
+ ENV["CFLAGS"] += " -I%s/deps/include" % dir
+
+ ENV["CXXFLAGS"] ||= ""
+ ENV["CXXFLAGS"] += " -I%s/deps/include" % dir
+ end
+
FileUtils.mkdir_p "input"
Dir.chdir("input") do
View
16 server/package.json
@@ -1,14 +1,14 @@
{
"name": "heroku-make",
"version": "0.0.1",
- "dependencies": [
- "connect-form",
- "cradle",
- "express",
- "knox",
- "node-uuid",
- "restler"
- ],
+ "dependencies": {
+ "connect-form": "0.2.x",
+ "cradle": "0.6.x",
+ "express": "2.x.x",
+ "knox": "0.0.x",
+ "node-uuid": "1.x.x",
+ "restler": "2.x.x"
+ },
"engines": {
"node": "0.6.x"
}
View
20 server/web.js
@@ -26,7 +26,7 @@ var couchdb_options = couchdb_url.auth ?
{ auth: { username: couchdb_url.auth.split(':')[0], password: couchdb_url.auth.split(':')[1] } } :
{ }
var db = new(cradle.Connection)(couchdb_url.hostname, couchdb_url.port || 5984, couchdb_options).database('make');
-db.create();
+db.create(function(){});
// POST /make starts a build
app.post('/make', function(request, response, next) {
@@ -53,20 +53,20 @@ app.post('/make', function(request, response, next) {
var id = uuid();
var command = fields.command;
var prefix = fields.prefix;
+ var deps = fields.deps;
// create a couchdb documents for this build
log_action(id, 'saving to couchdb');
- db.save(id, { command:command, prefix:prefix }, function(err, doc) {
+ db.save(id, { command:command, prefix:prefix, deps:deps }, function(err, doc) {
if (err) { log_error(id, util.inspect(err)); return next(err); }
// save the input tarball as an attachment
log_action(id, 'saving attachment - [id:' + doc.id + ', rev:' + doc.rev + ']')
- db.saveAttachment(
- doc.id,
- doc.rev,
- 'input',
- 'application/octet-stream',
- fs.createReadStream(files.code.path),
+ fs.createReadStream(files.code.path).pipe(db.saveAttachment(
+ {id: doc.id,
+ rev: doc.rev},
+ {name:'input',
+ 'Content-Type': 'application/octet-stream'},
function(err, data) {
if (err) {
// work around temporary problem with cloudant and document
@@ -99,7 +99,7 @@ app.post('/make', function(request, response, next) {
response.end();
});
}
- );
+ ));
// return the build id as a header
response.header('X-Make-Id', id);
@@ -114,7 +114,7 @@ app.post('/make', function(request, response, next) {
app.get('/output/:id', function(request, response, next) {
// from couchdb
- var stream = db.getAttachment(request.params.id, 'output');
+ var stream = db.getAttachment(request.params.id, 'output', function(){});
stream.on('error', function(err) {
console.log('download error: ' + err);
Something went wrong with that request. Please try again.