Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Artifacts #164

Closed
wants to merge 14 commits into from

3 participants

@richmeyers

This replaces #154.

Major changes:

  • Artifacts column length increased to 1000.
  • Artifacts are separated with semicolons, they have special meaning in the shell and are less likely to be used than commas both in unix and windows.
  • Expanded help text for artifacts.
  • More correctly escape and unescape artifact paths in urls.
  • Only serve artifacts that match configured artifact specifications.
@richmeyers richmeyers referenced this pull request
Closed

Artifacts #154

Shaun and others added some commits
Shaun implemented rough artifact support. d466f16
Shaun removed debugging lines, cleaned to request a pull into integrity/int…
…egrity

fixed buried artifacts breaking the display of artifacts and reverted lib/integrity/payload_builder.rb to pre-me derping with it state.
fa1f2b0
Shaun actually commit the modified views this time. 6d35384
Shaun moved the artifact logic into the rendering helper as #build_artifact…
…s, using %2F as the directory delimiter instead of ~, other small various things.
bfb7a44
Shaun fixed issue where projects with ( ) in the name were not having their…
… literal project path used and artifacts were 404'ing
1c3b6a4
@richmeyers richmeyers Use build.project to reference the project in build info partial ee46c21
@richmeyers richmeyers No need to pass project to build_artifacts 5f8bb00
@richmeyers richmeyers Set artifacts field length to 1000 chars.
It is very difficult to increase the length later,
set it to 1000 initially to be safe.
5ba2d47
@richmeyers richmeyers Artifacts field is nil for pre-artifacts builds bafa26e
@richmeyers richmeyers Reimplement artifact handling under builds.
Perform proper escaping of path components.

Sinatra treats %2F in urls as slashes and uses them to separate
path components. Escape artifact paths an extra time to work around this.

When retrieving artifacts, check that requested files are
in fact matched by one of the artifacts before serving them.
552a74b
@richmeyers richmeyers Artifacts are now always requested under a build c7e6cb6
@richmeyers richmeyers Add more documentation for artifacts field 6f9a575
@richmeyers richmeyers Separate artifacts with semicolons, this should cause fewer problems 398c2cf
@richmeyers richmeyers Update CHANGES 6193b33
@richmeyers

I have had this code running for a while now in our CI server, but we haven't actually used it.

My use case is nearly opposite from that of original submitter: when tests fail, I want to be able to access failure (exception) reports via integrity.

This means:

  1. Artifacts should be exposed regardless of build status, if they exist. Currently when a build fails artifacts are not exposed.
  2. Artifact wildcards are useful.
  3. If artifacts are specified via wildcards and none exist, no UI for them should be displayed. Currently there is an empty box for artifacts in my use case.
@grahamc
Owner

@richmeyers I agree on all three points. I'm also interested in exposing an entire directory of artifacts.

@p
Collaborator
p commented
@p p closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 10, 2012
  1. @richmeyers

    implemented rough artifact support.

    Shaun authored richmeyers committed
  2. @richmeyers

    removed debugging lines, cleaned to request a pull into integrity/int…

    Shaun authored richmeyers committed
    …egrity
    
    fixed buried artifacts breaking the display of artifacts and reverted lib/integrity/payload_builder.rb to pre-me derping with it state.
  3. @richmeyers

    actually commit the modified views this time.

    Shaun authored richmeyers committed
  4. @richmeyers

    moved the artifact logic into the rendering helper as #build_artifact…

    Shaun authored richmeyers committed
    …s, using %2F as the directory delimiter instead of ~, other small various things.
  5. @richmeyers

    fixed issue where projects with ( ) in the name were not having their…

    Shaun authored richmeyers committed
    … literal project path used and artifacts were 404'ing
  6. @richmeyers
  7. @richmeyers
  8. @richmeyers

    Set artifacts field length to 1000 chars.

    richmeyers authored
    It is very difficult to increase the length later,
    set it to 1000 initially to be safe.
  9. @richmeyers
  10. @richmeyers

    Reimplement artifact handling under builds.

    richmeyers authored
    Perform proper escaping of path components.
    
    Sinatra treats %2F in urls as slashes and uses them to separate
    path components. Escape artifact paths an extra time to work around this.
    
    When retrieving artifacts, check that requested files are
    in fact matched by one of the artifacts before serving them.
  11. @richmeyers
  12. @richmeyers
  13. @richmeyers
  14. @richmeyers

    Update CHANGES

    richmeyers authored
This page is out of date. Refresh to see the latest.
View
4 CHANGES
@@ -1,5 +1,9 @@
= master
+ * Integrity now can expose artifacts created by the builds.
+ Specify what files should be exposed in project settings.
+ Shell wildcards are allowed if your file names change from build to build.
+
* Integrity now has favicons for the home page, project pages and build pages
reflecting the respective build statuses.
View
24 lib/integrity/app.rb
@@ -1,3 +1,5 @@
+require 'cgi'
+
module Integrity
class App < Sinatra::Base
set :root, File.expand_path("../../..", __FILE__)
@@ -182,6 +184,28 @@ class App < Sinatra::Base
redirect build_url(@build).to_s
end
+ get "/:project/builds/:build/artifacts/:artifact" do |project, build, artifact|
+ login_required unless current_project.public?
+
+ artifact = CGI.unescape(artifact)
+
+ artifact_files = current_build.artifact_files
+ file = artifact_files.detect do |file|
+ file[:relative_path] == artifact
+ end
+
+ if file.nil?
+ halt 404
+ end
+
+ fs_path = current_build.build_directory.join(file[:relative_path])
+ unless File.exist?(fs_path)
+ halt 404
+ end
+
+ send_file fs_path, :filename => file[:name]
+ end
+
get "/:project/builds/:build" do
login_required unless current_project.public?
View
67 lib/integrity/build.rb
@@ -8,6 +8,8 @@ class Build
:pending => "%s hasn't been built yet",
:building => "%s is building"
}
+
+ GLOB_CHARS = '*?[]'
property :id, Serial
property :project_id, Integer # TODO :nullable => false
@@ -138,5 +140,70 @@ def status
def human_status
HUMAN_STATUS[status] % sha1_short
end
+
+ def build_directory
+ Pathname.new(Integrity.config.directory).join(self.id.to_s)
+ end
+
+ def escape_glob(path)
+ escaped = path
+ GLOB_CHARS.each do |char|
+ escaped = escaped.sub(char, "\\" + char)
+ end
+ escaped
+ end
+ private :escape_glob
+
+ def artifact_files
+ build_dir = build_directory
+=begin
+ # for file in build dir check below
+ build_dir = File.expand_path(build_dir)
+=end
+ escaped_build_dir = nil
+ all_files = []
+ project.get_artifacts.each do |artifact|
+ if GLOB_CHARS.split('').any? { |char| artifact.include?(char) }
+ if escaped_build_dir.nil?
+ escaped_build_dir = escape_glob(build_dir.to_s)
+ end
+
+ pattern = File.join(escaped_build_dir, artifact)
+ all_files += Dir[pattern]
+ else
+ file = build_dir.join(artifact).to_s
+ if File.exist?(file)
+ all_files << file
+ end
+ end
+ end
+
+ build_dir_with_slash = build_dir.to_s + '/'
+
+=begin
+ # check that all files are under the build dir
+ all_files.map! do |file|
+ File.expand_path(file)
+ end
+ all_files.delete_if do |file|
+ file[0, build_dir_with_slash.length] != build_dir_with_slash
+ end
+=end
+
+ all_files.map! do |file|
+ if file[0, build_dir_with_slash.length] == build_dir_with_slash
+ relative_path = file[build_dir_with_slash.length..-1]
+ else
+ relative_path = file
+ end
+ {
+ :name => File.basename(file),
+ :relative_path => relative_path,
+ }
+ end
+ all_files.sort do |a, b|
+ a[:name] <=> b[:name]
+ end
+ end
end
end
View
9 lib/integrity/helpers/resources.rb
@@ -16,6 +16,15 @@ def update_notifiers_of(project)
project.update_notifiers(params["enabled_notifiers"] || [], params["notifiers"])
end
end
+
+ def artifact_files(build)
+ files = build.artifact_files.dup
+ files.each do |file|
+ url = artifact_path(build, file[:relative_path])
+ file[:url] = url
+ end
+ files
+ end
end
end
end
View
6 lib/integrity/helpers/urls.rb
@@ -1,3 +1,5 @@
+require 'cgi'
+
module Integrity
module Helpers
module Urls
@@ -60,6 +62,10 @@ def build_url(build, *path)
def build_path(build, *path)
project_path(build.project, "builds", build.id, *path)
end
+
+ def artifact_path(build, artifact, *path)
+ build_path(build, 'artifacts', CGI.escape(CGI.escape(artifact)))
+ end
# Copyright (MIT) Eric Kidd -- http://github.com/emk/sinatra-url-for
#
View
9 lib/integrity/project.rb
@@ -11,6 +11,7 @@ class Project
property :uri, URI, :required => true, :length => 255
property :branch, String, :required => true, :length => 255, :default => "master"
property :command, String, :required => true, :length => 255, :default => "rake"
+ property :artifacts, String, :required => false, :length => 1000
property :public, Boolean, :default => true
property :last_build_id, Integer, :required => false
@@ -28,6 +29,14 @@ class Project
builds.destroy!
end
+ def get_artifacts
+ artifacts.split(";")
+ end
+
+ def artifacts_empty?
+ artifacts.nil? || artifacts.empty?
+ end
+
def repo
@repo ||= Repository.new(uri, branch)
end
View
2  public/integrity.css
@@ -79,6 +79,8 @@ a {
position: relative; }
#content form p.checkbox label {
margin-top: 0 !important; }
+ #content form p.explanation {
+ margin-left: 8.75em; }
#content form input.text, #content form textarea, #content form select {
width: 30em;
padding: 0.2em 0.4em;
View
12 views/_build_info.haml
@@ -20,10 +20,20 @@
%a{ :href => github_commit_url(build) } view on GitHub
- if build.completed?
+ - if build.successful? && ! build.project.artifacts_empty?
+ %h2
+ Artifacts
+ %blockquote
+ %ul#artifacts
+ - artifact_files(build).each do |file|
+ %a{ :title => file[:relative_path],
+ :href => file[:url] } #{file[:name]}
+ %br/
+
%h2
Build Output
%a{:href => build_path(build, :raw)} (raw)
-
+
%pre.output
:preserve
#{bash_color_codes h(build.output)}
View
2  views/integrity.sass
@@ -106,6 +106,8 @@ a
position: relative
&.checkbox label
margin-top: 0 !important
+ p.explanation
+ margin-left: 8.75em
input.text, textarea, select
width: 30em
padding: .2em .4em
View
6 views/new.haml
@@ -23,6 +23,12 @@
&== Build script #{errors_on(@project, :command)}
%textarea#project_build_script{ :name => "project_data[command]", :cols => 40, :rows => 2 }
&== #{@project.command.to_s}
+
+ %p.normal{ :class => error_class(@project, :artifacts) }
+ %label{ :for => "project_artifacts" }<
+ &== Artifacts to provide
+ %input.text#project_artifacts{ :name => "project_data[artifacts]", :type => "text", :value => h(@project.artifacts) }
+ %p.explanation Relative paths of files to expose per build. Separate multiple paths with semicolons. Wildcards are allowed (?, *, []) and must be escaped with a backslash if part of a filename.
%p.normal.checkbox
%label{ :for => "project_public" } Public project
Something went wrong with that request. Please try again.