This repository was archived by the owner on Dec 11, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 4
Structure
Dieter Adriaenssens edited this page Nov 1, 2015
·
53 revisions
-
init.sh: sets required variables (logfile name, root-dir, ...) -
timestamp.sh: logs a timestamp (to a file) (combine efforts with https://github.com/travis-ci/travis-build/pull/199 ?) -
analyse.py: calculates time spent between timestamps (lxml -> http://lxml.de/tutorial.html) -
generate_trend.py: create a trend from analysis results (current and previous) -> matplotlib : example, api doc
Each script (except init.sh) checks if init.sh was run.
-
service.py: powers Buildtime Trend as a Service on Heroku, see README.md for more information. -
utils/start_worker.py: starts the Celery worker executing jobs on the task queue, see README.md for more information.
filename : timestamps.csv
"stage1",timestamp1
"stage2",timestamp2
"end",timestamp3
Output of analysis.py, filename : trends/buildtimes.xml :
<builds>
<build
repo="ruleant/buildtime-trend" # repo name (since version 0.1)
build="1" # build id (since version 0.1)
job="1.1" # job id (since version 0.1)
result="passed" # build result : passed, failed (since version 0.1)
branch="master" # repo branch (since version 0.1)
started_at="2014-07-25T14:03:51Z" # timestamp when build started (since version 0.1)
ci_platform="travis"> # Continuous integration platform :
# travis, local, ... (since version 0.1)
<stages>
<stage
name="stage1" # stage name (since version 0.1)
duration="value2_1" /> # stage duration [sec] (since version 0.1)
...
</stages>
</build>
...
<builds>
Sent to Keen.io by analysis.py (python-client) or by Buildtime Trend as a Service.
Structure of each eventtype :
-
buildtime_trend
- project_name : project name is the unique identifier for this project, usually the repo name (since version 0.1 : keen)
-
version : Buildtime trend version generating the data(since version 0.1 : keen, renamed tolib_versionin version 0.2) -
lib_version : Buildtime trend library version generating the data (since version 0.2 : keen, renamed from
version) - client_version : Buildtime trend client version generating the data (since version 0.2 : keen)
- client : Name of Buildtime Trend client generating the data (since version 0.2 : keen)
- schema_version : Version of the schema structure (since version 0.1 : keen)
-
keen (added by Keen.io API)
-
timestamp : timestamp, equals to
created_atif not specified - created_at : when the event was created
- id : unique ID
-
timestamp : timestamp, equals to
In the build_jobs event collection, these are added :
-
job : see Build/job properties (since version 0.1 : keen, renamed in version 0.2 (previously
build))-
stages : list of stages (since version 0.1 : keen)
- stage : see Stage properties (since version 0.1 : keen)
-
stages : list of stages (since version 0.1 : keen)
In the build_stages event collection, these are added :
-
job : see Build/job properties (since version 0.1 : keen, renamed in version 0.2 (previously
build)) - stage : see Stage properties (since version 0.1 : keen)
In the build_substages event collection, these are added :
-
job : see Build/job properties (since version 0.1 : keen, renamed in version 0.2 (previously
build)) - stage : see Stage properties (since version 0.1 : keen)
- repo : repo name (since version 0.1 : native, keen)
- branch : repo branch (since version 0.1 : native, keen)
- build : build id (since version 0.1 : native, keen)
- job : job id (since version 0.1 : native, keen)
- result : build result : passed, failed (since version 0.1 : native, keen)
-
worker : CI worker information (since version 0.2 : keen) :
- hostname: worker hostname (since version 0.2 : keen)
- os: worker OS (since version 0.2 : keen)
-
started_at : timestamp when build started, see
Timestamp propertiesfor structure (since version 0.1, updated in version 0.2 : native, keen) -
finished_at : timestamp when build finished, see
Timestamp propertiesfor structure (since version 0.1, updated in version 0.2 : keen) - duration : build duration [sec] (since version 0.1 : keen)
- ci_platform : Continuous integration platform : travis, local, ... (since version 0.1 : native, keen)
- build_trigger : what triggered the build : push, pull_request (since version 0.3 : keen)
-
build_matrix : build matrix parameters (since version 0.3 : keen) :
- language : programming language, see Travis CI documentation for full list (since version 0.3 : keen)
- language_version : programming language version (since version 0.3 : keen)
- language_components : language components, fe.: android_20 (since version 0.3 : keen)
- jdk : Java JDK version (Java, Android, Groovy, Ruby, Scala), fe. : openjdk7 (since version 0.3 : keen)
- ghc : Haskell compiler (since version 0.3 : keen)
- lein : Lein version (Clojure) (since version 0.3 : keen)
- mono : Mono version (C#, F#, Visual Basic) (since version 0.3 : keen)
- node_js : Node JS version (Javascript) (since version 0.3 : keen)
- otp_release : OTP release version (Erlang) (since version 0.3 : keen)
- rvm: RVM version (Ruby, Objective-C) (since version 0.3 : keen)
- gemfile: Gemfile (Ruby, Objective-C) (since version 0.3 : keen)
- os : Operating system, fe. : linux, OS X (since version 0.3 : keen)
- compiler : Compiler, fe. : clang, gcc (since version 0.3 : keen)
- parameters : extra build matrix environment parameters (since version 0.3 : keen)
- xcode_sdk : Objective-C SDK (since version 0.3 : keen)
- xcode_scheme : Objective-C Scheme (since version 0.3 : keen)
- xcode_project : Objective-C project (since version 0.3 : keen)
- xcode_workspace : Objective-C workspace (since version 0.3 : keen)
- summary : Concatenation of all build matrix parameters into one string (since version 0.3 : keen)
-
pull_request : pull request information (since version 0.3 : keen)
- is_pull_request : true, false (since version 0.3 : keen)
- title : Pull request title (since version 0.3 : keen)
- number : Pull request number (since version 0.3 : keen)
- language : java, C, python, php, ... (not implemented yet)
- language version : openJDK 7, Python 2.7, PHP 5.5, ... (not implemented yet)
- framework : Spring, Android, Django, ... (not implemented yet)
- buildtool : make, ant, maven, gradle, ... (not implemented yet)
- commit (not implemented yet)
- name : stage name (since version 0.1 : native, keen)
-
started_at : timestamp when build started, see
Timestamp propertiesfor structure (since version 0.1, updated in version 0.2 : keen) -
finished_at : timestamp when build finished, see
Timestamp propertiesfor structure (since version 0.1, updated in version 0.2 : keen) - duration : build duration [sec] (since version 0.1 : keen)
- timestamp : iso formatted timestamp (since version 0.2 : keen)
- year : Year as 4 digit number (since version 0.2 : keen)
- month : Month as 2 digit number, with leading zero (01, 02, ..., 11, 12) (since version 0.2 : keen)
- month_full_en : Full name of the month in en_US (Januari, Februari, ...) (since version 0.2 : keen)
- month_short_en : Abbreviated name of the month in en_US (Jan, Feb, ...) (since version 0.2 : keen)
- day_of_month : Day of the month as 2 digit number, with leading zero (01, 02, ..., 30, 31) (since version 0.2 : keen)
- day_of_week : Day of week as a number (0 = Sunday, 1 = Monday, ..., 6 = Saturday) (since version 0.2 : keen)
- day_of_week_short_en : Abbreviated name of day of the week in en_US (Mon, Tue, ...) (since version 0.2 : keen)
- day_of_week_full_en : Full name of day of the week in en_US (Sunday, Monday, ...) (since version 0.2 : keen)
- hour_24 : Hour in 24-hour format as 2 digit number, with leading zero (00, 01, ..., 22, 23) (since version 0.2 : keen)
- hour_12 : Hour in 12-hour format as 2 digit number, with leading zero (01, 02, ..., 11, 12) (since version 0.2 : keen)
- hour_ampm : AM/PM (since version 0.2 : keen)
- minute : Minute as 2 digit number, with leading zero (00, 01, 02, ..., 58, 59) (since version 0.2 : keen)
- second : Second as 2 digit number, with leading zero (00, 01, 02, ..., 58, 59) (since version 0.2 : keen)
{
"buildtime_trend": {
"project_name": "ruleant/getback_gps",
"client": "buildtimetrend/service",
"client_version": "0.2-dev",
"lib_version": "0.2-dev",
"schema_version": "2"
},
"keen": {
"timestamp": "2014-08-22T15:01:49.782Z",
"created_at": "2014-08-22T15:01:49.782Z",
"id": "53f75b5d7d8cb9684aa6a904"
},
"job": {
"result": "passed",
"finished_at": {
"timestamp": "2014-08-22T15:01:46",
"day_of_week_short_en": "Fri",
"day_of_month": "22",
"day_of_week": "5",
"month": "08",
"month_full_en": "August",
"second": "46",
"month_short_en": "Aug",
"year": "2014",
"hour_24": "15",
"hour_12": "03",
"day_of_week_full_en": "Friday",
"minute": "01",
"hour_ampm": "PM"
},
"worker": {
"hostname": "worker-linux-docker-1bb6ca34.prod.travis-ci.org",
"os": "travis-linux-5"
},
"repo": "ruleant/getback_gps",
"job": "438.2",
"build": "438",
"branch": "test_travis_cache",
"duration": 522,
"ci_platform": "travis",
"started_at": {
"timestamp": "2014-08-22T14:51:15+00:00",
"day_of_week_short_en": "Fri",
"day_of_month": "22",
"day_of_week": "5",
"month": "08",
"month_full_en": "August",
"second": "15",
"month_short_en": "Aug",
"year": "2014",
"hour_24": "14",
"hour_12": "02",
"day_of_week_full_en": "Friday",
"minute": "51",
"hour_ampm": "PM"
}
},
"stage": {
"name": "update_javadoc",
"duration": 31,
"finished_at": {
"timestamp": "2014-08-22T15:01:46",
"day_of_week_short_en": "Fri",
"day_of_month": "22",
"day_of_week": "5",
"month": "08",
"month_full_en": "August",
"second": "46",
"month_short_en": "Aug",
"year": "2014",
"hour_24": "15",
"hour_12": "03",
"day_of_week_full_en": "Friday",
"minute": "01",
"hour_ampm": "PM"
},
"started_at": {
"timestamp": "2014-08-22T15:01:15",
"day_of_week_short_en": "Fri",
"day_of_month": "22",
"day_of_week": "5",
"month": "08",
"month_full_en": "August",
"second": "15",
"month_short_en": "Aug",
"year": "2014",
"hour_24": "15",
"hour_12": "03",
"day_of_week_full_en": "Friday",
"minute": "01",
"hour_ampm": "PM"
},
}
}
{
"buildtime_trend": {
"project_name": "ruleant/buildtime-trend",
"client": "buildtimetrend/service",
"client_version": "0.2-dev",
"lib_version": "0.2-dev",
"schema_version": "2"
},
"keen": {
"timestamp": "2014-08-22T10:20:15.295Z",
"created_at": "2014-08-22T10:20:15.295Z",
"id": "53f7195fc9e16340d0eed747"
},
"job": {
"result": "passed",
"started_at": {
"timestamp": "2014-08-22T10:17:44+00:00",
"day_of_week_short_en": "Fri",
"day_of_month": "22",
"day_of_week": "5",
"month": "08",
"month_full_en": "August",
"second": "44",
"month_short_en": "Aug",
"year": "2014",
"hour_24": "10",
"hour_12": "10",
"day_of_week_full_en": "Friday",
"minute": "17",
"hour_ampm": "AM"
},
"finished_at": {
"timestamp": "2014-08-22T10:20:12",
"day_of_week_short_en": "Fri",
"day_of_month": "22",
"day_of_week": "5",
"month": "08",
"month_full_en": "August",
"second": "12",
"month_short_en": "Aug",
"year": "2014",
"hour_24": "10",
"hour_12": "10",
"day_of_week_full_en": "Friday",
"minute": "20",
"hour_ampm": "AM"
},
"worker": {
"hostname": "worker-linux-docker-1bb6ca34.prod.travis-ci.org",
"os": "travis-linux-5"
},
"repo": "ruleant/buildtime-trend",
"job": "252.1",
"build": "252",
"branch": "master",
"duration": 145,
"ci_platform": "travis"
"stages": [
{
"duration": 136,
"finished_at": {
"timestamp": "2014-08-22T10:20:03",
"day_of_week_short_en": "Fri",
"day_of_month": "22",
"day_of_week": "5",
"month": "08",
"month_full_en": "August",
"second": "03",
"month_short_en": "Aug",
"year": "2014",
"hour_24": "10",
"hour_12": "10",
"day_of_week_full_en": "Friday",
"minute": "20",
"hour_ampm": "AM"
},
"started_at": {
"timestamp": "2014-08-22T10:17:47",
"day_of_week_short_en": "Fri",
"day_of_month": "22",
"day_of_week": "5",
"month": "08",
"month_full_en": "August",
"second": "47",
"month_short_en": "Aug",
"year": "2014",
"hour_24": "10",
"hour_12": "10",
"day_of_week_full_en": "Friday",
"minute": "17",
"hour_ampm": "AM"
},
"name": "install"
},
{
"duration": 5,
"finished_at": {
"timestamp": "2014-08-22T10:20:08",
"day_of_week_short_en": "Fri",
"day_of_month": "22",
"day_of_week": "5",
"month": "08",
"month_full_en": "August",
"second": "08",
"month_short_en": "Aug",
"year": "2014",
"hour_24": "10",
"hour_12": "10",
"day_of_week_full_en": "Friday",
"minute": "20",
"hour_ampm": "AM"
},
"started_at": {
"timestamp": "2014-08-22T10:20:03",
"day_of_week_short_en": "Fri",
"day_of_month": "22",
"day_of_week": "5",
"month": "08",
"month_full_en": "August",
"second": "03",
"month_short_en": "Aug",
"year": "2014",
"hour_24": "10",
"hour_12": "10",
"day_of_week_full_en": "Friday",
"minute": "20",
"hour_ampm": "AM"
},
"name": "tests"
}
],
}
}