Cloud Foundry - the open platform as a service project
Ruby Shell
Switch branches/tags
Nothing to show
Pull request Compare This branch is 19 commits ahead, 204 commits behind cloudfoundry-attic:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
assets @ 6e11f04

Yeti User Manual

What is Yeti

Yeti stands for "Yet Extraordinary Test Infrastructure" and is an rspec-based basic validation test for cloud foundry environments.

Under development

This repository contains tests for vcap.


  1. Ruby 1.9.2
  2. Bundle >= 1.1
  3. admin user/password is needed for parallel run, if you don't have it, run serial like: bundle exec rake full[1]
    It is recommended to manage Ruby runtime via RVM or rbenv

Tested Operating Systems

  1. Mac OS X 64bit, 10.6 and above
  2. Ubuntu 10.04 LTS 64bit

How to run it

  1. git clone git://
  2. cd vcap-yeti
  3. ./update ## this is not required for running administrative test cases
  4. bundle exec rake full
  5. During the first time, Yeti will prompt you for information about your environment:
    • target
    • test user/test password
    • admin user/admin password
      target should be a complete url, e.g. 'http://api.' or 'https://ccng.'. If no http or https is given, yeti will use https as default.
      This information except password is saved to ~/.bvt/config.yml file.
      When run the second time around, Yeti will not prompt for the information again.

Environment variables:

Yeti basic:

||Environment Variables       ||Function            ||Example                                             ||
|VCAP_BVT_TARGET              |target environment   |                                  |
|VCAP_BVT_USER                |test user            |                                       |
|VCAP_BVT_USER_PASSWD         |test user password   |<MY-PASSWORD>                                         |
|VCAP_BVT_ADMIN_USER          |admin user           |                                       |
|VCAP_BVT_ADMIN_USER_PASSWD   |admin user password  |<ADMIN-PASSWD>                                        |

Yeti advance:

||Environment Variables       ||Function            ||Example                                             ||
|VCAP_BVT_SHOW_PENDING        |show pending cases   |true                                                  |
|VCAP_BVT_LONGEVITY           |run testing N times  |100 (0 is endless)                                    |
|VCAP_BVT_CONFIG_FILE         |specify config file  |***/config.yml                                        |
|VCAP_BVT_CI_SINGLE_REPORT    |single reports for ci|true                                                  |

Service/App related:

||Environment Variables       ||Function            ||Example                                             ||
|VCAP_BVT_SERVICE_PG_MAXDBSIZE|service quota(MB)    |128                                                   |
|SERVICE_BROKER_TOKEN         |service broker token |<token>                                               |
|SERVICE_BROKER_URL           |service broker url   |http://...                                            |
|VCAP_BVT_SERVICE_PLAN        |service plan         |P100                                                  |
|VCAP_BVT_REDIS_MANIFEST      |service manifest     |{:vendor=>"redis", :version=>"2.2", :provider=>"core"}|
|VCAP_BVT_DEPLOY_MANIFEST     |deploy manifest      |<path to the file>                                    |

UAA related:

||Environment Variables       ||Function            ||Example                                             ||
|VCAP_BVT_ADMIN_CLIENT        |admin client of uaa  |admin                                                 |
|VCAP_BVT_ADMIN_SECRET        |admin secret of uaa  |adminsecret                                           |

Marketplace gateway related:

||Environment Variables       ||Function            ||Example                                             ||
|MPGW_TOKEN                   |specify mpgw token   |MPGW_TOKEN=testmarketplacetoken                       |
|MPGW_URL                     |specify mpgw url     |MPGW_URL=http://test-mpgw....                         |


  1. In order to support parallel running, and administrative test cases, Yeti will ask administrative account information.
    However, yeti will not abuse administrative privileges, just list users, create users,
    delete users created by the test script.
  2. rake full use parallel by default, you could run in serial by specifying thread number=1: bundle exec rake full[1]
  3. As dev setup has limited resources, we strongly recommend running 1-4 threads against dev_setup.


  1. What does "pending" mean and what is the correct number of pending cases?
    A: Pending means your target environment is missing some prerequisites, usually a service.
    The number of pending cases depends on your target environment and environment variables.

  2. What's the update file?
    A: For assets that need to be compiled such as Java applications, Yeti leverages a common blob store to hold all these precompiled assets. Therefore, users do not need maven to build java-based applications locally, the binaries just need to be sync'd from script automatically checks to see if there are new assets, so Yeti users need to run the script before running Yeti tests.

  3. What is an example?
    A: An Example is an RSpec naming convention for a test case scenario which may include several steps, if any of the steps within an example fail then the test case will be marked as failed.

  4. Where are binary assets stored?
    A: Binary assets are stored in which is a simple Sinatra application with blob service backend hosted on Cloud Foundry. These assets are then synchronized via the script into the .assets-binaries directory of vcap-yeti.

  5. How do I submit binary assets?
    A: Currently binaries are generated manually based on source code updates to vcap-assets. In the near future, source code updates will trigger a job to compile sources and update

  6. Where is the log file stored?
    A: There are runtime log and junit-format report.

    • Runtime log is stored in ~/.bvt/bvt.[target].log
    • Junit-format report is under [yeti_home]/reports. The junitResult.xml is the summary.
  7. What services should my environment have? Dev instance:

    • services: mongodb, mysql, postgresql, rabbitmq, redis, vblob


    • services: mongodb, mysql, postgresql, rabbitmq, redis

    (updated on Dec 17th, 2012)

  8. Runtime errors
    Sometimes runtime errors happen during Yeti execution,

    • 504 Gateway Error: Application fail to be started in 30 seconds
    • 502 Internal Error: Application fail to connect to service instance, including provision/un-provision bind/unbind.
    • 404 Not Found: Route fail to redirect request to specific application URL
  9. Run specific case
    User can run specific case via passing spec file with specify line number of an example or group as parameter For example: bundle exec rspec ./spec/simple/rails_console/ruby18_rails3_spec.rb:95

  10. Build java-based assets
    Please refer to docs/

Rake Tasks:

  • admin
    run admin test cases
  • tests
    run core tests in parallel, e.g. rake test[5] (default to 10, max = 16)
  • full
    run full tests in parallel, e.g. rake full[5] (default to 10, max = 16)
  • all
    run all tests in parallel, including cases in spec/ and services_spec/ e.g. rake all[5] (default to 10, max = 16)
  • random
    run all bvts randomly, e.g. rake random[1023] to re-run seed 1023
  • java
    run java tests (spring, java_web) in parallel
    e.g. rake java[5] (default to 10, max = 16)
  • jvm
    run jvm tests (spring, java_web, grails, lift) in parallel
    e.g. rake jvm[5] (default to 10, max = 16)
  • ruby
    run ruby tests (rails3, sinatra, rack) in parallel
    e.g. rake ruby[5] (default to 10, max = 16)
  • services
    run all service tests (mongodb/redis/mysql/postgres/rabbitmq/neo4j/vblob) in parallel
    e.g. rake services[5] (default to 10, max = 16)
  • core
    run core tests for verifying that an installation meets minimal Cloud Foundry compatibility requirements
    e.g. rake core[5] (default to 10, max = 16)
  • rerun
    rerun failed cases of the last run
    e.g. rake rerun[5] (default to 10, max = 16)
  • clean
    clean up test environment(only run this task after interruption).
    1, Remove all apps and services under test user
    2, Remove all test users created in admin_user_spec.rb
  • help
    list help commands