Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 181 lines (154 sloc) 8.556 kb
61a47e8 several bug fixes
michael zhang authored
1 Yeti User Manual
2 ================
ed20723 @pxie init commit about README.md file
pxie authored
3
61a47e8 several bug fixes
michael zhang authored
4 What is Yeti
5 ------------
8857163 update README and Rakefile on behalf of steven
michael zhang authored
6
7 Yeti stands for "Yet Extraordinary Test Infrastructure" and is an rspec-based
8 basic validation test for cloud foundry environments.
9
61a47e8 several bug fixes
michael zhang authored
10 <br>*Under development*
ed20723 @pxie init commit about README.md file
pxie authored
11
00bcdee @pxie Add readme for vcap-yeti repo
pxie authored
12 This repository contains tests for [vcap](https://github.com/cloudfoundry/vcap).
13
61a47e8 several bug fixes
michael zhang authored
14 ## Dependencies
eee297e @pxie update readme
pxie authored
15 1. Ruby 1.9.2
16 2. Bundle >= 1.1
f79bcf0 update README and Rakefile
michael zhang authored
17 3. admin user/password is needed for parallel run, if you don't have it, run serial like:
eb424ec @pxie Add README
pxie authored
18 ```bundle exec rake full[1]```
f79bcf0 update README and Rakefile
michael zhang authored
19 <br>It is recommended to manage Ruby runtime via RVM or rbenv
00bcdee @pxie Add readme for vcap-yeti repo
pxie authored
20
8857163 update README and Rakefile on behalf of steven
michael zhang authored
21 ## _Tested Operating Systems_
00bcdee @pxie Add readme for vcap-yeti repo
pxie authored
22 1. Mac OS X 64bit, 10.6 and above
23 2. Ubuntu 10.04 LTS 64bit
24
61a47e8 several bug fixes
michael zhang authored
25 How to run it
26 -------------
eb424ec @pxie Add README
pxie authored
27 1. ```git clone git://github.com/cloudfoundry/vcap-yeti.git```
61a47e8 several bug fixes
michael zhang authored
28 2. ```cd vcap-yeti```
8857163 update README and Rakefile on behalf of steven
michael zhang authored
29 3. ```./update.sh ## this is not required for running administrative test cases```
f79bcf0 update README and Rakefile
michael zhang authored
30 4. ```bundle exec rake full```
8857163 update README and Rakefile on behalf of steven
michael zhang authored
31 5. During the first time, Yeti will prompt you for information about your environment:
00bcdee @pxie Add readme for vcap-yeti repo
pxie authored
32 - target
8857163 update README and Rakefile on behalf of steven
michael zhang authored
33 - test user/test password
34 - admin user/admin password
35 <br>This information is saved to ~/.bvt/config.yml file.
36 <br>When run the second time around, Yeti will not prompt for the information again.
00bcdee @pxie Add readme for vcap-yeti repo
pxie authored
37
8857163 update README and Rakefile on behalf of steven
michael zhang authored
38 Notes:
61a47e8 several bug fixes
michael zhang authored
39 -----
44d5568 add env variable support for showing pending cases
michael zhang authored
40 1. To be compatible with BVT, some environment variables are preserved in Yeti:
61a47e8 several bug fixes
michael zhang authored
41 ```
49422e7 @jgong enable java7 cases
jgong authored
42 ||Environment Variables ||Function ||Example ||
43 |VCAP_BVT_TARGET |target environment |VCAP_BVT_TARGET=cloudfoundry.com |
44 |VCAP_BVT_USER |test user |VCAP_BVT_USER=pxie@vmware.com |
45 |VCAP_BVT_USER_PASSWD |test user password |VCAP_BVT_USER_PASSWD=<MY-PASSWORD> |
46 |VCAP_BVT_ADMIN_USER |admin user |VCAP_BVT_ADMIN_USER=admin@admin.com |
47 |VCAP_BVT_ADMIN_USER_PASSWD |admin user password |VCAP_BVT_ADMIN_USER_PASSWD=<ADMIN-PASSWD>|
48 |VCAP_BVT_SHOW_PENDING |show pending cases |VCAP_BVT_SHOW_PENDING=true |
49 |VCAP_BVT_SERVICE_PG_MAXDBSIZE|service quota(MB) |VCAP_BVT_SERVICE_PG_MAXDBSIZE=128 |
50 |VCAP_BVT_ADMIN_CLIENT |admin client of uaa |VCAP_BVT_ADMIN_CLIENT=admin |
51 |VCAP_BVT_ADMIN_SECRET |admin secret of uaa |VCAP_BVT_ADMIN_SECRET=adminsecret |
52 |ACM_URL |acm base url |ACM_URL=<URL> |
53 |ACM_USER |acm user |ACM_USER=<user> |
54 |ACM_PASSWORD |acm user password |ACM_PASSWORD=<***> |
55 |SERVICE_BROKER_TOKEN |service broker token |SERVICE_BROKER_TOKEN=<token> |
56 |SERVICE_BROKER_URL |service broker url |SERVICE_BROKER_URL=http://... |
eb424ec @pxie Add README
pxie authored
57 |VCAP_BVT_LONGEVITY |run testing N times |VCAP_BVT_LONGEVITY=100 |
ab189cb support config file parameter
michael zhang authored
58 |VCAP_BVT_CONFIG_FILE |specify config file |VCAP_BVT_CONFIG_FILE=***/config.yml |
61a47e8 several bug fixes
michael zhang authored
59 ```
60
c4af778 BVT for marketplace gateway
Harshawardhan Gadgil authored
61 To test marketplace gateway, specify the following environment variables:
62 ```
63 ||Environment Variables ||Function ||Example ||
64 |MPGW_TOKEN |specify mpgw token |MPGW_TOKEN=testmarketplacetoken |
65 |MPGW_URL |specify mpgw url |MPGW_URL=http://test-mpgw.... |
66
67 ```
68
eee297e @pxie update readme
pxie authored
69 2. In order to support parallel running, and administrative test cases, Yeti will ask administrative
70 account information.
8857163 update README and Rakefile on behalf of steven
michael zhang authored
71 <br>However, yeti will not abuse administrative privileges, just list users, create users,
72 <br>delete users created by the test script.
f79bcf0 update README and Rakefile
michael zhang authored
73 3. rake tests/full use parallel by default, you could run in serial by specifying thread number=1:
eb424ec @pxie Add README
pxie authored
74 ```bundle exec rake full[1]```
f79bcf0 update README and Rakefile
michael zhang authored
75 4. As dev setup has limited resources, we strongly recommend running 1-4 threads against dev_setup.
00bcdee @pxie Add readme for vcap-yeti repo
pxie authored
76
02ccb5a add the first faq to readme
michael zhang authored
77 FAQ:
61a47e8 several bug fixes
michael zhang authored
78 ----
8857163 update README and Rakefile on behalf of steven
michael zhang authored
79 1. What does "pending" mean and what is the correct number of pending cases?
80 <br>A: *Pending* means your target environment is missing some prerequisites, usually a service,
81 framework or runtime.
82 <br>The number of pending cases depends on your target environment and environment variables.
83 - For example, php/python is not available in the production environment so all php/python
84 related test cases should be pending.
85
86 2. What's update.sh?
87 <br>A: For assets that need to be compiled such as Java applications, Yeti leverages a common
88 blob store to hold all these precompiled assets. Therefore, users do not need maven to build
89 java-based applications locally, the binaries just need to be sync'd from blobs.cloudfoundry.com.
90 update.sh script automatically checks to see if there are new assets, so Yeti users need to
91 run the update.sh script before running Yeti tests.
92
93 3. What is an _example_?
94 <br>A: An Example is an RSpec naming convention for a test case scenario which may include several
95 steps, if any of the steps within an example fail then the test case will be marked as failed.
ed20723 @pxie init commit about README.md file
pxie authored
96
27326e2 @pxie Add FAQ
pxie authored
97 4. Where are binary assets stored?
eb424ec @pxie Add README
pxie authored
98 <br>A: Binary assets are stored in http://blobs.cloudfoundry.com which is a simple Sinatra application
99 with blob service backend hosted on Cloud Foundry. These assets are then synchronized via the
100 update.sh script into the .assets-binaries directory of vcap-yeti.
ed20723 @pxie init commit about README.md file
pxie authored
101
27326e2 @pxie Add FAQ
pxie authored
102 5. How do I submit binary assets?
eb424ec @pxie Add README
pxie authored
103 <br>A: Currently binaries are generated manually based on source code updates to vcap-assets. In
8857163 update README and Rakefile on behalf of steven
michael zhang authored
104 the near future, source code updates will trigger a job to compile sources and update
105 blobs.cloudfoundry.com.
27326e2 @pxie Add FAQ
pxie authored
106
107 6. Where is the log file stored?
8857163 update README and Rakefile on behalf of steven
michael zhang authored
108 <br>A: There two log files, runtime and error logs
109 - Runtime log is stored in ~/.bvt/bvt.log
f79bcf0 update README and Rakefile
michael zhang authored
110
8cbea6f update faq about environment
michael zhang authored
111 7. What runtimes/frameworks/services should my environment have?
112 <br>Dev setup:
eb424ec @pxie Add README
pxie authored
113 - runtimes: java, ruby18, ruby19, node, node06, node08, php, python2, erlangR14B01
8cbea6f update faq about environment
michael zhang authored
114 - frameworks: java_web, sinatra, grails, rack, play, lift, spring, rails3, node, standalone, php,
115 django, wsgi, otp_rebar
116 - services: mongodb, mysql, postgresql, rabbitmq, redis, vblob, filesystem
117 <br>(env variable CLOUD_FOUNDRY_EXCLUDED_COMPONENT can disable components, for details, please
eb424ec @pxie Add README
pxie authored
118 check README under vcap/dev_setup.)
8cbea6f update faq about environment
michael zhang authored
119
120 Dev instance:
eb424ec @pxie Add README
pxie authored
121 - runtimes: java, java7, ruby18, ruby19, node, node06, node08
8cbea6f update faq about environment
michael zhang authored
122 - frameworks: java_web, sinatra, grails, rack, play, lift, spring, rails3, node, standalone
123 - services: mongodb, mysql, postgresql, rabbitmq, redis, vblob
124
125 Production:
eb424ec @pxie Add README
pxie authored
126 - runtimes: java, java7, ruby18, ruby19, node, node06, node08
8cbea6f update faq about environment
michael zhang authored
127 - frameworks: java_web, sinatra, grails, rack, play, lift, spring, rails3, node, standalone
128 - services: mongodb, mysql, postgresql, rabbitmq, redis
129
eb424ec @pxie Add README
pxie authored
130 (updated on July 19th, 2012)
131
132 8. Runtime errors
133 <br>Sometimes runtime errors happen during Yeti execution,
134 - 504 Gateway Error: Application fail to be started in 30 seconds
135 - 502 Internal Error: Application fail to connect to service instance, including provision/un-provision
136 bind/unbind.
137 - 404 Not Found: Route fail to redirect request to specific application URL
138
139 9. Run specific case
140 <br>User can run specific case via passing spec file with specify line number of an example
141 or group as parameter
142 For example:
143 bundle exec rspec ./spec/simple/rails_console/ruby18_rails3_spec.rb:95
144
145 10. Build java-based assets
146 <br>Please refer to docs/how-to-build-assets.md
8cbea6f update faq about environment
michael zhang authored
147
f79bcf0 update README and Rakefile
michael zhang authored
148 Rake Tasks:
149 -----------
150 - admin
151 <br>run admin test cases
152 - tests
153 <br>run core tests in parallel, e.g. rake test\[5\] (default to 10, max = 16)
154 - full
155 <br>run full tests in parallel, e.g. rake full\[5\] (default to 10, max = 16)
156 - random
157 <br>run all bvts randomly, e.g. rake random\[1023\] to re-run seed 1023
158 - java
159 <br>run java tests (spring, java_web) in parallel
160 <br>e.g. rake java\[5\] (default to 10, max = 16)
161 - jvm
162 <br>run jvm tests (spring, java_web, grails, lift) in parallel
163 <br>e.g. rake jvm\[5\] (default to 10, max = 16)
164 - ruby
165 <br>run ruby tests (rails3, sinatra, rack) in parallel
166 <br>e.g. rake ruby\[5\] (default to 10, max = 16)
167 - services
168 <br>run service tests (mongodb/redis/mysql/postgres/rabbitmq/neo4j/vblob) in parallel
169 <br>e.g. rake services\[5\] (default to 10, max = 16)
170 - clean
171 <br>clean up test environment(only run this task after interruption).
172 <br>1, Remove all apps and services under test user
173 <br>2, Remove all test users created in admin_user_spec.rb
174 - help
175 <br>list help commands
176
94cb3e4 Update README to include Bug filing info per Deepika
michael zhang authored
177 File a Bug:
178 -----------
179 To file a bug against Cloud Foundry Open Source and its components, sign up and use our bug tracking
180 system: [http://cloudfoundry.atlassian.net](http://cloudfoundry.atlassian.net)
Something went wrong with that request. Please try again.