Open QA for Eucalyptus is to coordinate the Quality Assurance effort of Eucalyptus development with the community, welcoming active participation in improving Eucalyptus system.
Open QA for Eucalyptus includes the following areas:
Share all the build and test scripts with the community via Github
Provide a collaborative space to actively engage with the community to improve the quality of Eucalyptus.
Achieve complete transparency in QA process for Eucalyptus
Open QA Page
Open QA Prototype Page:
QA Test Result Page
A Sample of QA Test Results Page:
The daily operations would involve pushing the test results and their log files to qa.eucalyptus-systems.com once or twice a day.
1. How many tests are running per day?
In a given day, there are usually 8 to 10 tests running -- launched by QA engineers and developers.
2. How much log is generated per test?
Depending on the test configuration and its failure counts, a test produces between 100M to 1G of log files.
Running on AWS as instances.
Backend - Log File Server
on Eucalyptus Production Cloud at Core sites.
-> Embracing Hybrid Cloud between AWS and Euca
Recipe script to allow easy restart of the instance automagically
Instance space on production cloud
Bucket or volume space to hold results
How much bandwidth is needed between HQ <-> CoreSite for this?
In Need Now
1. More detailed information that explains:
2. Make all build and test scripts available to the community on Github.
3. Set up online mechanisms that allow the community members to engage directly with the QA process - i.e. comments, posts, contributions, etc.
Q: What is the page written in?
Q: Is it open as well?
A: Yes, it will be open-sourced; everything that touches OPEN QA will be open.
Q: Can we filter on a particular commit id/branch?
A: Twice a day, the results will be pushed out. At the time of push, we can filter out some tests if needed.
Q: Do we need to scrub any of our tests for possibly sensitive info?
A: We will be sure to review the tests to ensure that there are no sensitive information included in the tests.
Q: I'm still a bit concerned with information overload from this, it looks like there are plans for having an About page but Id rather have the info more upfront (possibly left side of page?).
A: Will be working on the content in the "About" page to describe what the visitors should expect from this page. Will let you guys know once the "About" page is ready.
Q: I also had a few questions on the Open QA initiative, are there any requirements from the Community team?
A: We are currently working closely with the Community team on defining what this project should look like as moving forward.
Q: What is the goal of this side of Open QA?
A: The goal is to bring in active community engagement via transparency and complete openness inside Eucalyptus.
Q: What does the community get out of this in the short/long term?
A: In short term, this project is to display our commitment toward the openness in Eucalyptus. In long term, the benefit to build trust in quality in Eucalyptus and crowd-sourcing of Eucalyptus QA process.
Q: What is the time frame for this task?
A: We would like to get the alpha testing of the project available within the company in about 2 weeks, and get the beta testing going in about a month, being very optimistic in the timeline here.
Q: How much time should the QA team devote to it (will it impact 3.2/3.3)?
A: It will be completely up to the individual level of dedication. It should never have impact on the release of 3.2 at least.
Q: What are the goals for this task in the future?
A: The long term goal is mentioned above.
Q: When should we get engineering involved to get their input on future systems?
A: Engineering should be exposed to the project as soon as possible, prior to the alpha testing.
Q: It would be nice to begin to track this effort through Jira so we can account for the work we are doing as well as providing a release flow (ie build, test, review, push).
A: The project will be tracked once the prototype and the system architecture is agreed upon the web and IT teams.