Skip to content

Commit

Permalink
[RC1] Fill all gaps already tracked as issues in CNTT (#918)
Browse files Browse the repository at this point in the history
* Fill all gaps already tracked as issues in CNTT

Port VTP test cases to Xtesting [1] is key because the TC doesn't conform
to the principles and requirements defined.

#917

Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

* Precise the gaps to avoid confusion

Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>

* Add tab for the new TC

Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
  • Loading branch information
collivier authored and rabiabdel committed Jan 10, 2020
1 parent 1692e64 commit 92b68ec
Showing 1 changed file with 42 additions and 4 deletions.
46 changes: 42 additions & 4 deletions doc/ref_cert/lfn/chapters/chapter11.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,52 @@
<a name="11.3"></a>
## 11.3 Automation Gaps

- Provide details of any automation gaps which require new/modified automation tools, or suites
At the time of writing,
[the yaml file](https://git.opnfv.org/releng/tree/jjb/airship/cntt.yaml)
configuring all RI Jenkins jobs are postprocessed by hand which requires lots
of skills especially about
[Jenkins job builder](https://docs.openstack.org/infra/jenkins-job-builder/).
The RI Cookbook will be deeply simplified if the deployment scripts are
delivered in a new
[Xtesting](https://xtesting.readthedocs.io/en/latest/)-based container as
proposed in
[new RI deployment container](https://github.com/cntt-n/CNTT/issues/828).
In addition to avoid configuring the jumphost, it will allow generating the
Jenkins jobs via a simple yaml file and
[Xtesting CI](https://galaxy.ansible.com/collivier/xtesting).

<a name="11.4"></a>
## 11.4 Test Case Gaps (analysis)
## 11.4 Test Case Gaps

- Provide details/analysis of Test Case gaps which required addition/editing of existing test suites, or creation of new suites altogether
OPNFV has developed many test cases in the different
[OPNFV test projects](https://wiki.opnfv.org/display/testing/TestPerf) which
can quickly improve RC. As highlighted in OPNFV Marketing Goals for 2020 and
listed in
[RC Test case integration requirements]({{ "/doc/ref_cert/lfn/chapters/chapter02.html" | relative_url }}),
porting all the existing testcases to Xtesting will unify the test case
execution and simplify the test integration as required by RC. Here are all the
related issues:
- [port YardStick testcases to Xtesting](https://github.com/cntt-n/CNTT/issues/509)
- [port Bottlenecks to Xtesting](https://github.com/cntt-n/CNTT/issues/511)
- [port StorPerf testcases to Xtesting](https://github.com/cntt-n/CNTT/issues/673)
- [port NFVbench testcases to Xtesting](https://github.com/cntt-n/CNTT/issues/865)

Here are the possible new test cases which could be integrated in the existing
OPNFV projects to improve RC:

| issues | requirements |
|---------------------------------------------------------------------------------------------------|-------------------|
| [update and integrate heat-tempest-plugin in Functest](https://github.com/cntt-n/CNTT/issues/483) | Heat API testing |
| [integrate KloudBuster in Functest](https://github.com/cntt-n/CNTT/issues/508) | disk benchmarking |
| [add tempest-stress in Functest](https://github.com/cntt-n/CNTT/issues/916) | stress testing |

<a name="11.5"></a>
## 11.5 Framework Gaps

- Provide details/analysis of Test Framework Gaps which required addition/editing of existing frameworks, incompatibility with frameworks, depreciation of frameworks.
As proposed in [port VTP test cases to Xtesting](https://github.com/cntt-n/CNTT/issues/917),
VTP selected in
[VNF E2E C&V Framework ]({{ "/doc/ref_cert/lfn/chapters/chapter05.html" | relative_url }})
requires small adaptations to fully fulfill the current
[RC Test case integration requirements]({{ "/doc/ref_cert/lfn/chapters/chapter02.html" | relative_url }}).
It seems trivial changes as VTP proposed a REST API but will ensure that
NFVI and VNF can be executed in the same CI toolchain very easily.

0 comments on commit 92b68ec

Please sign in to comment.