quality-assurance.md

Alkemio - Quality Assurance

This document provides an overview of the quality assurance and testing activities that are carried out on the Alkemio platform.

Process

• Perform functional/non-functional testing types on reviewed "github" issues
  • Depending on the "github" issue scope, testing is performed on different environments
• Log defects, communicate and retest them
• Perform regression testing as part of "github" issues testing
• Automate/maintain test cases on different levels
• Test data management
• Package dependency audit: regular review of reported vulnerabilities in packages dependent on and then address

Testing types and levels

Alkemio consists of multiple services. The following test types are being applied among the different test levels:

"Server" - testing types and levels:

First QA priority is the "Server" repository quality

• Functional testing:

  • Unit testing^*
  • GraphQL API integration tests
  • GraphQL Exploratory testing

• Non-functional testing:

  • Performance testing
  • Security testing
    • Authorization tests and accounts
      • admin@alkem.io
      • ecoverse.admin@alkem.io
  • Database migrations

• Static testing:

  • Code review
  • Static code analyses tools:

    • Build Status:

    • Build Quality:

"Client.Web" - testing types and levels:

Secondary QA priority is the "Client.Web" repository quality

• Functional testing:

  • Component testing^*
  • Integration testing^*
  • E2E automation testing^*
  • Exploratory testing

• Non-functional testing:

  • Performance testing^*
  • Security testing^*

• Static testing:

  • Code review
  • Static code analyses tools:

    • Build Status:

    • Build Quality:

Reporting

• Test cases

  • Number of test cases
  • Number of failed test cases per service

• Defects^*

  • Number of defects^*
  • Number of production environment defects by severity^*
  • Number of development environment defects by severity^*

Environments

• Local - setup for server and client-web
• Development - auto-deployed after each merge to develop branch
• Test - QA deploys from "github" repositories action, the branch version that is being tested
• Acceptance
• Production

Tools

• VS Code - IDE for automation test development
• Jest - javascript automation test framework
• Jmeter - performance test tool
• Postman - API testing tool
• MySQL 8 - query data and testing the systems integration
• Populator - test data population
• Test management system - for the moment, the manual testing is tracked within "github" issues
  • KIWI TCMS is being evaluated for manual test case management

NOTE

Test terminology meaning is based on: "ISTQB" syllabus

^*to be defined/implemented

Release - QA checklist

Pre-deployment

• Nightly test build is passing
• Server CI: passing
• Client-web CI: passing
• New features/bugs are tested manually and/or automated
  • Feature/Bug "x"
  • Feature/Bug "y"
  • Feature/Bug "y"
• Update scripts: tested
• Migration scripts: tested
• Performance/Security analyses, if required

Post-deployment

• System smoke test
  • Sign in/out
  • User registration
  • User profile update
  • User access to available resources
    • Space page
    • Subspace page
    • Subspace page
    • Search page
    • Admin page
      • Spaces
      • Users
      • Organizations
      • Authorization
• Update scripts: verify no data loss/coruption
• Migration scripts: verify no data loss/coruption
• Logs are error free
• Performance/Security analyses, if required