Work out test strategy and test plan

[skasberger]

Purpose

Work out test strategy and test plan for pyDataverse, with a test plan.

Scope

In:

• Detailed test plan for API module

Out:

• Detailed test plan for models and utils module.
• Test data

Requirements

• Create test strategy
• Create test plan

ACTIONS

0. Pre-Requisites

• Apply modern development practices
• Specification
• Risk Assessment Create risk assessment #82

1. Research

• API testing best practices
  • 3 Ways to Test API Client Applications in Python
  • Forming an API Test Strategy - Where to Start
  • Dependency injection
  • API testing strategy: Who is testing your API-driven product?
  • Quality Assurance, Quality Control and Testing — the Basics of Software Quality Management
  • Three aspects of software quality: functional, structural and process
  • Test Strategy and Test Plan
  • The 10 Minute Test Plan
  • Rapid Software Testing
  • A sneak peek into ‘test framework’, ‘test pyramid’ & ‘testing pyramid’
  • SoapUI
  • Software Testing - Wikipedia
    • Regression Testing - Wikipedia
    • Risk-based testing - Wikipedia
• Research other projects test strategy
  • Pandas
  • Requests
  • Spotify
  • Wikipedia API
  • Facebook
  • Twitter
  • Slack
  • Google Places
• Check solutions
  • vcrpy
  • VCR.py at Venmo
  • pytest
    • Check: re-use unit tests for integrity tests
    • Check: Plugins
    • Check: tmp directories
    • 5 Pytest Best Practices for Writing Great Python Tests
    • Tips and tricks for unit tests (with Python & Pytest)
• Check Pytest docs
  • Check Fixtures
  • Check parametrize
  • Check Monkeypatching
  • Check Hypothesis
• Watch python testing tutorials
• Read QA/Testing notes

2. Plan

• Define requirements
• Documentation of test strategy:
  • location: offline/online; GitHub, RTD
  • file format
  • content
• Required environment
  • Dataverse installation
    • Version
  • Continous Integration: Travis-CI

3. Implement

• Write tests
• Write/Update code
• Write/Update Docs
• Write/Update Docstrings
• Run pytest
• Run tox
• Run pylint
• Run mypy

4. Follow Ups

• Apply test strategy to
  • api Implement test strategy for API tests #83 (release 0.3.0)
  • models (release 0.4.0)
  • utils (later)
• Review extension of test strategy to
  • tox
  • linting
  • coverage
  • docstrings
  • docs
• Review
  • Code
  • Tests
  • Docs
  • Strategy
    • Content
    • Documentation

[pdurbin]

As discussed during the 2024-02-14 meeting of the pyDataverse working group, we are closing old milestones in favor of a new project board at https://github.com/orgs/gdcc/projects/1 and removing issues (like this one) from those old milestones. Please feel free to join the working group! You can find us at https://py.gdcc.io and https://dataverse.zulipchat.com/#narrow/stream/377090-python