Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test all major features #123

Closed
14 of 19 tasks
cereallarceny opened this issue May 27, 2020 · 1 comment
Closed
14 of 19 tasks

Test all major features #123

cereallarceny opened this issue May 27, 2020 · 1 comment
Assignees
Labels
Priority: 2 - High 😰 Should be fixed as quickly as possible, ideally within the current or following sprint Severity: 2 - High 😰 Does not cause a failure, but causes the system to be incorrect, incomplete, or inconsistent Status: In Progress 🌟 This is actively being worked on Type: Testing 🧪 Add testing or improving existing testing of a file, feature, or codebase

Comments

@cereallarceny
Copy link
Member

cereallarceny commented May 27, 2020

Description

We want to test all major features of the federated learning process in this repo. This includes:

  • Mocking PyGrid in order to run the test suite without the use of another server running (these endpoints will likely be what you want to mock)
  • Testing a flow with no authentication
  • Testing a flow with basic JWT authentication (failing)
  • Testing a flow with basic JWT authentication (passing)
  • Testing the acceptance of a worker in an FL cycle request
  • Testing the timeout for the re-application of an FL cycle request
  • Testing the graceful handling of an error whereby a plan or model does not exist in PyGrid from what is being requested
  • Testing the creation of one job
  • Testing the creation of multiple jobs
  • Testing that a job can be completed successfully
  • Testing that multiple jobs can be completed successfully
  • Testing that errors in a job can be caught by the onError handler and handled appropriately
  • Testing that we can successfully report a model diff to PyGrid
  • Any other related tests, or permutations of the above tests, that are required to pass for a developer to successfully use this library... in other words, be thorough and test everything you can think of that a developer might do.

Type of Test

  • Unit test (e.g. checking a loop, method, or function is working as intended)
  • Integration test (e.g. checking if a certain group or set of functionality is working as intended)
  • Regression test (e.g. checking if by adding or removing a module of code allows other systems to continue to function as intended)
  • Stress test (e.g. checking to see how well a system performs under various situations, including heavy usage)
  • Performance test (e.g. checking to see how efficient a system is as performing the intended task)
  • Other...

Expected Behavior

We expect all the above tests to pass.

Additional Context

None

@cereallarceny cereallarceny added Priority: 2 - High 😰 Should be fixed as quickly as possible, ideally within the current or following sprint Severity: 2 - High 😰 Does not cause a failure, but causes the system to be incorrect, incomplete, or inconsistent Status: In Progress 🌟 This is actively being worked on Type: Testing 🧪 Add testing or improving existing testing of a file, feature, or codebase labels May 27, 2020
@vvmnnnkv vvmnnnkv moved this from In progress to To do in Model-centric Federated Learning Jun 9, 2020
@vvmnnnkv vvmnnnkv moved this from To do to In progress in Model-centric Federated Learning Jun 15, 2020
@vvmnnnkv
Copy link
Member

Done in #170

Model-centric Federated Learning automation moved this from In progress to Done Jun 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority: 2 - High 😰 Should be fixed as quickly as possible, ideally within the current or following sprint Severity: 2 - High 😰 Does not cause a failure, but causes the system to be incorrect, incomplete, or inconsistent Status: In Progress 🌟 This is actively being worked on Type: Testing 🧪 Add testing or improving existing testing of a file, feature, or codebase
Development

No branches or pull requests

2 participants