Remove travis #3013
Merged
Remove travis #3013
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Feb 5, 2020
|
Wow, I'm in need of a Github Actions tutorial. @karlhigley or @systemshift can you guide me with this ? |
|
@ucalyptus: The Github Action docs are pretty good. |
karlhigley
approved these changes
Feb 5, 2020
| @@ -1,6 +1,6 @@ | |||
| # Introduction | |||
|
|
|||
| [](https://mybinder.org/v2/gh/OpenMined/PySyft/master) [](https://travis-ci.org/OpenMined/PySyft) [](https://openmined.slack.com/messages/team_pysyft) [](https://app.fossa.io/projects/git%2Bgithub.com%2Fmatthew-mcateer%2FPySyft?ref=badge_small) | |||
| [](https://mybinder.org/v2/gh/OpenMined/PySyft/master) [](https://openmined.slack.com/messages/team_pysyft) [](https://app.fossa.io/projects/git%2Bgithub.com%2Fmatthew-mcateer%2FPySyft?ref=badge_small) | |||
There was a problem hiding this comment.
We'll probably want to replace the Travis build status badge with a GH Actions build status badge.
There was a problem hiding this comment.
Done, Also added a second badge for notebooks, currently its broken, but should work once my other pull request is merged to master.
karlhigley
force-pushed
the
remove_travis
branch
from
ab94dde
to
b3c480e
Compare
LaRiffle
added a commit
that referenced
this pull request
Feb 20, 2020
* Migrate CI tests to Github Actions (#2936) * Create and test pysyft package on various Pythons Includes Python 2.7, 3.5, 3.6, 3.7 * Rework build script to run tests like Travis * Update Python versions to 3.6+ * Remove Python 3.8 since Torch doesn't support it * Only run the tutorial translation tests when the tutorials change The normal tutorial notebook tests still run on every build, which should catch the cases where changes to the code break the notebooks. When the main tutorial notebooks are changed, that will trigger the action that runs the notebook translation tests, which should surface the same failures in the whichever translated versions of the tutorials are affected. * Measure process time instead of wall time in efficiency tests This counts only 'user time' and excludes 'system time', and measures relative to the CPU clock speed, which should make the tests less flaky. * Increase `max_time` for `test_inv_sym` closer to original value * Rework "test all notebooks" tests to split off translation tests * Create a separate section in the action logs for coverage reports * Exclude translation tests from coverage checks * Bump `test_inv_sym` execution time limit to 30 * Omit `.github` directory from code coverage checking * Fix translation notebooks test to account for excluded notebooks * Move coverage exclusions into setup.cfg instead of .travis.yml This way they work everywhere that coverage runs, not just on Travis. * Omit the efficiency tests from Github Action check suite Omits them from both the tests and the test coverage checks * Unwind changes to the efficiency tests (now skipped in check suite) * Stop using efficiency test time assertions outside the efficiency tests * Standardize the name of the efficiency tests module * Fix efficiency assertion imports * Fix a missing wildcard in the coverage omission flag * Apply testing workflow to PR creation, synchronization, and reopening * Clean up coverage check inclusions and exclusions * Fix triggering events section * feat: Translation to Spanish of Part 08 - Introduction to Plans. (#2962) Changes made by @socd06 This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: #2775 See also: #2782 * Bump PyTorch to 1.4 (#2930) Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> * Modify conv implementation and corresponding test to get exact match (#2964) * Modify conv implementation and corresponding test to get exact match * Revert mkldnn to the original state after execution * Exorcise a Python `eval` and transmute into the lesser demon `getattr` (#2951) * Exorcise a Python `eval` and transmute into the lesser demon `getattr` * Remove the double split for optimization * Point to correct version (#2968) * Refactor Plan internals using PlaceHolder (#2910) * Add experiment about Plans with placeholders * Add first implementation of plan using placeholders * Update notebook of test * Improve local execution of plans * Support remote execution of plans * Rm unused function * Use Operation object in plan * Remove fixed precision and sharing from Plan's multiple fetch test These seem extraneous to the point of the test anyway, so may as well go ahead and remove them rather than disabling this test. * Disable encrypted and fixed precision Plans * Remove unused `plan` attribute on `State` * Make `test_fetch_plan_multiple_times` pass * Fix typo in test name * Update serde_full unit tests (WIP) * Add example of Plan problem * Update serde unit tests * Fix output handling in plans * Fix nested Plans handling * Apply formatting to serde helpers * Adjust serde tests to match the new plan structure * Update requirements.txt to avoid a merge conflict * Undo some changes to the Torch hook * Fix Long vs Float issue in `PointerPlan` test * Remove current PromiseTensor * Remove plan.procedure * Add Trace object to clean code * Rm promise notebook * Add support for fix_precision & MPC * Clean plan code and add a tracer decorator * Clean state and rm useless functions * Clean the tracer decorator and comment code * Add comments to Plan & State new methods * Fix ordering of arguments in Plan * Fix dummy error and fix wraps decorators * Fix * Add a few comments * Merge input/output placeholders and do light cleaning * Rm useless experimental notebook * Fix serde * Increase delay in a time test Co-authored-by: Karl Higley <kmhigley@gmail.com> Co-authored-by: Vova Manannikov <12518480+vvmnnnkv@users.noreply.github.com> * add Part 13a Hindi (#2958) Co-authored-by: Andrew Trask <liamtrask@gmail.com> * Bump version to `0.2.3.a1` (#2976) * Refactored code of SplitNN model and added evaluation on datasets (#2983) * Add Protobuf serialization for Placeholders, Plans, and States (#2972) * Add Protobuf serialization for Placeholders, Plans, and States * Bump `syft-proto` requirement to `0.1.1.a1.post5` Co-authored-by: Andrew Trask <liamtrask@gmail.com> * Make it possible to JIT trace a PySyft Plan to convert to Torchscript (#2988) * updated requirements.txt (#2970) * updated requirements.txt * Update requirements.txt Co-authored-by: George-Cristian Muraru <murarugeorgec@gmail.com> * Translation to Spanish of Tutorial Part 12 bis (#2979) feat: Translation to Spanish of Tutorial Part 12 bis - Encrypted Training on MNIST. This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: #2775 See also: #2782 Co-authored-by: Karl Higley <kmhigley@gmail.com> * Remove codefix for old pytorch version (#2999) * feat: Translation to Spanish of part 08 bis (#3011) This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: Track better the changes in every document. Avoid double work. Resolves: #2775 See also: #2782 * Added Bengali Translated Notebooks (#2992) Translated Notebooks: - Part 13a - Secure Classification with Syft Keras and TFE - Public Training.ipynb - Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb - Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Add Udacity Link to Readme (#3016) * Add sphinx documentation with automatic API building (#3017) * Add autoapi.extension to sphinx configuration * Add autoapi as dev dep. Add doc build to travis * Add doc testenv to github actions and travis * Fix formatting via black * Add readthedocs yml configuration * Configure sphinx to ignore "_static" * Update theme to sphinx-rtd-theme * Update index.rst Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> * Adding some notebooks translated to Portuguese (#3015) * Adding Part 12 tutorial in portuguese * Adding Parte13a tutorial in portuguese Co-authored-by: Andrew Trask <liamtrask@gmail.com> * trigger github action notebook checks (#3012) * use doubel qoutes instead * edit notebook just to trigger testing * add pull_request trigger condition * Revert "edit notebook just to trigger testing" This reverts commit 7e46d5b. * add path to pull_request * edit notebook again just to see if checks trigger * rename notebook .yml file * edit name string * change pull_request to master branch condition * master branch trigger condition * Revert "edit notebook again just to see if checks trigger" This reverts commit 60fee72. * small edit just to trigger notebook tests, AGAIN! * Revert "small edit just to trigger notebook tests, AGAIN!" This reverts commit 16a83d2. * Revert changes to `run-tests.yaml` This one is already working the way we want. * Make action workflows match Co-authored-by: Karl Higley <kmhigley@gmail.com> * Translation into Ukrainian language (#2854) * Add files via upload * Update Part 01 - Основні засоби приватного глибинного навчання.ipynb Co-authored-by: Karl Higley <kmhigley@gmail.com> Co-authored-by: George-Cristian Muraru <murarugeorgec@gmail.com> Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> * Remove travis (#3013) * delete travis.yml * remove travis from contributing.md * remoev travis from readme * add note for future contributers in test_notebooks * add github actions badge * edit typo * add notebook GH test badge Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * Hindi translations for Part 5, 6, 13b and 13c (#2909) * Hindi translations for Part 5, 6, 13b and 13c * Markdown changes applied and github link added * Fixed spaces and translatations * Update Part 06 - Federated Learning on MNIST using a CNN.ipynb * Made changes on translations and spaces * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Updated Part 5 - moving Pygrid Module to PySyft * Hindi translations for Part 4, 7 and 8 updated * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 08 - Introduction to Plans.ipynb * Hindi translations - 4, 7, 8, 8 bis,9 and 10 * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * (minor fix) fix broked badge for run notebook tests (#3028) * fix: failing tutorial translations should be ignored (#3034) * Add Bengali Translated Notebooks #Part 12 bis (#3038) Translated Notebooks: Part 12 bis - Encrypted Training on MNIST.ipynb * Add Bengali Translated Notebooks #Part 12 (#3039) Translated Notebooks: Part 12 - Train an Encrypted Neural Network on Encrypted Data.ipynb * fix: failing tutorial translations should be ignored (#3047) Exclude only the names, instead of full paths. This means it will effectively exclude translated titles too. * Translation to Portuguese of the tutorials Part 09, 10, and 11 (#2980) * add translation to Portuguese of the tutorial Part 02 * (enhance) translate example toy expression literally * (minor fixes) change 'portugues' to 'português' and 'nível superior' to 'parte superior' * add translation to Portuguese of the tutorial Part 03 * (fix) remove old folder name * add translation to Portuguese of the tutorial Part 04 * (update) rename 'portugues' to 'português' and remove old folder * add translation to Portuguese of the tutorial Part 05 * add translation to Portuguese of the tutorial Part 06 * (minor fixes) typo and verb correction * add translation to Portuguese of the tutorial Part 09 * add translation to Portuguese of the tutorial Part 10 * add translation to Portuguese of the tutorial Part 11 * (fix) a cell has been removed by mistake 😑 * (minor fixes) 🤞 🤞 🤞 * (minor fix) full stop added * (fixes) remove repeated paragraphs left in tutorials part 06 and 11 Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: George-Cristian Muraru <murarugeorgec@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * Portuguese translations Part7-8-8bis (#2977) * pt translation part07 * middle part 8 translation * fixing portuges * finishing translation part8 and part8 bis * fixing errors * finishing tutorial part 8bis * changing metadata kernel to python3 * update parte07-08 - fixing some errors * Update Parte 07 - Aprendizado Federado com Conjunto de Dados Federado.ipynb * Update Parte 08 - Introdução a Planos.ipynb Co-authored-by: Karl Higley <kmhigley@gmail.com> * Translated tutorial part 01 to Portuguese (#3035) Co-authored-by: Karl Higley <kmhigley@gmail.com> * Translate PySyft Tutorials to Portuguese (#3020) * Translate notebook Part 13b * translate notebook part 12 bis to portuguese * Update Parte 12 bis - Treinamento criptografado no MNIST.ipynb Co-authored-by: Karl Higley <kmhigley@gmail.com> * Portuguese translation 13c (#3023) * translate 13c * translate 13c update * translate 13c update 2 * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb * Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb Co-authored-by: Karl Higley <kmhigley@gmail.com> * No need for requires_grad for input data (#2998) * No need for requires_grad for input data * Increase error threshold Co-authored-by: Andrew Trask <liamtrask@gmail.com> * Put the test badges first in README (#3046) * Put the test badges first * Update the names of test workflows to clean up README badges Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> Co-authored-by: Andrew Trask <liamtrask@gmail.com> * Add Bengali Translated Notebooks #Part 11 (#3041) Translated Notebook: Part 11 - Secure Deep Learning Classification.ipynb Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * Add instruction for PySyft-Proto dependency and numbered the steps (#2818) * Add instruction for PySyft-Proto dependency and numbered the steps * Remove Step 5, it is handled in requirement_dev.txt * Update requirements files to point to syft-proto in PyPI * Remove duplicate dependency (syft-proto) outside requirements.txt * replace proto with syft-proto * Cleanup PATE code (#3033) * Format PATE docstrings Uniform indentation First pass of adding type hints Remove commented out TF flags code * Remove commented out print statements in PATE code * Change blank lines in PATE code No blank lines after function doc strings No blank lines immediately after control statements Break up larger chunks of code * Specify l and noise_eps when printing if cant compute sensitivity Before this commit, incorrect noise/moments spammed console with identical statements * Change torch/numpy sensitivity function type hints * Add notice required under Apache 2.0 license that changes have been made to code * Remove ellipses from List in type hints Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * update comment in test units (#3018) * remove test unit * Add a missing comma that `black` complains about * add ignore test back and update comment. Co-authored-by: Karl Higley <kmhigley@gmail.com> * Fix potential medium/high severity security holes (#3037) * remove security warning messages * Fix missing ast import * Flag the `eval` in `Plan` so that `bandit` security scan ignores it We do want to fix this and find a better way to do it, but don't want it to block adding security scans to, for example, catch future `eval`s added to the code base. * Black formatter wants two spaces before a comment Co-authored-by: Karl Higley <kmhigley@gmail.com> Co-authored-by: Andrew Trask <liamtrask@gmail.com> * CI security testing (#3036) * add bandit as dev dependancy * add job for bandit * only report medium and high severity * Match format of security task name with the others Co-authored-by: Karl Higley <kmhigley@gmail.com> * Only test translation notebooks if they have been changed (#3060) * Only test translated tutorials that have been changed * Add test_notebooks_basic_translations_diff for GHActions * Change GitHub Actions test for translations * Include git-diff file * Tutorial Notebook 10 note on numpy hook (#3022) * added note about notebook not working * phrasing Co-authored-by: Karl Higley <kmhigley@gmail.com> * Revert "Only test translation notebooks if they have been changed (#3060)" (#3062) This reverts commit 91c8114. * Split tutorial translation tests into separate Github Workflows (#3059) * Split tutorial translation tests into separate Github Workflows * Update the main tests workflow to exclude translation tests * Lower-case the directory name for Ukrainian translations * Add update-docker-image.yml to GitHub Actions workflow (#3030) * Remove pointer tags/description decode instructions (#3067) * Update GH checkout action to v2 (#3068) * Fix trigger paths in translation test workflows (#3065) * Fix trigger paths in translation test workflows * Update GH checkout action to v2 * Correct build context for dockerhub update via GitHub Actions (#3066) * fix build context for dockerhub * Update GH checkout action to v2 Co-authored-by: Karl Higley <kmhigley@gmail.com> * Update msgpack version requirement (#3073) * Reduce websocket timeout to one minute (#3072) This is making CI tests hang ~forever. I think 278 hours is a bit excessive, don't you? :) * Add wildcards to the tutorial translation test actions (#3071) * Translation to Spanish of Parts 09, 13 (a, b and c) (#3032) * feat: Translation to Spanish of Parts 09, 13 (a, b and c). Credits to: @thefirebanks This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: #2775 See also: #2782 * Fix typos in Spanish tutorials parts 9 and 13b Co-authored-by: Karl Higley <kmhigley@gmail.com> * Translations to Spanish of Tutorials Part 10 and 11 (#3031) * feat: Translation to Spanish of Part 10 - Federated Learning with Secure Aggregation. This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: #2775 See also: #2782 * feat: Translation to Spanish of Part 11 - Secure Deep Learning Classification This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: #2775 See also: #2782 * Fix typos in Spanish translations parts 10 and 11 Co-authored-by: Karl Higley <kmhigley@gmail.com> * Notebook 7 hindi translation (#3055) * Hindi translations for Part 5, 6, 13b and 13c * Markdown changes applied and github link added * Fixed spaces and translatations * Update Part 06 - Federated Learning on MNIST using a CNN.ipynb * Made changes on translations and spaces * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Updated Part 5 - moving Pygrid Module to PySyft * Hindi translations for Part 4, 7 and 8 updated * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 08 - Introduction to Plans.ipynb * Hindi translations - 4, 7, 8, 8 bis,9 and 10 * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb * Notebook 7 hindi translation * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb * Notebook 7 - hindi translation Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * Bengali 6 (#3052) * Bengali 6 Added * Fix markdown link syntax for OpenMined blog post Co-authored-by: Karl Higley <kmhigley@gmail.com> * Bengali 7 (#3053) * Bengali 7 Added * Fix link to PySyft project issues Co-authored-by: Karl Higley <kmhigley@gmail.com> * Split the (English) tutorial tests into a separate Github Action (#3075) * Split the (English) tutorial tests into a separate Github Action The tutorial tests take about 15 minutes to run, which is approximately equal to the running time of all the rest of the tests. Running them in parallel should speed up the PR checks. * Exclude the isolated file system test fixture from coverage checks It's only used by the notebook tests, which are run separately as of this pull request. * Run notebook tests on all PRs except translation tests Before this change, they would only run on PRs that changed notebooks. * Context of computation for crypten (#2963) * New message type for crypten party initialization * Handle CryptenInit message by running local party * Define a new context of computation for crypten The new context of computation can run parties that are distributed across syft workers. The communication of crypten parties remains the same, however, syft workers handle initialization and the serialization of return values. * updated docs * testing crypten context * fix: was setting the bad env variable DISTRIBUTED_BACKEND should have been set and not BACKEND * test serde of CryptenInit message * add crypten as core deps, this should change to extra * delete useless comment * fix: list CryptenInit in OBJ_SIMPLIFIER_AND_DETAILERS * fix msgpack tests * run black * don't cover empty function * Use tag for crypten data (#3003) Co-authored-by: Karl Higley <kmhigley@gmail.com> Co-authored-by: arturomf94 <arturomf94@gmail.com> Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> Co-authored-by: Arshjot Singh Khehra <arshjot@live.com> Co-authored-by: Vova Manannikov <12518480+vvmnnnkv@users.noreply.github.com> Co-authored-by: Yugandhar Tripathi <tripathi.yugandhar@gmail.com> Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: midokura-silvia <43242089+midokura-silvia@users.noreply.github.com> Co-authored-by: Carlos Salgado <51087091+socd06@users.noreply.github.com> Co-authored-by: Ricardo Pretelt <32759855+ricardopretelt@users.noreply.github.com> Co-authored-by: Sourav Das <31799654+adventuroussrv@users.noreply.github.com> Co-authored-by: Aiden V Johnson <aiden.dataminer@gmail.com> Co-authored-by: José Benardi de Souza Nunes <benardinunes@gmail.com> Co-authored-by: Marcus Vinicius <marcus.costa@ccc.ufcg.edu.br> Co-authored-by: systemshift <42102034+systemshift@users.noreply.github.com> Co-authored-by: Bogdan Ivanyuk <35574157+KyloRen1@users.noreply.github.com> Co-authored-by: Urvashi Bhattacharyya <raheja@users.noreply.github.com> Co-authored-by: Jeferson Ferreira <jefersonnpn@gmail.com> Co-authored-by: João Lucas <joao.felipe@ccc.ufcg.edu.br> Co-authored-by: Márcio Porto <6521281+MarcioPorto@users.noreply.github.com> Co-authored-by: Héricles Emanuel <hericles.me@gmail.com> Co-authored-by: Izabella Antonino <izabella.queiroz@ccc.ufcg.edu.br> Co-authored-by: Bolarinwa Saheed Olayemi <refactormyself@users.noreply.github.com> Co-authored-by: Tom <32938439+TTitcombe@users.noreply.github.com> Co-authored-by: froessler <11539266+fdroessler@users.noreply.github.com> Co-authored-by: Lina Tran <10761918+linamnt@users.noreply.github.com> Co-authored-by: Ionésio Junior <ionesiojr@gmail.com> Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com> Co-authored-by: Sayantan Das <36279638+ucalyptus@users.noreply.github.com> Co-authored-by: Ayoub Benaissa <ayouben9@gmail.com>
gmuraru
added a commit
to gmuraru/PySyft
that referenced
this pull request
Mar 3, 2020
* Migrate CI tests to Github Actions (OpenMined#2936) * Create and test pysyft package on various Pythons Includes Python 2.7, 3.5, 3.6, 3.7 * Rework build script to run tests like Travis * Update Python versions to 3.6+ * Remove Python 3.8 since Torch doesn't support it * Only run the tutorial translation tests when the tutorials change The normal tutorial notebook tests still run on every build, which should catch the cases where changes to the code break the notebooks. When the main tutorial notebooks are changed, that will trigger the action that runs the notebook translation tests, which should surface the same failures in the whichever translated versions of the tutorials are affected. * Measure process time instead of wall time in efficiency tests This counts only 'user time' and excludes 'system time', and measures relative to the CPU clock speed, which should make the tests less flaky. * Increase `max_time` for `test_inv_sym` closer to original value * Rework "test all notebooks" tests to split off translation tests * Create a separate section in the action logs for coverage reports * Exclude translation tests from coverage checks * Bump `test_inv_sym` execution time limit to 30 * Omit `.github` directory from code coverage checking * Fix translation notebooks test to account for excluded notebooks * Move coverage exclusions into setup.cfg instead of .travis.yml This way they work everywhere that coverage runs, not just on Travis. * Omit the efficiency tests from Github Action check suite Omits them from both the tests and the test coverage checks * Unwind changes to the efficiency tests (now skipped in check suite) * Stop using efficiency test time assertions outside the efficiency tests * Standardize the name of the efficiency tests module * Fix efficiency assertion imports * Fix a missing wildcard in the coverage omission flag * Apply testing workflow to PR creation, synchronization, and reopening * Clean up coverage check inclusions and exclusions * Fix triggering events section * feat: Translation to Spanish of Part 08 - Introduction to Plans. (OpenMined#2962) Changes made by @socd06 This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Bump PyTorch to 1.4 (OpenMined#2930) Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> * Modify conv implementation and corresponding test to get exact match (OpenMined#2964) * Modify conv implementation and corresponding test to get exact match * Revert mkldnn to the original state after execution * Exorcise a Python `eval` and transmute into the lesser demon `getattr` (OpenMined#2951) * Exorcise a Python `eval` and transmute into the lesser demon `getattr` * Remove the double split for optimization * Point to correct version (OpenMined#2968) * Refactor Plan internals using PlaceHolder (OpenMined#2910) * Add experiment about Plans with placeholders * Add first implementation of plan using placeholders * Update notebook of test * Improve local execution of plans * Support remote execution of plans * Rm unused function * Use Operation object in plan * Remove fixed precision and sharing from Plan's multiple fetch test These seem extraneous to the point of the test anyway, so may as well go ahead and remove them rather than disabling this test. * Disable encrypted and fixed precision Plans * Remove unused `plan` attribute on `State` * Make `test_fetch_plan_multiple_times` pass * Fix typo in test name * Update serde_full unit tests (WIP) * Add example of Plan problem * Update serde unit tests * Fix output handling in plans * Fix nested Plans handling * Apply formatting to serde helpers * Adjust serde tests to match the new plan structure * Update requirements.txt to avoid a merge conflict * Undo some changes to the Torch hook * Fix Long vs Float issue in `PointerPlan` test * Remove current PromiseTensor * Remove plan.procedure * Add Trace object to clean code * Rm promise notebook * Add support for fix_precision & MPC * Clean plan code and add a tracer decorator * Clean state and rm useless functions * Clean the tracer decorator and comment code * Add comments to Plan & State new methods * Fix ordering of arguments in Plan * Fix dummy error and fix wraps decorators * Fix * Add a few comments * Merge input/output placeholders and do light cleaning * Rm useless experimental notebook * Fix serde * Increase delay in a time test Co-authored-by: Karl Higley <kmhigley@gmail.com> Co-authored-by: Vova Manannikov <12518480+vvmnnnkv@users.noreply.github.com> * add Part 13a Hindi (OpenMined#2958) Co-authored-by: Andrew Trask <liamtrask@gmail.com> * Bump version to `0.2.3.a1` (OpenMined#2976) * Refactored code of SplitNN model and added evaluation on datasets (OpenMined#2983) * Add Protobuf serialization for Placeholders, Plans, and States (OpenMined#2972) * Add Protobuf serialization for Placeholders, Plans, and States * Bump `syft-proto` requirement to `0.1.1.a1.post5` Co-authored-by: Andrew Trask <liamtrask@gmail.com> * Make it possible to JIT trace a PySyft Plan to convert to Torchscript (OpenMined#2988) * updated requirements.txt (OpenMined#2970) * updated requirements.txt * Update requirements.txt Co-authored-by: George-Cristian Muraru <murarugeorgec@gmail.com> * Translation to Spanish of Tutorial Part 12 bis (OpenMined#2979) feat: Translation to Spanish of Tutorial Part 12 bis - Encrypted Training on MNIST. This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 Co-authored-by: Karl Higley <kmhigley@gmail.com> * Remove codefix for old pytorch version (OpenMined#2999) * feat: Translation to Spanish of part 08 bis (OpenMined#3011) This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: Track better the changes in every document. Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Added Bengali Translated Notebooks (OpenMined#2992) Translated Notebooks: - Part 13a - Secure Classification with Syft Keras and TFE - Public Training.ipynb - Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb - Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Add Udacity Link to Readme (OpenMined#3016) * Add sphinx documentation with automatic API building (OpenMined#3017) * Add autoapi.extension to sphinx configuration * Add autoapi as dev dep. Add doc build to travis * Add doc testenv to github actions and travis * Fix formatting via black * Add readthedocs yml configuration * Configure sphinx to ignore "_static" * Update theme to sphinx-rtd-theme * Update index.rst Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> * Adding some notebooks translated to Portuguese (OpenMined#3015) * Adding Part 12 tutorial in portuguese * Adding Parte13a tutorial in portuguese Co-authored-by: Andrew Trask <liamtrask@gmail.com> * trigger github action notebook checks (OpenMined#3012) * use doubel qoutes instead * edit notebook just to trigger testing * add pull_request trigger condition * Revert "edit notebook just to trigger testing" This reverts commit 7e46d5b. * add path to pull_request * edit notebook again just to see if checks trigger * rename notebook .yml file * edit name string * change pull_request to master branch condition * master branch trigger condition * Revert "edit notebook again just to see if checks trigger" This reverts commit 60fee72. * small edit just to trigger notebook tests, AGAIN! * Revert "small edit just to trigger notebook tests, AGAIN!" This reverts commit 16a83d2. * Revert changes to `run-tests.yaml` This one is already working the way we want. * Make action workflows match Co-authored-by: Karl Higley <kmhigley@gmail.com> * Translation into Ukrainian language (OpenMined#2854) * Add files via upload * Update Part 01 - Основні засоби приватного глибинного навчання.ipynb Co-authored-by: Karl Higley <kmhigley@gmail.com> Co-authored-by: George-Cristian Muraru <murarugeorgec@gmail.com> Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> * Remove travis (OpenMined#3013) * delete travis.yml * remove travis from contributing.md * remoev travis from readme * add note for future contributers in test_notebooks * add github actions badge * edit typo * add notebook GH test badge Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * Hindi translations for Part 5, 6, 13b and 13c (OpenMined#2909) * Hindi translations for Part 5, 6, 13b and 13c * Markdown changes applied and github link added * Fixed spaces and translatations * Update Part 06 - Federated Learning on MNIST using a CNN.ipynb * Made changes on translations and spaces * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Updated Part 5 - moving Pygrid Module to PySyft * Hindi translations for Part 4, 7 and 8 updated * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 08 - Introduction to Plans.ipynb * Hindi translations - 4, 7, 8, 8 bis,9 and 10 * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * (minor fix) fix broked badge for run notebook tests (OpenMined#3028) * fix: failing tutorial translations should be ignored (OpenMined#3034) * Add Bengali Translated Notebooks #Part 12 bis (OpenMined#3038) Translated Notebooks: Part 12 bis - Encrypted Training on MNIST.ipynb * Add Bengali Translated Notebooks #Part 12 (OpenMined#3039) Translated Notebooks: Part 12 - Train an Encrypted Neural Network on Encrypted Data.ipynb * fix: failing tutorial translations should be ignored (OpenMined#3047) Exclude only the names, instead of full paths. This means it will effectively exclude translated titles too. * Translation to Portuguese of the tutorials Part 09, 10, and 11 (OpenMined#2980) * add translation to Portuguese of the tutorial Part 02 * (enhance) translate example toy expression literally * (minor fixes) change 'portugues' to 'português' and 'nível superior' to 'parte superior' * add translation to Portuguese of the tutorial Part 03 * (fix) remove old folder name * add translation to Portuguese of the tutorial Part 04 * (update) rename 'portugues' to 'português' and remove old folder * add translation to Portuguese of the tutorial Part 05 * add translation to Portuguese of the tutorial Part 06 * (minor fixes) typo and verb correction * add translation to Portuguese of the tutorial Part 09 * add translation to Portuguese of the tutorial Part 10 * add translation to Portuguese of the tutorial Part 11 * (fix) a cell has been removed by mistake 😑 * (minor fixes) 🤞 🤞 🤞 * (minor fix) full stop added * (fixes) remove repeated paragraphs left in tutorials part 06 and 11 Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: George-Cristian Muraru <murarugeorgec@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * Portuguese translations Part7-8-8bis (OpenMined#2977) * pt translation part07 * middle part 8 translation * fixing portuges * finishing translation part8 and part8 bis * fixing errors * finishing tutorial part 8bis * changing metadata kernel to python3 * update parte07-08 - fixing some errors * Update Parte 07 - Aprendizado Federado com Conjunto de Dados Federado.ipynb * Update Parte 08 - Introdução a Planos.ipynb Co-authored-by: Karl Higley <kmhigley@gmail.com> * Translated tutorial part 01 to Portuguese (OpenMined#3035) Co-authored-by: Karl Higley <kmhigley@gmail.com> * Translate PySyft Tutorials to Portuguese (OpenMined#3020) * Translate notebook Part 13b * translate notebook part 12 bis to portuguese * Update Parte 12 bis - Treinamento criptografado no MNIST.ipynb Co-authored-by: Karl Higley <kmhigley@gmail.com> * Portuguese translation 13c (OpenMined#3023) * translate 13c * translate 13c update * translate 13c update 2 * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb * Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb Co-authored-by: Karl Higley <kmhigley@gmail.com> * No need for requires_grad for input data (OpenMined#2998) * No need for requires_grad for input data * Increase error threshold Co-authored-by: Andrew Trask <liamtrask@gmail.com> * Put the test badges first in README (OpenMined#3046) * Put the test badges first * Update the names of test workflows to clean up README badges Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> Co-authored-by: Andrew Trask <liamtrask@gmail.com> * Add Bengali Translated Notebooks #Part 11 (OpenMined#3041) Translated Notebook: Part 11 - Secure Deep Learning Classification.ipynb Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * Add instruction for PySyft-Proto dependency and numbered the steps (OpenMined#2818) * Add instruction for PySyft-Proto dependency and numbered the steps * Remove Step 5, it is handled in requirement_dev.txt * Update requirements files to point to syft-proto in PyPI * Remove duplicate dependency (syft-proto) outside requirements.txt * replace proto with syft-proto * Cleanup PATE code (OpenMined#3033) * Format PATE docstrings Uniform indentation First pass of adding type hints Remove commented out TF flags code * Remove commented out print statements in PATE code * Change blank lines in PATE code No blank lines after function doc strings No blank lines immediately after control statements Break up larger chunks of code * Specify l and noise_eps when printing if cant compute sensitivity Before this commit, incorrect noise/moments spammed console with identical statements * Change torch/numpy sensitivity function type hints * Add notice required under Apache 2.0 license that changes have been made to code * Remove ellipses from List in type hints Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * update comment in test units (OpenMined#3018) * remove test unit * Add a missing comma that `black` complains about * add ignore test back and update comment. Co-authored-by: Karl Higley <kmhigley@gmail.com> * Fix potential medium/high severity security holes (OpenMined#3037) * remove security warning messages * Fix missing ast import * Flag the `eval` in `Plan` so that `bandit` security scan ignores it We do want to fix this and find a better way to do it, but don't want it to block adding security scans to, for example, catch future `eval`s added to the code base. * Black formatter wants two spaces before a comment Co-authored-by: Karl Higley <kmhigley@gmail.com> Co-authored-by: Andrew Trask <liamtrask@gmail.com> * CI security testing (OpenMined#3036) * add bandit as dev dependancy * add job for bandit * only report medium and high severity * Match format of security task name with the others Co-authored-by: Karl Higley <kmhigley@gmail.com> * Only test translation notebooks if they have been changed (OpenMined#3060) * Only test translated tutorials that have been changed * Add test_notebooks_basic_translations_diff for GHActions * Change GitHub Actions test for translations * Include git-diff file * Tutorial Notebook 10 note on numpy hook (OpenMined#3022) * added note about notebook not working * phrasing Co-authored-by: Karl Higley <kmhigley@gmail.com> * Revert "Only test translation notebooks if they have been changed (OpenMined#3060)" (OpenMined#3062) This reverts commit 91c8114. * Split tutorial translation tests into separate Github Workflows (OpenMined#3059) * Split tutorial translation tests into separate Github Workflows * Update the main tests workflow to exclude translation tests * Lower-case the directory name for Ukrainian translations * Add update-docker-image.yml to GitHub Actions workflow (OpenMined#3030) * Remove pointer tags/description decode instructions (OpenMined#3067) * Update GH checkout action to v2 (OpenMined#3068) * Fix trigger paths in translation test workflows (OpenMined#3065) * Fix trigger paths in translation test workflows * Update GH checkout action to v2 * Correct build context for dockerhub update via GitHub Actions (OpenMined#3066) * fix build context for dockerhub * Update GH checkout action to v2 Co-authored-by: Karl Higley <kmhigley@gmail.com> * Update msgpack version requirement (OpenMined#3073) * Reduce websocket timeout to one minute (OpenMined#3072) This is making CI tests hang ~forever. I think 278 hours is a bit excessive, don't you? :) * Add wildcards to the tutorial translation test actions (OpenMined#3071) * Translation to Spanish of Parts 09, 13 (a, b and c) (OpenMined#3032) * feat: Translation to Spanish of Parts 09, 13 (a, b and c). Credits to: @thefirebanks This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Fix typos in Spanish tutorials parts 9 and 13b Co-authored-by: Karl Higley <kmhigley@gmail.com> * Translations to Spanish of Tutorials Part 10 and 11 (OpenMined#3031) * feat: Translation to Spanish of Part 10 - Federated Learning with Secure Aggregation. This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * feat: Translation to Spanish of Part 11 - Secure Deep Learning Classification This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Fix typos in Spanish translations parts 10 and 11 Co-authored-by: Karl Higley <kmhigley@gmail.com> * Notebook 7 hindi translation (OpenMined#3055) * Hindi translations for Part 5, 6, 13b and 13c * Markdown changes applied and github link added * Fixed spaces and translatations * Update Part 06 - Federated Learning on MNIST using a CNN.ipynb * Made changes on translations and spaces * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Updated Part 5 - moving Pygrid Module to PySyft * Hindi translations for Part 4, 7 and 8 updated * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 08 - Introduction to Plans.ipynb * Hindi translations - 4, 7, 8, 8 bis,9 and 10 * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb * Notebook 7 hindi translation * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb * Notebook 7 - hindi translation Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * Bengali 6 (OpenMined#3052) * Bengali 6 Added * Fix markdown link syntax for OpenMined blog post Co-authored-by: Karl Higley <kmhigley@gmail.com> * Bengali 7 (OpenMined#3053) * Bengali 7 Added * Fix link to PySyft project issues Co-authored-by: Karl Higley <kmhigley@gmail.com> * Split the (English) tutorial tests into a separate Github Action (OpenMined#3075) * Split the (English) tutorial tests into a separate Github Action The tutorial tests take about 15 minutes to run, which is approximately equal to the running time of all the rest of the tests. Running them in parallel should speed up the PR checks. * Exclude the isolated file system test fixture from coverage checks It's only used by the notebook tests, which are run separately as of this pull request. * Run notebook tests on all PRs except translation tests Before this change, they would only run on PRs that changed notebooks. * Context of computation for crypten (OpenMined#2963) * New message type for crypten party initialization * Handle CryptenInit message by running local party * Define a new context of computation for crypten The new context of computation can run parties that are distributed across syft workers. The communication of crypten parties remains the same, however, syft workers handle initialization and the serialization of return values. * updated docs * testing crypten context * fix: was setting the bad env variable DISTRIBUTED_BACKEND should have been set and not BACKEND * test serde of CryptenInit message * add crypten as core deps, this should change to extra * delete useless comment * fix: list CryptenInit in OBJ_SIMPLIFIER_AND_DETAILERS * fix msgpack tests * run black * don't cover empty function * Use tag for crypten data (OpenMined#3003) Co-authored-by: Karl Higley <kmhigley@gmail.com> Co-authored-by: arturomf94 <arturomf94@gmail.com> Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> Co-authored-by: Arshjot Singh Khehra <arshjot@live.com> Co-authored-by: Vova Manannikov <12518480+vvmnnnkv@users.noreply.github.com> Co-authored-by: Yugandhar Tripathi <tripathi.yugandhar@gmail.com> Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: midokura-silvia <43242089+midokura-silvia@users.noreply.github.com> Co-authored-by: Carlos Salgado <51087091+socd06@users.noreply.github.com> Co-authored-by: Ricardo Pretelt <32759855+ricardopretelt@users.noreply.github.com> Co-authored-by: Sourav Das <31799654+adventuroussrv@users.noreply.github.com> Co-authored-by: Aiden V Johnson <aiden.dataminer@gmail.com> Co-authored-by: José Benardi de Souza Nunes <benardinunes@gmail.com> Co-authored-by: Marcus Vinicius <marcus.costa@ccc.ufcg.edu.br> Co-authored-by: systemshift <42102034+systemshift@users.noreply.github.com> Co-authored-by: Bogdan Ivanyuk <35574157+KyloRen1@users.noreply.github.com> Co-authored-by: Urvashi Bhattacharyya <raheja@users.noreply.github.com> Co-authored-by: Jeferson Ferreira <jefersonnpn@gmail.com> Co-authored-by: João Lucas <joao.felipe@ccc.ufcg.edu.br> Co-authored-by: Márcio Porto <6521281+MarcioPorto@users.noreply.github.com> Co-authored-by: Héricles Emanuel <hericles.me@gmail.com> Co-authored-by: Izabella Antonino <izabella.queiroz@ccc.ufcg.edu.br> Co-authored-by: Bolarinwa Saheed Olayemi <refactormyself@users.noreply.github.com> Co-authored-by: Tom <32938439+TTitcombe@users.noreply.github.com> Co-authored-by: froessler <11539266+fdroessler@users.noreply.github.com> Co-authored-by: Lina Tran <10761918+linamnt@users.noreply.github.com> Co-authored-by: Ionésio Junior <ionesiojr@gmail.com> Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com> Co-authored-by: Sayantan Das <36279638+ucalyptus@users.noreply.github.com> Co-authored-by: Ayoub Benaissa <ayouben9@gmail.com>
gmuraru
added a commit
to gmuraru/PySyft
that referenced
this pull request
Mar 16, 2020
* Migrate CI tests to Github Actions (OpenMined#2936) * Create and test pysyft package on various Pythons Includes Python 2.7, 3.5, 3.6, 3.7 * Rework build script to run tests like Travis * Update Python versions to 3.6+ * Remove Python 3.8 since Torch doesn't support it * Only run the tutorial translation tests when the tutorials change The normal tutorial notebook tests still run on every build, which should catch the cases where changes to the code break the notebooks. When the main tutorial notebooks are changed, that will trigger the action that runs the notebook translation tests, which should surface the same failures in the whichever translated versions of the tutorials are affected. * Measure process time instead of wall time in efficiency tests This counts only 'user time' and excludes 'system time', and measures relative to the CPU clock speed, which should make the tests less flaky. * Increase `max_time` for `test_inv_sym` closer to original value * Rework "test all notebooks" tests to split off translation tests * Create a separate section in the action logs for coverage reports * Exclude translation tests from coverage checks * Bump `test_inv_sym` execution time limit to 30 * Omit `.github` directory from code coverage checking * Fix translation notebooks test to account for excluded notebooks * Move coverage exclusions into setup.cfg instead of .travis.yml This way they work everywhere that coverage runs, not just on Travis. * Omit the efficiency tests from Github Action check suite Omits them from both the tests and the test coverage checks * Unwind changes to the efficiency tests (now skipped in check suite) * Stop using efficiency test time assertions outside the efficiency tests * Standardize the name of the efficiency tests module * Fix efficiency assertion imports * Fix a missing wildcard in the coverage omission flag * Apply testing workflow to PR creation, synchronization, and reopening * Clean up coverage check inclusions and exclusions * Fix triggering events section * feat: Translation to Spanish of Part 08 - Introduction to Plans. (OpenMined#2962) Changes made by @socd06 This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Bump PyTorch to 1.4 (OpenMined#2930) Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> * Modify conv implementation and corresponding test to get exact match (OpenMined#2964) * Modify conv implementation and corresponding test to get exact match * Revert mkldnn to the original state after execution * Exorcise a Python `eval` and transmute into the lesser demon `getattr` (OpenMined#2951) * Exorcise a Python `eval` and transmute into the lesser demon `getattr` * Remove the double split for optimization * Point to correct version (OpenMined#2968) * Refactor Plan internals using PlaceHolder (OpenMined#2910) * Add experiment about Plans with placeholders * Add first implementation of plan using placeholders * Update notebook of test * Improve local execution of plans * Support remote execution of plans * Rm unused function * Use Operation object in plan * Remove fixed precision and sharing from Plan's multiple fetch test These seem extraneous to the point of the test anyway, so may as well go ahead and remove them rather than disabling this test. * Disable encrypted and fixed precision Plans * Remove unused `plan` attribute on `State` * Make `test_fetch_plan_multiple_times` pass * Fix typo in test name * Update serde_full unit tests (WIP) * Add example of Plan problem * Update serde unit tests * Fix output handling in plans * Fix nested Plans handling * Apply formatting to serde helpers * Adjust serde tests to match the new plan structure * Update requirements.txt to avoid a merge conflict * Undo some changes to the Torch hook * Fix Long vs Float issue in `PointerPlan` test * Remove current PromiseTensor * Remove plan.procedure * Add Trace object to clean code * Rm promise notebook * Add support for fix_precision & MPC * Clean plan code and add a tracer decorator * Clean state and rm useless functions * Clean the tracer decorator and comment code * Add comments to Plan & State new methods * Fix ordering of arguments in Plan * Fix dummy error and fix wraps decorators * Fix * Add a few comments * Merge input/output placeholders and do light cleaning * Rm useless experimental notebook * Fix serde * Increase delay in a time test Co-authored-by: Karl Higley <kmhigley@gmail.com> Co-authored-by: Vova Manannikov <12518480+vvmnnnkv@users.noreply.github.com> * add Part 13a Hindi (OpenMined#2958) Co-authored-by: Andrew Trask <liamtrask@gmail.com> * Bump version to `0.2.3.a1` (OpenMined#2976) * Refactored code of SplitNN model and added evaluation on datasets (OpenMined#2983) * Add Protobuf serialization for Placeholders, Plans, and States (OpenMined#2972) * Add Protobuf serialization for Placeholders, Plans, and States * Bump `syft-proto` requirement to `0.1.1.a1.post5` Co-authored-by: Andrew Trask <liamtrask@gmail.com> * Make it possible to JIT trace a PySyft Plan to convert to Torchscript (OpenMined#2988) * updated requirements.txt (OpenMined#2970) * updated requirements.txt * Update requirements.txt Co-authored-by: George-Cristian Muraru <murarugeorgec@gmail.com> * Translation to Spanish of Tutorial Part 12 bis (OpenMined#2979) feat: Translation to Spanish of Tutorial Part 12 bis - Encrypted Training on MNIST. This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 Co-authored-by: Karl Higley <kmhigley@gmail.com> * Remove codefix for old pytorch version (OpenMined#2999) * feat: Translation to Spanish of part 08 bis (OpenMined#3011) This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: Track better the changes in every document. Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Added Bengali Translated Notebooks (OpenMined#2992) Translated Notebooks: - Part 13a - Secure Classification with Syft Keras and TFE - Public Training.ipynb - Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb - Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Add Udacity Link to Readme (OpenMined#3016) * Add sphinx documentation with automatic API building (OpenMined#3017) * Add autoapi.extension to sphinx configuration * Add autoapi as dev dep. Add doc build to travis * Add doc testenv to github actions and travis * Fix formatting via black * Add readthedocs yml configuration * Configure sphinx to ignore "_static" * Update theme to sphinx-rtd-theme * Update index.rst Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> * Adding some notebooks translated to Portuguese (OpenMined#3015) * Adding Part 12 tutorial in portuguese * Adding Parte13a tutorial in portuguese Co-authored-by: Andrew Trask <liamtrask@gmail.com> * trigger github action notebook checks (OpenMined#3012) * use doubel qoutes instead * edit notebook just to trigger testing * add pull_request trigger condition * Revert "edit notebook just to trigger testing" This reverts commit 7e46d5b. * add path to pull_request * edit notebook again just to see if checks trigger * rename notebook .yml file * edit name string * change pull_request to master branch condition * master branch trigger condition * Revert "edit notebook again just to see if checks trigger" This reverts commit 60fee72. * small edit just to trigger notebook tests, AGAIN! * Revert "small edit just to trigger notebook tests, AGAIN!" This reverts commit 16a83d2. * Revert changes to `run-tests.yaml` This one is already working the way we want. * Make action workflows match Co-authored-by: Karl Higley <kmhigley@gmail.com> * Translation into Ukrainian language (OpenMined#2854) * Add files via upload * Update Part 01 - Основні засоби приватного глибинного навчання.ipynb Co-authored-by: Karl Higley <kmhigley@gmail.com> Co-authored-by: George-Cristian Muraru <murarugeorgec@gmail.com> Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> * Remove travis (OpenMined#3013) * delete travis.yml * remove travis from contributing.md * remoev travis from readme * add note for future contributers in test_notebooks * add github actions badge * edit typo * add notebook GH test badge Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * Hindi translations for Part 5, 6, 13b and 13c (OpenMined#2909) * Hindi translations for Part 5, 6, 13b and 13c * Markdown changes applied and github link added * Fixed spaces and translatations * Update Part 06 - Federated Learning on MNIST using a CNN.ipynb * Made changes on translations and spaces * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Updated Part 5 - moving Pygrid Module to PySyft * Hindi translations for Part 4, 7 and 8 updated * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 08 - Introduction to Plans.ipynb * Hindi translations - 4, 7, 8, 8 bis,9 and 10 * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * (minor fix) fix broked badge for run notebook tests (OpenMined#3028) * fix: failing tutorial translations should be ignored (OpenMined#3034) * Add Bengali Translated Notebooks #Part 12 bis (OpenMined#3038) Translated Notebooks: Part 12 bis - Encrypted Training on MNIST.ipynb * Add Bengali Translated Notebooks #Part 12 (OpenMined#3039) Translated Notebooks: Part 12 - Train an Encrypted Neural Network on Encrypted Data.ipynb * fix: failing tutorial translations should be ignored (OpenMined#3047) Exclude only the names, instead of full paths. This means it will effectively exclude translated titles too. * Translation to Portuguese of the tutorials Part 09, 10, and 11 (OpenMined#2980) * add translation to Portuguese of the tutorial Part 02 * (enhance) translate example toy expression literally * (minor fixes) change 'portugues' to 'português' and 'nível superior' to 'parte superior' * add translation to Portuguese of the tutorial Part 03 * (fix) remove old folder name * add translation to Portuguese of the tutorial Part 04 * (update) rename 'portugues' to 'português' and remove old folder * add translation to Portuguese of the tutorial Part 05 * add translation to Portuguese of the tutorial Part 06 * (minor fixes) typo and verb correction * add translation to Portuguese of the tutorial Part 09 * add translation to Portuguese of the tutorial Part 10 * add translation to Portuguese of the tutorial Part 11 * (fix) a cell has been removed by mistake 😑 * (minor fixes) 🤞 🤞 🤞 * (minor fix) full stop added * (fixes) remove repeated paragraphs left in tutorials part 06 and 11 Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: George-Cristian Muraru <murarugeorgec@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * Portuguese translations Part7-8-8bis (OpenMined#2977) * pt translation part07 * middle part 8 translation * fixing portuges * finishing translation part8 and part8 bis * fixing errors * finishing tutorial part 8bis * changing metadata kernel to python3 * update parte07-08 - fixing some errors * Update Parte 07 - Aprendizado Federado com Conjunto de Dados Federado.ipynb * Update Parte 08 - Introdução a Planos.ipynb Co-authored-by: Karl Higley <kmhigley@gmail.com> * Translated tutorial part 01 to Portuguese (OpenMined#3035) Co-authored-by: Karl Higley <kmhigley@gmail.com> * Translate PySyft Tutorials to Portuguese (OpenMined#3020) * Translate notebook Part 13b * translate notebook part 12 bis to portuguese * Update Parte 12 bis - Treinamento criptografado no MNIST.ipynb Co-authored-by: Karl Higley <kmhigley@gmail.com> * Portuguese translation 13c (OpenMined#3023) * translate 13c * translate 13c update * translate 13c update 2 * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb * Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb Co-authored-by: Karl Higley <kmhigley@gmail.com> * No need for requires_grad for input data (OpenMined#2998) * No need for requires_grad for input data * Increase error threshold Co-authored-by: Andrew Trask <liamtrask@gmail.com> * Put the test badges first in README (OpenMined#3046) * Put the test badges first * Update the names of test workflows to clean up README badges Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> Co-authored-by: Andrew Trask <liamtrask@gmail.com> * Add Bengali Translated Notebooks #Part 11 (OpenMined#3041) Translated Notebook: Part 11 - Secure Deep Learning Classification.ipynb Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * Add instruction for PySyft-Proto dependency and numbered the steps (OpenMined#2818) * Add instruction for PySyft-Proto dependency and numbered the steps * Remove Step 5, it is handled in requirement_dev.txt * Update requirements files to point to syft-proto in PyPI * Remove duplicate dependency (syft-proto) outside requirements.txt * replace proto with syft-proto * Cleanup PATE code (OpenMined#3033) * Format PATE docstrings Uniform indentation First pass of adding type hints Remove commented out TF flags code * Remove commented out print statements in PATE code * Change blank lines in PATE code No blank lines after function doc strings No blank lines immediately after control statements Break up larger chunks of code * Specify l and noise_eps when printing if cant compute sensitivity Before this commit, incorrect noise/moments spammed console with identical statements * Change torch/numpy sensitivity function type hints * Add notice required under Apache 2.0 license that changes have been made to code * Remove ellipses from List in type hints Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * update comment in test units (OpenMined#3018) * remove test unit * Add a missing comma that `black` complains about * add ignore test back and update comment. Co-authored-by: Karl Higley <kmhigley@gmail.com> * Fix potential medium/high severity security holes (OpenMined#3037) * remove security warning messages * Fix missing ast import * Flag the `eval` in `Plan` so that `bandit` security scan ignores it We do want to fix this and find a better way to do it, but don't want it to block adding security scans to, for example, catch future `eval`s added to the code base. * Black formatter wants two spaces before a comment Co-authored-by: Karl Higley <kmhigley@gmail.com> Co-authored-by: Andrew Trask <liamtrask@gmail.com> * CI security testing (OpenMined#3036) * add bandit as dev dependancy * add job for bandit * only report medium and high severity * Match format of security task name with the others Co-authored-by: Karl Higley <kmhigley@gmail.com> * Only test translation notebooks if they have been changed (OpenMined#3060) * Only test translated tutorials that have been changed * Add test_notebooks_basic_translations_diff for GHActions * Change GitHub Actions test for translations * Include git-diff file * Tutorial Notebook 10 note on numpy hook (OpenMined#3022) * added note about notebook not working * phrasing Co-authored-by: Karl Higley <kmhigley@gmail.com> * Revert "Only test translation notebooks if they have been changed (OpenMined#3060)" (OpenMined#3062) This reverts commit 91c8114. * Split tutorial translation tests into separate Github Workflows (OpenMined#3059) * Split tutorial translation tests into separate Github Workflows * Update the main tests workflow to exclude translation tests * Lower-case the directory name for Ukrainian translations * Add update-docker-image.yml to GitHub Actions workflow (OpenMined#3030) * Remove pointer tags/description decode instructions (OpenMined#3067) * Update GH checkout action to v2 (OpenMined#3068) * Fix trigger paths in translation test workflows (OpenMined#3065) * Fix trigger paths in translation test workflows * Update GH checkout action to v2 * Correct build context for dockerhub update via GitHub Actions (OpenMined#3066) * fix build context for dockerhub * Update GH checkout action to v2 Co-authored-by: Karl Higley <kmhigley@gmail.com> * Update msgpack version requirement (OpenMined#3073) * Reduce websocket timeout to one minute (OpenMined#3072) This is making CI tests hang ~forever. I think 278 hours is a bit excessive, don't you? :) * Add wildcards to the tutorial translation test actions (OpenMined#3071) * Translation to Spanish of Parts 09, 13 (a, b and c) (OpenMined#3032) * feat: Translation to Spanish of Parts 09, 13 (a, b and c). Credits to: @thefirebanks This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Fix typos in Spanish tutorials parts 9 and 13b Co-authored-by: Karl Higley <kmhigley@gmail.com> * Translations to Spanish of Tutorials Part 10 and 11 (OpenMined#3031) * feat: Translation to Spanish of Part 10 - Federated Learning with Secure Aggregation. This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * feat: Translation to Spanish of Part 11 - Secure Deep Learning Classification This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Fix typos in Spanish translations parts 10 and 11 Co-authored-by: Karl Higley <kmhigley@gmail.com> * Notebook 7 hindi translation (OpenMined#3055) * Hindi translations for Part 5, 6, 13b and 13c * Markdown changes applied and github link added * Fixed spaces and translatations * Update Part 06 - Federated Learning on MNIST using a CNN.ipynb * Made changes on translations and spaces * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Updated Part 5 - moving Pygrid Module to PySyft * Hindi translations for Part 4, 7 and 8 updated * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 08 - Introduction to Plans.ipynb * Hindi translations - 4, 7, 8, 8 bis,9 and 10 * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb * Notebook 7 hindi translation * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb * Notebook 7 - hindi translation Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: Karl Higley <kmhigley@gmail.com> * Bengali 6 (OpenMined#3052) * Bengali 6 Added * Fix markdown link syntax for OpenMined blog post Co-authored-by: Karl Higley <kmhigley@gmail.com> * Bengali 7 (OpenMined#3053) * Bengali 7 Added * Fix link to PySyft project issues Co-authored-by: Karl Higley <kmhigley@gmail.com> * Split the (English) tutorial tests into a separate Github Action (OpenMined#3075) * Split the (English) tutorial tests into a separate Github Action The tutorial tests take about 15 minutes to run, which is approximately equal to the running time of all the rest of the tests. Running them in parallel should speed up the PR checks. * Exclude the isolated file system test fixture from coverage checks It's only used by the notebook tests, which are run separately as of this pull request. * Run notebook tests on all PRs except translation tests Before this change, they would only run on PRs that changed notebooks. * Context of computation for crypten (OpenMined#2963) * New message type for crypten party initialization * Handle CryptenInit message by running local party * Define a new context of computation for crypten The new context of computation can run parties that are distributed across syft workers. The communication of crypten parties remains the same, however, syft workers handle initialization and the serialization of return values. * updated docs * testing crypten context * fix: was setting the bad env variable DISTRIBUTED_BACKEND should have been set and not BACKEND * test serde of CryptenInit message * add crypten as core deps, this should change to extra * delete useless comment * fix: list CryptenInit in OBJ_SIMPLIFIER_AND_DETAILERS * fix msgpack tests * run black * don't cover empty function * Use tag for crypten data (OpenMined#3003) Co-authored-by: Karl Higley <kmhigley@gmail.com> Co-authored-by: arturomf94 <arturomf94@gmail.com> Co-authored-by: Théo Ryffel <theo.leffyr@gmail.com> Co-authored-by: Arshjot Singh Khehra <arshjot@live.com> Co-authored-by: Vova Manannikov <12518480+vvmnnnkv@users.noreply.github.com> Co-authored-by: Yugandhar Tripathi <tripathi.yugandhar@gmail.com> Co-authored-by: Andrew Trask <liamtrask@gmail.com> Co-authored-by: midokura-silvia <43242089+midokura-silvia@users.noreply.github.com> Co-authored-by: Carlos Salgado <51087091+socd06@users.noreply.github.com> Co-authored-by: Ricardo Pretelt <32759855+ricardopretelt@users.noreply.github.com> Co-authored-by: Sourav Das <31799654+adventuroussrv@users.noreply.github.com> Co-authored-by: Aiden V Johnson <aiden.dataminer@gmail.com> Co-authored-by: José Benardi de Souza Nunes <benardinunes@gmail.com> Co-authored-by: Marcus Vinicius <marcus.costa@ccc.ufcg.edu.br> Co-authored-by: systemshift <42102034+systemshift@users.noreply.github.com> Co-authored-by: Bogdan Ivanyuk <35574157+KyloRen1@users.noreply.github.com> Co-authored-by: Urvashi Bhattacharyya <raheja@users.noreply.github.com> Co-authored-by: Jeferson Ferreira <jefersonnpn@gmail.com> Co-authored-by: João Lucas <joao.felipe@ccc.ufcg.edu.br> Co-authored-by: Márcio Porto <6521281+MarcioPorto@users.noreply.github.com> Co-authored-by: Héricles Emanuel <hericles.me@gmail.com> Co-authored-by: Izabella Antonino <izabella.queiroz@ccc.ufcg.edu.br> Co-authored-by: Bolarinwa Saheed Olayemi <refactormyself@users.noreply.github.com> Co-authored-by: Tom <32938439+TTitcombe@users.noreply.github.com> Co-authored-by: froessler <11539266+fdroessler@users.noreply.github.com> Co-authored-by: Lina Tran <10761918+linamnt@users.noreply.github.com> Co-authored-by: Ionésio Junior <ionesiojr@gmail.com> Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com> Co-authored-by: Sayantan Das <36279638+ucalyptus@users.noreply.github.com> Co-authored-by: Ayoub Benaissa <ayouben9@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
From a discussion with @karlhigley , now that the project has migrated to github actions, travis still takes a long time to run. Leading to errors on checks even if everything passes, which prevents us from merging good pull requests.