Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Http client for downloading files, pygrid requests #51

Merged
merged 11 commits into from
Mar 8, 2020

Conversation

vkkhare
Copy link
Member

@vkkhare vkkhare commented Feb 24, 2020

This PR is built upon the work of #50

@vkkhare vkkhare requested a review from mccorby February 24, 2020 01:13
@vkkhare vkkhare marked this pull request as ready for review March 3, 2020 20:25
@vkkhare vkkhare requested a review from mccorby March 3, 2020 20:25
@vkkhare
Copy link
Member Author

vkkhare commented Mar 3, 2020

removed the test files. Will need to be rewritten according to the subscriber pattern. I have made the socket api and retrofit api similar in use case. PR closes #46 and #18

Copy link
Member

@mccorby mccorby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of comments.

@vkkhare vkkhare requested a review from mccorby March 6, 2020 23:28
@mccorby
Copy link
Member

mccorby commented Mar 7, 2020

This PR is growing too much. I like having small PRs with closed functionality even if it's not the whole thing.
I would suggest to merge it as soon as we're happy with the status of the code and create issues for the missing parts.

val file = File(destination, fileName)
file.outputStream()
.use { fileName -> input.copyTo(fileName) }
val file = File(fileName)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can skip the val file

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

skipping it as in?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

        with(File(destination, fileName)) {
            outputStream().use { outputFile ->
                input.copyTo(outputFile)
            }
            emitter.onSuccess(absolutePath)
        }

@@ -9,7 +10,13 @@ import org.openmined.syft.threading.ProcessSchedulers

@ExperimentalUnsignedTypes
fun main() {
val schedulers = object : ProcessSchedulers {
val networkingSchedulers = object : ProcessSchedulers {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need two different sets of schedulers? If using just one, wouldn't it work as expected?

Copy link
Member Author

@vkkhare vkkhare Mar 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is we need different type of thread scheduling for computation and networking. We can spawn multiple threads for networking however compute usually would be restricted to single thread. Then many would require the subscriber to run on mainThread example for network response. While a single compute thread might be busy. This makes the distinction clear for the user and provides larger flexibility

@@ -0,0 +1,14 @@
package org.openmined.syft.processes

open class JobStatusSubscriber {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have the feeling that we are mixing a reactive approach with callbacks and creating what it seems to me unnecessary complexity.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for mixing this is we dont want the end user of our library to understand the entire reactive approach. Plus it goes with how the roadmap specifies things to be

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But JobStatusSubscriber is used below Syft. The client (the app) doesn't need to know about this, does it?
If we need to notify the upper layer about something from the job(s), shouldn't that go through Syft?
At least this is how I thought of the architecture of the library.

@vkkhare vkkhare requested a review from mccorby March 7, 2020 13:00
Copy link
Member

@mccorby mccorby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vkkhare vkkhare merged commit a025df4 into OpenMined:dev Mar 8, 2020
@vkkhare vkkhare deleted the http_client branch March 8, 2020 12:49
menzi101 pushed a commit to menzi101/KotlinSyft that referenced this pull request Jul 11, 2020
* scaffolding basic job scheduling

* job worker API implemented

* Added http client. Refactored common api usage

* Made socket client and http client uniform in usage

* WIP downloader

* added file saver

* rebase with dev

* updated standalone-demo

* added throwable for jobs and other minor changes

* added subscriber class for jobs,addressed comments

* code clean up
cereallarceny added a commit that referenced this pull request Jul 11, 2020
* Added publish script

* WebRTC first tests

* Improve expected value

* Added badges for CI and version

* Change badge version

* Draft with okHttp

* Socket with okhttp plus previous work on networking

Co-authored-by: Varun Khare  varunkhare1234@gmail.com

* Address review comments

* Addressed comments

* Fixing style guidelines (#25)

* added project style file

* reformat code

* Added roadmap objectives from openmined (#27)

* Check validity of protocol and timeout (#26)

* Check validity of protocol and timeout

* Added test. Addressed comments

* Changes with style

* Refactor using protocol and injecting signalling client

* Fix Bintray badge link (#38)

* Auto versioning with nebula (#22)

* Add dependency on PySyft protobuf classes (#45)

* refactoring strings from json data, setting up Job and syft API (#41)

* refactoring strings from json requests, setting up Job API

* add response handler and new API types

* added deserialization for API endpoints

* Kotlin serialization is love. Everything removed with polymorphic deserialization

* reformat code

* polymorphic serialization only for report. Implemented custom deserialization for the rest

* added unit tests

* Ci action build check (#48)

* Create android.yml

* added deploy script

* updated CI badge

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>

* Update github action to fire on pull request (#52)

* extracted dependency version to a separate file (#49)

* extracted library version to a separate file

* applied requested changes

* applied requested changes

* Minor typo changes (#55)

* Http client for downloading files, pygrid requests (#51)

* scaffolding basic job scheduling

* job worker API implemented

* Added http client. Refactored common api usage

* Made socket client and http client uniform in usage

* WIP downloader

* added file saver

* rebase with dev

* updated standalone-demo

* added throwable for jobs and other minor changes

* added subscriber class for jobs,addressed comments

* code clean up

* updated federated learning network API (#60)

* updated federated learning network API

* added status field in authentication

* fixed bug data serialization

* added client config module

* remove version field

* no version in cycle response from pygrid

* Protobuf infrastructure (#54)

* Merge from upstream

* Unbuffer plan operations

* Deserialise script module

* Remove not-need lib

* Wee refactor

* Remove rogue activity. It's not the time yet

* Restore deleted files

* Wire up MNIST

* Added vm

* Remove long files

* Working model

* Refactor with repository pattern

* Wiring up

* Addressed comments

* Demo app (#63)

* added pygrid integration demo and logging

* debugging pygrid integration

* Wip streaming downloads don't work but direct succeeds

* updated syftlib to have new job status callbacks

* cleaning up code

* Syft instance (#65)

* auto initialise communication client

* added HTTP protocol and made snetworking clients to be modifiable

* added test for socket client initialization

* reformat code

* cleared duplicate http protocol

* updated demo app to use new syft worker

* added tensor serialization (#68)

* added tensor serialization

* removed lateinit initialization, SyftModel integeration into job

* reformat code

* cleared duplicate http protocol

* addressed comments and rebasing

* remove message processor test

* removed pytorch tensor wrapper since we wont perform computation in kotlin

* added syftmodel test

* updated order of schedulers

* Wire up cycle with data training and process (#67)

* Draft

* Logger

* Attempt to read mnist script

* Almost there

* Added logger

* Almost there

* Fix data loading

* Move out zip

* And it is working

* More wiring

* Show loss in chart

* Pass floats directly

* made UI responsive (#73)

* updated syft-proto (#77)

* updated syft-proto

* removed syft-proto from jetifier

* Add demo steps to README (#76)

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* added batch support, UI tweaks (#74)

* added batch support, UI tweaks

* addressed comments Added real mnist training data

* added training data to git lfs

* code clean up

* remove lfs and made loading batches clear

* updated readme

* updated pytorch (1.5.0) and gradle version

* Docs protobuf serialization (#80)

* refactor readme (#86)

* Docs Plan Execution (#85)

* plan documentation

* changes

* bug

* minor changes

* fixes

* speed test functional

speed test error handling

correct syft tests

created netowrk state manager

* Syft config (#89)

* added configuration parameter

* added syft config and device monitor

* refactor datasource for device status

* added syft status subscriber

* made class constructors internal for testing support

* register callbacks for broadcasts

* updated demo app

* corrected syft test

* network cache bug cleared

* correct default schedulers

* Test (#92)

* initiated testing files and added tests for JobStatusSubscriber

* added socket client test

* updated readme (#95)

* updated readme

* Update README.md

Co-authored-by: Patrick Cason <me@patrickcason.com>

* docs: add mccorby as a contributor (#181)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#180)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add cereallarceny as a contributor (#182)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add codeboy5 as a contributor (#183)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#185)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#186)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add cereallarceny as a contributor (#187)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#188)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#189)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#190)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* added test coverage and moved tests to a new action (#179)

* added test coverage and moved tests to a new action

* updated readme badge

Co-authored-by: Patrick Cason <me@patrickcason.com>

* NDK cache in CI build (#191)

* added test coverage and moved tests to a new action

* added ndk caching for faster builds

* tests badge rename

* corrected url space

* corrected badge position and ndk cache permissions (#194)

* added network monitoring (#195)

* added network monitoring

* added device state checks

* removed redundant JobStatusMessage.JobError

* corrected demo app call to stop execution on fail

* disposing jobs functional

* updated demo app to accomodate dispose

* corrected tests

* added unit tests for device monitor

Co-authored-by: Patrick Cason <me@patrickcason.com>

* Docs Update http-client (#193)

* Clean mnist (#199)

* cleaning mnist

* corrected syft worker disposal

* updated readme

* made login UI full screen

* updated readme for sleep/wake support

* Charge detection (#200)

* added battery listener

* solved battery intent action bug

* corrected device monitor tests

* added robolectric tests

* remove usage of junit5 engine

* Dont skip ci to get code coverage (#203)

* docs: add galalen as a contributor (#202)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add erksch as a contributor (#205)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Documentation for SocketAPI, SocketClient, and SyftWebSocket (#201)

* Documentation for SocketAPI, SocketClient, and SyftWebSocket

* add links and additional info to documentation

* follow KDoc standard in documenting SyftWebSocket constructor params

* fix README typo

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Docs HttpAPI interface (#192)

* Docs HttpAPI

* update

* update

* feature testing (#207)

* complete run through added

* reformat

* made auth token optional

* correct handling multiple disposals

* correct singleton syft worker creation

* WIP adding tests

* constraint validation added

* correct looper robolectric

* multiple jobs tested

* add disposing in test to clear worker

* SyftJob tests (#209)

* Added first tests. Refactor SyftJob

* First tests for JobDownloader

* Added conditions to download

* Download using CycleAccept response data

* Make requestKey available to other parts of the workflow

* Test report

* Improved test

* Added test for throwError

* JobDownloader test

* Refactor downloader to use data sources and add tests

* Restrict visibility of data sources to internal

* Tests for jobDownloader

* Tests for RemoteDataSource

* Remove unused method

* Remove duplicated logic

* Pass the correct object to remote data source

* Rename and place classes

* Fix failing test

* Remove activity monitoring and API clean up (#211)

* activity detection removed

* syft job docs, restricted public APIs

* describe multi threading support (#210)

* Test for jobLocalDataSource. Refactor IO (#212)

* Tests for SyftTensor (#213)

* Test for SyftTensor deserialisation

* Fix recursive calls

* Refactor deserialize name

* Pytorch Tensor to SyftTensor test

* Cover content types

* Tests for serialize and getTorchTensor

* Model tests

* Check size of new params

* Report diff (#217)

* diff reporting added

* update running mutliple cycles

* remove open tag from configuration

* correct tests

* added tests for files

* addressing comments

* refactor asset reading

* remove tests for createDiff of repository

* added new tests for jobRepository and jobLocalDataSource

* added jobLocaldatasource test

* cosmetic change

* Speed tests optional (#219)

* Speed tests optional

* Introduce Either to handle failure when checking conditions

* No need for requiresSpeedTest to be lateinit

* Add feature flag to allow execution while the server is updated

* Remove ignored files

* Too many deletions

* Update syft-proto version

* Property for buildConfig

* Revert to 0.4.1

* correct version of SyftProto

* Remove feature flags as too many issues with integration tests

* Remove unused methods

* Small changes

* Failing test

* Leave only in use methods

* Background demo (#220)

* first draft of background execution

* add UI

* removed boilerplate code

* update syft proto dependency

* refactor paarameter mismatch

* correct default device monitoring status

* code cleanup

* work manager logging

* corrected tests

* remove unsubscribe for background tasks

* refactoring WIP

* job worker functional

* refactor attachmnistlogger

* added tests for diff coverage

* refactor view models

* rebase

* empty commit for codecov

* docs: add mccorby as a contributor (#223)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#224)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* restrict file scope (#227)

* documentation for JobStatusSubscriber (#229)

* documentation for JobStatusSubscriber

* add description of the class

* correct syft model doc

* adding tests for codecov

* Add dokka and docs dir (#225)

* Add dokka and docs dir

* Remove docs in wrong place

* Docs to root

* Documentatino workflow

* GH format

* Doc public only/

* Add docs

* Update docs and go

* Fix wrong directory

* Internals missing

* docs: add galalen as a contributor (#231)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#232)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* General config (#233)

* added variable client config

* refactor plan

* regenerate the docs

* document Client Config

* correct doc references

* regenerate docs

* add tests for SyftModel

* add some more tests

* corrected http handler and added jwt token (#234)

* corrected http handler and added jwt token

* added plan name

* updated tests

* update docs

* single socket per job

* plan updated

* demo working

* added small database

* updated tests

* remove changes in network and add params tester

* README Change

Co-authored-by: varun khare <varunkhare1234@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>
vkkhare added a commit that referenced this pull request Jul 23, 2020
* Preparation for release 0.1.0 (#236)

* Added publish script

* WebRTC first tests

* Improve expected value

* Added badges for CI and version

* Change badge version

* Draft with okHttp

* Socket with okhttp plus previous work on networking

Co-authored-by: Varun Khare  varunkhare1234@gmail.com

* Address review comments

* Addressed comments

* Fixing style guidelines (#25)

* added project style file

* reformat code

* Added roadmap objectives from openmined (#27)

* Check validity of protocol and timeout (#26)

* Check validity of protocol and timeout

* Added test. Addressed comments

* Changes with style

* Refactor using protocol and injecting signalling client

* Fix Bintray badge link (#38)

* Auto versioning with nebula (#22)

* Add dependency on PySyft protobuf classes (#45)

* refactoring strings from json data, setting up Job and syft API (#41)

* refactoring strings from json requests, setting up Job API

* add response handler and new API types

* added deserialization for API endpoints

* Kotlin serialization is love. Everything removed with polymorphic deserialization

* reformat code

* polymorphic serialization only for report. Implemented custom deserialization for the rest

* added unit tests

* Ci action build check (#48)

* Create android.yml

* added deploy script

* updated CI badge

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>

* Update github action to fire on pull request (#52)

* extracted dependency version to a separate file (#49)

* extracted library version to a separate file

* applied requested changes

* applied requested changes

* Minor typo changes (#55)

* Http client for downloading files, pygrid requests (#51)

* scaffolding basic job scheduling

* job worker API implemented

* Added http client. Refactored common api usage

* Made socket client and http client uniform in usage

* WIP downloader

* added file saver

* rebase with dev

* updated standalone-demo

* added throwable for jobs and other minor changes

* added subscriber class for jobs,addressed comments

* code clean up

* updated federated learning network API (#60)

* updated federated learning network API

* added status field in authentication

* fixed bug data serialization

* added client config module

* remove version field

* no version in cycle response from pygrid

* Protobuf infrastructure (#54)

* Merge from upstream

* Unbuffer plan operations

* Deserialise script module

* Remove not-need lib

* Wee refactor

* Remove rogue activity. It's not the time yet

* Restore deleted files

* Wire up MNIST

* Added vm

* Remove long files

* Working model

* Refactor with repository pattern

* Wiring up

* Addressed comments

* Demo app (#63)

* added pygrid integration demo and logging

* debugging pygrid integration

* Wip streaming downloads don't work but direct succeeds

* updated syftlib to have new job status callbacks

* cleaning up code

* Syft instance (#65)

* auto initialise communication client

* added HTTP protocol and made snetworking clients to be modifiable

* added test for socket client initialization

* reformat code

* cleared duplicate http protocol

* updated demo app to use new syft worker

* added tensor serialization (#68)

* added tensor serialization

* removed lateinit initialization, SyftModel integeration into job

* reformat code

* cleared duplicate http protocol

* addressed comments and rebasing

* remove message processor test

* removed pytorch tensor wrapper since we wont perform computation in kotlin

* added syftmodel test

* updated order of schedulers

* Wire up cycle with data training and process (#67)

* Draft

* Logger

* Attempt to read mnist script

* Almost there

* Added logger

* Almost there

* Fix data loading

* Move out zip

* And it is working

* More wiring

* Show loss in chart

* Pass floats directly

* made UI responsive (#73)

* updated syft-proto (#77)

* updated syft-proto

* removed syft-proto from jetifier

* Add demo steps to README (#76)

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* added batch support, UI tweaks (#74)

* added batch support, UI tweaks

* addressed comments Added real mnist training data

* added training data to git lfs

* code clean up

* remove lfs and made loading batches clear

* updated readme

* updated pytorch (1.5.0) and gradle version

* Docs protobuf serialization (#80)

* refactor readme (#86)

* Docs Plan Execution (#85)

* plan documentation

* changes

* bug

* minor changes

* fixes

* speed test functional

speed test error handling

correct syft tests

created netowrk state manager

* Syft config (#89)

* added configuration parameter

* added syft config and device monitor

* refactor datasource for device status

* added syft status subscriber

* made class constructors internal for testing support

* register callbacks for broadcasts

* updated demo app

* corrected syft test

* network cache bug cleared

* correct default schedulers

* Test (#92)

* initiated testing files and added tests for JobStatusSubscriber

* added socket client test

* updated readme (#95)

* updated readme

* Update README.md

Co-authored-by: Patrick Cason <me@patrickcason.com>

* docs: add mccorby as a contributor (#181)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#180)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add cereallarceny as a contributor (#182)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add codeboy5 as a contributor (#183)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#185)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#186)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add cereallarceny as a contributor (#187)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#188)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#189)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#190)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* added test coverage and moved tests to a new action (#179)

* added test coverage and moved tests to a new action

* updated readme badge

Co-authored-by: Patrick Cason <me@patrickcason.com>

* NDK cache in CI build (#191)

* added test coverage and moved tests to a new action

* added ndk caching for faster builds

* tests badge rename

* corrected url space

* corrected badge position and ndk cache permissions (#194)

* added network monitoring (#195)

* added network monitoring

* added device state checks

* removed redundant JobStatusMessage.JobError

* corrected demo app call to stop execution on fail

* disposing jobs functional

* updated demo app to accomodate dispose

* corrected tests

* added unit tests for device monitor

Co-authored-by: Patrick Cason <me@patrickcason.com>

* Docs Update http-client (#193)

* Clean mnist (#199)

* cleaning mnist

* corrected syft worker disposal

* updated readme

* made login UI full screen

* updated readme for sleep/wake support

* Charge detection (#200)

* added battery listener

* solved battery intent action bug

* corrected device monitor tests

* added robolectric tests

* remove usage of junit5 engine

* Dont skip ci to get code coverage (#203)

* docs: add galalen as a contributor (#202)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add erksch as a contributor (#205)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Documentation for SocketAPI, SocketClient, and SyftWebSocket (#201)

* Documentation for SocketAPI, SocketClient, and SyftWebSocket

* add links and additional info to documentation

* follow KDoc standard in documenting SyftWebSocket constructor params

* fix README typo

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Docs HttpAPI interface (#192)

* Docs HttpAPI

* update

* update

* feature testing (#207)

* complete run through added

* reformat

* made auth token optional

* correct handling multiple disposals

* correct singleton syft worker creation

* WIP adding tests

* constraint validation added

* correct looper robolectric

* multiple jobs tested

* add disposing in test to clear worker

* SyftJob tests (#209)

* Added first tests. Refactor SyftJob

* First tests for JobDownloader

* Added conditions to download

* Download using CycleAccept response data

* Make requestKey available to other parts of the workflow

* Test report

* Improved test

* Added test for throwError

* JobDownloader test

* Refactor downloader to use data sources and add tests

* Restrict visibility of data sources to internal

* Tests for jobDownloader

* Tests for RemoteDataSource

* Remove unused method

* Remove duplicated logic

* Pass the correct object to remote data source

* Rename and place classes

* Fix failing test

* Remove activity monitoring and API clean up (#211)

* activity detection removed

* syft job docs, restricted public APIs

* describe multi threading support (#210)

* Test for jobLocalDataSource. Refactor IO (#212)

* Tests for SyftTensor (#213)

* Test for SyftTensor deserialisation

* Fix recursive calls

* Refactor deserialize name

* Pytorch Tensor to SyftTensor test

* Cover content types

* Tests for serialize and getTorchTensor

* Model tests

* Check size of new params

* Report diff (#217)

* diff reporting added

* update running mutliple cycles

* remove open tag from configuration

* correct tests

* added tests for files

* addressing comments

* refactor asset reading

* remove tests for createDiff of repository

* added new tests for jobRepository and jobLocalDataSource

* added jobLocaldatasource test

* cosmetic change

* Speed tests optional (#219)

* Speed tests optional

* Introduce Either to handle failure when checking conditions

* No need for requiresSpeedTest to be lateinit

* Add feature flag to allow execution while the server is updated

* Remove ignored files

* Too many deletions

* Update syft-proto version

* Property for buildConfig

* Revert to 0.4.1

* correct version of SyftProto

* Remove feature flags as too many issues with integration tests

* Remove unused methods

* Small changes

* Failing test

* Leave only in use methods

* Background demo (#220)

* first draft of background execution

* add UI

* removed boilerplate code

* update syft proto dependency

* refactor paarameter mismatch

* correct default device monitoring status

* code cleanup

* work manager logging

* corrected tests

* remove unsubscribe for background tasks

* refactoring WIP

* job worker functional

* refactor attachmnistlogger

* added tests for diff coverage

* refactor view models

* rebase

* empty commit for codecov

* docs: add mccorby as a contributor (#223)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#224)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* restrict file scope (#227)

* documentation for JobStatusSubscriber (#229)

* documentation for JobStatusSubscriber

* add description of the class

* correct syft model doc

* adding tests for codecov

* Add dokka and docs dir (#225)

* Add dokka and docs dir

* Remove docs in wrong place

* Docs to root

* Documentatino workflow

* GH format

* Doc public only/

* Add docs

* Update docs and go

* Fix wrong directory

* Internals missing

* docs: add galalen as a contributor (#231)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#232)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* General config (#233)

* added variable client config

* refactor plan

* regenerate the docs

* document Client Config

* correct doc references

* regenerate docs

* add tests for SyftModel

* add some more tests

* corrected http handler and added jwt token (#234)

* corrected http handler and added jwt token

* added plan name

* updated tests

* update docs

* single socket per job

* plan updated

* demo working

* added small database

* updated tests

* remove changes in network and add params tester

* README Change

Co-authored-by: varun khare <varunkhare1234@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>

* update gif (#237) (#243)

* update gif

* add installation instructions

* update readme language

* add slack badge

* Update README.md

* Update README.md

* publish action should fire only on release tags

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>
vkkhare added a commit that referenced this pull request Jul 24, 2020
* Preparation for release 0.1.0 (#236)

* Added publish script

* WebRTC first tests

* Improve expected value

* Added badges for CI and version

* Change badge version

* Draft with okHttp

* Socket with okhttp plus previous work on networking

Co-authored-by: Varun Khare  varunkhare1234@gmail.com

* Address review comments

* Addressed comments

* Fixing style guidelines (#25)

* added project style file

* reformat code

* Added roadmap objectives from openmined (#27)

* Check validity of protocol and timeout (#26)

* Check validity of protocol and timeout

* Added test. Addressed comments

* Changes with style

* Refactor using protocol and injecting signalling client

* Fix Bintray badge link (#38)

* Auto versioning with nebula (#22)

* Add dependency on PySyft protobuf classes (#45)

* refactoring strings from json data, setting up Job and syft API (#41)

* refactoring strings from json requests, setting up Job API

* add response handler and new API types

* added deserialization for API endpoints

* Kotlin serialization is love. Everything removed with polymorphic deserialization

* reformat code

* polymorphic serialization only for report. Implemented custom deserialization for the rest

* added unit tests

* Ci action build check (#48)

* Create android.yml

* added deploy script

* updated CI badge

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>

* Update github action to fire on pull request (#52)

* extracted dependency version to a separate file (#49)

* extracted library version to a separate file

* applied requested changes

* applied requested changes

* Minor typo changes (#55)

* Http client for downloading files, pygrid requests (#51)

* scaffolding basic job scheduling

* job worker API implemented

* Added http client. Refactored common api usage

* Made socket client and http client uniform in usage

* WIP downloader

* added file saver

* rebase with dev

* updated standalone-demo

* added throwable for jobs and other minor changes

* added subscriber class for jobs,addressed comments

* code clean up

* updated federated learning network API (#60)

* updated federated learning network API

* added status field in authentication

* fixed bug data serialization

* added client config module

* remove version field

* no version in cycle response from pygrid

* Protobuf infrastructure (#54)

* Merge from upstream

* Unbuffer plan operations

* Deserialise script module

* Remove not-need lib

* Wee refactor

* Remove rogue activity. It's not the time yet

* Restore deleted files

* Wire up MNIST

* Added vm

* Remove long files

* Working model

* Refactor with repository pattern

* Wiring up

* Addressed comments

* Demo app (#63)

* added pygrid integration demo and logging

* debugging pygrid integration

* Wip streaming downloads don't work but direct succeeds

* updated syftlib to have new job status callbacks

* cleaning up code

* Syft instance (#65)

* auto initialise communication client

* added HTTP protocol and made snetworking clients to be modifiable

* added test for socket client initialization

* reformat code

* cleared duplicate http protocol

* updated demo app to use new syft worker

* added tensor serialization (#68)

* added tensor serialization

* removed lateinit initialization, SyftModel integeration into job

* reformat code

* cleared duplicate http protocol

* addressed comments and rebasing

* remove message processor test

* removed pytorch tensor wrapper since we wont perform computation in kotlin

* added syftmodel test

* updated order of schedulers

* Wire up cycle with data training and process (#67)

* Draft

* Logger

* Attempt to read mnist script

* Almost there

* Added logger

* Almost there

* Fix data loading

* Move out zip

* And it is working

* More wiring

* Show loss in chart

* Pass floats directly

* made UI responsive (#73)

* updated syft-proto (#77)

* updated syft-proto

* removed syft-proto from jetifier

* Add demo steps to README (#76)

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* added batch support, UI tweaks (#74)

* added batch support, UI tweaks

* addressed comments Added real mnist training data

* added training data to git lfs

* code clean up

* remove lfs and made loading batches clear

* updated readme

* updated pytorch (1.5.0) and gradle version

* Docs protobuf serialization (#80)

* refactor readme (#86)

* Docs Plan Execution (#85)

* plan documentation

* changes

* bug

* minor changes

* fixes

* speed test functional

speed test error handling

correct syft tests

created netowrk state manager

* Syft config (#89)

* added configuration parameter

* added syft config and device monitor

* refactor datasource for device status

* added syft status subscriber

* made class constructors internal for testing support

* register callbacks for broadcasts

* updated demo app

* corrected syft test

* network cache bug cleared

* correct default schedulers

* Test (#92)

* initiated testing files and added tests for JobStatusSubscriber

* added socket client test

* updated readme (#95)

* updated readme

* Update README.md

Co-authored-by: Patrick Cason <me@patrickcason.com>

* docs: add mccorby as a contributor (#181)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#180)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add cereallarceny as a contributor (#182)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add codeboy5 as a contributor (#183)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#185)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#186)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add cereallarceny as a contributor (#187)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#188)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#189)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#190)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* added test coverage and moved tests to a new action (#179)

* added test coverage and moved tests to a new action

* updated readme badge

Co-authored-by: Patrick Cason <me@patrickcason.com>

* NDK cache in CI build (#191)

* added test coverage and moved tests to a new action

* added ndk caching for faster builds

* tests badge rename

* corrected url space

* corrected badge position and ndk cache permissions (#194)

* added network monitoring (#195)

* added network monitoring

* added device state checks

* removed redundant JobStatusMessage.JobError

* corrected demo app call to stop execution on fail

* disposing jobs functional

* updated demo app to accomodate dispose

* corrected tests

* added unit tests for device monitor

Co-authored-by: Patrick Cason <me@patrickcason.com>

* Docs Update http-client (#193)

* Clean mnist (#199)

* cleaning mnist

* corrected syft worker disposal

* updated readme

* made login UI full screen

* updated readme for sleep/wake support

* Charge detection (#200)

* added battery listener

* solved battery intent action bug

* corrected device monitor tests

* added robolectric tests

* remove usage of junit5 engine

* Dont skip ci to get code coverage (#203)

* docs: add galalen as a contributor (#202)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add erksch as a contributor (#205)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Documentation for SocketAPI, SocketClient, and SyftWebSocket (#201)

* Documentation for SocketAPI, SocketClient, and SyftWebSocket

* add links and additional info to documentation

* follow KDoc standard in documenting SyftWebSocket constructor params

* fix README typo

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Docs HttpAPI interface (#192)

* Docs HttpAPI

* update

* update

* feature testing (#207)

* complete run through added

* reformat

* made auth token optional

* correct handling multiple disposals

* correct singleton syft worker creation

* WIP adding tests

* constraint validation added

* correct looper robolectric

* multiple jobs tested

* add disposing in test to clear worker

* SyftJob tests (#209)

* Added first tests. Refactor SyftJob

* First tests for JobDownloader

* Added conditions to download

* Download using CycleAccept response data

* Make requestKey available to other parts of the workflow

* Test report

* Improved test

* Added test for throwError

* JobDownloader test

* Refactor downloader to use data sources and add tests

* Restrict visibility of data sources to internal

* Tests for jobDownloader

* Tests for RemoteDataSource

* Remove unused method

* Remove duplicated logic

* Pass the correct object to remote data source

* Rename and place classes

* Fix failing test

* Remove activity monitoring and API clean up (#211)

* activity detection removed

* syft job docs, restricted public APIs

* describe multi threading support (#210)

* Test for jobLocalDataSource. Refactor IO (#212)

* Tests for SyftTensor (#213)

* Test for SyftTensor deserialisation

* Fix recursive calls

* Refactor deserialize name

* Pytorch Tensor to SyftTensor test

* Cover content types

* Tests for serialize and getTorchTensor

* Model tests

* Check size of new params

* Report diff (#217)

* diff reporting added

* update running mutliple cycles

* remove open tag from configuration

* correct tests

* added tests for files

* addressing comments

* refactor asset reading

* remove tests for createDiff of repository

* added new tests for jobRepository and jobLocalDataSource

* added jobLocaldatasource test

* cosmetic change

* Speed tests optional (#219)

* Speed tests optional

* Introduce Either to handle failure when checking conditions

* No need for requiresSpeedTest to be lateinit

* Add feature flag to allow execution while the server is updated

* Remove ignored files

* Too many deletions

* Update syft-proto version

* Property for buildConfig

* Revert to 0.4.1

* correct version of SyftProto

* Remove feature flags as too many issues with integration tests

* Remove unused methods

* Small changes

* Failing test

* Leave only in use methods

* Background demo (#220)

* first draft of background execution

* add UI

* removed boilerplate code

* update syft proto dependency

* refactor paarameter mismatch

* correct default device monitoring status

* code cleanup

* work manager logging

* corrected tests

* remove unsubscribe for background tasks

* refactoring WIP

* job worker functional

* refactor attachmnistlogger

* added tests for diff coverage

* refactor view models

* rebase

* empty commit for codecov

* docs: add mccorby as a contributor (#223)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#224)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* restrict file scope (#227)

* documentation for JobStatusSubscriber (#229)

* documentation for JobStatusSubscriber

* add description of the class

* correct syft model doc

* adding tests for codecov

* Add dokka and docs dir (#225)

* Add dokka and docs dir

* Remove docs in wrong place

* Docs to root

* Documentatino workflow

* GH format

* Doc public only/

* Add docs

* Update docs and go

* Fix wrong directory

* Internals missing

* docs: add galalen as a contributor (#231)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#232)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* General config (#233)

* added variable client config

* refactor plan

* regenerate the docs

* document Client Config

* correct doc references

* regenerate docs

* add tests for SyftModel

* add some more tests

* corrected http handler and added jwt token (#234)

* corrected http handler and added jwt token

* added plan name

* updated tests

* update docs

* single socket per job

* plan updated

* demo working

* added small database

* updated tests

* remove changes in network and add params tester

* README Change

Co-authored-by: varun khare <varunkhare1234@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>

* update gif (#237) (#243)

* update gif

* add installation instructions

* update readme language

* add slack badge

* Update README.md

* Update README.md

* publish action should fire only on release tags

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>
vkkhare added a commit to vkkhare/KotlinSyft that referenced this pull request Jul 30, 2020
* Added publish script

* WebRTC first tests

* Improve expected value

* Added badges for CI and version

* Change badge version

* Draft with okHttp

* Socket with okhttp plus previous work on networking

Co-authored-by: Varun Khare  varunkhare1234@gmail.com

* Address review comments

* Addressed comments

* Fixing style guidelines (OpenMined#25)

* added project style file

* reformat code

* Added roadmap objectives from openmined (OpenMined#27)

* Check validity of protocol and timeout (OpenMined#26)

* Check validity of protocol and timeout

* Added test. Addressed comments

* Changes with style

* Refactor using protocol and injecting signalling client

* Fix Bintray badge link (OpenMined#38)

* Auto versioning with nebula (OpenMined#22)

* Add dependency on PySyft protobuf classes (OpenMined#45)

* refactoring strings from json data, setting up Job and syft API (OpenMined#41)

* refactoring strings from json requests, setting up Job API

* add response handler and new API types

* added deserialization for API endpoints

* Kotlin serialization is love. Everything removed with polymorphic deserialization

* reformat code

* polymorphic serialization only for report. Implemented custom deserialization for the rest

* added unit tests

* Ci action build check (OpenMined#48)

* Create android.yml

* added deploy script

* updated CI badge

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>

* Update github action to fire on pull request (OpenMined#52)

* extracted dependency version to a separate file (OpenMined#49)

* extracted library version to a separate file

* applied requested changes

* applied requested changes

* Minor typo changes (OpenMined#55)

* Http client for downloading files, pygrid requests (OpenMined#51)

* scaffolding basic job scheduling

* job worker API implemented

* Added http client. Refactored common api usage

* Made socket client and http client uniform in usage

* WIP downloader

* added file saver

* rebase with dev

* updated standalone-demo

* added throwable for jobs and other minor changes

* added subscriber class for jobs,addressed comments

* code clean up

* updated federated learning network API (OpenMined#60)

* updated federated learning network API

* added status field in authentication

* fixed bug data serialization

* added client config module

* remove version field

* no version in cycle response from pygrid

* Protobuf infrastructure (OpenMined#54)

* Merge from upstream

* Unbuffer plan operations

* Deserialise script module

* Remove not-need lib

* Wee refactor

* Remove rogue activity. It's not the time yet

* Restore deleted files

* Wire up MNIST

* Added vm

* Remove long files

* Working model

* Refactor with repository pattern

* Wiring up

* Addressed comments

* Demo app (OpenMined#63)

* added pygrid integration demo and logging

* debugging pygrid integration

* Wip streaming downloads don't work but direct succeeds

* updated syftlib to have new job status callbacks

* cleaning up code

* Syft instance (OpenMined#65)

* auto initialise communication client

* added HTTP protocol and made snetworking clients to be modifiable

* added test for socket client initialization

* reformat code

* cleared duplicate http protocol

* updated demo app to use new syft worker

* added tensor serialization (OpenMined#68)

* added tensor serialization

* removed lateinit initialization, SyftModel integeration into job

* reformat code

* cleared duplicate http protocol

* addressed comments and rebasing

* remove message processor test

* removed pytorch tensor wrapper since we wont perform computation in kotlin

* added syftmodel test

* updated order of schedulers

* Wire up cycle with data training and process (OpenMined#67)

* Draft

* Logger

* Attempt to read mnist script

* Almost there

* Added logger

* Almost there

* Fix data loading

* Move out zip

* And it is working

* More wiring

* Show loss in chart

* Pass floats directly

* made UI responsive (OpenMined#73)

* updated syft-proto (OpenMined#77)

* updated syft-proto

* removed syft-proto from jetifier

* Add demo steps to README (OpenMined#76)

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* added batch support, UI tweaks (OpenMined#74)

* added batch support, UI tweaks

* addressed comments Added real mnist training data

* added training data to git lfs

* code clean up

* remove lfs and made loading batches clear

* updated readme

* updated pytorch (1.5.0) and gradle version

* Docs protobuf serialization (OpenMined#80)

* refactor readme (OpenMined#86)

* Docs Plan Execution (OpenMined#85)

* plan documentation

* changes

* bug

* minor changes

* fixes

* speed test functional

speed test error handling

correct syft tests

created netowrk state manager

* Syft config (OpenMined#89)

* added configuration parameter

* added syft config and device monitor

* refactor datasource for device status

* added syft status subscriber

* made class constructors internal for testing support

* register callbacks for broadcasts

* updated demo app

* corrected syft test

* network cache bug cleared

* correct default schedulers

* Test (OpenMined#92)

* initiated testing files and added tests for JobStatusSubscriber

* added socket client test

* updated readme (OpenMined#95)

* updated readme

* Update README.md

Co-authored-by: Patrick Cason <me@patrickcason.com>

* docs: add mccorby as a contributor (OpenMined#181)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (OpenMined#180)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add cereallarceny as a contributor (OpenMined#182)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add codeboy5 as a contributor (OpenMined#183)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (OpenMined#185)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (OpenMined#186)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add cereallarceny as a contributor (OpenMined#187)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (OpenMined#188)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (OpenMined#189)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (OpenMined#190)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* added test coverage and moved tests to a new action (OpenMined#179)

* added test coverage and moved tests to a new action

* updated readme badge

Co-authored-by: Patrick Cason <me@patrickcason.com>

* NDK cache in CI build (OpenMined#191)

* added test coverage and moved tests to a new action

* added ndk caching for faster builds

* tests badge rename

* corrected url space

* corrected badge position and ndk cache permissions (OpenMined#194)

* added network monitoring (OpenMined#195)

* added network monitoring

* added device state checks

* removed redundant JobStatusMessage.JobError

* corrected demo app call to stop execution on fail

* disposing jobs functional

* updated demo app to accomodate dispose

* corrected tests

* added unit tests for device monitor

Co-authored-by: Patrick Cason <me@patrickcason.com>

* Docs Update http-client (OpenMined#193)

* Clean mnist (OpenMined#199)

* cleaning mnist

* corrected syft worker disposal

* updated readme

* made login UI full screen

* updated readme for sleep/wake support

* Charge detection (OpenMined#200)

* added battery listener

* solved battery intent action bug

* corrected device monitor tests

* added robolectric tests

* remove usage of junit5 engine

* Dont skip ci to get code coverage (OpenMined#203)

* docs: add galalen as a contributor (OpenMined#202)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add erksch as a contributor (OpenMined#205)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Documentation for SocketAPI, SocketClient, and SyftWebSocket (OpenMined#201)

* Documentation for SocketAPI, SocketClient, and SyftWebSocket

* add links and additional info to documentation

* follow KDoc standard in documenting SyftWebSocket constructor params

* fix README typo

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Docs HttpAPI interface (OpenMined#192)

* Docs HttpAPI

* update

* update

* feature testing (OpenMined#207)

* complete run through added

* reformat

* made auth token optional

* correct handling multiple disposals

* correct singleton syft worker creation

* WIP adding tests

* constraint validation added

* correct looper robolectric

* multiple jobs tested

* add disposing in test to clear worker

* SyftJob tests (OpenMined#209)

* Added first tests. Refactor SyftJob

* First tests for JobDownloader

* Added conditions to download

* Download using CycleAccept response data

* Make requestKey available to other parts of the workflow

* Test report

* Improved test

* Added test for throwError

* JobDownloader test

* Refactor downloader to use data sources and add tests

* Restrict visibility of data sources to internal

* Tests for jobDownloader

* Tests for RemoteDataSource

* Remove unused method

* Remove duplicated logic

* Pass the correct object to remote data source

* Rename and place classes

* Fix failing test

* Remove activity monitoring and API clean up (OpenMined#211)

* activity detection removed

* syft job docs, restricted public APIs

* describe multi threading support (OpenMined#210)

* Test for jobLocalDataSource. Refactor IO (OpenMined#212)

* Tests for SyftTensor (OpenMined#213)

* Test for SyftTensor deserialisation

* Fix recursive calls

* Refactor deserialize name

* Pytorch Tensor to SyftTensor test

* Cover content types

* Tests for serialize and getTorchTensor

* Model tests

* Check size of new params

* Report diff (OpenMined#217)

* diff reporting added

* update running mutliple cycles

* remove open tag from configuration

* correct tests

* added tests for files

* addressing comments

* refactor asset reading

* remove tests for createDiff of repository

* added new tests for jobRepository and jobLocalDataSource

* added jobLocaldatasource test

* cosmetic change

* Speed tests optional (OpenMined#219)

* Speed tests optional

* Introduce Either to handle failure when checking conditions

* No need for requiresSpeedTest to be lateinit

* Add feature flag to allow execution while the server is updated

* Remove ignored files

* Too many deletions

* Update syft-proto version

* Property for buildConfig

* Revert to 0.4.1

* correct version of SyftProto

* Remove feature flags as too many issues with integration tests

* Remove unused methods

* Small changes

* Failing test

* Leave only in use methods

* Background demo (OpenMined#220)

* first draft of background execution

* add UI

* removed boilerplate code

* update syft proto dependency

* refactor paarameter mismatch

* correct default device monitoring status

* code cleanup

* work manager logging

* corrected tests

* remove unsubscribe for background tasks

* refactoring WIP

* job worker functional

* refactor attachmnistlogger

* added tests for diff coverage

* refactor view models

* rebase

* empty commit for codecov

* docs: add mccorby as a contributor (OpenMined#223)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (OpenMined#224)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* restrict file scope (OpenMined#227)

* documentation for JobStatusSubscriber (OpenMined#229)

* documentation for JobStatusSubscriber

* add description of the class

* correct syft model doc

* adding tests for codecov

* Add dokka and docs dir (OpenMined#225)

* Add dokka and docs dir

* Remove docs in wrong place

* Docs to root

* Documentatino workflow

* GH format

* Doc public only/

* Add docs

* Update docs and go

* Fix wrong directory

* Internals missing

* docs: add galalen as a contributor (OpenMined#231)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (OpenMined#232)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* General config (OpenMined#233)

* added variable client config

* refactor plan

* regenerate the docs

* document Client Config

* correct doc references

* regenerate docs

* add tests for SyftModel

* add some more tests

* corrected http handler and added jwt token (OpenMined#234)

* corrected http handler and added jwt token

* added plan name

* updated tests

* update docs

* single socket per job

* plan updated

* demo working

* added small database

* updated tests

* remove changes in network and add params tester

* README Change

Co-authored-by: varun khare <varunkhare1234@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>
cereallarceny added a commit that referenced this pull request Jul 30, 2020
* Preparation for release 0.1.0 (#236)

* Added publish script

* WebRTC first tests

* Improve expected value

* Added badges for CI and version

* Change badge version

* Draft with okHttp

* Socket with okhttp plus previous work on networking

Co-authored-by: Varun Khare  varunkhare1234@gmail.com

* Address review comments

* Addressed comments

* Fixing style guidelines (#25)

* added project style file

* reformat code

* Added roadmap objectives from openmined (#27)

* Check validity of protocol and timeout (#26)

* Check validity of protocol and timeout

* Added test. Addressed comments

* Changes with style

* Refactor using protocol and injecting signalling client

* Fix Bintray badge link (#38)

* Auto versioning with nebula (#22)

* Add dependency on PySyft protobuf classes (#45)

* refactoring strings from json data, setting up Job and syft API (#41)

* refactoring strings from json requests, setting up Job API

* add response handler and new API types

* added deserialization for API endpoints

* Kotlin serialization is love. Everything removed with polymorphic deserialization

* reformat code

* polymorphic serialization only for report. Implemented custom deserialization for the rest

* added unit tests

* Ci action build check (#48)

* Create android.yml

* added deploy script

* updated CI badge

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>

* Update github action to fire on pull request (#52)

* extracted dependency version to a separate file (#49)

* extracted library version to a separate file

* applied requested changes

* applied requested changes

* Minor typo changes (#55)

* Http client for downloading files, pygrid requests (#51)

* scaffolding basic job scheduling

* job worker API implemented

* Added http client. Refactored common api usage

* Made socket client and http client uniform in usage

* WIP downloader

* added file saver

* rebase with dev

* updated standalone-demo

* added throwable for jobs and other minor changes

* added subscriber class for jobs,addressed comments

* code clean up

* updated federated learning network API (#60)

* updated federated learning network API

* added status field in authentication

* fixed bug data serialization

* added client config module

* remove version field

* no version in cycle response from pygrid

* Protobuf infrastructure (#54)

* Merge from upstream

* Unbuffer plan operations

* Deserialise script module

* Remove not-need lib

* Wee refactor

* Remove rogue activity. It's not the time yet

* Restore deleted files

* Wire up MNIST

* Added vm

* Remove long files

* Working model

* Refactor with repository pattern

* Wiring up

* Addressed comments

* Demo app (#63)

* added pygrid integration demo and logging

* debugging pygrid integration

* Wip streaming downloads don't work but direct succeeds

* updated syftlib to have new job status callbacks

* cleaning up code

* Syft instance (#65)

* auto initialise communication client

* added HTTP protocol and made snetworking clients to be modifiable

* added test for socket client initialization

* reformat code

* cleared duplicate http protocol

* updated demo app to use new syft worker

* added tensor serialization (#68)

* added tensor serialization

* removed lateinit initialization, SyftModel integeration into job

* reformat code

* cleared duplicate http protocol

* addressed comments and rebasing

* remove message processor test

* removed pytorch tensor wrapper since we wont perform computation in kotlin

* added syftmodel test

* updated order of schedulers

* Wire up cycle with data training and process (#67)

* Draft

* Logger

* Attempt to read mnist script

* Almost there

* Added logger

* Almost there

* Fix data loading

* Move out zip

* And it is working

* More wiring

* Show loss in chart

* Pass floats directly

* made UI responsive (#73)

* updated syft-proto (#77)

* updated syft-proto

* removed syft-proto from jetifier

* Add demo steps to README (#76)

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* added batch support, UI tweaks (#74)

* added batch support, UI tweaks

* addressed comments Added real mnist training data

* added training data to git lfs

* code clean up

* remove lfs and made loading batches clear

* updated readme

* updated pytorch (1.5.0) and gradle version

* Docs protobuf serialization (#80)

* refactor readme (#86)

* Docs Plan Execution (#85)

* plan documentation

* changes

* bug

* minor changes

* fixes

* speed test functional

speed test error handling

correct syft tests

created netowrk state manager

* Syft config (#89)

* added configuration parameter

* added syft config and device monitor

* refactor datasource for device status

* added syft status subscriber

* made class constructors internal for testing support

* register callbacks for broadcasts

* updated demo app

* corrected syft test

* network cache bug cleared

* correct default schedulers

* Test (#92)

* initiated testing files and added tests for JobStatusSubscriber

* added socket client test

* updated readme (#95)

* updated readme

* Update README.md

Co-authored-by: Patrick Cason <me@patrickcason.com>

* docs: add mccorby as a contributor (#181)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#180)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add cereallarceny as a contributor (#182)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add codeboy5 as a contributor (#183)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#185)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#186)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add cereallarceny as a contributor (#187)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#188)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#189)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#190)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* added test coverage and moved tests to a new action (#179)

* added test coverage and moved tests to a new action

* updated readme badge

Co-authored-by: Patrick Cason <me@patrickcason.com>

* NDK cache in CI build (#191)

* added test coverage and moved tests to a new action

* added ndk caching for faster builds

* tests badge rename

* corrected url space

* corrected badge position and ndk cache permissions (#194)

* added network monitoring (#195)

* added network monitoring

* added device state checks

* removed redundant JobStatusMessage.JobError

* corrected demo app call to stop execution on fail

* disposing jobs functional

* updated demo app to accomodate dispose

* corrected tests

* added unit tests for device monitor

Co-authored-by: Patrick Cason <me@patrickcason.com>

* Docs Update http-client (#193)

* Clean mnist (#199)

* cleaning mnist

* corrected syft worker disposal

* updated readme

* made login UI full screen

* updated readme for sleep/wake support

* Charge detection (#200)

* added battery listener

* solved battery intent action bug

* corrected device monitor tests

* added robolectric tests

* remove usage of junit5 engine

* Dont skip ci to get code coverage (#203)

* docs: add galalen as a contributor (#202)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add erksch as a contributor (#205)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Documentation for SocketAPI, SocketClient, and SyftWebSocket (#201)

* Documentation for SocketAPI, SocketClient, and SyftWebSocket

* add links and additional info to documentation

* follow KDoc standard in documenting SyftWebSocket constructor params

* fix README typo

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Docs HttpAPI interface (#192)

* Docs HttpAPI

* update

* update

* feature testing (#207)

* complete run through added

* reformat

* made auth token optional

* correct handling multiple disposals

* correct singleton syft worker creation

* WIP adding tests

* constraint validation added

* correct looper robolectric

* multiple jobs tested

* add disposing in test to clear worker

* SyftJob tests (#209)

* Added first tests. Refactor SyftJob

* First tests for JobDownloader

* Added conditions to download

* Download using CycleAccept response data

* Make requestKey available to other parts of the workflow

* Test report

* Improved test

* Added test for throwError

* JobDownloader test

* Refactor downloader to use data sources and add tests

* Restrict visibility of data sources to internal

* Tests for jobDownloader

* Tests for RemoteDataSource

* Remove unused method

* Remove duplicated logic

* Pass the correct object to remote data source

* Rename and place classes

* Fix failing test

* Remove activity monitoring and API clean up (#211)

* activity detection removed

* syft job docs, restricted public APIs

* describe multi threading support (#210)

* Test for jobLocalDataSource. Refactor IO (#212)

* Tests for SyftTensor (#213)

* Test for SyftTensor deserialisation

* Fix recursive calls

* Refactor deserialize name

* Pytorch Tensor to SyftTensor test

* Cover content types

* Tests for serialize and getTorchTensor

* Model tests

* Check size of new params

* Report diff (#217)

* diff reporting added

* update running mutliple cycles

* remove open tag from configuration

* correct tests

* added tests for files

* addressing comments

* refactor asset reading

* remove tests for createDiff of repository

* added new tests for jobRepository and jobLocalDataSource

* added jobLocaldatasource test

* cosmetic change

* Speed tests optional (#219)

* Speed tests optional

* Introduce Either to handle failure when checking conditions

* No need for requiresSpeedTest to be lateinit

* Add feature flag to allow execution while the server is updated

* Remove ignored files

* Too many deletions

* Update syft-proto version

* Property for buildConfig

* Revert to 0.4.1

* correct version of SyftProto

* Remove feature flags as too many issues with integration tests

* Remove unused methods

* Small changes

* Failing test

* Leave only in use methods

* Background demo (#220)

* first draft of background execution

* add UI

* removed boilerplate code

* update syft proto dependency

* refactor paarameter mismatch

* correct default device monitoring status

* code cleanup

* work manager logging

* corrected tests

* remove unsubscribe for background tasks

* refactoring WIP

* job worker functional

* refactor attachmnistlogger

* added tests for diff coverage

* refactor view models

* rebase

* empty commit for codecov

* docs: add mccorby as a contributor (#223)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#224)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* restrict file scope (#227)

* documentation for JobStatusSubscriber (#229)

* documentation for JobStatusSubscriber

* add description of the class

* correct syft model doc

* adding tests for codecov

* Add dokka and docs dir (#225)

* Add dokka and docs dir

* Remove docs in wrong place

* Docs to root

* Documentatino workflow

* GH format

* Doc public only/

* Add docs

* Update docs and go

* Fix wrong directory

* Internals missing

* docs: add galalen as a contributor (#231)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#232)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* General config (#233)

* added variable client config

* refactor plan

* regenerate the docs

* document Client Config

* correct doc references

* regenerate docs

* add tests for SyftModel

* add some more tests

* corrected http handler and added jwt token (#234)

* corrected http handler and added jwt token

* added plan name

* updated tests

* update docs

* single socket per job

* plan updated

* demo working

* added small database

* updated tests

* remove changes in network and add params tester

* README Change

Co-authored-by: varun khare <varunkhare1234@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>

* update gif (#237) (#243)

* update gif

* add installation instructions

* update readme language

* add slack badge

* Update README.md

* Update README.md

* Rebasing dev to master for future merges (#248)

* Preparation for release 0.1.0 (#236)

* Added publish script

* WebRTC first tests

* Improve expected value

* Added badges for CI and version

* Change badge version

* Draft with okHttp

* Socket with okhttp plus previous work on networking

Co-authored-by: Varun Khare  varunkhare1234@gmail.com

* Address review comments

* Addressed comments

* Fixing style guidelines (#25)

* added project style file

* reformat code

* Added roadmap objectives from openmined (#27)

* Check validity of protocol and timeout (#26)

* Check validity of protocol and timeout

* Added test. Addressed comments

* Changes with style

* Refactor using protocol and injecting signalling client

* Fix Bintray badge link (#38)

* Auto versioning with nebula (#22)

* Add dependency on PySyft protobuf classes (#45)

* refactoring strings from json data, setting up Job and syft API (#41)

* refactoring strings from json requests, setting up Job API

* add response handler and new API types

* added deserialization for API endpoints

* Kotlin serialization is love. Everything removed with polymorphic deserialization

* reformat code

* polymorphic serialization only for report. Implemented custom deserialization for the rest

* added unit tests

* Ci action build check (#48)

* Create android.yml

* added deploy script

* updated CI badge

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>

* Update github action to fire on pull request (#52)

* extracted dependency version to a separate file (#49)

* extracted library version to a separate file

* applied requested changes

* applied requested changes

* Minor typo changes (#55)

* Http client for downloading files, pygrid requests (#51)

* scaffolding basic job scheduling

* job worker API implemented

* Added http client. Refactored common api usage

* Made socket client and http client uniform in usage

* WIP downloader

* added file saver

* rebase with dev

* updated standalone-demo

* added throwable for jobs and other minor changes

* added subscriber class for jobs,addressed comments

* code clean up

* updated federated learning network API (#60)

* updated federated learning network API

* added status field in authentication

* fixed bug data serialization

* added client config module

* remove version field

* no version in cycle response from pygrid

* Protobuf infrastructure (#54)

* Merge from upstream

* Unbuffer plan operations

* Deserialise script module

* Remove not-need lib

* Wee refactor

* Remove rogue activity. It's not the time yet

* Restore deleted files

* Wire up MNIST

* Added vm

* Remove long files

* Working model

* Refactor with repository pattern

* Wiring up

* Addressed comments

* Demo app (#63)

* added pygrid integration demo and logging

* debugging pygrid integration

* Wip streaming downloads don't work but direct succeeds

* updated syftlib to have new job status callbacks

* cleaning up code

* Syft instance (#65)

* auto initialise communication client

* added HTTP protocol and made snetworking clients to be modifiable

* added test for socket client initialization

* reformat code

* cleared duplicate http protocol

* updated demo app to use new syft worker

* added tensor serialization (#68)

* added tensor serialization

* removed lateinit initialization, SyftModel integeration into job

* reformat code

* cleared duplicate http protocol

* addressed comments and rebasing

* remove message processor test

* removed pytorch tensor wrapper since we wont perform computation in kotlin

* added syftmodel test

* updated order of schedulers

* Wire up cycle with data training and process (#67)

* Draft

* Logger

* Attempt to read mnist script

* Almost there

* Added logger

* Almost there

* Fix data loading

* Move out zip

* And it is working

* More wiring

* Show loss in chart

* Pass floats directly

* made UI responsive (#73)

* updated syft-proto (#77)

* updated syft-proto

* removed syft-proto from jetifier

* Add demo steps to README (#76)

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* added batch support, UI tweaks (#74)

* added batch support, UI tweaks

* addressed comments Added real mnist training data

* added training data to git lfs

* code clean up

* remove lfs and made loading batches clear

* updated readme

* updated pytorch (1.5.0) and gradle version

* Docs protobuf serialization (#80)

* refactor readme (#86)

* Docs Plan Execution (#85)

* plan documentation

* changes

* bug

* minor changes

* fixes

* speed test functional

speed test error handling

correct syft tests

created netowrk state manager

* Syft config (#89)

* added configuration parameter

* added syft config and device monitor

* refactor datasource for device status

* added syft status subscriber

* made class constructors internal for testing support

* register callbacks for broadcasts

* updated demo app

* corrected syft test

* network cache bug cleared

* correct default schedulers

* Test (#92)

* initiated testing files and added tests for JobStatusSubscriber

* added socket client test

* updated readme (#95)

* updated readme

* Update README.md

Co-authored-by: Patrick Cason <me@patrickcason.com>

* docs: add mccorby as a contributor (#181)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#180)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add cereallarceny as a contributor (#182)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add codeboy5 as a contributor (#183)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#185)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#186)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add cereallarceny as a contributor (#187)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#188)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#189)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#190)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* added test coverage and moved tests to a new action (#179)

* added test coverage and moved tests to a new action

* updated readme badge

Co-authored-by: Patrick Cason <me@patrickcason.com>

* NDK cache in CI build (#191)

* added test coverage and moved tests to a new action

* added ndk caching for faster builds

* tests badge rename

* corrected url space

* corrected badge position and ndk cache permissions (#194)

* added network monitoring (#195)

* added network monitoring

* added device state checks

* removed redundant JobStatusMessage.JobError

* corrected demo app call to stop execution on fail

* disposing jobs functional

* updated demo app to accomodate dispose

* corrected tests

* added unit tests for device monitor

Co-authored-by: Patrick Cason <me@patrickcason.com>

* Docs Update http-client (#193)

* Clean mnist (#199)

* cleaning mnist

* corrected syft worker disposal

* updated readme

* made login UI full screen

* updated readme for sleep/wake support

* Charge detection (#200)

* added battery listener

* solved battery intent action bug

* corrected device monitor tests

* added robolectric tests

* remove usage of junit5 engine

* Dont skip ci to get code coverage (#203)

* docs: add galalen as a contributor (#202)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add erksch as a contributor (#205)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Documentation for SocketAPI, SocketClient, and SyftWebSocket (#201)

* Documentation for SocketAPI, SocketClient, and SyftWebSocket

* add links and additional info to documentation

* follow KDoc standard in documenting SyftWebSocket constructor params

* fix README typo

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Docs HttpAPI interface (#192)

* Docs HttpAPI

* update

* update

* feature testing (#207)

* complete run through added

* reformat

* made auth token optional

* correct handling multiple disposals

* correct singleton syft worker creation

* WIP adding tests

* constraint validation added

* correct looper robolectric

* multiple jobs tested

* add disposing in test to clear worker

* SyftJob tests (#209)

* Added first tests. Refactor SyftJob

* First tests for JobDownloader

* Added conditions to download

* Download using CycleAccept response data

* Make requestKey available to other parts of the workflow

* Test report

* Improved test

* Added test for throwError

* JobDownloader test

* Refactor downloader to use data sources and add tests

* Restrict visibility of data sources to internal

* Tests for jobDownloader

* Tests for RemoteDataSource

* Remove unused method

* Remove duplicated logic

* Pass the correct object to remote data source

* Rename and place classes

* Fix failing test

* Remove activity monitoring and API clean up (#211)

* activity detection removed

* syft job docs, restricted public APIs

* describe multi threading support (#210)

* Test for jobLocalDataSource. Refactor IO (#212)

* Tests for SyftTensor (#213)

* Test for SyftTensor deserialisation

* Fix recursive calls

* Refactor deserialize name

* Pytorch Tensor to SyftTensor test

* Cover content types

* Tests for serialize and getTorchTensor

* Model tests

* Check size of new params

* Report diff (#217)

* diff reporting added

* update running mutliple cycles

* remove open tag from configuration

* correct tests

* added tests for files

* addressing comments

* refactor asset reading

* remove tests for createDiff of repository

* added new tests for jobRepository and jobLocalDataSource

* added jobLocaldatasource test

* cosmetic change

* Speed tests optional (#219)

* Speed tests optional

* Introduce Either to handle failure when checking conditions

* No need for requiresSpeedTest to be lateinit

* Add feature flag to allow execution while the server is updated

* Remove ignored files

* Too many deletions

* Update syft-proto version

* Property for buildConfig

* Revert to 0.4.1

* correct version of SyftProto

* Remove feature flags as too many issues with integration tests

* Remove unused methods

* Small changes

* Failing test

* Leave only in use methods

* Background demo (#220)

* first draft of background execution

* add UI

* removed boilerplate code

* update syft proto dependency

* refactor paarameter mismatch

* correct default device monitoring status

* code cleanup

* work manager logging

* corrected tests

* remove unsubscribe for background tasks

* refactoring WIP

* job worker functional

* refactor attachmnistlogger

* added tests for diff coverage

* refactor view models

* rebase

* empty commit for codecov

* docs: add mccorby as a contributor (#223)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#224)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* restrict file scope (#227)

* documentation for JobStatusSubscriber (#229)

* documentation for JobStatusSubscriber

* add description of the class

* correct syft model doc

* adding tests for codecov

* Add dokka and docs dir (#225)

* Add dokka and docs dir

* Remove docs in wrong place

* Docs to root

* Documentatino workflow

* GH format

* Doc public only/

* Add docs

* Update docs and go

* Fix wrong directory

* Internals missing

* docs: add galalen as a contributor (#231)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#232)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* General config (#233)

* added variable client config

* refactor plan

* regenerate the docs

* document Client Config

* correct doc references

* regenerate docs

* add tests for SyftModel

* add some more tests

* corrected http handler and added jwt token (#234)

* corrected http handler and added jwt token

* added plan name

* updated tests

* update docs

* single socket per job

* plan updated

* demo working

* added small database

* updated tests

* remove changes in network and add params tester

* README Change

Co-authored-by: varun khare <varunkhare1234@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>

* update gif (#237) (#243)

* update gif

* add installation instructions

* update readme language

* add slack badge

* Update README.md

* Update README.md

* publish action should fire only on release tags

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>

* Tutorial for reading custom dataset  (#247)

* custom dataset tutorial

* Update Part X - Train on custom dataset.ipynb

* tutorial in md

* rm notebook file

* Update Part X - Train on custom dataset.md

* Update Part X - Train on custom dataset.md

Co-authored-by: Pengyuan Zhou <pengyuan.zhou@helsinki.fi>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add pengyuan-zhou as a contributor (#250)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* change tagging system to be compatible with nebula (#251)

* write unit tests for network status (#208)

write unit tests for network status

* revise readme (#240)

* Namespacing changes and changes from static to model centric

* updated state and model class to store ivalues

* doc update

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>
Co-authored-by: Pengyuan Zhou <zpymyyn@gmail.com>
Co-authored-by: Pengyuan Zhou <pengyuan.zhou@helsinki.fi>
Co-authored-by: Mohammed Galalen <mohammedgalalen@gmail.com>
vkkhare added a commit that referenced this pull request Jul 30, 2020
* Rebasing dev to master for future merges (#248)

* Preparation for release 0.1.0 (#236)

* Added publish script

* WebRTC first tests

* Improve expected value

* Added badges for CI and version

* Change badge version

* Draft with okHttp

* Socket with okhttp plus previous work on networking

Co-authored-by: Varun Khare  varunkhare1234@gmail.com

* Address review comments

* Addressed comments

* Fixing style guidelines (#25)

* added project style file

* reformat code

* Added roadmap objectives from openmined (#27)

* Check validity of protocol and timeout (#26)

* Check validity of protocol and timeout

* Added test. Addressed comments

* Changes with style

* Refactor using protocol and injecting signalling client

* Fix Bintray badge link (#38)

* Auto versioning with nebula (#22)

* Add dependency on PySyft protobuf classes (#45)

* refactoring strings from json data, setting up Job and syft API (#41)

* refactoring strings from json requests, setting up Job API

* add response handler and new API types

* added deserialization for API endpoints

* Kotlin serialization is love. Everything removed with polymorphic deserialization

* reformat code

* polymorphic serialization only for report. Implemented custom deserialization for the rest

* added unit tests

* Ci action build check (#48)

* Create android.yml

* added deploy script

* updated CI badge

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>

* Update github action to fire on pull request (#52)

* extracted dependency version to a separate file (#49)

* extracted library version to a separate file

* applied requested changes

* applied requested changes

* Minor typo changes (#55)

* Http client for downloading files, pygrid requests (#51)

* scaffolding basic job scheduling

* job worker API implemented

* Added http client. Refactored common api usage

* Made socket client and http client uniform in usage

* WIP downloader

* added file saver

* rebase with dev

* updated standalone-demo

* added throwable for jobs and other minor changes

* added subscriber class for jobs,addressed comments

* code clean up

* updated federated learning network API (#60)

* updated federated learning network API

* added status field in authentication

* fixed bug data serialization

* added client config module

* remove version field

* no version in cycle response from pygrid

* Protobuf infrastructure (#54)

* Merge from upstream

* Unbuffer plan operations

* Deserialise script module

* Remove not-need lib

* Wee refactor

* Remove rogue activity. It's not the time yet

* Restore deleted files

* Wire up MNIST

* Added vm

* Remove long files

* Working model

* Refactor with repository pattern

* Wiring up

* Addressed comments

* Demo app (#63)

* added pygrid integration demo and logging

* debugging pygrid integration

* Wip streaming downloads don't work but direct succeeds

* updated syftlib to have new job status callbacks

* cleaning up code

* Syft instance (#65)

* auto initialise communication client

* added HTTP protocol and made snetworking clients to be modifiable

* added test for socket client initialization

* reformat code

* cleared duplicate http protocol

* updated demo app to use new syft worker

* added tensor serialization (#68)

* added tensor serialization

* removed lateinit initialization, SyftModel integeration into job

* reformat code

* cleared duplicate http protocol

* addressed comments and rebasing

* remove message processor test

* removed pytorch tensor wrapper since we wont perform computation in kotlin

* added syftmodel test

* updated order of schedulers

* Wire up cycle with data training and process (#67)

* Draft

* Logger

* Attempt to read mnist script

* Almost there

* Added logger

* Almost there

* Fix data loading

* Move out zip

* And it is working

* More wiring

* Show loss in chart

* Pass floats directly

* made UI responsive (#73)

* updated syft-proto (#77)

* updated syft-proto

* removed syft-proto from jetifier

* Add demo steps to README (#76)

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* added batch support, UI tweaks (#74)

* added batch support, UI tweaks

* addressed comments Added real mnist training data

* added training data to git lfs

* code clean up

* remove lfs and made loading batches clear

* updated readme

* updated pytorch (1.5.0) and gradle version

* Docs protobuf serialization (#80)

* refactor readme (#86)

* Docs Plan Execution (#85)

* plan documentation

* changes

* bug

* minor changes

* fixes

* speed test functional

speed test error handling

correct syft tests

created netowrk state manager

* Syft config (#89)

* added configuration parameter

* added syft config and device monitor

* refactor datasource for device status

* added syft status subscriber

* made class constructors internal for testing support

* register callbacks for broadcasts

* updated demo app

* corrected syft test

* network cache bug cleared

* correct default schedulers

* Test (#92)

* initiated testing files and added tests for JobStatusSubscriber

* added socket client test

* updated readme (#95)

* updated readme

* Update README.md

Co-authored-by: Patrick Cason <me@patrickcason.com>

* docs: add mccorby as a contributor (#181)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#180)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add cereallarceny as a contributor (#182)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add codeboy5 as a contributor (#183)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#185)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#186)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add cereallarceny as a contributor (#187)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#188)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#189)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#190)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* added test coverage and moved tests to a new action (#179)

* added test coverage and moved tests to a new action

* updated readme badge

Co-authored-by: Patrick Cason <me@patrickcason.com>

* NDK cache in CI build (#191)

* added test coverage and moved tests to a new action

* added ndk caching for faster builds

* tests badge rename

* corrected url space

* corrected badge position and ndk cache permissions (#194)

* added network monitoring (#195)

* added network monitoring

* added device state checks

* removed redundant JobStatusMessage.JobError

* corrected demo app call to stop execution on fail

* disposing jobs functional

* updated demo app to accomodate dispose

* corrected tests

* added unit tests for device monitor

Co-authored-by: Patrick Cason <me@patrickcason.com>

* Docs Update http-client (#193)

* Clean mnist (#199)

* cleaning mnist

* corrected syft worker disposal

* updated readme

* made login UI full screen

* updated readme for sleep/wake support

* Charge detection (#200)

* added battery listener

* solved battery intent action bug

* corrected device monitor tests

* added robolectric tests

* remove usage of junit5 engine

* Dont skip ci to get code coverage (#203)

* docs: add galalen as a contributor (#202)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add erksch as a contributor (#205)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Documentation for SocketAPI, SocketClient, and SyftWebSocket (#201)

* Documentation for SocketAPI, SocketClient, and SyftWebSocket

* add links and additional info to documentation

* follow KDoc standard in documenting SyftWebSocket constructor params

* fix README typo

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Docs HttpAPI interface (#192)

* Docs HttpAPI

* update

* update

* feature testing (#207)

* complete run through added

* reformat

* made auth token optional

* correct handling multiple disposals

* correct singleton syft worker creation

* WIP adding tests

* constraint validation added

* correct looper robolectric

* multiple jobs tested

* add disposing in test to clear worker

* SyftJob tests (#209)

* Added first tests. Refactor SyftJob

* First tests for JobDownloader

* Added conditions to download

* Download using CycleAccept response data

* Make requestKey available to other parts of the workflow

* Test report

* Improved test

* Added test for throwError

* JobDownloader test

* Refactor downloader to use data sources and add tests

* Restrict visibility of data sources to internal

* Tests for jobDownloader

* Tests for RemoteDataSource

* Remove unused method

* Remove duplicated logic

* Pass the correct object to remote data source

* Rename and place classes

* Fix failing test

* Remove activity monitoring and API clean up (#211)

* activity detection removed

* syft job docs, restricted public APIs

* describe multi threading support (#210)

* Test for jobLocalDataSource. Refactor IO (#212)

* Tests for SyftTensor (#213)

* Test for SyftTensor deserialisation

* Fix recursive calls

* Refactor deserialize name

* Pytorch Tensor to SyftTensor test

* Cover content types

* Tests for serialize and getTorchTensor

* Model tests

* Check size of new params

* Report diff (#217)

* diff reporting added

* update running mutliple cycles

* remove open tag from configuration

* correct tests

* added tests for files

* addressing comments

* refactor asset reading

* remove tests for createDiff of repository

* added new tests for jobRepository and jobLocalDataSource

* added jobLocaldatasource test

* cosmetic change

* Speed tests optional (#219)

* Speed tests optional

* Introduce Either to handle failure when checking conditions

* No need for requiresSpeedTest to be lateinit

* Add feature flag to allow execution while the server is updated

* Remove ignored files

* Too many deletions

* Update syft-proto version

* Property for buildConfig

* Revert to 0.4.1

* correct version of SyftProto

* Remove feature flags as too many issues with integration tests

* Remove unused methods

* Small changes

* Failing test

* Leave only in use methods

* Background demo (#220)

* first draft of background execution

* add UI

* removed boilerplate code

* update syft proto dependency

* refactor paarameter mismatch

* correct default device monitoring status

* code cleanup

* work manager logging

* corrected tests

* remove unsubscribe for background tasks

* refactoring WIP

* job worker functional

* refactor attachmnistlogger

* added tests for diff coverage

* refactor view models

* rebase

* empty commit for codecov

* docs: add mccorby as a contributor (#223)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#224)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* restrict file scope (#227)

* documentation for JobStatusSubscriber (#229)

* documentation for JobStatusSubscriber

* add description of the class

* correct syft model doc

* adding tests for codecov

* Add dokka and docs dir (#225)

* Add dokka and docs dir

* Remove docs in wrong place

* Docs to root

* Documentatino workflow

* GH format

* Doc public only/

* Add docs

* Update docs and go

* Fix wrong directory

* Internals missing

* docs: add galalen as a contributor (#231)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#232)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* General config (#233)

* added variable client config

* refactor plan

* regenerate the docs

* document Client Config

* correct doc references

* regenerate docs

* add tests for SyftModel

* add some more tests

* corrected http handler and added jwt token (#234)

* corrected http handler and added jwt token

* added plan name

* updated tests

* update docs

* single socket per job

* plan updated

* demo working

* added small database

* updated tests

* remove changes in network and add params tester

* README Change

Co-authored-by: varun khare <varunkhare1234@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>

* update gif (#237) (#243)

* update gif

* add installation instructions

* update readme language

* add slack badge

* Update README.md

* Update README.md

* publish action should fire only on release tags

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>

* Tutorial for reading custom dataset  (#247)

* custom dataset tutorial

* Update Part X - Train on custom dataset.ipynb

* tutorial in md

* rm notebook file

* Update Part X - Train on custom dataset.md

* Update Part X - Train on custom dataset.md

Co-authored-by: Pengyuan Zhou <pengyuan.zhou@helsinki.fi>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add pengyuan-zhou as a contributor (#250)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* change tagging system to be compatible with nebula (#251)

* write unit tests for network status (#208)

write unit tests for network status

* revise readme (#240)

* Namespacing changes and changes from static to model centric

* updated state and model class to store ivalues (#252)

* Preparation for release 0.1.0 (#236)

* Added publish script

* WebRTC first tests

* Improve expected value

* Added badges for CI and version

* Change badge version

* Draft with okHttp

* Socket with okhttp plus previous work on networking

Co-authored-by: Varun Khare  varunkhare1234@gmail.com

* Address review comments

* Addressed comments

* Fixing style guidelines (#25)

* added project style file

* reformat code

* Added roadmap objectives from openmined (#27)

* Check validity of protocol and timeout (#26)

* Check validity of protocol and timeout

* Added test. Addressed comments

* Changes with style

* Refactor using protocol and injecting signalling client

* Fix Bintray badge link (#38)

* Auto versioning with nebula (#22)

* Add dependency on PySyft protobuf classes (#45)

* refactoring strings from json data, setting up Job and syft API (#41)

* refactoring strings from json requests, setting up Job API

* add response handler and new API types

* added deserialization for API endpoints

* Kotlin serialization is love. Everything removed with polymorphic deserialization

* reformat code

* polymorphic serialization only for report. Implemented custom deserialization for the rest

* added unit tests

* Ci action build check (#48)

* Create android.yml

* added deploy script

* updated CI badge

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>

* Update github action to fire on pull request (#52)

* extracted dependency version to a separate file (#49)

* extracted library version to a separate file

* applied requested changes

* applied requested changes

* Minor typo changes (#55)

* Http client for downloading files, pygrid requests (#51)

* scaffolding basic job scheduling

* job worker API implemented

* Added http client. Refactored common api usage

* Made socket client and http client uniform in usage

* WIP downloader

* added file saver

* rebase with dev

* updated standalone-demo

* added throwable for jobs and other minor changes

* added subscriber class for jobs,addressed comments

* code clean up

* updated federated learning network API (#60)

* updated federated learning network API

* added status field in authentication

* fixed bug data serialization

* added client config module

* remove version field

* no version in cycle response from pygrid

* Protobuf infrastructure (#54)

* Merge from upstream

* Unbuffer plan operations

* Deserialise script module

* Remove not-need lib

* Wee refactor

* Remove rogue activity. It's not the time yet

* Restore deleted files

* Wire up MNIST

* Added vm

* Remove long files

* Working model

* Refactor with repository pattern

* Wiring up

* Addressed comments

* Demo app (#63)

* added pygrid integration demo and logging

* debugging pygrid integration

* Wip streaming downloads don't work but direct succeeds

* updated syftlib to have new job status callbacks

* cleaning up code

* Syft instance (#65)

* auto initialise communication client

* added HTTP protocol and made snetworking clients to be modifiable

* added test for socket client initialization

* reformat code

* cleared duplicate http protocol

* updated demo app to use new syft worker

* added tensor serialization (#68)

* added tensor serialization

* removed lateinit initialization, SyftModel integeration into job

* reformat code

* cleared duplicate http protocol

* addressed comments and rebasing

* remove message processor test

* removed pytorch tensor wrapper since we wont perform computation in kotlin

* added syftmodel test

* updated order of schedulers

* Wire up cycle with data training and process (#67)

* Draft

* Logger

* Attempt to read mnist script

* Almost there

* Added logger

* Almost there

* Fix data loading

* Move out zip

* And it is working

* More wiring

* Show loss in chart

* Pass floats directly

* made UI responsive (#73)

* updated syft-proto (#77)

* updated syft-proto

* removed syft-proto from jetifier

* Add demo steps to README (#76)

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* added batch support, UI tweaks (#74)

* added batch support, UI tweaks

* addressed comments Added real mnist training data

* added training data to git lfs

* code clean up

* remove lfs and made loading batches clear

* updated readme

* updated pytorch (1.5.0) and gradle version

* Docs protobuf serialization (#80)

* refactor readme (#86)

* Docs Plan Execution (#85)

* plan documentation

* changes

* bug

* minor changes

* fixes

* speed test functional

speed test error handling

correct syft tests

created netowrk state manager

* Syft config (#89)

* added configuration parameter

* added syft config and device monitor

* refactor datasource for device status

* added syft status subscriber

* made class constructors internal for testing support

* register callbacks for broadcasts

* updated demo app

* corrected syft test

* network cache bug cleared

* correct default schedulers

* Test (#92)

* initiated testing files and added tests for JobStatusSubscriber

* added socket client test

* updated readme (#95)

* updated readme

* Update README.md

Co-authored-by: Patrick Cason <me@patrickcason.com>

* docs: add mccorby as a contributor (#181)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#180)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add cereallarceny as a contributor (#182)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add codeboy5 as a contributor (#183)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#185)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#186)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add cereallarceny as a contributor (#187)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#188)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#189)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#190)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* added test coverage and moved tests to a new action (#179)

* added test coverage and moved tests to a new action

* updated readme badge

Co-authored-by: Patrick Cason <me@patrickcason.com>

* NDK cache in CI build (#191)

* added test coverage and moved tests to a new action

* added ndk caching for faster builds

* tests badge rename

* corrected url space

* corrected badge position and ndk cache permissions (#194)

* added network monitoring (#195)

* added network monitoring

* added device state checks

* removed redundant JobStatusMessage.JobError

* corrected demo app call to stop execution on fail

* disposing jobs functional

* updated demo app to accomodate dispose

* corrected tests

* added unit tests for device monitor

Co-authored-by: Patrick Cason <me@patrickcason.com>

* Docs Update http-client (#193)

* Clean mnist (#199)

* cleaning mnist

* corrected syft worker disposal

* updated readme

* made login UI full screen

* updated readme for sleep/wake support

* Charge detection (#200)

* added battery listener

* solved battery intent action bug

* corrected device monitor tests

* added robolectric tests

* remove usage of junit5 engine

* Dont skip ci to get code coverage (#203)

* docs: add galalen as a contributor (#202)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add erksch as a contributor (#205)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Documentation for SocketAPI, SocketClient, and SyftWebSocket (#201)

* Documentation for SocketAPI, SocketClient, and SyftWebSocket

* add links and additional info to documentation

* follow KDoc standard in documenting SyftWebSocket constructor params

* fix README typo

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Docs HttpAPI interface (#192)

* Docs HttpAPI

* update

* update

* feature testing (#207)

* complete run through added

* reformat

* made auth token optional

* correct handling multiple disposals

* correct singleton syft worker creation

* WIP adding tests

* constraint validation added

* correct looper robolectric

* multiple jobs tested

* add disposing in test to clear worker

* SyftJob tests (#209)

* Added first tests. Refactor SyftJob

* First tests for JobDownloader

* Added conditions to download

* Download using CycleAccept response data

* Make requestKey available to other parts of the workflow

* Test report

* Improved test

* Added test for throwError

* JobDownloader test

* Refactor downloader to use data sources and add tests

* Restrict visibility of data sources to internal

* Tests for jobDownloader

* Tests for RemoteDataSource

* Remove unused method

* Remove duplicated logic

* Pass the correct object to remote data source

* Rename and place classes

* Fix failing test

* Remove activity monitoring and API clean up (#211)

* activity detection removed

* syft job docs, restricted public APIs

* describe multi threading support (#210)

* Test for jobLocalDataSource. Refactor IO (#212)

* Tests for SyftTensor (#213)

* Test for SyftTensor deserialisation

* Fix recursive calls

* Refactor deserialize name

* Pytorch Tensor to SyftTensor test

* Cover content types

* Tests for serialize and getTorchTensor

* Model tests

* Check size of new params

* Report diff (#217)

* diff reporting added

* update running mutliple cycles

* remove open tag from configuration

* correct tests

* added tests for files

* addressing comments

* refactor asset reading

* remove tests for createDiff of repository

* added new tests for jobRepository and jobLocalDataSource

* added jobLocaldatasource test

* cosmetic change

* Speed tests optional (#219)

* Speed tests optional

* Introduce Either to handle failure when checking conditions

* No need for requiresSpeedTest to be lateinit

* Add feature flag to allow execution while the server is updated

* Remove ignored files

* Too many deletions

* Update syft-proto version

* Property for buildConfig

* Revert to 0.4.1

* correct version of SyftProto

* Remove feature flags as too many issues with integration tests

* Remove unused methods

* Small changes

* Failing test

* Leave only in use methods

* Background demo (#220)

* first draft of background execution

* add UI

* removed boilerplate code

* update syft proto dependency

* refactor paarameter mismatch

* correct default device monitoring status

* code cleanup

* work manager logging

* corrected tests

* remove unsubscribe for background tasks

* refactoring WIP

* job worker functional

* refactor attachmnistlogger

* added tests for diff coverage

* refactor view models

* rebase

* empty commit for codecov

* docs: add mccorby as a contributor (#223)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#224)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* restrict file scope (#227)

* documentation for JobStatusSubscriber (#229)

* documentation for JobStatusSubscriber

* add description of the class

* correct syft model doc

* adding tests for codecov

* Add dokka and docs dir (#225)

* Add dokka and docs dir

* Remove docs in wrong place

* Docs to root

* Documentatino workflow

* GH format

* Doc public only/

* Add docs

* Update docs and go

* Fix wrong directory

* Internals missing

* docs: add galalen as a contributor (#231)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#232)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* General config (#233)

* added variable client config

* refactor plan

* regenerate the docs

* document Client Config

* correct doc references

* regenerate docs

* add tests for SyftModel

* add some more tests

* corrected http handler and added jwt token (#234)

* corrected http handler and added jwt token

* added plan name

* updated tests

* update docs

* single socket per job

* plan updated

* demo working

* added small database

* updated tests

* remove changes in network and add params tester

* README Change

Co-authored-by: varun khare <varunkhare1234@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>

* update gif (#237) (#243)

* update gif

* add installation instructions

* update readme language

* add slack badge

* Update README.md

* Update README.md

* Rebasing dev to master for future merges (#248)

* Preparation for release 0.1.0 (#236)

* Added publish script

* WebRTC first tests

* Improve expected value

* Added badges for CI and version

* Change badge version

* Draft with okHttp

* Socket with okhttp plus previous work on networking

Co-authored-by: Varun Khare  varunkhare1234@gmail.com

* Address review comments

* Addressed comments

* Fixing style guidelines (#25)

* added project style file

* reformat code

* Added roadmap objectives from openmined (#27)

* Check validity of protocol and timeout (#26)

* Check validity of protocol and timeout

* Added test. Addressed comments

* Changes with style

* Refactor using protocol and injecting signalling client

* Fix Bintray badge link (#38)

* Auto versioning with nebula (#22)

* Add dependency on PySyft protobuf classes (#45)

* refactoring strings from json data, setting up Job and syft API (#41)

* refactoring strings from json requests, setting up Job API

* add response handler and new API types

* added deserialization for API endpoints

* Kotlin serialization is love. Everything removed with polymorphic deserialization

* reformat code

* polymorphic serialization only for report. Implemented custom deserialization for the rest

* added unit tests

* Ci action build check (#48)

* Create android.yml

* added deploy script

* updated CI badge

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>

* Update github action to fire on pull request (#52)

* extracted dependency version to a separate file (#49)

* extracted library version to a separate file

* applied requested changes

* applied requested changes

* Minor typo changes (#55)

* Http client for downloading files, pygrid requests (#51)

* scaffolding basic job scheduling

* job worker API implemented

* Added http client. Refactored common api usage

* Made socket client and http client uniform in usage

* WIP downloader

* added file saver

* rebase with dev

* updated standalone-demo

* added throwable for jobs and other minor changes

* added subscriber class for jobs,addressed comments

* code clean up

* updated federated learning network API (#60)

* updated federated learning network API

* added status field in authentication

* fixed bug data serialization

* added client config module

* remove version field

* no version in cycle response from pygrid

* Protobuf infrastructure (#54)

* Merge from upstream

* Unbuffer plan operations

* Deserialise script module

* Remove not-need lib

* Wee refactor

* Remove rogue activity. It's not the time yet

* Restore deleted files

* Wire up MNIST

* Added vm

* Remove long files

* Working model

* Refactor with repository pattern

* Wiring up

* Addressed comments

* Demo app (#63)

* added pygrid integration demo and logging

* debugging pygrid integration

* Wip streaming downloads don't work but direct succeeds

* updated syftlib to have new job status callbacks

* cleaning up code

* Syft instance (#65)

* auto initialise communication client

* added HTTP protocol and made snetworking clients to be modifiable

* added test for socket client initialization

* reformat code

* cleared duplicate http protocol

* updated demo app to use new syft worker

* added tensor serialization (#68)

* added tensor serialization

* removed lateinit initialization, SyftModel integeration into job

* reformat code

* cleared duplicate http protocol

* addressed comments and rebasing

* remove message processor test

* removed pytorch tensor wrapper since we wont perform computation in kotlin

* added syftmodel test

* updated order of schedulers

* Wire up cycle with data training and process (#67)

* Draft

* Logger

* Attempt to read mnist script

* Almost there

* Added logger

* Almost there

* Fix data loading

* Move out zip

* And it is working

* More wiring

* Show loss in chart

* Pass floats directly

* made UI responsive (#73)

* updated syft-proto (#77)

* updated syft-proto

* removed syft-proto from jetifier

* Add demo steps to README (#76)

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* added batch support, UI tweaks (#74)

* added batch support, UI tweaks

* addressed comments Added real mnist training data

* added training data to git lfs

* code clean up

* remove lfs and made loading batches clear

* updated readme

* updated pytorch (1.5.0) and gradle version

* Docs protobuf serialization (#80)

* refactor readme (#86)

* Docs Plan Execution (#85)

* plan documentation

* changes

* bug

* minor changes

* fixes

* speed test functional

speed test error handling

correct syft tests

created netowrk state manager

* Syft config (#89)

* added configuration parameter

* added syft config and device monitor

* refactor datasource for device status

* added syft status subscriber

* made class constructors internal for testing support

* register callbacks for broadcasts

* updated demo app

* corrected syft test

* network cache bug cleared

* correct default schedulers

* Test (#92)

* initiated testing files and added tests for JobStatusSubscriber

* added socket client test

* updated readme (#95)

* updated readme

* Update README.md

Co-authored-by: Patrick Cason <me@patrickcason.com>

* docs: add mccorby as a contributor (#181)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#180)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add cereallarceny as a contributor (#182)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add codeboy5 as a contributor (#183)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#185)

* docs: update README.md [skip ci]

* docs: create .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add IamRavikantSingh as a contributor (#186)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add cereallarceny as a contributor (#187)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#188)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#189)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#190)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* added test coverage and moved tests to a new action (#179)

* added test coverage and moved tests to a new action

* updated readme badge

Co-authored-by: Patrick Cason <me@patrickcason.com>

* NDK cache in CI build (#191)

* added test coverage and moved tests to a new action

* added ndk caching for faster builds

* tests badge rename

* corrected url space

* corrected badge position and ndk cache permissions (#194)

* added network monitoring (#195)

* added network monitoring

* added device state checks

* removed redundant JobStatusMessage.JobError

* corrected demo app call to stop execution on fail

* disposing jobs functional

* updated demo app to accomodate dispose

* corrected tests

* added unit tests for device monitor

Co-authored-by: Patrick Cason <me@patrickcason.com>

* Docs Update http-client (#193)

* Clean mnist (#199)

* cleaning mnist

* corrected syft worker disposal

* updated readme

* made login UI full screen

* updated readme for sleep/wake support

* Charge detection (#200)

* added battery listener

* solved battery intent action bug

* corrected device monitor tests

* added robolectric tests

* remove usage of junit5 engine

* Dont skip ci to get code coverage (#203)

* docs: add galalen as a contributor (#202)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add erksch as a contributor (#205)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Documentation for SocketAPI, SocketClient, and SyftWebSocket (#201)

* Documentation for SocketAPI, SocketClient, and SyftWebSocket

* add links and additional info to documentation

* follow KDoc standard in documenting SyftWebSocket constructor params

* fix README typo

Co-authored-by: varun khare <varunkhare1234@gmail.com>

* Docs HttpAPI interface (#192)

* Docs HttpAPI

* update

* update

* feature testing (#207)

* complete run through added

* reformat

* made auth token optional

* correct handling multiple disposals

* correct singleton syft worker creation

* WIP adding tests

* constraint validation added

* correct looper robolectric

* multiple jobs tested

* add disposing in test to clear worker

* SyftJob tests (#209)

* Added first tests. Refactor SyftJob

* First tests for JobDownloader

* Added conditions to download

* Download using CycleAccept response data

* Make requestKey available to other parts of the workflow

* Test report

* Improved test

* Added test for throwError

* JobDownloader test

* Refactor downloader to use data sources and add tests

* Restrict visibility of data sources to internal

* Tests for jobDownloader

* Tests for RemoteDataSource

* Remove unused method

* Remove duplicated logic

* Pass the correct object to remote data source

* Rename and place classes

* Fix failing test

* Remove activity monitoring and API clean up (#211)

* activity detection removed

* syft job docs, restricted public APIs

* describe multi threading support (#210)

* Test for jobLocalDataSource. Refactor IO (#212)

* Tests for SyftTensor (#213)

* Test for SyftTensor deserialisation

* Fix recursive calls

* Refactor deserialize name

* Pytorch Tensor to SyftTensor test

* Cover content types

* Tests for serialize and getTorchTensor

* Model tests

* Check size of new params

* Report diff (#217)

* diff reporting added

* update running mutliple cycles

* remove open tag from configuration

* correct tests

* added tests for files

* addressing comments

* refactor asset reading

* remove tests for createDiff of repository

* added new tests for jobRepository and jobLocalDataSource

* added jobLocaldatasource test

* cosmetic change

* Speed tests optional (#219)

* Speed tests optional

* Introduce Either to handle failure when checking conditions

* No need for requiresSpeedTest to be lateinit

* Add feature flag to allow execution while the server is updated

* Remove ignored files

* Too many deletions

* Update syft-proto version

* Property for buildConfig

* Revert to 0.4.1

* correct version of SyftProto

* Remove feature flags as too many issues with integration tests

* Remove unused methods

* Small changes

* Failing test

* Leave only in use methods

* Background demo (#220)

* first draft of background execution

* add UI

* removed boilerplate code

* update syft proto dependency

* refactor paarameter mismatch

* correct default device monitoring status

* code cleanup

* work manager logging

* corrected tests

* remove unsubscribe for background tasks

* refactoring WIP

* job worker functional

* refactor attachmnistlogger

* added tests for diff coverage

* refactor view models

* rebase

* empty commit for codecov

* docs: add mccorby as a contributor (#223)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add vkkhare as a contributor (#224)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* restrict file scope (#227)

* documentation for JobStatusSubscriber (#229)

* documentation for JobStatusSubscriber

* add description of the class

* correct syft model doc

* adding tests for codecov

* Add dokka and docs dir (#225)

* Add dokka and docs dir

* Remove docs in wrong place

* Docs to root

* Documentatino workflow

* GH format

* Doc public only/

* Add docs

* Update docs and go

* Fix wrong directory

* Internals missing

* docs: add galalen as a contributor (#231)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* docs: add mccorby as a contributor (#232)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* General config (#233)

* added variable client config

* refactor plan

* regenerate the docs

* document Client Config

* correct doc references

* regenerate docs

* add tests for SyftModel

* add some more tests

* corrected http handler and added jwt token (#234)

* corrected http handler and added jwt token

* added plan name

* updated tests

* update docs

* single socket per job

* plan updated

* demo working

* added small database

* updated tests

* remove changes in network and add params tester

* README Change

Co-authored-by: varun khare <varunkhare1234@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>

* update gif (#237) (#243)

* update gif

* add installation instructions

* update readme language

* add slack badge

* Update README.md

* Update README.md

* publish action should fire only on release tags

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>

* Tutorial for reading custom dataset  (#247)

* custom dataset tutorial

* Update Part X - Train on custom dataset.ipynb

* tutorial in md

* rm notebook file

* Update Part X - Train on custom dataset.md

* Update Part X - Train on custom dataset.md

Co-authored-by: Pengyuan Zhou <pengyuan.zhou@helsinki.fi>
Co-authored-by: varun khare <varunkhare1234@gmail.com>

* docs: add pengyuan-zhou as a contributor (#250)

* docs: update README.md

* docs: update .all-contributorsrc

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* change tagging system to be compatible with nebula (#251)

* write unit tests for network status (#208)

write unit tests for network status

* revise readme (#240)

* Namespacing changes and changes from static to model centric

* updated state and model class to store ivalues

* doc update

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>
Co-authored-by: Pengyuan Zhou <zpymyyn@gmail.com>
Co-authored-by: Pengyuan Zhou <pengyuan.zhou@helsinki.fi>
Co-authored-by: Mohammed Galalen <mohammedgalalen@gmail.com>

Co-authored-by: Jose A. Corbacho <mccorby@gmail.com>
Co-authored-by: Henning Dodenhof <henning.dodenhof@gmail.com>
Co-authored-by: Ravikant Singh <40258150+IamRavikantSingh@users.noreply.github.com>
Co-authored-by: Saksham Rastogi <40931412+codeboy5@users.noreply.github.com>
Co-authored-by: Patrick Cason <me@patrickcason.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Mohammed Galalen <m.galalen96@gmail.com>
Co-authored-by: Ravikant Singh <ravikantsingh2308@gmail.com>
Co-authored-by: Pengyuan Zhou <zpymyyn@gmail.com>
Co-authored-by: Pengyuan Zhou <pengyuan.zhou@helsinki.fi>
Co-authored-by: Mohammed Galalen <mohammedgalalen@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

implement Http endpoints for PyGrid Use Connectable observable for signalling client
2 participants