Skip to content

Add support for ORG's blocked.org.uk scheduler#406

Merged
hellais merged 2 commits intoooni:masterfrom
dantheta:org_queue_PR
Feb 5, 2016
Merged

Add support for ORG's blocked.org.uk scheduler#406
hellais merged 2 commits intoooni:masterfrom
dantheta:org_queue_PR

Conversation

@dantheta
Copy link
Copy Markdown
Contributor

Add:
* ORG Queue integration
* Support for additional values to send to ORG backend

Refactor director startup
* Moved collector setup to shared scope
* Moved deck setup to shared scope
* Moved annotations setup to shared scope
* Moved global options to shared scope

* fix some naming convention errors

* move global counter to more sensible scope
* fixes from pull request code review
* remove another mobile browser
* remove iPhone useragent to prevent mobile versions being returned to client

* add HTTPS support for reporter
* add authentication info from environment
* import PROBE_ vars from environment into report
* allow https collector in regex
* AMQP connection details as cmdline parameter
* more queue state, looping listen call
* remove old bin script
* exit after bounded random number of test decks
* improve error handling
* use proper custom exception for queue timeout reason
* reduce duplicate serialization
* add flag to prevent response body being written to report
* add subargs to queue createdeck
* fix global args passing
* reuse cached geoip
* reuse existing TOR connection
* simplify for one-shot use
* extend and report lifetime
* probe exit on AMQP error
* remove ooniresource from setup script

@hellais
Copy link
Copy Markdown
Member

hellais commented Sep 2, 2015

Thanks a lot for this good work.

I will review this PR in the upcoming days.

@hellais
Copy link
Copy Markdown
Member

hellais commented Sep 3, 2015

@dantheta Is it correct to assume that this PR superseeds #395?

@dantheta
Copy link
Copy Markdown
Contributor Author

dantheta commented Sep 3, 2015

Yes, that's right. The refactoring changes inside ooni/oonicli.py were quite substantial.

@hellais hellais mentioned this pull request Sep 3, 2015
@bassosimone bassosimone self-assigned this Nov 2, 2015
@bassosimone
Copy link
Copy Markdown
Contributor

Hi @dantheta, there is a merge conflict in geoip.py and as such this pull request does not merge cleanly. Could you please rebase against master and force push? Thanks!

    Add support for ORG's blocked.org.uk scheduler

    Add:
    * ORG Queue integration
    * Support for additional values to send to ORG backend

    Refactor director startup
    * Moved collector setup to shared scope
    * Moved deck setup to shared scope
    * Moved annotations setup to shared scope
    * Moved global options to shared scope

    * fix some naming convention errors

    * move global counter to more sensible scope
    * fixes from pull request code review
    * remove another mobile browser
    * remove iPhone useragent to prevent mobile versions being returned to client

    * add HTTPS support for reporter
    * add authentication info from environment
    * import PROBE_ vars from environment into report
    * allow https collector in regex
    * AMQP connection details as cmdline parameter
    * more queue state, looping listen call
    * remove old bin script
    * exit after bounded random number of test decks
    * improve error handling
    * use proper custom exception for queue timeout reason
    * reduce duplicate serialization
    * add flag to prevent response body being written to report
    * add subargs to queue createdeck
    * fix global args passing
    * reuse cached geoip
    * reuse existing TOR connection
    * simplify for one-shot use
    * extend and report lifetime
    * probe exit on AMQP error
    * remove ooniresource from setup script
@dantheta
Copy link
Copy Markdown
Contributor Author

dantheta commented Nov 3, 2015

That's done. Many thanks!

@bassosimone
Copy link
Copy Markdown
Contributor

Thanks! I'm now about at 75% of reading-the-diff-and-reasoning-about it. Next, I'll provide comments and start testing the code.

Comment thread ooni/oonicli.py
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: the two functions above have been moved here without modifications from the old runWithDirector() code.

@bassosimone
Copy link
Copy Markdown
Contributor

Have made progress with the review, added many comments, checked that the most substantial comments to #395 are also applied here. Still need to review the daemon director part and do testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants