Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement ASN database - backend side #406

Closed
FedericoCeratto opened this issue May 11, 2020 · 6 comments
Closed

Implement ASN database - backend side #406

FedericoCeratto opened this issue May 11, 2020 · 6 comments
Assignees
@FedericoCeratto
Labels
ooni/backend Issues related to https://github.com/ooni/backend priority/high
Milestone
Sprint 17 - Θέτις

Comments

@FedericoCeratto
Copy link
Contributor

Related to ooni/probe-engine#584
This issue track the backend-related work on the ASN database

  • Investigate data quality
  • Automate generation and test
    • Deploy the perl library on github actions or other CI system
    • Run generation based on cronjobs
    • Store historical versions on S3 [in a later issue]
  • Deploy the ASN db in the pipeline and use it when processing msmts [in a later issue]
@FedericoCeratto FedericoCeratto added this to the Sprint 12 - Beluga milestone May 11, 2020
@FedericoCeratto FedericoCeratto self-assigned this May 11, 2020
@bassosimone bassosimone mentioned this issue May 11, 2020
Closed
2 tasks
@FedericoCeratto FedericoCeratto added priority/high ooni/backend Issues related to https://github.com/ooni/backend labels May 11, 2020
@bassosimone bassosimone mentioned this issue May 22, 2020
Closed
4 tasks
@FedericoCeratto
Copy link
Contributor Author

First fully functional version of the database generated: it contains networks, ASNs and organization names and the data are very consistent with other sources.
Next steps: automate generation, store output in s3 or elsewhere

@FedericoCeratto
Copy link
Contributor Author

Lowering estimate to 5 as most of the work was done

@FedericoCeratto
Copy link
Contributor Author

Scripts updated to also support using RIPE ASN->AS name maps instead of CAIDA

@bassosimone bassosimone mentioned this issue May 29, 2020
Closed
2 tasks
@bassosimone bassosimone mentioned this issue Jun 9, 2020
Closed
1 task
@FedericoCeratto
Copy link
Contributor Author

TODO: While pending updates on copyright on ASN data, publish the script with instructions on manual run.

@FedericoCeratto
Copy link
Contributor Author

Updated scripts to use CAIDA data. Generated new db for the next release.

@bassosimone bassosimone mentioned this issue Jun 22, 2020
Closed
4 tasks
@FedericoCeratto
Copy link
Contributor Author

Next steps: automate monthly generation and publishing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@FedericoCeratto FedericoCeratto

Labels
ooni/backend Issues related to https://github.com/ooni/backend priority/high
Projects
None yet
Milestone
Sprint 17 - Θέτις
Development

No branches or pull requests
1 participant
@FedericoCeratto