Refactor the crawler to use Developers Italia API #302
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bfabio
force-pushed
the
developers-italia-api
branch
3 times, most recently
from
914e2dd
to
bef2a25
Compare
bfabio
force-pushed
the
developers-italia-api
branch
2 times, most recently
from
e090a87
to
0ae85a9
Compare
bfabio
force-pushed
the
developers-italia-api
branch
from
0ae85a9
to
4965539
Compare
CanonicalURL represents the canonical URL of a repository that the code hosting platforms resolves to when given an alias (fe. old name before a repo rename).
bfabio
force-pushed
the
developers-italia-api
branch
6 times, most recently
from
1956f28
to
fc85ab6
Compare
bfabio
changed the title
sebbalex
approved these changes
Oct 13, 2022
Add an API client with convenience methods to use the Developers Italia API.
The delete command is just an API call now, having it in the crawler is not more convenient than just using curl.
Remove blacklists, they are handled at API level with the "active" fields in software and publishers. Fix italia#179.
The filename is not expected to change, since it's standard mandated.
Introduce SKIP_VITALITY configuration to skip the repository clone and vitality calculation.
Don't panic if config.toml is missing, but instead fallback to environment variables (as per README).
bfabio
force-pushed
the
developers-italia-api
branch
from
f8517fb
to
ab40b4d
Compare
This was referenced Oct 17, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adapt the crawler to use Developers Italia API