[enhancement] Added WhoIs details alongside devices #1065
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
DragnEmperor
moved this to In Progress
in [GSoC25] WHOIS Information and IP Address-Based Geolocation
|
There are some commits which are related to another PR #1054. Once its merged, this branch will be rebased. |
Added WHOIS model with the required fields. Implemented a new celery task to fetch WHOIS details using the `geoip2` web service. In order to trigger the task whenever `last_ip` changes, using existing logic of `_changed_fields` to track the changes. Closes #1032 Closes #1033 Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
Added tests for checking if `fetch_whois_details` task is called properly or not, and for checking creation of WHOIS record for a device if the last_ip recorded is public. As now whois record is also fetched whenever fetching a device, there is an increase in query count in some of the tests. Closes #1045 Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
Closes #1037 Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
Using JSONField for address gives the flexibility to format address as per convenience. Added fallbacks for typical address fields in `fetch_whois_details`. Updated docs for WhoIs feature with steps to obtain and setup the required Credentials. Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
To reduce data redundancy for duplicate IPs, WhoIs model now uses IP address as primary key and devices are mapped to it via manual lookups. Updated the test cases to reflect the same. Formatting changes in WhoIs doc page and settings variables. Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
To improve modularity and flexibility, all WhoIs related code is kept under `whois` subdirectory under `config` module. This includes tests, tasks, on delete handlers as well. Added `cache` to prevent repeated lookups during multiple WhoIs details fetch during device listing. Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
Moved all tasks to service file to keep one source. Refactored the way whois_info is fetcehd for a device. Modified the order of whois checks to keep expensive checks at the bottom. As in views like `DeviceChecksumView` whole device instance is being cached, now org settings will be fetched from db rather than through device to maintain consistency. This has led to increase in queries in some test cases. Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
Org config settings are now cached to ensure DeviceChecksumview is not degraded. This will be invalidated on org settings update/delete which are rare. Refactored code and tests for readability. Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
Migrated all receivers related to WhoIs to the model class. Includes minor refactoring as well. Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
The imports are restructured to avoid app registry errors and aid whois task discovery. Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
There was an issue that the task was triggered for a device whose last_ip is updated but without checking if we already have WhoIs for the latest ip. Have added a check to return from the task if that is the case. Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
The checks in `_need_who_is_lookup` are split for better readability Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
Added new test cases and validator for who_is model fields. Also refactored the way address is being computed Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
To get who_is details in device listing and detail views, added BriefWhoIsSerializer and WhoIsSerializer respectively. As these are accessed via SerializerMethodField, created a Mixin to reduce duplicated logic. Closes #1026 Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
DragnEmperor
force-pushed
the
issues/1026-whois-display
branch
from
b50598f to
a9b7891
Compare
Signed-off-by: DragnEmperor <dragnemperor@gmail.com>
DragnEmperor
force-pushed
the
issues/1026-whois-display
branch
from
a9b7891 to
dddfb45
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checklist
Reference to Existing Issue
Closes #1026.
Description of Changes
To fetch WhoIs details in device REST APIs, created two serializers :
BriefWhoIsSerializerandWhoIsSerializerfor listing and detail endpoints respectively