{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":641533329,"defaultBranch":"main","name":"VTurlAnalyzerPy","ownerLogin":"PKHarsimran","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2023-05-16T17:07:18.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/22066581?v=4","public":true,"private":false,"isOrgOwned":false},"refInfo":{"name":"","listCacheKey":"v0:1720038241.0","currentOid":""},"activityList":{"items":[{"before":"c6183db9b0b7573d0969e3dead45f80c3d6622f7","after":"673417aa470983c3130a7c5fbdcb350237e9c19a","ref":"refs/heads/main","pushedAt":"2024-07-03T20:33:46.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update VTUAnlayzer.py\n\n#### Overview\r\nThe script has been significantly optimized and enhanced to ensure robust performance, better error handling, and maintainability. Detailed comments have been added to provide clarity on each part of the script. The major changes include automatic package installation, improved logging, progress tracking, and handling of interruptions or unexpected errors.\r\n\r\n#### Changes and Enhancements\r\n\r\n1. **Automatic Package Installation**\r\n   - **install_and_import function:** This function attempts to import a specified package and installs it using `pip` if it is not already installed. This ensures that all required packages are available when the script runs.\r\n   - **install_requirements function:** This function reads the `requirements.txt` file and installs all listed packages using the `install_and_import` function. This streamlines the setup process and ensures that dependencies are managed efficiently.\r\n\r\n2. **Improved Logging**\r\n   - **Logging Configuration:** The logging configuration is set up at the beginning of the script to log detailed information, including debug messages, errors, and warnings, to a file named `vt_analysis.log`. This helps in monitoring the script's execution and troubleshooting issues.\r\n   - **Detailed Log Messages:** The script logs detailed messages at various steps, such as when making API requests, receiving responses, processing URLs, and handling errors. This provides a clear trace of the script's activity.\r\n\r\n3. **Progress Tracking**\r\n   - **Progress Bar:** The script uses `tqdm` to display a progress bar that shows the real-time progress of URL analysis. This includes the number of URLs processed, the estimated time remaining, and the current URL being analyzed. This enhances user experience by providing visual feedback on the script's progress.\r\n\r\n4. **Error Handling and Interruption Management**\r\n   - **Exception Handling:** The script includes comprehensive error handling to manage different types of exceptions. If an HTTP error occurs during an API request, it logs the error and continues processing the remaining URLs.\r\n   - **Keyboard Interrupt Handling:** If the user interrupts the script (e.g., using Ctrl+C), the script catches the interruption, saves the partial results to a CSV file named `vt_results_partial.csv`, and logs the interruption.\r\n   - **Unexpected Error Handling:** Any unexpected errors during execution are caught, logged, and the partial results are saved to ensure no data is lost.\r\n\r\n5. **Function Documentation**\r\n   - **Detailed Comments:** Each function in the script is thoroughly documented with comments explaining its purpose, parameters, and the steps it performs. This improves code readability and maintainability.\r\n\r\n6. **Core Functions**\r\n   - **get_url_report function:** Makes a request to the VirusTotal API to retrieve the report for a given URL, parses the JSON response, extracts relevant information, and returns it as a dictionary.\r\n   - **analyze_urls function:** Iterates over a list of URLs, uses the `get_url_report` function to analyze each URL, and logs the progress. It handles rate limiting by ensuring at least 15 seconds between requests.\r\n   - **save_partial_results function:** Saves the partial results to a CSV file if the script is interrupted or an error occurs.\r\n   - **main function:** Reads the input CSV file, extracts URLs, analyzes them using the `analyze_urls` function, and saves the final results to a CSV file.\r\n\r\nThese changes and enhancements ensure that the script is more robust, user-friendly, and easier to maintain, providing detailed logging and error management capabilities.","shortMessageHtmlLink":"Update VTUAnlayzer.py"}},{"before":"0b2f646cface2878ac6fe71c347faa674048416f","after":"c6183db9b0b7573d0969e3dead45f80c3d6622f7","ref":"refs/heads/main","pushedAt":"2024-07-03T20:26:27.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"23ca1619f838630ef5a477f916a66080a171dcfd","after":"0b2f646cface2878ac6fe71c347faa674048416f","ref":"refs/heads/main","pushedAt":"2024-07-03T20:25:00.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"225a203814256c45e232620b0567f82a3472797e","after":null,"ref":"refs/heads/ErrorHandlingImprov","pushedAt":"2024-07-03T20:24:01.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"}},{"before":"a5408bb0927c94a9daba3021a8f4e0a6365cfe36","after":"23ca1619f838630ef5a477f916a66080a171dcfd","ref":"refs/heads/main","pushedAt":"2024-07-03T20:23:59.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Merge pull request #2 from PKHarsimran/ErrorHandlingImprov\n\nfeat: Enhance VirusTotal URL Analysis Script with robust error handli…","shortMessageHtmlLink":"Merge pull request <a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2389407540\" data-permission-text=\"Title is private\" data-url=\"https://github.com/PKHarsimran/VTurlAnalyzerPy/issues/2\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/PKHarsimran/VTurlAnalyzerPy/pull/2/hovercard\" href=\"https://github.com/PKHarsimran/VTurlAnalyzerPy/pull/2\">#2</a> from PKHarsimran/ErrorHandlingImprov"}},{"before":"a5408bb0927c94a9daba3021a8f4e0a6365cfe36","after":"225a203814256c45e232620b0567f82a3472797e","ref":"refs/heads/ErrorHandlingImprov","pushedAt":"2024-07-03T20:23:32.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"feat: Enhance VirusTotal URL Analysis Script with robust error handling, detailed logging, and dynamic package installation\n\n- Added function to install required packages dynamically from requirements.txt.\n- Implemented detailed logging for each step of the script, including HTTP requests and responses.\n- Enhanced progress reporting with tqdm progress bar.\n- Added robust error handling for HTTP errors, unexpected responses, and script interruptions.\n- Saved partial results in case of script interruption or errors.\n- Improved readability and maintainability with detailed comments throughout the script.\n- Updated README.md to reflect new features and usage instructions.\n- Created requirements.txt with necessary dependencies.\n\nChanges:\n- install_and_import: Function to install packages dynamically.\n- install_requirements: Function to install packages from requirements.txt.\n- Logging improvements: Detailed logging for each URL processed.\n- Error handling: Handled KeyboardInterrupt and other exceptions to save partial results.\n- tqdm progress bar: Added to show real-time progress.","shortMessageHtmlLink":"feat: Enhance VirusTotal URL Analysis Script with robust error handli…"}},{"before":null,"after":"a5408bb0927c94a9daba3021a8f4e0a6365cfe36","ref":"refs/heads/ErrorHandlingImprov","pushedAt":"2024-07-03T19:53:34.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Delete url.txt","shortMessageHtmlLink":"Delete url.txt"}},{"before":"b5afd55b5aea37dc7d1532dd8fab81b8fdd21c8f","after":"a5408bb0927c94a9daba3021a8f4e0a6365cfe36","ref":"refs/heads/main","pushedAt":"2024-07-03T17:51:19.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Delete url.txt","shortMessageHtmlLink":"Delete url.txt"}},{"before":"27852eb71070ed5b8984ae09f0d5aeb16e67db40","after":"b5afd55b5aea37dc7d1532dd8fab81b8fdd21c8f","ref":"refs/heads/main","pushedAt":"2024-07-03T17:50:41.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update VTUAnlayzer.py","shortMessageHtmlLink":"Update VTUAnlayzer.py"}},{"before":"27000aece4c9c94cab426b74d87235b766424750","after":"27852eb71070ed5b8984ae09f0d5aeb16e67db40","ref":"refs/heads/main","pushedAt":"2024-07-03T17:46:25.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Create LICENSE","shortMessageHtmlLink":"Create LICENSE"}},{"before":"327612ca35bbfa428926f4302f0599be06902cae","after":"27000aece4c9c94cab426b74d87235b766424750","ref":"refs/heads/main","pushedAt":"2024-07-03T17:46:08.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Create CODE_OF_CONDUCT.md","shortMessageHtmlLink":"Create CODE_OF_CONDUCT.md"}},{"before":"0a8cbcb7f78d9f469893a3dd26e7df16693c8253","after":null,"ref":"refs/heads/OptionToReadCSV","pushedAt":"2024-07-03T17:43:01.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"}},{"before":"b77418fd100b955b7ba6425cb484d957d4547126","after":"327612ca35bbfa428926f4302f0599be06902cae","ref":"refs/heads/main","pushedAt":"2024-07-03T17:42:57.000Z","pushType":"pr_merge","commitsCount":3,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Merge pull request #1 from PKHarsimran/OptionToReadCSV\n\nAdd Detailed Documentation and Improve VirusTotal URL Analysis Script","shortMessageHtmlLink":"Merge pull request <a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2389145447\" data-permission-text=\"Title is private\" data-url=\"https://github.com/PKHarsimran/VTurlAnalyzerPy/issues/1\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/PKHarsimran/VTurlAnalyzerPy/pull/1/hovercard\" href=\"https://github.com/PKHarsimran/VTurlAnalyzerPy/pull/1\">#1</a> from PKHarsimran/OptionToReadCSV"}},{"before":"cd1bf71985dc1492d018a72cf42b410d1be1561a","after":"0a8cbcb7f78d9f469893a3dd26e7df16693c8253","ref":"refs/heads/OptionToReadCSV","pushedAt":"2024-07-03T17:40:34.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"feat: Add detailed README and dynamic package installation to VirusTotal URL Analysis Script\n\n- Added a comprehensive README.md with sections on features, requirements, installation, setup, usage, output, logging, troubleshooting, license, and acknowledgments.\n- Implemented dynamic package installation to automatically install required packages if they are not already installed.\n- Improved error handling and logging for better debugging and user experience.\n- Ensured detailed output CSV and log files for analysis and troubleshooting.","shortMessageHtmlLink":"feat: Add detailed README and dynamic package installation to VirusTo…"}},{"before":"b77418fd100b955b7ba6425cb484d957d4547126","after":"cd1bf71985dc1492d018a72cf42b410d1be1561a","ref":"refs/heads/OptionToReadCSV","pushedAt":"2024-07-03T17:07:22.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"feat: Update script to read domains from CSV and analyze using VirusTotal API\n\n- Added pandas library to read CSV file.\n- Modified script to extract domains from the 'Domain' column in the CSV.\n- Updated analyze_urls function to process the extracted list of domains.\n- Ensured compatibility with existing VirusTotal API request and response handling.\n- Enhanced main function to read the CSV file and initiate URL analysis.\n\nThis update allows the script to directly read domain data from a CSV file, improving usability and efficiency in processing domain reports from VirusTotal.","shortMessageHtmlLink":"feat: Update script to read domains from CSV and analyze using VirusT…"}},{"before":null,"after":"b77418fd100b955b7ba6425cb484d957d4547126","ref":"refs/heads/OptionToReadCSV","pushedAt":"2023-05-23T16:09:43.569Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update VTUAnlayzer.py","shortMessageHtmlLink":"Update VTUAnlayzer.py"}},{"before":"f934bc2bc51637bf0f54e7a639353d61b6b8a519","after":"b77418fd100b955b7ba6425cb484d957d4547126","ref":"refs/heads/main","pushedAt":"2023-05-22T15:05:21.243Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update VTUAnlayzer.py","shortMessageHtmlLink":"Update VTUAnlayzer.py"}},{"before":"3593ed69dc2e5ac0fef73084c1a24036e1fea352","after":"f934bc2bc51637bf0f54e7a639353d61b6b8a519","ref":"refs/heads/main","pushedAt":"2023-05-22T14:43:07.887Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"d526b9c727a125c350e5c5c0c758cf22f80c9a9c","after":"3593ed69dc2e5ac0fef73084c1a24036e1fea352","ref":"refs/heads/main","pushedAt":"2023-05-22T14:42:49.779Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"4a1345b4182a3e8e47587efeded2124d12f33a92","after":"d526b9c727a125c350e5c5c0c758cf22f80c9a9c","ref":"refs/heads/main","pushedAt":"2023-05-22T14:26:01.138Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"7aa6507234429b19e9142e7684829d94757bff52","after":"4a1345b4182a3e8e47587efeded2124d12f33a92","ref":"refs/heads/main","pushedAt":"2023-05-22T14:25:42.881Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"e15113f6a0b2785988fa3f3acc370f3443d52c69","after":"7aa6507234429b19e9142e7684829d94757bff52","ref":"refs/heads/main","pushedAt":"2023-05-22T14:23:33.477Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"fa1fd601b5082f2ff945cc2336198cf0d61810e4","after":"e15113f6a0b2785988fa3f3acc370f3443d52c69","ref":"refs/heads/main","pushedAt":"2023-05-22T14:22:22.821Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"22814233c70b28ef7fdda513f254d758d0aa3e7d","after":"fa1fd601b5082f2ff945cc2336198cf0d61810e4","ref":"refs/heads/main","pushedAt":"2023-05-22T14:22:09.398Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"880b2bab886d3cd9212cdfeb4b1386b58acd16b7","after":"22814233c70b28ef7fdda513f254d758d0aa3e7d","ref":"refs/heads/main","pushedAt":"2023-05-22T14:21:55.392Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"d35a4287978817e1b1ae5b8ccb36e7f462133c68","after":"880b2bab886d3cd9212cdfeb4b1386b58acd16b7","ref":"refs/heads/main","pushedAt":"2023-05-22T14:21:45.715Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"1daacb64059f5638ebd9c9eab692f51729942036","after":"d35a4287978817e1b1ae5b8ccb36e7f462133c68","ref":"refs/heads/main","pushedAt":"2023-05-22T14:19:42.202Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"Update VTUAnlayzer.py","shortMessageHtmlLink":"Update VTUAnlayzer.py"}},{"before":"9c5834fa69de5e8c70a9562dbded08bb54c55006","after":"1daacb64059f5638ebd9c9eab692f51729942036","ref":"refs/heads/main","pushedAt":"2023-05-22T03:07:04.370Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"comments added","shortMessageHtmlLink":"comments added"}},{"before":"95e467e84d83e4616b77e034d2c70cf7cfd86a45","after":"9c5834fa69de5e8c70a9562dbded08bb54c55006","ref":"refs/heads/main","pushedAt":"2023-05-22T03:02:04.846Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"11","shortMessageHtmlLink":"11"}},{"before":"3804a978d1c57eca0d5172f014b3a541aab00946","after":"95e467e84d83e4616b77e034d2c70cf7cfd86a45","ref":"refs/heads/main","pushedAt":"2023-05-22T03:01:19.953Z","pushType":"push","commitsCount":1,"pusher":{"login":"PKHarsimran","name":"Harsimran Sidhu","path":"/PKHarsimran","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22066581?s=80&v=4"},"commit":{"message":"test","shortMessageHtmlLink":"test"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEdkgJUQA","startCursor":null,"endCursor":null}},"title":"Activity · PKHarsimran/VTurlAnalyzerPy"}