-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mike feature/split crawler logging #13
Conversation
As per toddy's request. OFAScraper now belongs to Dao
The output was producing incorrect results. Should be fixed now.
Split up the crawler and scraper to create more specialized module's.
Changed how description is written in output.
Code was only looking at 1 url no matter what. now its fixed.
Logging done, need review.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Content is readable and easy to use. Works well for me.
Felt I needed to switch the name now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Logging logic output to terminal is easy to understand how and where we at with the scrapping. I suggest Mike to add empty line between each link for more readable.
Crawler was split for Dao and I added proper logging. Please review.