Basically, a web crawler that people know is a computer program that explores the World Wide Web in an organized and automated way. The program we're trying to create is to create a crawler in a way that users type in to find the information they want. Our program is free and anyone can use it.
Come in and watch it a lot.
How to use
There is no need to inststall!
1.Just connect htt[://172.31.37.114:5000
2.So the program will run in the web server environment.
3.next Run Window
4.Web init Design
5.Enter a Word or http address and Click the Run button!
File list
test_naver.py
-Requset
imformation from http.
-beautifulsoup
html to python.
-Flask Web Server play into Server.
-JSON data
Whenever JSON data is used, it can be easily used as a variable without visiting the key, value.
-Play
press F12.. and checked yello press
You want to press category.
click right button. select copy - copy selector
put the to copy thing. Scrap HTML Tag and Site Link Address
example>
-Result
-Mantis
How to repot bugs
For Windows, IIS, Apache, PHP, and MySQL must be installed.
But there is a freeware APM that installs at once.
You can find more information in the repository.
Contribute
There are many ways in which you can participate in the project, for example:
-Submit bugs and feature requests, and help us verify as they are checked in
-Review source code changes
-Review the documentation and make pull requests for anything from typos to new content
If you are interested in fixing issues and contributing directly to the code base, please see the document How to Contribute, which covers the following:
-How to build and run from source
-The development workflow, including debugging and running tests
-Coding guidelines
-Submitting pull requests
-Contributing to translations
License
GPL
You can see all data in the repository.
Essentials before joining the project
-Python Installation
Python Installation Site - https://www.python.org/
-Module installation - can be installed with Python pipe.
BeautifulSoup4
c:\ >pip install beatsoup4
-Request
c:\ >pip install requests