Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve post details scraping #343

Closed
9 of 10 tasks
DEENUU1 opened this issue Oct 4, 2023 · 0 comments
Closed
9 of 10 tasks

Improve post details scraping #343

DEENUU1 opened this issue Oct 4, 2023 · 0 comments
Labels
todo

Comments

@DEENUU1
Copy link
Owner

DEENUU1 commented Oct 4, 2023 •
edited
Loading

  • Scrape number of likes
  • Scrape number of shares and comments
  • Scrape content (text)
  • Scrape images (URL)
  • Scrape post's author
  • Update pipeline
  • Refactor
  • Update Post model
  • Save scraped data to database
  • Fix schemas and fastapi server for new Post model
@DEENUU1 DEENUU1 added bug Something isn't working todo and removed bug Something isn't working labels Oct 4, 2023
DEENUU1 pushed a commit that referenced this issue Oct 4, 2023
Improve PostDetail scraper - for 'posts' scrape - all images, content…
750640b 
…, number of likes and author (#343)
DEENUU1 pushed a commit that referenced this issue Oct 4, 2023
Code refactor (#343)
17a1938
DEENUU1 pushed a commit that referenced this issue Oct 4, 2023
Add image_urls as a JSON field to Post model (#343)
0fb6cba
DEENUU1 pushed a commit that referenced this issue Oct 4, 2023
Add pipeline for scraping post details for specified Person or just b…
d65993b 
…y passing an url for Post (#343)
DEENUU1 pushed a commit that referenced this issue Oct 4, 2023
Display Post data in fastapi application (#343)
876151c
DEENUU1 pushed a commit that referenced this issue Oct 4, 2023
Update pytest for Post model (#343)
a1a5776
@DEENUU1 DEENUU1 closed this as completed Oct 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
todo
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@DEENUU1