Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In GShop, parsing Price Missing #5

Closed
spsyco opened this issue Aug 7, 2021 · 9 comments
Closed

In GShop, parsing Price Missing #5

spsyco opened this issue Aug 7, 2021 · 9 comments

Comments

@spsyco
Copy link

spsyco commented Aug 7, 2021

Gshop json is not able to show price , can anyone tell what is the problem in it.
i guess while scraping wrong scraping parameter was taken.
i request Michael, please look in it and help to solve this or give me any idea what to do,,,
i thing due to site changes in google web pages changes their xpath or class or id so that money not able to scrap

@michael-act
Copy link
Owner

michael-act commented Aug 7, 2021

Please kindly check new video below.
I have added more parameters to define Price Code in your Country.

https://youtu.be/aIZFELGtfWY?t=146

Just additional other information:
I'm guessing you're a data practitioner, you can use to_pd() to export into DataFrame.

Thank you :)

@spsyco
Copy link
Author

spsyco commented Aug 7, 2021

= RESTART: C:\Users\Spsyco\Downloads\glowing-pulse-form\Senginta.py-main\Senginta.py-main\senginta\tests.py
title 40
price 0
e-commerce 40
src-link 40
img-link 40
Traceback (most recent call last):
File "C:\Users\Spsyco\Downloads\glowing-pulse-form\Senginta.py-main\Senginta.py-main\senginta\tests.py", line 25, in
print(GoogleShops.to_json())
File "C:\Users\Spsyco\Downloads\glowing-pulse-form\Senginta.py-main\Senginta.py-main\senginta\static\search.py", line 108, in to_json
result = self.to_pd().to_json(orient='index')
File "C:\Users\Spsyco\Downloads\glowing-pulse-form\Senginta.py-main\Senginta.py-main\senginta\static\search.py", line 93, in to_pd
self.dataframe = pd.DataFrame(self.get_all())
File "C:\Users\Spsyco\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\frame.py", line 614, in init
mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy, typ=manager)
File "C:\Users\Spsyco\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\internals\construction.py", line 464, in dict_to_mgr
return arrays_to_mgr(
File "C:\Users\Spsyco\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\internals\construction.py", line 119, in arrays_to_mgr
index = _extract_index(arrays)
File "C:\Users\Spsyco\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\internals\construction.py", line 635, in _extract_index
raise ValueError("All arrays must be of the same length")
ValueError: All arrays must be of the same length

Seems like error so i tried with the get_all() and to_pd() as well but in the ouput same issue found the price key not added values prices missing

title 40
price 0
e-commerce 40
src-link 40
img-link 40

above is the output for to_pd and get_all() kindly check not able to scrap the price of the product

@spsyco
Copy link
Author

spsyco commented Aug 7, 2021

['Tv Remote', 'Tv TV TV Remote', 'TV', '₹199.00', 'Remote', 'TV Remote', '₹348.00', 'Remote', 'Tv', 'Remote', 'Remote', '₹110.00', 'Remote', 'TV', '₹449.00', 'TV Remote', 'TV', 'Remote', '₹230.00', 'Tv', 'Remote', 'Tv', 'Remote', 'Tv', 'TV', '₹209.00', 'TV', 'Remote', '₹280.00', 'Remote', 'Remote', 'REMOTE', 'Remote', '₹209.00', 'remote', 'Remotes', 'TV', '₹270.00', 'Remote', 'TV', 'TV', 'tv', '₹249.00', 'Tv Remote', 'Tv TV TV Remote', 'tv', '₹349.00', 'Remotes', 'Remote', 'Remote', 'Remotes', 'REMOTE', 'Remote', '₹209.00', 'Tv Remote', 'TV TV Remote', '₹209.00', 'Remote', 'Remote', '₹209.00', 'Tv', 'Remote', 'Remote', 'TV', 'Remote', 'Remote', '₹209.00', 'Tv', 'Remote', '₹199.00', 'Remote', '₹155.00', 'Tv', 'Remote', '₹249.00', 'Remote', 'Remote', 'REMOTE', '₹209.00', 'Remote', 'Tv', 'TV', '₹189.00', 'Tv', 'Remote', 'Tv', 'Remote', '₹258.00', 'Tv Remote', 'Tv', 'TV TV Remote', 'TV', '₹349.00', 'TV Remote', '₹249.00', 'Remote', 'TV', 'Remote', '₹209.00', 'Remote', 'Remote', '₹399.00', 'Tv', 'Remote', 'Remote', '₹227.00', 'Remote', 'Remote', 'Remote', '₹209.00', 'Tv Remote', 'TV TV Remote', '₹209.00', 'Remote', 'TV', 'Remote', 'TV', 'Remote', '₹301.00', 'Remote', 'TV', 'Remote', '₹249.00', 'Remote', 'Remote', 'Remote', 'TV', 'Remote', '₹209.00', 'Remote', 'Remote', 'TV', '₹299.00', 'Remote', 'TV', 'Remote', '₹379.00', 'TV', 'Remote', 'TV', '₹299.00', 'Remote', 'TV', 'TV', '₹339.00', 'Remote', 'Remote', 'TV', 'Remote', '₹399.00', 'Remote', 'TV', '₹215.00', 'Remote', 'Remote', 'TV', 'Remote', '₹219.00', 'Tv Remote', 'Tv TV TV Remote', 'TV', '₹239.00', 'TV', 'Remote', 'TV', '₹289.00', 'Remote', 'Remote', 'TV', 'Remote', 'Tv', 'Tv', 'Tv Remote', 'Remote', 'Tv', 'Remote Remote', 'Tv', 'Remote', 'Remote', 'Remote', 'Tv', 'Remote', 'Remote', 'Tv', 'Remote', 'Tv', 'Remote', 'Tv', 'Remote', 'Remote', 'Tv', 'Remote', 'Tv', 'Remote', 'Tv', 'Remote', 'Tv', 'Remote', 'TV Remote', 'Tv', 'Remote', 'Tv', 'Remote', 'Tv Remote', 'Tv', 'Remote', 'Tv', 'Remote', 'TV', 'Remote', 'Remote', 'TV', 'Remote', 'tv', 'Tv', 'Remote', 'Tv', 'Remote', 'Tv Remote', 'Tv TV TV Remote', 'TV', 'Remote', 'Tv', 'Tv', 'Remote', 'Tv', 'Remote', 'TV Remote', 'Remote', 'Tv Remote', 'Remote', 'TV', 'Remote', 'Tv Remote', 'Remote', 'Remote', 'Remote', 'Remote', 'Remote', 'Tv', 'TV Remote', 'Remote', 'TV Remote', 'Remote', 'tv', 'Remote', 'Remote', 'Remote', 'TV', 'Tv', 'Remote', 'Tv', 'Remote', 'Remote', 'Remote', 'Remote', 'Remote', 'Remote', 'TV Remote', 'Remote', 'REMOTE', 'Remote', 'TV Remote', 'Tv Remote', 'Remote', 'TV', 'Remote', 'TV Remote', 'TV', 'Tv Remote', 'Remote', 'TV', 'Remote', 'Remote', 'Remote', 'TV', 'Remote', 'TV', 'Remote', 'Remote', 'Tv Remote', 'Remote', 'Remote', 'Remote', 'Remote', 'TV Remotes', 'Remote', 'TV Remote', 'Remote', 'TV Remote', 'Remote', 'Remote', 'Remote', 'Remote', 'TV', 'Remote', 'Tv Remote', 'Remote', 'Remote', 'Remote', 'Tv', 'Remote', 'Remote', 'Remote', 'tv', 'Remote', 'Remote', 'Remote', 'TV']

Output for B tag while calling get_price() so that prices is coming but seems like other unnecessary tags data also coming

@michael-act
Copy link
Owner

michael-act commented Aug 8, 2021

Have you tried out this code?

GShop('Remove TV', '₹', 1, 2)

@spsyco
Copy link
Author

spsyco commented Aug 8, 2021

Traceback (most recent call last):
File "C:\Users\Spsyco\Downloads\glowing-pulse-form\ftest-main\ftest-main\testr.py", line 3, in
print(GoogleShops.get_all('Remove TV', '₹', 1, 2))
TypeError: get_all() takes 1 positional argument but 5 were given

May be Price is not able to scrap from webpage, can you try once in your system please
get_main(), get_price tags getting lots of unnecessary data with price

@michael-act
Copy link
Owner

michael-act commented Aug 8, 2021

Haha, sorry. I mean GShop('Remove TV', '₹', 1, 2) .

@spsyco
Copy link
Author

spsyco commented Aug 8, 2021

Hey Michael, It is working thanks
what was my fault is i have not replaced the right currency symbol while calling Gshop

GoogleShops = GShop('Remote TV', '₹', 1, 3)
print(GoogleShops.to_pd())

the above is right for india and it is working fine thanks Michael again and this will help me in my collage final year project

@spsyco spsyco closed this as completed Aug 8, 2021
@michael-act
Copy link
Owner

Thank's. It's my old project, I'm sorry I don't really remember what is defined on the code. So, I'm sorry for not becoming a good Repository Owner :(.

Nice to see somebody who using this for Final Year Project. Can you tell me through this issue if the project have been built?

@spsyco
Copy link
Author

spsyco commented Aug 8, 2021

yeah sure, when it will be completed, I'll remind you don't worry

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants