Use static scraping libraries instead of headless browser to make it more performant

[jayantkatia]

Current implementation

chromedp which uses headless browser to scrap information.

Better solution

Use static scraping libraries and make same network calls which the website makes internally (passing same query params and request headers).

Example,

curl --location --request GET 'https://www.91mobiles.com/template/category_finder/finder_ajax.php?show_next=1&ord=0.17428677812670812&excludeId=&hash=&search=&hidFrmSubFlag=1&page=1&category=mobile&unique_sort=ga_views&gaCategory=Upcoming+Mobiles+Price+List+in+India-filter&requestType=1&showPagination=1&listType=list&listType_v3=list&listType_v1=list&listType_v2=list&listType_v4=list&listType_v5=list&listType_v6=list&page_type=upcoming&finderRuleUrl=&selMobSort=ga_views&hdnCategory=mobile&user_search=&url_feat_rule=upcoming-mobiles-in-india&buygaCat=upcoming-mob&amount=0%3B200000&sCatName=mobile&price_range_apply=1&tr_fl%5B%5D=mob_market_status_filter.marketstatus_filter%3Aupcoming&tr_fl%5B%5D=mob_market_status_filter.marketstatus_filter%3Arumoured' \
--header 'x-requested-with: XMLHttpRequest' \
--header 'user-agent: Mozilla/5.0'

Massive overhaul

Since this will lead to massive changes, i propose using another branch