adding proxy to x-ray node js

[bassemAmous]

Subject of the issue

Describe your issue here.

Your environment

• version of node: run node --version
• version of npm: run npm --version

Steps to reproduce

Tell us how to reproduce this issue.

Expected behaviour

Tell us what should happen.

Actual behaviour

Tell us what happens instead.

[bassemAmous]

I am developing a node js application and I am scraping a web site using x-ray and I think the blocked my ip address so how can I configure a proxy or hide my Ip address for that ? here is my code:

var Xray = require('x-ray');
var x = Xray()
x('https://www.myurl.com',  {


    title: x('#cm_cr-review_list .a-section.review', [{,
        blogs:"..a-text-bold"
    }]),
})
    .paginate('li.a-l a@href')
    .write('result.json')

[0xgeert]

1. x-ray allows for custom [drivers]( https://github.com/matthewmueller/x-ray#xraydriverdriver) .. E.g.: see prev link to a [request](https://github.com/request/request) driver. 2. adapt request driver to point to a [proxy]( https://github.com/request/request#proxies) 3. You can choose proxies yourself randomly/ round-robin, etc. (you need to build that into your driver) or choose a managed rotating-proxy. Advantage of the latter is that you just need to point the `proxy` argument of your request-driver to 1 static url and be done with it. There's paid ones, there's also [this](https://github.com/mattes/rotating-proxy) which sets up a socks5 proxy using Haproxy on your own server using TOR. Hth

…

On Thu, Jan 11, 2018 at 12:34 PM, bassemAmous ***@***.***> wrote: I am developing a node js application and I am scraping a web site using x-ray and I think the blocked my ip address so how can I configure a proxy or hide my Ip address for that ? here is my code: var Xray = require('x-ray'); var x = Xray() x('https://www.myurl.com', { title: x('#cm_cr-review_list .a-section.review', [{, blogs:"..a-text-bold" }]), }) .paginate('li.a-l ***@***.***') .write('result.json') — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#299 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAYAK5zpthH16iRYTmRyxD5vnsQ0dsgOks5tJfGvgaJpZM4RatV6> .

[bassemAmous]

@gebrits Thank you for your answer but I didn't really understand the second part could you please send me an example for that ?

[0xgeert]

Something like this. Untested: (second option in the comments)

https://gist.github.com/gebrits/57689768eceaec43ae0ddd17949d7503

[bassemAmous]

Thank you I just resolved the problem.