New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The proxy plugin is never instantiated/run in Crawljax 3.1 #286

Closed
amesbah opened this Issue Jun 20, 2013 · 3 comments

Comments

Projects
None yet
3 participants
@amesbah
Member

amesbah commented Jun 20, 2013

See my comments in PR #282

@alexnederlof

This comment has been minimized.

Show comment
Hide comment
@alexnederlof

alexnederlof Jun 21, 2013

Member

Ok I see what the code does.

There are two scenarios for a Proxy configuration:

  1. You want to dynamically start and stop a proxy (as you do with all the interceptor plugins).
  2. You want to use an already initiated proxy. This might even be on a different host.

In both scenario's you don't need a ProxyPlugin. In scenario 1 you can use the PreCrawlingPlugin just like all the other stuff you might want to set-up before running a crawl. You can clean-up/tear-down the proxy server when you are done in a PostCrawling plugin. Stopping the service isn't even covered in the current proxy plugin.

If you want to use a dynamic port for the proxy (like I do) because you don't know which ports are available on a given computer, you need to setup the proxy before you configure Crawljax. This is because the configuration is immutable. A proxy plugin doesn't help here either.

In scenario 2 you don't need a plugin either, because you just pass the proxy configuration to the configuration builder and you're done.

Member

alexnederlof commented Jun 21, 2013

Ok I see what the code does.

There are two scenarios for a Proxy configuration:

  1. You want to dynamically start and stop a proxy (as you do with all the interceptor plugins).
  2. You want to use an already initiated proxy. This might even be on a different host.

In both scenario's you don't need a ProxyPlugin. In scenario 1 you can use the PreCrawlingPlugin just like all the other stuff you might want to set-up before running a crawl. You can clean-up/tear-down the proxy server when you are done in a PostCrawling plugin. Stopping the service isn't even covered in the current proxy plugin.

If you want to use a dynamic port for the proxy (like I do) because you don't know which ports are available on a given computer, you need to setup the proxy before you configure Crawljax. This is because the configuration is immutable. A proxy plugin doesn't help here either.

In scenario 2 you don't need a plugin either, because you just pass the proxy configuration to the configuration builder and you're done.

@amesbah amesbah closed this in #287 Jun 26, 2013

@taibd

This comment has been minimized.

Show comment
Hide comment
@taibd

taibd Feb 11, 2014

you can show me to how about using PreCrawlingPlugin and PostCrawling. I can't find that. You can write a demo example..thank you so much!!!

taibd commented Feb 11, 2014

you can show me to how about using PreCrawlingPlugin and PostCrawling. I can't find that. You can write a demo example..thank you so much!!!

@alexnederlof

This comment has been minimized.

Show comment
Hide comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment