Bug when page is redirected?

[ghost]

It's happening for me with:
$ rawler http://arbitragem.pt/links/index.php

...rawler says that lots of links are broken.

You see http://arbitragem.pt/index.php changes the location to http://arbitragem.pt/noticias/index.php
...I think that's what's causing the problem.

Let me know if you need more info.

[ghost]

Also:

$ rawler http://arbitragem.pt/

should work. But it does not.

[ghost]

Or maybe the 1st problem is that rawler doesn't properly process "../" -prefixed URLs in links?

[oscardelben]

Hi devtime, that's something I'll work on asap.

[oscardelben]

Hi, I've pushed this to master, it'll be included in the next release. Thanks and let me know if you find any other issue.

[ghost]

Thanks, Oscar! I'll test it as soon as the gem gets updated.

And what about the other (possible) problem:

that rawler doesn't properly process "../" -prefixed URLs in links?

Or this problem doesn't exist? ...If you publish the gem, I'll test it all, and let you know.

Thanks again!