url greater than 255 chars are not saved (was: urls with redirection(s) silently fail to be downloaded)

[pmaziere]

When sending wallabag a url that ends up to be a redirection (HTTP 3xx) to another url, it seems that wallabag does not follow the redirection, and page download fails silently wihout even a failure notice

[tcitworld]

Hi,
Could you give an example of page ?
I used for instance the goo.gl service which gives HTTP 301 redirection and it works for me.

[pmaziere]

here is a link coming from my ttrss instance: using the the ttrss wallabag plugin or pasting it in the search box of wallabag UI result in the same failure

http://news.google.com/news/url?sa=t&fd=R&ct2=fr&usg=AFQjCNHJgjNo3aondjF0E3S_vUEQ3o4Y5A&clid=c3a7d30bb8a4878e06b80cf16b898331&cid=52778860338942&ei=vrNwU9iCO4e1iAatCg&url=http://www.metronews.fr/info/et-si-vous-controliez-vos-reves-grace-a-l-electricite/mnel!0B6VnlzEtMG6w/

[tcitworld]

Well, I've been trying for 15 minutes with no success, but it works now. I guess it was a problem on MetroNews' side.

Can you confirm it ?

[pmaziere]

nope it does not work
this link leads to 2 redirections before reaching the final page: I can always get to the final page when entering the link in my browser address bar, but it always fail to be downloaded with wallabag.

I did not check in details what wallabag was doing, but I guess I will have to get a closer look since you are not able to reproduce this issue

[tcitworld]

Well, it didn't work at the beginning, as with the original Google URL and with the final MetroNews URL : http://www.metronews.fr/info/et-si-vous-controliez-vos-reves-grace-a-l-electricite/mnel!0B6VnlzEtMG6w/ but now they both work. I think they had issues on their side because once I got a 500 server error.
Try again in a few hours in case.

[pmaziere]

2014-05-12 17:55:33: (mod_fastcgi.c.2676) FastCGI-stderr: DEBUG POCHE : storage type postgres
DEBUG POCHE : display home view

2014-05-12 17:55:50: (mod_fastcgi.c.2676) FastCGI-stderr: DEBUG POCHE : storage type postgres
DEBUG POCHE : execute query error : SQLSTATE[22001]: String data, right truncated: 7 ERROR: value too long for type character varying(255)
DEBUG POCHE : error during insertion : the link wasn't added http://news.google.com/news/url?sa=t&fd=R&ct2=fr&usg=AFQjCNHJgjNo3aondjF0E3S_vUEQ3o4Y5A&clid=c3a7d30bb8a4878e06b80cf16b898331&cid=52778860338942&ei=vrNwU9iCO4e1iAatCg&url=http://www.metronews.fr/info/et-si-vous-controliez-vos-reves-grace-a-l-electricite/mnel!0B6VnlzEtMG6w/
DEBUG POCHE : redirect to ?view=home&closewin=true

it seems to be an issue with postgresql "entries" table definition: url is limited to 255 characters while this url is 274. This is also the case for mysql, while the TEXT data type in sqlite does not seem to be limited. I guess there should not be any limit to url and title length

so for postgresql and mysql, VARCHAR(255) should be replaced by TEXT

[tcitworld]

Okay. I was using it with SQLite which has no character limitation, that's why.
I wonder if we better save the final URL into DB rather than the given one in case of redirections.
But you're right however, we've got to change this into a TEXT field.

[pmaziere]

It depends on the redirection: if it's a 301 or 308 error code, then the final URL should be saved.
Otherwise the first URL should be considered as a permanent link and the redirection as a link that may change in the future for whatever reasons

[tcitworld]

That's totally right.

[tcitworld]

Since it implies database changes, this goes to v2 (a upgrade script would have been required either).

[j0k3r]

url column are now defined as text in the v2 branch.

@ORM\Column(name="url", type="text", nullable=true)