Manga loading failing

[fuzetsu]

A fairly serious issue seems to have appeared in the stable branch of AMR. People are reporting it on the forums, facebook, and the community page. Seemingly randomly AMR stops loading manga pages in the continuous style, as if it wasn't injecting the content scripts any more.

I have experienced this as well but bizarrely I am able to get it working again whenever it happens by disabling and re-enabling AMR.

I have been watching the background page for a while to see if any strange errors are popping up but I haven't noticed anything so far.

Is anyone else experiencing this? Any thoughts @AllMangasReader-dev/developers ?

EDIT:
The error occurs when AMR tries to refresh the manga list looking for new chapters. When it tries to do this it is unable to request the manga and times out. After this when you try to load a manga page the content script can't send a request to the background page and never executes the callback (response.isOk and all that).

[fuzetsu]

I think I have identified the point at which is fails. This appeared in the console when AMR stopped loading pages.

I'm not sure if it is related, but as you can see there is an error in the MangaChapter.js implementation due to calling replace on undefined (when getting the title). After that the extension is unable to refresh the user's mangas.

EDIT:
I think the error in MangaChapter.js is unrelated, I fixed it on my end and the time-out still occurred.

[mexicano21]

Do you read anything on MangaChapter? I'm bored today and thought I could
try to fix something after months (maybe a year?), but do not want to just
randomly wait until something goes wrong...

2013/8/3 fuzetsu notifications@github.com

I think I have identified the point at which is fails. This appeared in
the console when AMR stopped loading pages.
[image: image]https://f.cloud.github.com/assets/848652/906450/be62d3e0-fca3-11e2-8060-7f2b835fbea6.png
I'm not sure if it is related, but as you can see there is an error in the
MangaChapter.js implementation due to calling replace on undefined (when
getting the title). After that the extension is unable to refresh the
mirrors.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22065003
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[fuzetsu]

Yeah, I'm reading a few things on manga chapter. It seems like the only thing that's broken there is the method of getting the chapter name and adding it to the array. Right now they're just showing up as "No chapters found yet..."

[mexicano21]

It's sad how the same code that always break in the lab runs fine if I run
it manually in the browser console.

2013/8/3 fuzetsu notifications@github.com

Yeah, I'm reading a few things on manga chapter. It seems like the only
thing that broken there is the method of getting the chapter name and
adding it to the array. Right now they're just showing up as "No chapters
found yet..."

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22065303
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[mexicano21]

Anyway, that code is broken even if is was not for this, because it doesn't
handle mangas with more than one page list of chapters.

Maybe the page that I'm looking is not the same that AMR loads? It would be
strange as I copied the link in the lab, but ...

2013/8/3 Fábio de Godoy mexicano21@gmail.com

It's sad how the same code that always break in the lab runs fine if I run
it manually in the browser console.

2013/8/3 fuzetsu notifications@github.com

Yeah, I'm reading a few things on manga chapter. It seems like the only
thing that broken there is the method of getting the chapter name and
adding it to the array. Right now they're just showing up as "No chapters
found yet..."

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22065303
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[fuzetsu]

By the way, in case it isn't clear this issue is saying that all manga sites are not loading properly. The MangaChapter issue is separate as far as I can tell.

[braiam]

I just bumped jQuerry version to 2.0 (issue #83) that was working on my box since weeks ago, nobody bothered to check. Also pushed a small change to popup.js 1359b2f, but nothings seems related. Nothing on my background but error about stupid images...

[fuzetsu]

Yeah I don't think it's related to your commits, they were on the dev branch anyway right? This appears to be affecting the master branch...

[braiam]

@fuzetsu you sure your internet was up and running when you updated your list? it's reproducible? Call refreshManga() and lets see what happens.

[mexicano21]

It is jQuery. And I'm stupid to not have know this before.

The jQuery API says:

"As of jQuery 1.6, the .attr() method returns undefined for attributes that
have not been set."

Before we used a ancient jQuery version (1.4.2, I think), that returns a
empty string in that same case. We should check all the scripts for attr()
method.

2013/8/3 Braiam Peguero notifications@github.com

@fuzetsu https://github.com/fuzetsu you sure your internet was up and
running when you updated your list? it's reproducible? Call refreshManga()and lets see what happens.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22065527
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[fuzetsu]

@braiam
It is definitely up and running, I can't really call that function directly because it needs the manga objects passed to it. I can reproduce the error by going into the popup and clicking refresh beside chapter lists

@mexicano21
It may have just been sloppy code, I mean unless some of the HTML changes there is not reason to think that the attribute would not be there. I'm pretty sure that the stable branch has been using jQuery 1.9 for a while before this..

[braiam]

HUH?! We were using .prop() some time now instead .attr() ( @plduhoux mentioned this in one of his commits) for the options page, and what not. Anyways I just checked out and the use of .attr() is only used to change the title and src attributes. https://github.com/AllMangasReader-dev/AMR/search?q=.attr&ref=cmdform&type=Code

[braiam]

@fuzetsu silly me is refreshAllLasts() also, you checked out those site that weren't down for you?

[fuzetsu]

@braiam
Yeah, that's the same function called when you click refresh manga list in the popup I'm guessing.
Same issue, could it be related to my manga list? When I try a fresh install with the master branch in Canary the error doesn't occur..

[mexicano21]

Maybe we are using a newer jQuery version on AMR for a while, BUT
MangaChapter implementation is old and relies on a older version. If that
problem is new probably MangaChapter changed recently, which would explain
too why the implementation doesn't care for multiple page chapter lists.

In a way, everything is chained =D

2013/8/3 Braiam Peguero notifications@github.com

HUH?! We were using .prop() some time now instead .attr() ( @plduhouxhttps://github.com/plduhouxmentioned this in one of his commits) for the options page, and what not.
Anyways I just checked out and the use of .attr() is only used to change
the title and src attributes.
https://github.com/AllMangasReader-dev/AMR/search?q=.attr&ref=cmdform&type=Code

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22065588
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[braiam]

Anyone is up to check MangaChapter script on the community site? I just checked adding some of the mangas you had in your list and it worked just fine and cozy. If you are using git copy try git pull to pull my changes, unless you installed the .crx.

[mexicano21]

It breaks only when you try to load the chapters of a manga with a multiple
page chapter list, because page and next page links doesn't have title
attributes set on them.

I'm already fixing it.

2013/8/3 Braiam Peguero notifications@github.com

Anyone is up to check MangaChapter script on the community site? I just
checked adding some of the mangas you had in your list and it worked just
fine and cozy. If you are using git copy try git pull to pull my changes,
unless you installed the .crx.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22065635
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[mexicano21]

Changing $(this).attr("title").replace(mangaName, "")
to ($(this).attr("title") || "").replace(mangaName, "") did the trick. Just
to be sure, I changed every attr() call to attr() || "".

Now I'm working in the multiple page chapter list issue.

2013/8/3 Fábio de Godoy mexicano21@gmail.com

It breaks only when you try to load the chapters of a manga with a
multiple page chapter list, because page and next page links doesn't have
title attributes set on them.

I'm already fixing it.

2013/8/3 Braiam Peguero notifications@github.com

Anyone is up to check MangaChapter script on the community site? I just
checked adding some of the mangas you had in your list and it worked just
fine and cozy. If you are using git copy try git pull to pull my
changes, unless you installed the .crx.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22065635
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[fuzetsu]

@braiam
Like I said, with a fresh install using the git master (freshly cloned) I don't have any problems.
But on my normal install with my full manga list I am experiencing this issue.

@mexicano21
That looks good, but just a suggestion, I would go a step further in the MangaChapter.js implementation and change it to:

$(".mangadata a[title]", div).each(
    function(index){
        var currentChapter = (this.title || '').replace(mangaName, "");
        res[res.length] = [currentChapter.trim(), $(this).attr("href")];
    }
);

Mainly the [title], just so that it only gets links that actually have the title attr defined.

[braiam]

@fuzetsu wouldn't that ignore some mangas then? Also, I bet is your manga list that's corrupted. I'm working in a diagnostic method right now...

[fuzetsu]

@braiam
I don't think it would ignore any, from what I can see on manga chapter all the mangas have title attributes, whereas the navigation links (for the multiple pages of chapters) don't have title attributes.
Using just $('.mangadata a') also gets the navigation elements (which we don not want).

[mexicano21]

Yes, I thought on changing the selector, but I will use it as is to handle
multiple pages =)

2013/8/3 fuzetsu notifications@github.com

@mexicano21 https://github.com/mexicano21
I don't think it would ignore any, from what I can see on manga chapter
all the mangas have title attributes, whereas the navigation links (for the
multiple pages of chapters) don't have title links.
Using just $('.mangadata a') also gets the navigation elements (which we
don not want).

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22065766
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[fuzetsu]

@mexicano21 Ahh, good point, in order to get the link to the next page and continue on, good thinking.

[braiam]

@fuzetsu Please try this on your background page. It should point where your list went south:

var i,
    test = mangaList;
for (i = 0; test.length > i; i += 1) {
    if (typeof test[i].name !== "string" && typeof test[i].url !== "string" && typeof test[i].lastChapterReadURL !== "string" && typeof test[i].lastChapterReadName !== "string" && typeof test[i].mirror !== "string") {
        console.log("We don't know what " + test[i].name + " is doing here. You should delete this entry test[" + i + "] completely. Check http://stackoverflow.com/a/6234289/792066 for method");
    }
    if (typeof test[i].cats !== "object" && typeof test[i].listChaps !== "object") {
        console.log("Found an error in " + mangaList[i].name + "list. Fixing now! Warning the fix could have deleted your categories.");
        //mangaList[i].cats = [];
    }
    if (typeof test[i].consult !== "fuction" && typeof test[i].refreshLast !== "function") {
        console.error("We found a serious problem on " + mangaList[i].name + "! You should delete the entry and readd it.");
    }
}

[fuzetsu]

@braiam My list comes out clear... none of those checks go through.

[fuzetsu]

Ok, looking through the networks requests made on a refresh it looks like a bunch of them do go through but all the ones after the MangaChapter error timeout. Is it possible that the error is making them fail and the pending requests are keeping the background page tied up making it ignore the requests of the content scripts? I noticed that after it finishes timing all the requests out, that the background page finally receives the sendRequest to validate the URL from the manga pages that wouldn't load, but it logs an error because "there is not tab with that ID", because the tab that made the request closed (maybe). I figured it would all be asynchronous but I guess it's possible that something it's messing it up.
EDIT: Well it looks like it makes it all the way to $.loadScript, but once it gets there it doesn't actually call the callback to send the response to the tab.

[braiam]

Then lets drop your mangalist database... it's just that it's not possible that anything is wrong. Do a backup of your data first! Then go to the resources tab, hit WebSQL > AMR, you will get a console, then just type DELETE FROM manga and DELETE FROM websites update your implementations, manga lists and manga chapter lists.

@mexicano21 Is the script ready so I can validate it?

[mexicano21]

Almost ready. Do you believe that the image on the last page of every
chapter, and only that, have a different id even the last page being equal
to any other page?

2013/8/4 Braiam Peguero notifications@github.com

Then lets drop your mangalist database... it's just that it's not possible
that anything is wrong. Do a backup of your data first! Then go to the
resources tab, hit WebSQL > AMR, you will get a console, then just type DELETE
FROM manga and DELETE FROM websites update your implementations, manga
lists and manga chapter lists.

@mexicano21 https://github.com/mexicano21 Is the script ready so I can
validate it?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22066275
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[mexicano21]

Done!

• Fixed the attr() returns "undefined" bug
• Handle multiple pages chapters list
• I actually changed that selector to "a[title]". It was easier to load the
  navigation links separated
• Added back domain address to image list (they must have removed added it
  at some point because that code was just commented but now don't have again)
• Changed image selector to match the weird different id only the images of
  the last pages of the chapters have

2013/8/4 Fábio de Godoy mexicano21@gmail.com

Almost ready. Do you believe that the image on the last page of every
chapter, and only that, have a different id even the last page being equal
to any other page?

2013/8/4 Braiam Peguero notifications@github.com

Then lets drop your mangalist database... it's just that it's not
possible that anything is wrong. Do a backup of your data first! Then go to
the resources tab, hit WebSQL > AMR, you will get a console, then just type DELETE
FROM manga and DELETE FROM websites update your implementations, manga
lists and manga chapter lists.

@mexicano21 https://github.com/mexicano21 Is the script ready so I can
validate it?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22066275
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[fuzetsu]

@braiam
Ok I'll try that, but regarding the mangas not loading I've tracked down why it isn't working.

Basically the flow is like this:

1. manga page loads
2. content script loads
3. back.js makes request to background page to check if the page location is a manga location
4. if it is valid we'll have the mirror name and implementation URL
5. load the script using the jquery plugin 'loadScript'
6. loadScript checks the scripts on the page to see if the script is already loaded
7. if the script is already loaded just call the callback which sends the 'OK' response back to the manga page otherwise use ajax to get the script and send the script text to the callback
8. take script text execute it and send the response.

The problem is that at step 7 locaScript is executing the ajax but neither the success or error functions are being called. I think it's because of all the timed out requests.

[fuzetsu]

Also, even after clearing the manga and website tables I still have the same problem...

[fuzetsu]

Another thing I've noticed.. It seems that whenever AMR loads it tries to fetch the manga list for "manga panda", it fetches www.mangapanda.com/alphabetical which leads to a warning on chrome about malware. I think that the request is being blocked and clogging up the other requests, it just shows up as pending in the network tab and never completes whereas all the requests before it go through no problem.

[fuzetsu]

OK, that was the issue, I had one manga from manga panda in my list which meant that the site had to be enabled. I removed that manga and disabled manga panda and it no longer tries to load it's manga list. Now refreshing seems to work fine. Still seems strange that not everyone is affected by it though...

@braiam I'm still having issues with MangaChapter, has @mexicano21's fix gone through?

@AllMangasReader-dev/developers
And speaking of implementations I kind of feel like the hosting / update method for them could be improved. I think it would be better to host them in a github repo which is kept up to date on the server in order to make it easier to make modifications and track issues. The community site is not bad but it is hard to beat Github at its own game.
I think I'll make a new issue to discuss this..

[mexicano21]

While we are still at it, seens like Manga Traders and Manga Eden have the
undefined problem too. Yesterday my AMR started to freeze whenever it tried
to refresh manga chapters, and today I looked in the console of the
extension and saw them there blocking everyone. It may be one manga in my
list though, I still need to debug this - one way or another, this should
never happen. And on another note I saw Manga Eden, MangaFox and EatManga
loading unnecessary images. I'll fix this too.

2013/8/4 fuzetsu notifications@github.com

OK, that was the issue, I had one manga from manga panda in my list which
meant that the site had to be enabled. I removed that manga and disabled
manga panda and it no longer tries to load it's manga list. Now refreshing
seems to work fine. Still seems strange that not everyone is affected by it
though...

@braiam https://github.com/braiam I'm still having issues with
MangaChapter, has @mexicano21 https://github.com/mexicano21's fix gone
through?

@AllMangasReader-dev/developers
And speaking of implementations I kind of feel like the hosting / update
method for them could be improved. I think it would be better to host them
in a github repo which is kept up to date on the server in order to make it
easier to make modifications and track issues. The community site is not
bad but it is hard to beat Github at its own game.
I think I'll make a new issue to discuss this..

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22073905
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[fuzetsu]

@mexicano21
Yeah this is the issue I've been facing since yesterday, along with a lot of other people it seems.
Have you been able to import implementations into AMR using the community site? I was trying to make some changes to the manga panda implementation yesterday through the community site but it wouldn't let me import the script...

Also, does $.ajax not have a default timeout? If not we might need to add it in some places, if bad requests really are the thing messing up the updates..

[mexicano21]

The "Import in AMR" is not working, but "Test in Lab" is, and it do import
too, so it's ok for me.

I'll see about $.ajax.

2013/8/4 fuzetsu notifications@github.com

@mexicano21 https://github.com/mexicano21
Yeah this is the issue I've been facing since yesterday, along with a lot
of other people it seems.
Have you been able to import implementations into AMR using the community
site? I was trying to make some changes to the manga panda implementation
yesterday through the community site but it wouldn't let me import the
script...

Also, does $.ajax not have a default timeout? If not we might need to add
it in some places, if bad requests really are the thing messing up the
updates..

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22074253
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[mexicano21]

jQuery has a timeout parameter to $.ajax calls. But (
http://api.jquery.com/jQuery.ajax/):

timeout
Type: Number http://api.jquery.com/Types/#Number
Set a timeout (in milliseconds) for the request. This will override any
global timeout set with $.ajaxSetup()http://api.jquery.com/jQuery.ajaxSetup.
The timeout period starts at the point the $.ajax call is made; if several
other requests are in progress and the browser has no connections
available, it is possible for a request to time out before it can be sent.

My emphasys.

2013/8/4 Fábio de Godoy mexicano21@gmail.com

The "Import in AMR" is not working, but "Test in Lab" is, and it do import
too, so it's ok for me.

I'll see about $.ajax.

2013/8/4 fuzetsu notifications@github.com

@mexicano21 https://github.com/mexicano21
Yeah this is the issue I've been facing since yesterday, along with a lot
of other people it seems.
Have you been able to import implementations into AMR using the community
site? I was trying to make some changes to the manga panda implementation
yesterday through the community site but it wouldn't let me import the
script...

Also, does $.ajax not have a default timeout? If not we might need to add
it in some places, if bad requests really are the thing messing up the
updates..

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22074253
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[fuzetsu]

Interesting, I wonder if this is what we're experiencing, I can't imagine that it's running out of connections through... It was only like 30 requests or so for me.

Also, I wasn't able to get test in the lab to work either. I'll have to try again I guess.

[mexicano21]

Turns out that neither Manga Traders nor Manga Eden didn't have the
"undefined" bug. I had a manga (the same in both mirrors, and the first in
my manga list) with a broken link in them. When refreshing them make AMR
freeze. I don't know what broke the links - maybe this was the Manga
Chapter bug doing?

Anyway, Manga Traders did has another bug, which I fixed.

2013/8/4 fuzetsu notifications@github.com

Interesting, I wonder if this is what we're experiencing, I can't imagine
that it's running out of connections through... It was only like 30
requests or so.

Also, I wasn't able to get test in the lab to work either. I'll have to
try again I guess.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/84#issuecomment-22075161
.

Fábio de Godoy
http://www.animenewsnetwork.com/MyManga/?user=mexicano21

[braiam]

@mexicano21 I modified mangapanda script so it wont ask for the full manga list, and instead search for it, but probably we should delete it (the site is dangerous as is and we should prevent infect our users with viruses, etc.), I'm looking for a way to make that all the sites dissable their keybindings (yeah, so we don't use per-site solutions) without too much luck with the back.js.

@fuzetsu Yeah, I will test right now, remember to force an implementation update or you will have to wait for the scripts to expire.

[fuzetsu]

@braiam Yeah, if they don't fix the issue they'll have to be removed. It's likely something to do with their ad provider. According to their facebook page they'll have it fixed soon.

[braiam]

@fuzetsu it works right now... so publishing the script.

[fuzetsu]

Yeah, looks like mangapanda fixed the issue :-)

[braiam]

@fuzetsu are you able to reproduce the initial issue? If not, just close this...

[fuzetsu]

@braiam I think we should keep this open until it is confirmed that manga panda was the culprit, because that still doesn't seem 100% clear. We should also see if the other people that were reporting the issue are no longer experiencing it.

[fuzetsu]

Closing this issue since the error hasn't reoccured, meaning that the fix worked.