Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarification on the meaning of 'class" Error (Pixiv Tags Translation feature related) #200

Closed
ghost opened this issue Feb 20, 2017 · 6 comments

Comments

@ghost
Copy link

ghost commented Feb 20, 2017

The program is not downloading select images with no specific error code displayed.
Processing Image Id: 7575556 Image ID (7575556): 'class' Image #31
Processing Image Id: 189902 Image ID (189902): 'class' Image #32

Some images will download just fine, but not these among many others.

I should add, viewing in a regular browser will still work.

@Nandaka
Copy link
Owner

Nandaka commented Feb 20, 2017

Looks like they do something on the way the tags is being represented (something related to translation?)

@NHOrus
Copy link
Contributor

NHOrus commented Feb 20, 2017

Experiencing same error, multiple times. Say, here

I got about 300 in log file from today's attempt to crawl 3700 artists.

@fredgido
Copy link

Getting that error too.
Nothing changed here, probably something changed on pixiv site side?

@naharamas
Copy link

Same issue here. I'm using last release.
A lot of images don't download because of this.
Artist ids 2208819(SFW) or 15426672(NSFW) for instance misses a lot of pictures.

Pixiv website seems to have some issues currently tough. I'm unable to follow anyone right now, pressing the follow button redirects me on my bookmarks page. This is not related of course, but there could be other similar issues preventing the downloading.

@pixtrix
Copy link
Contributor

pixtrix commented Feb 20, 2017

Tags must have changed yesterday or today. Not all of the tags are in class="text" links any more. I used the data-tooltip="Articles related to (tag_name)" links to pull the tag names that weren't in the old style links.

I changed the code in the ParseTags method in PixivImage in PixivModel.py to:

    def ParseTags(self, page):
        del self.imageTags[:]
        temp = page.find(attrs={'class': 'tags'})
        if temp is not None and len(temp) > 0:
            temp2 = temp.findAll('a')
            if temp2 is not None and len(temp2) > 0:
                for tag in temp2:
                    if tag.has_key('class'):
                        if tag['class'] == 'text' and tag.string is not None:
                            tag_name = unicode(tag.string)
                            if tag_name not in self.imageTags:
                                self.imageTags.append(tag_name)
                        elif tag['class'] == 'icon-pixpedia _ui-tooltip':
                            tag_name = unicode(re.sub('^Articles related to ', '', tag['data-tooltip']))
                            if tag_name not in self.imageTags:
                                self.imageTags.append(tag_name)

But I'm sure it could be fixed better. And if this new style of tag links is permanent, the old style links might need to be discarded. But their code was changing while I was working on this fix, so I think that these changes aren't set in stone yet.

@Nandaka
Copy link
Owner

Nandaka commented Feb 20, 2017

Pixiv are doing a/b testing related to tags translation, for now I will take the japanese version.

@Nandaka Nandaka changed the title Clarification on the meaning of 'class" Error Clarification on the meaning of 'class" Error (Pixiv Tags Translation feature related) Feb 28, 2017
@Nandaka Nandaka closed this as completed Oct 5, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants