New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarification on the meaning of 'class" Error (Pixiv Tags Translation feature related) #200

Closed
DevPaws opened this Issue Feb 20, 2017 · 6 comments

Comments

Projects
None yet
6 participants
@DevPaws

DevPaws commented Feb 20, 2017

The program is not downloading select images with no specific error code displayed.
Processing Image Id: 7575556 Image ID (7575556): 'class' Image #31
Processing Image Id: 189902 Image ID (189902): 'class' Image #32

Some images will download just fine, but not these among many others.

I should add, viewing in a regular browser will still work.

@Nandaka Nandaka added the Enhancement label Feb 20, 2017

@Nandaka

This comment has been minimized.

Owner

Nandaka commented Feb 20, 2017

Looks like they do something on the way the tags is being represented (something related to translation?)

@NHOrus

This comment has been minimized.

Contributor

NHOrus commented Feb 20, 2017

Experiencing same error, multiple times. Say, here

I got about 300 in log file from today's attempt to crawl 3700 artists.

@fredgido

This comment has been minimized.

fredgido commented Feb 20, 2017

Getting that error too.
Nothing changed here, probably something changed on pixiv site side?

@naharamas

This comment has been minimized.

naharamas commented Feb 20, 2017

Same issue here. I'm using last release.
A lot of images don't download because of this.
Artist ids 2208819(SFW) or 15426672(NSFW) for instance misses a lot of pictures.

Pixiv website seems to have some issues currently tough. I'm unable to follow anyone right now, pressing the follow button redirects me on my bookmarks page. This is not related of course, but there could be other similar issues preventing the downloading.

@pixtrix

This comment has been minimized.

Contributor

pixtrix commented Feb 20, 2017

Tags must have changed yesterday or today. Not all of the tags are in class="text" links any more. I used the data-tooltip="Articles related to (tag_name)" links to pull the tag names that weren't in the old style links.

I changed the code in the ParseTags method in PixivImage in PixivModel.py to:

    def ParseTags(self, page):
        del self.imageTags[:]
        temp = page.find(attrs={'class': 'tags'})
        if temp is not None and len(temp) > 0:
            temp2 = temp.findAll('a')
            if temp2 is not None and len(temp2) > 0:
                for tag in temp2:
                    if tag.has_key('class'):
                        if tag['class'] == 'text' and tag.string is not None:
                            tag_name = unicode(tag.string)
                            if tag_name not in self.imageTags:
                                self.imageTags.append(tag_name)
                        elif tag['class'] == 'icon-pixpedia _ui-tooltip':
                            tag_name = unicode(re.sub('^Articles related to ', '', tag['data-tooltip']))
                            if tag_name not in self.imageTags:
                                self.imageTags.append(tag_name)

But I'm sure it could be fixed better. And if this new style of tag links is permanent, the old style links might need to be discarded. But their code was changing while I was working on this fix, so I think that these changes aren't set in stone yet.

@Nandaka

This comment has been minimized.

Owner

Nandaka commented Feb 20, 2017

Pixiv are doing a/b testing related to tags translation, for now I will take the japanese version.

@Nandaka Nandaka changed the title from Clarification on the meaning of 'class" Error to Clarification on the meaning of 'class" Error (Pixiv Tags Translation feature related) Feb 28, 2017

@Nandaka Nandaka closed this Oct 5, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment