Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expected str instance nonetype found #168

Closed
NIkitabala opened this issue Nov 2, 2020 · 10 comments
Closed

Expected str instance nonetype found #168

NIkitabala opened this issue Nov 2, 2020 · 10 comments

Comments

@NIkitabala
Copy link

My code

from lyricsgenius import Genius

genius = Genius(xxx)


genius.remove_section_headers = True

artist = genius.search_artist(xxx)
artist.save_lyrics(extension = 'txt', binary_encoding=True)

Sometimes script can't find lyrics to one song, passes it and goes to another one. But when saving file, it gives this error:

Error at line XX: Expected str instance nonetype found genius lyrics

How do I fix this error so I'm able to save the lyrics, even if one or more is missing?

@NIkitabala NIkitabala changed the title Expected str instance nonetype found genius lyrics Expected str instance nonetype found Nov 2, 2020
@NIkitabala
Copy link
Author

Here's full traceback:

Found 89 songs.
Lyrics_МотMot.txt already exists. Overwrite?
(y/n): y
Traceback (most recent call last):
  File "/data/user/0/ru.iiec.pydroid3/files/accomp_files/iiec_run/iiec_run.py", line 31, in <module>
    start(fakepyfile,mainpyfile)
  File "/data/user/0/ru.iiec.pydroid3/files/accomp_files/iiec_run/iiec_run.py", line 30, in start
    exec(open(mainpyfile).read(),  __main__.__dict__)
  File "<string>", line 9, in <module>
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/lyricsgenius/artist.py", line 298, in save_lyrics
    self.to_text(filename, binary_encoding=binary_encoding)
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/lyricsgenius/artist.py", line 209, in to_text
    data = ' '.join(song.lyrics for song in self.songs)
TypeError: sequence item 70: expected str instance, NoneType found

[Program finished]

@allerter
Copy link
Collaborator

allerter commented Nov 2, 2020

Hi. You might be using an outdated version of the package. What version of LyricsGenius are you using?

@NIkitabala
Copy link
Author

@allerter Hi, the newest one, I've updated it today.

@allerter
Copy link
Collaborator

allerter commented Nov 2, 2020

Please provide a minimal script so that I reproduce the error.

@NIkitabala
Copy link
Author

I'm using this script:

from lyricsgenius import Genius

genius = Genius(xxx)


genius.remove_section_headers = True

artist = genius.search_artist(xxx)
artist.save_lyrics(extension = 'txt', binary_encoding=True)

And I'm getting this:

Searching for songs by Мот...

Changing artist name to 'Мот (Mot)'
Song 1: "Ливень (Downpour)"
Song 2: "Когда исчезнет Слово (KIS)"
Song 3: "Талисман (Talisman)"
Song 4: "Паруса"
Song 5: "Санторини (Santorini)"
Song 6: "Соло (Solo)"
Song 7: "Раз, два, три, …, шесть / Ссора (Quarrel)"
Song 8: "Капкан (Trap)"
Song 9: "Молодость (Youth)"
Song 10: "По буквам (Spell)"
Song 11: "Сопрано (Soprano)"
Song 12: "Далласский клуб злопыхателей (DKZ)"
Song 13: "Сталактит (Stalactite)"
Song 14: "Опус Магнум (Opus Magnum)"
Song 15: "Тарантино (Tarantino)"
Song 16: "Добрая музыка клавиш (DMK)"
Song 17: "Она не твоя (She is not yours)"
Song 18: "Звуки пианино (Sounds of the Piano)"
Song 19: "Абсолютно всё (Absolutely Everything)"
Song 20: "Перекрёстки (Crossroads)"
Song 21: "Ча-Ча Ленд (Cha-cha land)"
Song 22: "Парабола (Parabola)"
Song 23: "Муссоны (Monsoon)"
Song 24: "Общество модных поэтов (Fashion poets society)"
Song 25: "Мысли (Thoughts)"
Song 26: "Шаманы (Shamans)"
Song 27: "Очень. Осень (Very. Fall)"
Song 28: "Карты, деньги, две тарелки (Cards, money, two plates)"
Song 29: "Бумажный дом (House of Paper)"
Song 30: "Сколько лет (How Many Years)"
Song 31: "Космос - это синяки 2 (Kosmos - eto sinyaki 2)"
Song 32: "На дне (At the Bottom)"
Song 33: "До мурашек (Before the shivers)"
Song 34: "День и ночь (Day and night)"
Song 35: "Космос - это синяки (Kosmos - eto sinyaki)"
Song 36: "Гудки (Beeps)"
Song 37: "Разбуди меня шепотом (Wake Me Up in a Whisper)"          Song 38: "92 дня (92 Days)"
Song 39: "Кислород (Kislorod)"
Song 40: "Белые ночи (White nights)"
Song 41: "Суперзвезда (Superstar)"                                 Song 42: "Изюм (Raisin)"
Song 43: "Словами с привкусом Мёртвого моря.. (Words with a taste of the Dead Sea..)"
Song 44: "Невероятно (Incredibly)"
Song 45: "Для своих (Dlya svoih)"
Song 46: "Все чувства - в Латынь.. (All feelings - in Latin)"
Song 47: "На юга (To the south)"
Song 48: "Principi"
Song 49: "Пролетая над коттеджами Барвихи (One flew over the cottages Barvikha)"
Song 50: "Yellow Song"
Song 51: "Будь моей женой (Be My Wife)"
Song 52: "С ней (With Her)"
Song 53: "Побег из Шоубиза (Escape from Showbiz)"
Song 54: "На йух (Na yuh)"
Song 55: "Наизнанку (Inside Out) [Intro]"
Song 56: "Фантастика (Fantastic)"
Song 57: "Оригами-ми-ми (Origami-mi-mi)"
Song 58: "Люди (People)"
Song 59: "Тиффани (Tiffany)"
Song 60: "Файа-Файа-Файа (Faya-Faya-Faya)"
Song 61: "В джазе только девушки? (Are there only girls in jazz?)"
Song 62: "Disco Disci"
Couldn't find the lyrics section.
Song 63: "Дай Джазу! (Give me Jazz!)"
Song 64: "Мечтаю (I Dream)"
Song 65: "Маврикий (Mauritius)"
Song 66: "Подмигни (Wink)"
Couldn't find the lyrics section.
Song 67: "Понедельник-Вторник (Monday-Tueseday)"
Song 68: "18+"
Song 69: "911"
Song 70: "Rendez-vous"
Couldn't find the lyrics section.
Song 71: "Эдем (Eden)"
Song 72: "Veni! Vidi! Vici!"
Song 73: "Мажоры-миноры (Major-Minors)"
Couldn't find the lyrics section.
Song 74: "Бенджамин (Benjamin)"
Song 75: "Апостроф’ (Apostrophe’)"
Song 76: "Потрачено (Spent)"
Song 77: "24-7"
Couldn't find the lyrics section.
Song 78: "Топ (Top)"
Song 79: "Останови! (Stop!)"
Couldn't find the lyrics section.
Song 80: "Папа, дай ей денег (Dad, Give Her Money)"
Song 81: "Медленно (Slowly)"
Couldn't find the lyrics section.
Song 82: "Барон Мюнхгаузен (Baron Munchausen)"
Song 83: "#Мотсейчасвклубе (#Motseychasvklube)"
Song 84: "Мать-И-Мачеха (Coltsfoot)"
Song 85: "Чёрный день (Black Day)"
Song 86: "Молодая кровь (Young Blood)"
Song 87: "8-е Чудо Света (8th Wonder of the world)"
Couldn't find the lyrics section.
Song 88: "Страна OZ (Land OZ)"
Song 89: "Женщина (Woman)"
Done. Found 89 songs.
Lyrics_МотMot.txt already exists. Overwrite?
(y/n): y
Traceback (most recent call last):
  File "/data/user/0/ru.iiec.pydroid3/files/accomp_files/iiec_run/iiec_run.py", line 31, in <module>
    start(fakepyfile,mainpyfile)
  File "/data/user/0/ru.iiec.pydroid3/files/accomp_files/iiec_run/iiec_run.py", line 30, in start
    exec(open(mainpyfile).read(),  __main__.__dict__)
  File "<string>", line 9, in <module>
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/lyricsgenius/artist.py", line 298, in save_lyrics
    self.to_text(filename, binary_encoding=binary_encoding)
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/lyricsgenius/artist.py", line 209, in to_text
    data = ' '.join(song.lyrics for song in self.songs)
TypeError: sequence item 66: expected str instance, NoneType found

[Program finished]

It happens only after searching for all lyrics. Some issue with empty lyrics (classified as None), but I can't understand, how to skip them, so script wouldn't try to save them. I thought some problem with encoding, but script worked fine for previous artist. Now it's not working for everyone.

@allerter
Copy link
Collaborator

allerter commented Nov 2, 2020

Currently, there is no way to make artist.save_lyrics work if one or more of the songs have empty lyrics (None). But you could do what it does yourself. Also there's no straightforward way to solve the timeout issue since this also happens inside the method (genius.search_artist). I recommend doing this instead:

from lyricsgenius import Genius
from lyricsgenius.song import Song
genius = Genius(xxx)
genius.remove_section_headers = True

artist = genius.search_artist(xxx, max_songs=0)
page = 1
while page:
    res = genius.artist_songs(artist._id,
                              sort='popularity',
                              per_page=50,
                              page=page)
    if res is None:
        continue
    for hit in res['songs']:
        if hit['lyrics_state'] == 'complete':
            lyrics = genius.lyrics(hit['url'])
            song = Song(hit, lyrics)
            # this will add both the songs where the artist is the primary artist and the ones where he is a feature artist
            artist.songs.append(song)
            # replace the line above with artist.add_song(song) if you only want to add songs where the artist is the primary artist
            print('Added: {}'.format(hit['title']))
        else:
            print('Skipped: {}'.format(hit['title']))

    page = res['next_page']


data = ' '.join(song.lyrics for song in artist.songs if song.lyrics is not None)
filename = 'lyrics.txt'
with open(filename, 'w', encoding='utf8') as f:
    f.write(data)

Both of the issues (the timeout and artist.save_lyrics) will be solved in the next version. But it could be some time before it comes out. So I suggest using the script above.

@NIkitabala
Copy link
Author

I've tried your solution, but still there is some kind of error:

Timeout raised and caught:
HTTPSConnectionPool(host='api.genius.com', port=443): Read timed out. (read timeout=5)
Traceback (most recent call last):
  File "/data/user/0/ru.iiec.pydroid3/files/accomp_files/iiec_run/iiec_run.py", line 31, in <module>
    start(fakepyfile,mainpyfile)
  File "/data/user/0/ru.iiec.pydroid3/files/accomp_files/iiec_run/iiec_run.py", line 30, in start
    exec(open(mainpyfile).read(),  __main__.__dict__)
  File "<string>", line 6, in <module>
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/lyricsgenius/genius.py", line 622, in search_artist
    song = Song(info, lyrics)
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/lyricsgenius/song.py", line 20, in __init__
    self._body = json_dict['song'] if 'song' in json_dict else json_dict
TypeError: argument of type 'NoneType' is not iterable

[Program finished]

This happens randomly, at different songs. In your code I only changed part with maximum songs.

@allerter
Copy link
Collaborator

allerter commented Nov 2, 2020

You shouldn't change the max_songs parameter as we're adding the songs manually, so all we need is an empty Artist object. I edited the script above and improved it a bit.

@NIkitabala
Copy link
Author

I've tried a few more times, and last time it actually worked. I guess, time-out is Genius-related issue, since their API is not created for scraping songs lyrics. Thanks you!

@allerter
Copy link
Collaborator

allerter commented Nov 2, 2020

This particular issue happens because we're making a lot of requests and as you mentioned their API wasn't made for lyrics, so we have to download each song page separately. This causes the number of requests to rise and time-outs are bound to happen when we make so many HTTP requests. In your case, the timeout happens when the package tries to fetch the full song info (because the get_full_info parameter is True), but at the same time, the package doesn't handle the timeouts as it should. But all that is handled in the next version that's coming out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants