Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Solved] getSongs() doesn't return all songs from an artist #8

Closed
brandhsu opened this issue Aug 12, 2020 · 3 comments
Closed

[Solved] getSongs() doesn't return all songs from an artist #8

brandhsu opened this issue Aug 12, 2020 · 3 comments
Labels
bug Something isn't working

Comments

@brandhsu
Copy link

brandhsu commented Aug 12, 2020

Describe the bug
getSongs() fails to retrieve all songs from an artist

To Reproduce

from azapi import AZlyrics
import pprint

API = AZlyrics('google', accuracy=0.5)
API.artist = 'bangtanboys'
songs = API.getSongs()
pprint.pprint(songs, indent=2)

printed json

{
  "24/7=Heaven": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/247heaven.html",
    "year": "2014"
  },
  "2nd Grade (2학년)": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/2ndgrade2.html",
    "year": "2014"
  },
  "Attack On Bangtan (진격의 방탄; Jingyeogui Bangtan)": {
    "album": "O!RUL8,2?",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/attackonbangtanjingyeoguibangtan.html",
    "year": "2013"
  },
  "BTS Cypher Pt. 1": {
    "album": "O!RUL8,2?",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/btscypherpt1.html",
    "year": "2013"
  },
  "BTS Cypher Pt. 2: Triptych": {
    "album": "Skool Luv Affair",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/btscypherpt2triptych.html",
    "year": "2014"
  },
  "BTS Cypher Pt. 3: Killer": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/btscypherpt3killer.html",
    "year": "2014"
  },
  "Blanket Kick (이불킥)": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/blanketkick.html",
    "year": "2014"
  },
  "Boy In Luv (상남자; Sang-Namja)": {
    "album": "Skool Luv Affair",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/boyinluvsangnamja.html",
    "year": "2014"
  },
  "Can You Turn Off Your Cellphone? (핸드폰 좀 꺼줄래)": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/canyouturnoffyourcellphone.html",
    "year": "2014"
  },
  "Coffee": {
    "album": "O!RUL8,2?",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/coffee.html",
    "year": "2013"
  },
  "Danger": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/danger.html",
    "year": "2014"
  },
  "Does That Make Sense? (Outro) (그게 말이 돼? (Outro))": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/doesthatmakesenseoutrooutro.html",
    "year": "2014"
  },
  "Hip Hop Lover (힙합성애자)": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/hiphoplover.html",
    "year": "2014"
  },
  "I Like It (Slow Jam Remix) (좋아요; Joh-Ayo)": {
    "album": "Skool Luv Affair",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/ilikeitslowjamremixjohayo.html",
    "year": "2014"
  },
  "I Like It (좋아요; Joayo)": {
    "album": "2 Cool 4 Skool",
    "type": "single",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/ilikeitjoayo.html",
    "year": "2013"
  },
  "If I Ruled The World": {
    "album": "O!RUL8,2?",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/ifiruledtheworld.html",
    "year": "2013"
  },
  "Interlude: What Are You Doing? (Interlude:뭐해; Interlude: Mwohae)": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/interludewhatareyoudoinginterludeinterludemwohae.html",
    "year": "2014"
  },
  "Intro: 2 Cool 4 Skool": {
    "album": "2 Cool 4 Skool",
    "type": "single",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/intro2cool4skool.html",
    "year": "2013"
  },
  "Intro: O!RUL8,2": {
    "album": "O!RUL8,2?",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/introorul82.html",
    "year": "2013"
  },
  "Intro: Skool Luv Affair": {
    "album": "Skool Luv Affair",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/introskoolluvaffair.html",
    "year": "2014"
  },
  "Jump": {
    "album": "Skool Luv Affair",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/jump.html",
    "year": "2014"
  },
  "Just One Day (하루만; Haruman)": {
    "album": "Skool Luv Affair",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/justonedayharuman.html",
    "year": "2014"
  },
  "Let Me Know": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/letmeknow.html",
    "year": "2014"
  },
  "Look Here (여기 봐)": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/lookhere.html",
    "year": "2014"
  },
  "Miss Right": {
    "album": "Skool Luv Affair",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/missright.html",
    "year": "2014"
  },
  "N.O": {
    "album": "O!RUL8,
    2?",
"type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/no.html",
    "year": "2013"
  },
  "No More Dream": {
    "album": "2 Cool 4 Skool",
    "type": "single",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/nomoredream.html",
    "year": "2013"
  },
  "Outro: Luv In Skool": {
    "album": "O!RUL8,
    2?",
          "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/outroluvinskool.html",
    "year": "2013"
  },
  "Outro: Propose": {
    "album": "Skool Luv Affair",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/outropropose.html",
    "year": "2014"
  },
  "Path (길; Gil)": {
    "album": "2 Cool 4 Skool",
    "type": "single",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/pathgil.html",
    "year": "2013"
  },
  "Rain": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/rain.html",
    "year": "2014"
  },
  "Satoori Rap (팔도강산; Paldogangsan)": {
    "album": "O!RUL8,2?",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/satoorirappaldogangsan.html",
    "year": "2013"
  },
  "Skit: Soulmate": {
    "album": "Skool Luv Affair",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/skitsoulmate.html",
    "year": "2014"
  },
  "Spine Breaker (등골브레이커; Deung-Gol Beuleikeo)": {
    "album": "Skool Luv Affair",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/spinebreakerdeunggolbeuleikeo.html",
    "year": "2014"
  },
  "Tomorrow": {
    "album": "Skool Luv Affair",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/tomorrow.html",
    "year": "2014"
  },
  "War Of Hormone (호르몬 전쟁)": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/warofhormone.html",
    "year": "2014"
  },
  "We Are Bulletproof Pt. 2": {
    "album": "2 Cool 4 Skool",
    "type": "single",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/wearebulletproofpt2.html",
    "year": "2013"
  },
  "We On": {
    "album": "O!RUL8,2?",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/weon.html",
    "year": "2013"
  },
  "What Am I To You (Intro)": {
    "album": "Dark & Wild",
    "type": "album",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/whatamitoyouintro.html",
    "year": "2014"
  },
  "Where Did You Come From? (어디에서 왔는지; Eodieseo Wassneunji)": {
    "album": "Skool Luv Affair",
    "type": "EP",
    "url": "http: //www.azlyrics.com/lyrics/bangtanboys/wheredidyoucomefromeodieseowassneunji.html",
    "year": "2014"
  }
}

Expected behavior
When going to the url: https://www.azlyrics.com/b/bangtanboys.html
we can see how songs have been chopped off in the print statement when compared to the actual list of songs on AZlyrics.com.
I believe this issue has to do the the ParseSongs() method in tools.py.
AZlyrics seems to include google adds in one under a certain div tag and I think this is causing early termination possibly.

Screenshots
image

Desktop (please complete the following information):
Using macOS Majove

Additional context
Add any other context about the problem here.

@elmoiv
Copy link
Owner

elmoiv commented Aug 13, 2020

Thanks @brandhsu,
I will do my best to solve this issue ASAP.

@elmoiv
Copy link
Owner

elmoiv commented Aug 13, 2020

I tested azapi and I found that in tools.py inside ParseSongs function, I wrote this within the main loop over listAlbum items:

...
if elmnt.name == 'script':
    break
...

And because AZLyrics.com updated their website with new Google Ads mechanism which now exists between songs under listAlbum dev and uses a script tag, the loop terminates itself when facing the first Google Ad causing many songs not to be included.

Problem was fixed when this code is removed.

I will update azapi to v3.0.3 ASAP

@elmoiv
Copy link
Owner

elmoiv commented Aug 13, 2020

Now pip install azapi==3.0.3 to fix your issue :)

@elmoiv elmoiv closed this as completed Aug 13, 2020
@elmoiv elmoiv added the bug Something isn't working label Aug 13, 2020
@elmoiv elmoiv changed the title getSongs() doesn't return all songs from an artist [Solved] getSongs() doesn't return all songs from an artist Aug 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants