Search raises 403 HTTP error asking for captcha on VPS

[thepeshka]

Lib should use api.genius.com/search if token provided.

Expected behavior
Lyrics returned

To Reproduce

>>> from lyricsgenius import Genius
>>> genius = Genius(TOKEN)
>>> genius.search_song("Hello", "Adele")
Searching for "Hello" by Adele...
Traceback (most recent call last):
  File "/root/testvenv/lib/python3.6/site-packages/lyricsgenius/api/base.py", line 80, in _make_request
    response.raise_for_status()
  File "/root/testvenv/lib/python3.6/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://genius.com/api/search/multi?q=Hello+Adele

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/testvenv/lib/python3.6/site-packages/lyricsgenius/genius.py", line 401, in search_song
    search_response = self.search_all(search_term)
  File "/root/testvenv/lib/python3.6/site-packages/lyricsgenius/api/public_methods/search.py", line 210, in search_all
    return self.search(search_term, per_page, page, endpoint)
  File "/root/testvenv/lib/python3.6/site-packages/lyricsgenius/api/public_methods/search.py", line 45, in search
    return self._make_request(path, params_=params, public_api=True)
  File "/root/testvenv/lib/python3.6/site-packages/lyricsgenius/api/base.py", line 88, in _make_request
    raise HTTPError(response.status_code, error)
requests.exceptions.HTTPError: [Errno 403] 403 Client Error: Forbidden for url: https://genius.com/api/search/multi?q=Hello+Adele

# curl https://genius.com/api/search/multi?q=Hello+Adele
...
<div class="line2">Sorry, we have to make sure you're a human before we can show you this page</div>
...

Version info

• Package version 3.0.0
• OS: Ubuntu 18.04.4 LTS

Additional context
Reproduced on VPS located in Germany. Locally everything works fine.

[allerter]

Genius.search_song is a convenience method and calls the following methods:

genius.search_all
genius.song
genius.lyrics

But as you have experienced, VPS and proxy users are very likely to get a 403 error because of Genius's captcha service. Even If we switched to using api.genius.com/search, you would still get a 403 error in genius.lyrics.
I suggest either going behind a proxy that works for Genius or limiting your calls to developers' API methods. For example, if you wanted to search for a song and get its info, you could try this which only uses developers' API methods:

songs = genius.search_songs("Eminem Rap God")["hits"]
for song in songs:
    song = song["result"]
    if song['title'] == "Rap God":
        song_id = song['id']
song = genius.song(song_id)

You can view the list of developers' API methods in the docs.

[thepeshka]

@allerter I didn't thought about this. That's sad. Btw, thanks for explanation!

[thedustyrover]

fwiw, it seems the authenticated APIs behind api.genius.com work fine behind the VPS, it's just the public ones without auth that run into the 403 issue.

There is an authenticated version of the Search API with a similar response format that the library could probably be refactored to, however the auth API is also missing other APIs like /albums so this is an incomplete solution overall.

[Lionheartxx]

I have the same error!!!
If i deploy my telegram bot project to server i get 403 forbidden error but when i run local in PyCharm everything works
This my Code:

import numpy as np
from requests.exceptions import HTTPError, Timeout
import lyricsgenius

from aiogram import types
from data.config import GENIUS_ACCESS_TOKEN,GENIUS_CLIENT_ID,GENIUS_CLIENT_SECRET,GENIUS_REDIRECT_URI
from keyboards.inline.SelectButton import SelectSong
from aiogram.types import CallbackQuery
from aiogram.dispatcher.filters import Command
from aiogram.dispatcher.filters.state import State, StatesGroup
from aiogram.dispatcher import FSMContext

from oauthlib.oauth2 import BackendApplicationClient
from requests_oauthlib import OAuth2Session

from loader import dp

class OAuth2Genius(StatesGroup):
    waiting_for_code = State()


GENIUS_API_BASE_URL = 'https://api.genius.com'

def create_genius_session(token=None, state=None):
    client = BackendApplicationClient(client_id=GENIUS_CLIENT_ID)
    genius3 = OAuth2Session(client=client, token=token, redirect_uri=GENIUS_REDIRECT_URI, state=state)
    return genius3

@dp.message_handler(Command("start"))
async def start_auth(message: types.Message):
    # Start the Genius OAuth2 flow
    genius3 = create_genius_session()
    authorization_url, state = genius3.authorization_url('https://api.genius.com/oauth/authorize')

    await message.answer("To authorize, click on this [link]({})".format(authorization_url), parse_mode='Markdown')
    await OAuth2Genius.waiting_for_code.set()

@dp.message_handler(state=OAuth2Genius.waiting_for_code)
async def receive_auth_code(message: types.Message, state: FSMContext):
    # Receive the Genius OAuth2 code and exchange it for a token
    genius3 = create_genius_session(state=state)
    token = genius3.fetch_token('https://api.genius.com/oauth/token', authorization_response=message.text,
                               client_secret=GENIUS_CLIENT_SECRET)

    # Save the token to use later in your requests
    await state.finish()
    await message.answer("Authorization successful! You can now use Genius API.")
    # Store the token somewhere secure for future use (e.g., a database)

    global genius2
    genius2 = create_genius_session(token=token)

genius = lyricsgenius.Genius(GENIUS_ACCESS_TOKEN)
numbered_songs = []

@dp.message_handler()
async def Send_List(message: types.Message):
    def ListOfSongs(search_songs):
        try:
            song = genius.search_songs(f"{search_songs}")

            list_of_artist = []
            list_of_song = []

            for hit in song['hits']:
                list_of_artist.append(hit['result']['primary_artist']['name'])
                list_of_song.append(hit['result']['title'])

            arr = np.dstack((list_of_artist, list_of_song))
            len_arr = len(arr[0])

            for res in arr[0][range(0, len_arr)]:
                numbered_songs.append(f"{res[0]} - {res[1]}")

        except HTTPError as e:
            print(e.errno)
            print(e.args[0])
            print(e.args[1])
        except Timeout:
            pass

    if len(numbered_songs) == 0:
        ListOfSongs(search_songs=message.text)
    else:
        numbered_songs.clear()
        ListOfSongs(search_songs=message.text)

    result_count = min(len(numbered_songs), 10)

    if result_count >= 1:
        await message.answer(
            f"<b>Результаты 1-{result_count}</b>\n\n"
            + "\n".join(f"{i + 1}.  {numbered_songs[i]}" for i in range(result_count)),
            reply_markup=SelectSong
        )
    else:
        await message.answer("Такой песен не найдено 🤔")

@dp.callback_query_handler(lambda call: call.data.isdigit())
async def Send_Lyric(call: CallbackQuery):
    index = int(call.data)

    if 1 <= index <= len(numbered_songs):
        await call.message.delete()
        await call.message.answer("🔎 Ищу тексты песен...")

        full_text = await GetText(numbered_songs[index - 1])

        if full_text:
            # Telegramning maksimal belgilar soni 4040 ta
            if len(full_text) <= 4020:
                await call.message.answer(full_text)
            else:
                short_text = full_text[:4020]
                await call.message.answer(short_text + f"...\n\n<i>через @MusixMBot</i>")
        else:
            await call.message.answer("Извините, такого текста не найдено 🫥")

async def GetText(song_name):
    artist_name, song_get_name = map(str.strip, song_name.split("-"))

    try:
        artist = genius.search_artist(artist_name, max_songs=1)
        song_text = artist.song(song_get_name)
        text = song_text.lyrics

        return f"<code>{artist_name} - {song_get_name}</code>\n\n{text[:-5]}\n\n<i>через @MusixMBot</i>"

    except HTTPError as e:
        print(e.errno)
        print(e.args[0])
        print(e.args[1])

    except Timeout:
        pass

    return None

I used the liyricsgenius package. But I don't know how to integrate Genius authentication method into Code. If anyone knows, please help!!!