-
-
Notifications
You must be signed in to change notification settings - Fork 160
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #153 from Allerter/add-documentation
Created online docs, fixed div class issue used to scrape lyrics.
- Loading branch information
Showing
31 changed files
with
1,562 additions
and
200 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# Minimal makefile for Sphinx documentation | ||
# | ||
|
||
# You can set these variables from the command line, and also | ||
# from the environment for the first two. | ||
SPHINXOPTS ?= | ||
SPHINXBUILD ?= sphinx-build | ||
SOURCEDIR = src | ||
BUILDDIR = build | ||
|
||
# Put it first so that "make" without argument is like "make help". | ||
help: | ||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) | ||
|
||
.PHONY: help Makefile | ||
|
||
# Catch-all target: route all unknown targets to Sphinx using the new | ||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). | ||
%: Makefile | ||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
@ECHO OFF | ||
|
||
pushd %~dp0 | ||
|
||
REM Command file for Sphinx documentation | ||
|
||
if "%SPHINXBUILD%" == "" ( | ||
set SPHINXBUILD=sphinx-build | ||
) | ||
set SOURCEDIR=src | ||
set BUILDDIR=build | ||
|
||
if "%1" == "" goto help | ||
|
||
%SPHINXBUILD% >NUL 2>NUL | ||
if errorlevel 9009 ( | ||
echo. | ||
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx | ||
echo.installed, then set the SPHINXBUILD environment variable to point | ||
echo.to the full path of the 'sphinx-build' executable. Alternatively you | ||
echo.may add the Sphinx directory to PATH. | ||
echo. | ||
echo.If you don't have Sphinx installed, grab it from | ||
echo.http://sphinx-doc.org/ | ||
exit /b 1 | ||
) | ||
|
||
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% | ||
goto end | ||
|
||
:help | ||
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% | ||
|
||
:end | ||
popd |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
sphinx==3.2.0 | ||
sphinx-rtd-theme==0.5.0 |
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
.. image:: 404.png | ||
|
||
|
||
Whoops! | ||
======== | ||
The page you're looking for doesn't exist. Let's go | ||
:ref:`home <index>`. | ||
|
||
.. toctree:: | ||
:maxdepth: 1 | ||
:hidden: | ||
:caption: Library | ||
|
||
reference | ||
release_notes | ||
|
||
.. toctree:: | ||
:maxdepth: 1 | ||
:hidden: | ||
:caption: Guide | ||
|
||
setup | ||
usage | ||
contributing | ||
|
||
|
||
.. _Genius.com: https://www.genius.com |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
import os | ||
import sys | ||
|
||
sys.path.insert(0, os.path.abspath('../..')) | ||
|
||
# -- Project information ----------------------------------------------------- | ||
|
||
project = 'lyricsgenius' | ||
copyright = '2020, John W. Miller, Allerter' | ||
author = 'John W. Miller, Allerter' | ||
|
||
# -- General configuration --------------------------------------------------- | ||
|
||
extensions = [ | ||
'sphinx_rtd_theme', | ||
'sphinx.ext.napoleon', | ||
'sphinx.ext.autodoc', | ||
'sphinx.ext.autosummary', | ||
'sphinx.ext.extlinks', | ||
] | ||
|
||
exclude_patterns = ['build'] | ||
master_doc = 'index' | ||
autosummary_generate = True | ||
# -- Options for HTML output ------------------------------------------------- | ||
|
||
highlight_language = 'python3' | ||
html_theme = 'sphinx_rtd_theme' | ||
|
||
# -- Other ------------------------------------------------------------------- | ||
|
||
extlinks = { | ||
'issue': ('https://github.com/johnwmillr/LyricsGenius/issues/%s', '#'), | ||
'commit': ('https://github.com/johnwmillr/LyricsGenius/commit/%s', '') | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
Contributing | ||
------------ | ||
Please contribute! Genius has a lot of `undocumented API endpoints`_, and | ||
the ones in that link are only some of them. You could try to implement the | ||
ones in the link or look through Genius yourself to uncover new ones, and | ||
implement them in LyricsGenius to make this client richer in features, and | ||
the access it offers to Genius.com. | ||
|
||
If you want to fix a bug, suggest improvements, or | ||
add new features to the project, just `open an issue`_ on GitHub. | ||
|
||
|
||
|
||
.. _open an issue: https://github.com/johnwmillr/LyricsGenius/issues | ||
.. _undocumented API endpoints: https://github.com/shaedrich/ | ||
geniusly/wiki/Undocumented-API-endpoints |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
Example projects | ||
================== | ||
|
||
- `Trucks and Beer: A textual analysis of popular country music`_ | ||
- `Neural machine translation: Explaining the Meaning Behind Lyrics`_ | ||
- `What makes some blink-182 songs more popular than others?`_ | ||
- `Sentiment analysis on hip-hop lyrics`_ | ||
- `Does Country Music Drink More Than Other Genres?`_ | ||
- `49 Years of Lyrics: Why So Angry?`_ | ||
|
||
.. _`Trucks and Beer: A textual analysis of popular country music`: http://www.johnwmillr.com/trucks-and-beer/ | ||
.. _`Neural machine translation: Explaining the Meaning Behind Lyrics`: https://github.com/tsandefer/dsi_capstone_3 | ||
.. _What makes some blink-182 songs more popular than others?: http://jdaytn.com/posts/download-blink-182-data/ | ||
.. _Sentiment analysis on hip-hop lyrics: https://github.com/Hugo-Nattagh/2017-Hip-Hop | ||
.. _Does Country Music Drink More Than Other Genres?: https://towardsdatascience.com/does-country-music-drink-more-than-other-genres-a21db901940b | ||
.. _`49 Years of Lyrics: Why So Angry?`: https://towardsdatascience.com/49-years-of-lyrics-why-so-angry-1adf0a3fa2b4 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,73 @@ | ||
.. _snippets: | ||
|
||
Snippets | ||
================== | ||
Here are some snippets showcasing how the library can be used. | ||
|
||
|
||
- `All the songs of an artist`_ | ||
- `Artist's least popular song`_ | ||
- `YouTube URL of artist's songs`_ | ||
- `Searching for a song by lyrics`_ | ||
|
||
|
||
All the songs of an artist | ||
------------------------------ | ||
|
||
.. code:: python | ||
from lyricsgenius import Genius | ||
genius = Genius(token) | ||
genius.search_artist('Andy Shauf') | ||
artist.save_lyrics() | ||
Artist's least popular song | ||
---------------------------- | ||
.. code:: python | ||
genius = Genius(token) | ||
artist = genius.search_artist('Andy Shauf', max_songs=1) | ||
page = 1 | ||
songs = [] | ||
while page: | ||
request = genius.get_artist_songs(artist._id, | ||
sort='popularity', | ||
per_page=50, | ||
page=page) | ||
songs.extend(request['songs']) | ||
page = request['next_page'] | ||
least_popular_song = genius.search_song(songs[-1]['title'], artist.name) | ||
print(least_popular_song.lyrics) | ||
YouTube URL of artist's songs | ||
------------------------------ | ||
.. code:: python | ||
import json | ||
# we have saved the songs with artist.save_lyrics() before this | ||
with open('saved_file.json') as f: | ||
data = json.load(f) | ||
for song in data['songs']: | ||
links = song['media'] | ||
if links: | ||
for media in links: | ||
if media['provider'] == 'youtube': | ||
print(print(['song'] + ': ' + media['url']) | ||
break | ||
Searching for a song by lyrics | ||
------------------------------- | ||
.. code:: python | ||
from lyricsgenius import Genius | ||
genius = Genius(token) | ||
request = genius.search_genius_web('Jeremy can we talk a minute?') | ||
for hit in request['sections'][2]['hits']: | ||
print(hit['result']['title']) |
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
.. _how-it-works: | ||
|
||
|
||
How It Works | ||
############# | ||
LyricsGenius makes two kinds of calls to the Genius API. The first one is | ||
the API that requires an access token, and the other one to the public API. | ||
|
||
The API | ||
********** | ||
This API is available through | ||
`api.genius.com <http://api.genius.com>`_, and provides access to features like | ||
searching songs, getting song or annotation data by | ||
providing their ID, and some other features. But the problem with this API | ||
is that Genius has only exposed a little of the whole API. | ||
This is not the case for the second kind of calls: the ones to the public API. | ||
|
||
The Public API | ||
*************** | ||
The public API is the API that doesn't require an access token and can be | ||
accessed by anyone (end-users use this on their browsers). The public API | ||
can be called at `genius.com/api <http://genius.com/api>`_. This API | ||
exposes a whole other set of features (and sometimes access to the same) than | ||
the *token API*, prominently searching Genius web. Although these two calls | ||
can provide you with a lot of information, they still don't let you anywhere | ||
near what's probably the most important thing you may want from Genius: | ||
the lyrics. | ||
|
||
|
||
The Lyrics | ||
********** | ||
Genius has legal agreements with music publishers and considers the lyrics | ||
on their website to be a legal property of Genius, and won't allow you | ||
to re-use their lyrics without explicit licensing. They even | ||
`sued Google on grounds of stolen lyrics`_, asking for $50 million in damages, | ||
but `to no avail`_. So it shouldn't come as a surprise if they don't | ||
provide lyrics in calls to | ||
the API. So how does LyricsGenius get the lyrics? | ||
|
||
LyricsGenius uses a web-scraping library called `Beautiful Soup`_ | ||
to scrape lyrics from the song's page on Genius. Scraping the lyrics in | ||
this way violates Genius' terms of service. If you intend to use the lyrics for | ||
personal purposes, that shouldn't be cause for trouble, but other than that, | ||
you should inquire what happens when you violate the terms this way. | ||
As a reminder, LyricsGenius is not responsible for your usage of the library. | ||
|
||
|
||
.. _the Genius API: http://genius.com/api-clients\ | ||
.. _create a new API client: https://genius.com/api-clients/new | ||
.. _sued Google on grounds of stolen lyrics: https://www.theverge.com/ | ||
2020/8/11/21363692/google-genius-lyrics-lawsuit-scraping-copyright | ||
-yelp-antitrust-competition | ||
.. _to no avail: https://www.theverge.com/2020/8/11/21363692/ | ||
google-genius-lyrics-lawsuit-scraping-copyright-yelp-antitrust-competition | ||
.. _Beautiful Soup: https://pypi.org/project/beautifulsoup4/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,86 @@ | ||
.. _index: | ||
.. image:: header.png | ||
|
||
|
||
LyricsGenius: a Python client for the Genius.com API | ||
==================================================== | ||
.. image:: https://travis-ci.org/johnwmillr/LyricsGenius.svg?branch=master | ||
:target: https://travis-ci.org/johnwmillr/LyricsGenius | ||
.. image:: https://badge.fury.io/py/lyricsgenius.svg | ||
:target: https://pypi.org/project/lyricsgenius/ | ||
.. image:: https://img.shields.io/badge/python-3.x-brightgreen.svg | ||
:target: https://pypi.org/project/lyricsgenius/ | ||
|
||
`Genius.com`_ is a fun website. If you aren’t familiar with it, Genius hosts a | ||
bunch of song lyrics and lets users highlight and annotate passages with | ||
interpretations, explanations, and references. Originally called RapGenius.com | ||
and devoted to lyrics from rap and hip-hop songs, the website now includes | ||
lyrics and annotations from all genres of music. You can figure out what | ||
`“Words are flowing out like endless rain into a paper cup”`_ from Across the | ||
Universe really means, or what Noname was referring to when she said `“Moses | ||
wrote my name in gold and Kanye did the eulogy”`_. | ||
|
||
It’s actually not too difficult to start pulling data from the Genius website. | ||
Genius is hip enough to provide a free application programming interface (API) | ||
that makes it easy for nerds to programmatically access song and artist data | ||
from their website. What the Genius API doesn’t provide, however, | ||
is a way to download the lyrics themselves. With a little help from | ||
`Beautiful Soup`_ though, it’s possible to grab the song lyrics without too | ||
much extra work. And LyricsGenius has done all of that for you already. | ||
|
||
.. | ||
source::https://www.johnwmillr.com/scraping-genius-lyrics/ | ||
|
||
``lyricsgenius`` provides a simple interface to the song, artist, and | ||
lyrics data stored on `Genius.com`_. | ||
|
||
Using this library you can convienently access the content on Genius.com | ||
And much more using the public API. | ||
|
||
You can use ``pip`` to install lyricsgenius: | ||
|
||
.. code:: bash | ||
pip install lyricsgenius | ||
LyricsGenius provides lots of features to work with. For example, let's | ||
download all the lyrics of an artist's songs, and save them to a JSON | ||
file: | ||
|
||
.. code:: python | ||
from lyricsgenius import Genius | ||
genius = Genius(token) | ||
genius.search_artist('Andy Shauf') | ||
artist.save_lyrics() | ||
But before using the library you will need to get an access token. Head over | ||
to :ref:`setup` to get started. | ||
|
||
.. toctree:: | ||
:maxdepth: 1 | ||
:hidden: | ||
:caption: Library | ||
|
||
reference | ||
release_notes | ||
|
||
.. toctree:: | ||
:maxdepth: 1 | ||
:caption: Guide | ||
|
||
setup | ||
usage | ||
how_it_works | ||
text_formatting | ||
contributing | ||
|
||
|
||
.. _Genius.com: https://www.genius.com | ||
.. _“Words are flowing out like endless rain into a paper cup”: | ||
https://genius.com/3287551 | ||
.. _“Moses wrote my name in gold and Kanye did the eulogy”: | ||
https://genius.com/10185147 | ||
.. _`Beautiful Soup`: https://www.crummy.com/software/BeautifulSoup/ |
Oops, something went wrong.