Merge pull request #5 from pelledrijver/dev

Merge changes into master branch

.github/workflows/python-publish.yml

@@ -0,0 +1,45 @@
+name: Publish Python 🐍 distribution 📦 to PyPI and inform the Discord community
+
+on:
+  release:
+    types: [created]
+
+jobs:
+  build-n-publish:
+    name: Build and publish Python 🐍 distributions 📦 to PyPI
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python
+      uses: actions/setup-python@v2
+      with:
+        python-version: '3.x'
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install setuptools wheel
+    - name: Build distributions
+      run: |
+        python setup.py sdist bdist_wheel
+    - name: Publish distribution 📦 to PyPI
+      uses: pypa/gh-action-pypi-publish@master
+      with:
+        password: ${{ secrets.PYPI_API_TOKEN }}
+
+  discord-notification:
+    name: Send an announcement to the Discord community channel 💬
+    runs-on: ubuntu-latest
+    steps:
+    - name: Discord notification
+      env:
+        DISCORD_WEBHOOK: ${{ secrets.DISCORD_WEBHOOK }}
+      uses: Ilshidur/action-discord@master
+      with:
+        args: |+
+          :loudspeaker: **Release Alert!** :loudspeaker:
+          Hi everyone! A new version of the twitch-highlights package has been released: ${{ github.event.release.tag_name }} 📦
+
+          Take a look at the new features and the updated documentation at: https://github.com/pelledrijver/highlights-bot
+        
+  
+

README.md

@@ -1,4 +1,9 @@
 # twitch-highlights
+[![GitHub](https://img.shields.io/github/license/pelledrijver/twitch-highlights)](https://github.com/pelledrijver/twitch-highlights/blob/master/LICENSE)
+[![PyPI](https://img.shields.io/pypi/v/twitch-highlights)](https://pypi.org/project/twitch-highlights/)
+[![GitHub Repo stars](https://img.shields.io/github/stars/pelledrijver/twitch-highlights)](https://github.com/pelledrijver/twitch-highlights/stargazers)
+[![Discord](https://img.shields.io/discord/829297778324537384?color=%237289da&label=discord)](https://discord.gg/SPCj7TURj7)
+
 An OS-independent and easy-to-use module for creating highlight videos from trending Twitch clips. Twitch highlight videos can be created by either specifying a category or a list of streamer names.  
 
 ## Getting started
@@ -25,7 +30,7 @@ Arguments:
 - **twitch_credentials**: *(optional)* Dictionary storing the *client_id* and *client_sectet* keys.
 
 ### login_twitch
-Performs the proper authentication steps using Twitch's OAuth procedure to get access to its API. This method must be called before any other methods on the TwitchHighlights object are called.
+Performs the proper authentication steps using Twitch's OAuth procedure to get access to its API. This method must be called before any other methods on the *TwitchHighlights* object are called. Information on how to obtain these credentials can be found [here](https://dev.twitch.tv/docs/authentication).
 
 ```python
 highlight_generator = TwitchHighlights()
@@ -46,12 +51,14 @@ highlight_generator.make_video_by_category(category = "Just Chatting", language
 ```
 Arguments:
 - **category**: Name of the category from which the clips are gathered (case-insensitive).
-- **output_name**: Name of the generated output mp4 file. Defaults to "*output_video*".
+- **output_name**: Name of the generated output mp4 file. Defaults to *"output_video"*.
 - **language**: Preferred language of the clips to be included in the video. Note that the clip's language tag might not actually match the language spoken in the clip. Defaults to *None*, which means that no clips are removed.
-- **video_length**: Minimum length of the video to be created in seconds. Clips are added to the combined video until this length is reached. Defaults to 300.
+- **video_length**: Minimum length of the video to be created in seconds. Clips are added to the combined video until this length is reached. Defaults to *300*.
 - **started_at**: Starting date/time for included clips as a datetime object in the UTC standard. Defaults to exactly one day before the time at which the method is called.
 - **ended_at**: Ending date/time for included clips as a datetime object in the UTC standard. Defaults to the time at which the method is called.
-- **target_resolution**: Tuple containing (*desired_height*, *desired_width*) to which the resolution is resized. Defaults to (1080, 1920)
+- **render_settings**: Dictionary containing information used for rendering and combining the clips. More information [here](#render_settings). Defaults to *None*.
+- **sort_by**: Preferred ordering of clips (*"popularity", "chronologically", or "random"*). Defaults to *"popularity"*.
+
 
 ### make_video_by_streamer
 Creates a highlight video consisting of trending clip from the provided category in the current directory.
@@ -60,27 +67,38 @@ highlight_generator.make_video_by_streamer(streamers = ["Ninja", "Myth"])
 ```
 Arguments:
 - **streamers**: List of streamer names to gather clips from.
-- **output_name**: Name of the generated output mp4 file. Defaults to "*output_video*".
+- **output_name**: Name of the generated output mp4 file. Defaults to *"output_video"*.
 - **language**: Preferred language of the clips to be included in the video. Note that the clip's language tag might not actually match the language spoken in the clip. Defaults to *None*, which means that no clips are removed.
-- **video_length**: Minimum length of the video to be created in seconds. Clips are added to the combined video until this length is reached. Defaults to 300.
+- **video_length**: Minimum length of the video to be created in seconds. Clips are added to the combined video until this length is reached. Defaults to *300*.
 - **started_at**: Starting date/time for included clips as a datetime object in the UTC standard. Defaults to exactly one day before the time at which the method is called.
 - **ended_at**: Ending date/time for included clips as a datetime object in the UTC standard. Defaults to the time at which the method is called.
-- **target_resolution**: Tuple containing (*desired_height*, *desired_width*) to which the resolution is resized. Defaults to (1080, 1920)
+- **render_settings**: Dictionary containing information used for rendering and combining the clips. More information [here](#render_settings). Defaults to *None*.
+- **sort_by**: Preferred ordering of clips (*"popularity", "chronologically", or "random"*). Defaults to *"popularity"*.
+
 
 ### get_top_categories
 Returns a list of the names of the most trending categories on Twitch at that moment. 
 ```python
 highlight_generator.get_top_categories(5)
 ```
 Arguments:
-- **amount**: Maximum number of categories to return. Maximum: 100. Defaults to 20.
+- **amount**: Maximum number of categories to return. Maximum: 100. Defaults to *20*.
+
+
+### render_settings
+Dictionary containing information used for rendering and combining the clips. When None is passed or any of the keys are missing, the default values are used. 
+
+Keys:
+- **intro_path**: Path to the file containing the intro video that has to be added to the start of the generated video. If not specified, no intro is added.
+- **transition_path**: Path to the file containing the transition video that has to be added between each of the clips in the generated video. If not specified, no transitions are added.
+- **outro_path**: Path to the file containing the outro video that has to be added to the end of the generated video. If not specified, no outro is added.
+- **target_resolution**: Tuple containing (desired_height, desired_width) to which the resolution is resized. Defaults to *(1080, 1920)*.
+- **fps**: Number of frames per second. Defaults to *60*.
 
 ## License
-Apache-2.0
+[Apache-2.0](https://github.com/pelledrijver/twitch-highlights/blob/dev/LICENSE)
 
 ## Contributing
-So far, I have been the only one who has worked on the project and it would be great if I could get an extra pair of hands. Feel free to contact me if you have any great ideas and would like contribute to this project. New features I'm currently working on are:
+So far, I have been the only one who has worked on the project and it would be great if I could get an extra pair of hands. Feel free to contact me if you have any great ideas and would like to contribute to this project. New features I'm currently working on are:
 - Copyright music detection
-- Adding an intro/outro to the generated video
-- Uploading the created video directly to YouTube
-- An extra ordering feature in which the user can specify a specific ordering of clips (by number of views, chronological order, etc)
+- Uploading the created video directly to YouTube

setup.py

@@ -8,15 +8,15 @@
     author_email='pelledrijver@gmail.com',
     url='https://github.com/pelledrijver/twitch-highlights',
     name='twitch-highlights',
-    version='0.0.1',
+    version='1.0.0',
     long_description=long_description,
     long_description_content_type="text/markdown",
     description = "An OS-independent and easy-to-use module for creating highlight videos from trending Twitch clips. Twitch highlight videos can be created by either specifying a category or a list of streamer names.",
     py_modules={"twitch_highlights"},
     install_requires = [
         'requests', 
         'datetime',
-        'moviepy>=1.0.3'       
+        'moviepy>=1.0.3'
     ],
     package_dir={'':'src'},
     packages=find_packages(),

src/twitch_highlights.py

@@ -1,33 +1,94 @@
+from moviepy.editor import VideoFileClip, concatenate_videoclips
+from datetime import datetime, timedelta
 import tempfile
-import os
 import requests
 import shutil
-from datetime import datetime, timedelta
-from moviepy.editor import VideoFileClip, concatenate_videoclips
+import os
+from tqdm import tqdm
+import random
+import proglog
+
 
 def _sort_clips_chronologically(clips):
     clips.sort(key=lambda k : k["created_at"])
 
 
 def _sort_clips_popularity(clips):
-    clips.sort(key=lambda k : k["view_count"])
+    clips.sort(key=lambda k : k["view_count"], reverse = True)
+
+
+def _sort_clips_randomly(clips):
+    clips.sort(random.shuffle(clips))
 
 
-def _merge_videos(clip_list, output_name):
+def _add_clip(clip_list, file_path, render_settings):
+    if len(clip_list) == 0 and 'intro_path' in render_settings:
+        clip_list.append(VideoFileClip(render_settings['intro_path'], target_resolution=render_settings["target_resolution"]))
+
+    if 'transition_path' in render_settings and len(clip_list) != 0:
+        clip_list.append(VideoFileClip(render_settings['transition_path'], target_resolution=render_settings["target_resolution"]))
+
+    clip_list.append(VideoFileClip(file_path, target_resolution=render_settings["target_resolution"]))
+
+def _get_combined_video_length(clip_list):
+    sum = 0
+    for clip in clip_list:
+        sum += clip.duration
+    return sum
+
+
+def _merge_videos(clip_list, output_name, render_settings):  
+    if 'outro_path' in render_settings:
+        clip_list.append(VideoFileClip(render_settings['outro_path'], target_resolution=render_settings["target_resolution"]))
+
     merged_video = concatenate_videoclips(clip_list, method="compose")
 
+    print(f"Writing video file to {output_name}.mp4")
+
     merged_video.write_videofile(
             f"{output_name}.mp4",
             codec="libx264",
-            fps=60,
+            fps=render_settings['fps'],
             temp_audiofile="temp-audio.m4a",
             remove_temp=True,
-            audio_codec="aac")
+            audio_codec="aac",
+            logger=proglog.TqdmProgressBarLogger(print_messages=False))
 
     merged_video.close() 
     for clip in clip_list:
         clip.close()
+
+    print(f'Succesfully generated highlight video {output_name}!')
+
+
+
+def _check_render_settings(render_settings):
+    if render_settings is None:
+        render_settings = dict()
+        render_settings["fps"] = 60
+        render_settings["target_resolution"] = (1080, 1920)
+        return render_settings
+
+    if 'fps' not in render_settings:
+        render_settings['fps'] = 60
 
+    if 'target_resolution' not in render_settings:
+        render_settings['target_resolution'] = (1080, 1920)    
+
+    if 'intro_path' in render_settings:
+        temp = VideoFileClip(render_settings['intro_path'], target_resolution=render_settings["target_resolution"])
+        temp.close()
+
+    if 'outro_path' in render_settings:
+        temp = VideoFileClip(render_settings['outro_path'], target_resolution=render_settings["target_resolution"])
+        temp.close()
+
+    if 'transition_path' in render_settings:
+        temp = VideoFileClip(render_settings['transition_path'], target_resolution=render_settings["target_resolution"])
+        temp.close()
+
+    return render_settings
+
 
 class TwitchHighlights:
     _TWITCH_OAUTH_ENDPOINT = "https://id.twitch.tv/oauth2/token"
@@ -38,18 +99,28 @@ class TwitchHighlights:
 
 
     def __init__(self, twitch_credentials = None):
-        self.tmpdir = tempfile.TemporaryDirectory()
+        self.tmpdir = tempfile.mkdtemp()
 
         if(twitch_credentials):
             self.login_twitch(twitch_credentials)   
 
+    def __del__(self):
+        if hasattr(self, "clip_list"):
+            for clip in self.clip_list:
+                clip.close() 
+
+        shutil.rmtree(self.tmpdir)
 
     def login_twitch(self, twitch_credentials):
         twitch_client_id = twitch_credentials["client_id"]
         twitch_client_secret =  twitch_credentials["client_secret"]
         query_parameters = f'?client_id={twitch_client_id}&client_secret={twitch_client_secret}&grant_type=client_credentials'
 
         response = requests.post(self._TWITCH_OAUTH_ENDPOINT + query_parameters)
+
+        if(response.status_code != 200):
+            raise Exception(response.json())
+
         twitch_token = response.json()['access_token']
 
         self.twitch_oauth_header = {"Client-ID": twitch_client_id,
@@ -66,55 +137,67 @@ def get_top_categories(self, amount = 20):
         return categories    
 
 
-    def make_video_by_category(self, category, output_name = "output_video", language = None, video_length = 300, started_at = datetime.utcnow() - timedelta(days=1), ended_at = datetime.utcnow(), target_resolution=(1080,1920)):
+    def make_video_by_category(self, category, output_name = "output_video", language = None, video_length = 300, started_at = datetime.utcnow() - timedelta(days=1), ended_at = datetime.utcnow(), render_settings = None, sort_by = "popularity"):
         clips = self._get_clips_by_category(category, started_at, ended_at)
-        self._create_video_from_json(clips, output_name, language, video_length, target_resolution)
+        self._create_video_from_json(clips, output_name, language, video_length, render_settings, sort_by)
 
 
-    def make_video_by_streamer(self, streamers, output_name = "output_video", language = None, video_length = 300, started_at = datetime.utcnow() - timedelta(days=1), ended_at = datetime.utcnow(), target_resolution=(1080,1920)):
+    def make_video_by_streamer(self, streamers, output_name = "output_video", language = None, video_length = 300, started_at = datetime.utcnow() - timedelta(days=1), ended_at = datetime.utcnow(), render_settings = None, sort_by = "popularity"):
         clips = []
 
         for streamer in streamers:
             clips += self._get_clips_by_streamer(streamer, started_at, ended_at)
 
-        _sort_clips_popularity(clips)
-        self._create_video_from_json(clips, output_name, language, video_length, target_resolution)
+        self._create_video_from_json(clips, output_name, language, video_length, render_settings, sort_by)
+
 
+    def _create_video_from_json(self, clips, output_name, language, video_length, render_settings, sort_by):
+        print("Succesfully fetched clip data")
 
-    def _create_video_from_json(self, clips, output_name, language, video_length, target_resolution):
-        print("Succesfully fetched clip data") 
         self._preprocess_clips(clips, language)
+        render_settings = _check_render_settings(render_settings)
+
+        if sort_by == "random":
+            _sort_clips_randomly(clips)
+        elif sort_by == "popularity":
+            _sort_clips_popularity(clips)
+        elif sort_by == "chronologically":
+            _sort_clips_chronologically(clips)
+        else:
+            Exception(f'Sorting method {sort_by} not recognized.')
 
         clip_list = []
-        combined_length = 0
+        self.clip_list = clip_list
 
         with requests.Session() as s:
             for i, clip in enumerate(clips):
-                if combined_length >= video_length:
+                if _get_combined_video_length(clip_list) >= video_length:
                     break
 
                 print(f'Downloading clip: {clip["broadcaster_name"]} - {clip["title"]}')
                 file_path = self._download_clip(s, clip, i)
-                newVideoFileClip = VideoFileClip(file_path, target_resolution=target_resolution)
-                clip_list.append(newVideoFileClip)
-                combined_length += newVideoFileClip.duration
 
-        _merge_videos(clip_list, output_name)
+                _add_clip(clip_list, file_path, render_settings)
+
+        _merge_videos(clip_list, output_name, render_settings)
 
-        for file in os.listdir(self.tmpdir.name):
-            os.remove(os.path.join(self.tmpdir.name, file))
+        for file in os.listdir(self.tmpdir):
+            os.remove(os.path.join(self.tmpdir, file))
 
 
     def _check_twitch_authentication(self):
         if not hasattr(self, "twitch_oauth_header"):
-            raise Exception("Twitch authentication incomplete. Please authenticate using the login() method.")
+            raise Exception("Twitch authentication incomplete. Please authenticate using the login_twitch() method.")
 
 
     def _get_request(self, endpoint_url, query_parameters, error_message = "An error occurred"):
         self._check_twitch_authentication()
 
         response = requests.get(endpoint_url + query_parameters, headers=self.twitch_oauth_header)
 
+        if(response.status_code != 200):
+            raise Exception(response.json())
+
         if response.json()["data"] == None:
             raise Exception(error_message)
 
@@ -133,19 +216,20 @@ def _preprocess_clips(self, clips, language):
 
 
     def _download_clip(self, session, clip, id):
-        if not hasattr(self, "tmpdir"):
-            raise Exception("No temporary file storage available for downloaded clips.")
-
         video_url = clip["video_url"]
+        file_path = f'{self.tmpdir}/{id}.mp4'
 
-        file_path = f'{self.tmpdir.name}/{id}.mp4'
+        response = session.get(video_url, stream=True)
+        total_size_in_bytes= int(response.headers.get('content-length', 0))
+        progress_bar = tqdm(total=total_size_in_bytes, unit='iB', unit_scale=True)
 
-        r=session.get(video_url)
         f=open(file_path,'wb')
-        for chunk in r.iter_content(chunk_size=1024*1024):
+        for chunk in response.iter_content(chunk_size=1024*1024):
             if chunk:
+                progress_bar.update(len(chunk))
                 f.write(chunk)
         f.close()
+        progress_bar.close()
 
         return file_path