-
Notifications
You must be signed in to change notification settings - Fork 0
2. Problems, Solutions, and What I've Learned
Prior to starting this project, I created a similar app that a user can run locally with their own Client ID and Client Secret. For this project, I used the Spotipy Python library for user authentication and to make GET and POST requests to read user data. While making this early version of Selectify, I relied a lot on Spotipy's documentation to understand what the library had to offer and how to use it. I also used the documentation that Spotify provides on their developer website to learn about different authentication flows and which may work best for me, API keys, tokens, requests, etc., and what they are and how they work with Spotify's API.
When I first started to work on Selectify, I thought that the Spotipy library would be sufficient for user auth and getting user data. I knew nothing about user authentication and felt that Spotipy wouldn't be flexible enough with what I was planning, so I decided to get user authentication and data without the use of it. In hindsight, Spotipy likely would have been fine for this, if not better. However, writing the user authentication code myself gave me a deeper understanding of how APIs work.
While researching how I could use Python for the backend of my project, I learned about Flask, a lightweight micro web framework that allows users to build web apps in Python. I followed Spotify's Web API documentation to learn how to make GET and POST requests. Reading this and implementing it was a challenge because most of Spotify's example code is written in JavaScript, a language I don't currently know. So, I had to first read and understand the examples they provided then figure out how to write them in Python (i.e., Figure out specific libraries that are required such as the requests library for making GET and POST requests). I also learned about HTML files and how they can be implemented with a Flask app, cookies, sessions, and how info is stored from users. Eventually, I got the app working on my laptop locally via flask.
During the early stages of development, my decision to host the app locally or from a cloud service changed a lot. This happened as I did more research about deploying apps because I would read about something that would make me favor one option over the other.
My first plan was to host the app via GitHub Pages. While figuring out how to set it up, I learned about GitHub Pages and its general use, Jekyll and how it is implemented with GitHub Pages, and static vs. dynamic HTML pages. I realized I couldn't use it since Pages only supports static sites and my page is dynamic.
Next, I tried using my old Raspberry PI 3. I couldn't get its operating system to update, but I did learn about port forwarding while trying to set it up. I went on and looked into various cloud hosting services such as AWS, Firebase, Supabase, Google Cloud, etc. None of them seemed to have what I was looking for or were too advanced for what I was trying to do. I decided to revisit my Raspberry PI. First, I wiped the micro SD card which contained its OS and replaced it with the latest version. Then I started learning about local hosting and common ways it is done. I learned about Apache, Ubuntu, Wordpress, and XAMPP. During this, I ran into two issues. The first was that many of the tutorials I read and watched would talk about using relational databases which turned me away because I didn't think I would need it. The other issue was that my Raspberry PI is an older model from 2017 and only has 1 GB of RAM. Since the OS I downloaded was the newest version and had a desktop environment, 60% of the PI's RAM would be used while the PI was idle. This meant that running any app or program while hosting the app would require more memory than what was available and it would likely result in the PI crashing or having other issues.
I turned back to hosting the app through a cloud service and settled on using Supabase. While doing research into Supabase, I began to think that I would need somewhere to store user data such as their username, access token and refresh token. I read about Supabase's built-in Postgres database. Very quickly, this became too confusing for me and I again decided to host the app locally, this time on my laptop. I started watching tutorials and reading about using MySQL with Python because I wanted to host the database on my laptop as well. During this, I realized that a database is not necessary since the app doesn't store any information about the user (e.g., their saved albums) and Spotify stores information such as if a user has authorized my app before. Each time a user requests a random album on the app, it makes a new GET request to Spotify for their saved album data. When Spotify sees that the app has already been authorized, it returns the user's data. If the user hasn't given prior authorization, the app requests it.
Next, I looked into how to port forward since it would be necessary to allow other users to access my site. This turned out to be very confusing due to how my home Wi-Fi is set up, but I learned the difference between static and dynamic IP addresses. I also researched SSL/TLS certificates and realized that to safely deploy my app, I should use a Web Server Gateway Interface (WSGI) server. Flask has a list of recommended WSGI platforms for self-hosting. I chose Gunicorn which recommended I use Nginx as a reverse proxy in conjunction with it. I learned that PowerShell isn't as compatible as other command languages like Bash, Linux or Zsh. Since this was the case, I decided to host the website in a virtual environment using Gunicorn and Nginx until I could get a new Raspberry PI with enough ram to function.
In the meantime, I learned about VirtualBox and started to experiment with using virtual environments. Luckily, I managed to purchase a new PI soon after and decided to set it up with a headless OS and control it with my Mac. I had planned to host the app from a virtual environment on my PI, but learned that there aren't many type-2 hypervisors (e.g., VirtualBox) that have support for the Raspberry PI OS. I then read about venv, a Python module used to create virtual environments for Python apps and chose to use it.
When the PI came in, I set it up and got the app working locally in a virtual environment. After encountering a variety of issues, I got Gunicorn and Nginx properly configured and made the app accessible to anyone. One of the biggest issues I faced involved the Unix socket configuration. A tutorial I followed used a Unix socket to connect Gunicorn and Nginx to each other to communicate. This way of communication was direct via the socket file instead of using IP addresses or network ports. This is typically prioritized over other forms of communication, but it prevents outside devices from connecting. Instead, I had Gunicorn run off of my private IP through a port (e.g., 7000). Then, I had Nginx serve as a reverse proxy meaning it receives incoming requests through a different port (e.g., 80) on my private IP, forwards the requests to Gunicorn which is running the app. When Gunicorn generates a response, it sends it to Nginx which turns around and sends it back to the user.
Next, I used Certbot to get a SSL certificate and set up HTTPS for the app for increased security. I assumed I incorrectly set up the SSL certificate because some devices were able to load the website while others could not, but I could not figure out how to fix it.
The front end of the website was pretty ugly. I started by changing the background color, text color, size, and location, and added a login button. Next, I added a Bootstrap navbar that would be present on every page using template extending. I worked on how the selected album would be displayed. During this, I realized that all of the devices that could not load the website were connected to my Wi-Fi, while all devices that could load it were not (e.g., On my phone, I couldn't load the website while it was connected to my Wi-Fi. However, when I turned Wi-Fi off and used cell data instead, the website would load.). While debugging this, I tried to connect to a VPN on my laptop and loading the website, and it successfully loaded. For displaying the selected album, I used the iFrame API and embedded the selected album onto the website. Lastly, I added a button that opens the album in the Spotify app, and created an "about" and "remove access" page which can be accessed in the navbar.
At the end of August, I will be going back to school and plan to bring my PI with me. Since I won't be living at home, if I want to continue to host Selectify, I will have to reconfigure a lot of the Gunicorn and Nginx files to match the new Wi-Fi connection and IP addresses. This is not ideal since I will also have to reconfigure the files anytime I visit home and if I want the bring the PI away from the house, it won't be able to host the app since it won't be connected to the home network. Instead, I have set up the website to be hosted from PythonAnywhere. The domain that I own had custom records which pointed towards the PI's IP address, with Nginx handling requests and responses between the user and the Gunicorn WSGI server. Instead, I now have the domain forwarding to my PythonAnywhere page.
Although I learned a lot about full-stack development during this project, there is still a lot for me to learn and there are many things I could have done differently and better. For instance, choosing a random album to be selected and displayed is fast for users with a few albums saved, but it is quite slow for longer lists and needs to be scaled handle them. I had planned to add more features to the website but decided against it due to time constraints and Flask's limitations compared to other frameworks such as React JS. I hope to one day revisit this project using a different framework and implement the other features I had planned to add.