Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start using Bun! #6

Merged
merged 80 commits into from
Sep 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
635b46e
Use bun
Eltik Sep 5, 2023
3bce81f
Proxies working
Eltik Sep 5, 2023
cd522e2
Added extractor stuff
Eltik Sep 5, 2023
becdb54
Add mappings code
Eltik Sep 5, 2023
19bc6e9
Mappings working!
Eltik Sep 6, 2023
ff88e5b
Bug fixes and mappings updates
Eltik Sep 6, 2023
11507d5
Bug fixes with SQLite
Eltik Sep 6, 2023
f961275
Updates to database
Eltik Sep 6, 2023
f113461
Added a lot of providers
Eltik Sep 6, 2023
6c041d4
Add manga providers
Eltik Sep 6, 2023
dcc2f46
Searching
Eltik Sep 6, 2023
6619687
Bug fixes to searching
Eltik Sep 6, 2023
f52e540
Use different base providers
Eltik Sep 7, 2023
d942771
Use multiple base providers
Eltik Sep 7, 2023
2efcdbb
Use MangaDex instead of ComicK
Eltik Sep 7, 2023
04394bf
Fix MangaDex
Eltik Sep 7, 2023
f3ee2f2
Use AniList for novels
Eltik Sep 7, 2023
76c61bb
Episode fetching
Eltik Sep 7, 2023
1377e1b
Added chapters
Eltik Sep 7, 2023
9d5b8c8
Skip times!
Eltik Sep 8, 2023
37dc033
PDF creation
Eltik Sep 9, 2023
eb1b15d
Update routing
Eltik Sep 9, 2023
d1256cd
Add seasonal/recent
Eltik Sep 9, 2023
5dfc2e4
Bug fixes to sources
Eltik Sep 9, 2023
bab5323
Fix MangaDex and seasonal route
Eltik Sep 9, 2023
20ac584
Fix __dirname
Eltik Sep 10, 2023
9ee1ea3
Bug fixes to MangaDex
Eltik Sep 11, 2023
b2f05c1
Add redis caching
Eltik Sep 11, 2023
740aa47
PDFs workin!
Eltik Sep 12, 2023
fbfd4aa
Merged backend recode to main folder
Eltik Sep 12, 2023
958367d
Added recode for manager
Eltik Sep 12, 2023
1d8d211
Merge manager recode to manager
Eltik Sep 13, 2023
7e45637
Squashed commit of the following:
Eltik Sep 13, 2023
fab6c9f
Merge branch 'main' into bun
Eltik Sep 13, 2023
bcf1946
Fix merge
Eltik Sep 13, 2023
f5f2d15
Update manager
Eltik Sep 13, 2023
11136f8
Update auth and manager
Eltik Sep 14, 2023
7832aa4
Add crawling file
Eltik Sep 15, 2023
ac0522e
Shutdown page
Eltik Sep 15, 2023
5086952
Update
Eltik Sep 15, 2023
e3ebcf0
Reverted commit
Eltik Sep 15, 2023
c8d60de
Update crawling efficiency
Eltik Sep 15, 2023
8197a12
Add MangaFire (broken)
Eltik Sep 18, 2023
5ad3d54
Fix MangaDex
Eltik Sep 18, 2023
84e6683
Fix searching
Eltik Sep 18, 2023
c7e5a8e
Use Jimp instead of canvas
Eltik Sep 19, 2023
42d47f3
Improve mapping accuracy + fix MangaFire
Eltik Sep 19, 2023
d7fbe38
Add TMDB as info provider
Eltik Sep 20, 2023
2fedad6
Add TVDB content fetching
Eltik Sep 20, 2023
9886681
Optimized TVDB
Eltik Sep 20, 2023
fe67611
Add AniDB as provider
Eltik Sep 20, 2023
8c793f4
Add AniDB for content
Eltik Sep 20, 2023
cc7d1d3
Content metadata
Eltik Sep 20, 2023
3917667
Fix AniDb
Eltik Sep 20, 2023
adc8cc5
Bug fixes with mappings
Eltik Sep 21, 2023
faa646e
Bug fixes with searching
Eltik Sep 21, 2023
14446fe
Add scripts and update README
Eltik Sep 21, 2023
ca7df73
Update frontend/bug fixes
Eltik Sep 21, 2023
4add83a
Support novel PDFs
Eltik Sep 22, 2023
bb9d756
Remove images from epub craetion
Eltik Sep 22, 2023
a0316c5
Update how novel/manga upload works
Eltik Sep 22, 2023
eec567f
Update how Mixdrop is stored
Eltik Sep 22, 2023
f1fee09
Update 9anime
Eltik Sep 23, 2023
7bb846d
Fix 9anime
Eltik Sep 23, 2023
f070287
Update crawl script
Eltik Sep 23, 2023
18743c2
Update epub.ts
Eltik Sep 23, 2023
ac6bfe6
make recent() much more type safe
skearya Sep 23, 2023
0dcb4ac
Merge branch 'bun' of https://github.com/Eltik/Anify into bun
skearya Sep 23, 2023
9a65dac
Add MangaFox
Eltik Sep 23, 2023
0c8382e
Merge pull request #5 from skearya/bun
Eltik Sep 23, 2023
f630b6d
Update workflows
Eltik Sep 23, 2023
2151230
Update workflows
Eltik Sep 23, 2023
d712f55
Fix workflow again (v2)
Eltik Sep 23, 2023
5016e8b
Use NovelUpdates as base provider
Eltik Sep 23, 2023
cf4521f
Update NovelUpdates
Eltik Sep 24, 2023
22fa434
Bug fixes
Eltik Sep 25, 2023
bb5934d
More fixes
Eltik Sep 25, 2023
2f131f1
Add startup folder
Eltik Sep 25, 2023
f8ab69d
Update manager!
Eltik Sep 25, 2023
0a5de94
Update README.md
Eltik Sep 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
22 changes: 7 additions & 15 deletions .github/workflows/node.js.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,27 +5,19 @@ name: Node.js CI

on:
push:
branches: [ "main" ]
branches: [ "main", "bun" ]
pull_request:
branches: [ "main" ]
branches: [ "main", "bun" ]

jobs:
build:

runs-on: ubuntu-latest

strategy:
matrix:
node-version: [16.x, 18.x]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/

steps:
- uses: actions/checkout@v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- run: npm ci
- run: npm run install && npm run build:ts
#- run: npm test
- uses: oven-sh/setup-bun@v1

- run: bun run install
- run: bun run lint
- run: bun run build
9 changes: 8 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,11 @@ yarn-error.log*
/dist
manami.json
database.json
/manga
/manga
**/manga
bun.lockb
goodProxies.json
proxies.json
/builds
db.sqlite
dump.rdb
66 changes: 12 additions & 54 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,70 +1,28 @@
# Anify
Robust anime and manga web-app.
Robust anime, manga, and light novel web-app.

## Installation
**NOTE:** This project utilizes [Bun](https://bun.sh). Please install it before attempting to run Anify.
1. Clone this repository and `cd` into the directory.
```bash
git clone https://github.com/Eltik/Anify.git
```
2. Run `npm run build`. This will install both backend and frontend `node_modules` as well as build the databases for the backend and frontend.
3. View each directory's `README.MD` file for additional installation/requirements.
4. `cd` into the `anify-manager` directory.
5. Run `npm run dev`.
2. Run `bun i`. This will install the `node_modules` dependencies for all sub-repos.
3. Run `bun run build` to build all directories.
4. View each directory's `README.MD` file for additional installation/requirements.
5. Run `bun start` to startup all services using pm2. If you wish to individually startup a specific service, you can `cd` into a directory and run `bun start`.

## How it Works
Anify has both a backend and a frontend as you may notice from the two directories, `anify-frontend` and `anify-backend`. The first directory is a simple NextJS web server that sends request locally to the backend server. Server-side rendering becomes super easy as a result, allowing for data to be taken from that folder and then rendered server-side. This means interaction with a separate API server client-side is not needed, and CloudFlare under-attack-mode can be turned on to prevent web scraping.
Anify's core components are the backend and frontend. Using custom mappings without the use of external API's such as [Simkl](https://simkl.com) and [MALSync](https://malsync.moe), the backend maps all types of Japanese media and gathers information before storing the mappings in a database. For creating a full-stack website, the frontend then requests data on the server-side to fetch episodes/chapters and sources/pages from the backend allowing for a smooth and secure experience watching anime and reading manga/light novels. Additionally, there is an authentication server for tracking your favorite shows and series on a preferred website.

## Pre-Requisites
### NodeJS
As you may expect, Anify requires NodeJS. Version 18 is recommended.
## Basic Pre-Requisites
### Bun
As you may expect, Anify requires NodeJS. However, for performance sake, Anify utilizes [Bun](https://bun.sh) for the fastest response times possible.
```bash
sudo apt-add-repository -r ppa:chris-lea/node.js
sudo apt update -q
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash - &&\
sudo apt-get install -y nodejs
```
### PostgreSQL
For database management, Anify requires PostgreSQL version 15 and some custom plugins.
```bash
# File repo config
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'

# Import repository signing key thingamajig
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -

# Update package lists
sudo apt-get update

# Install
sudo apt-get -y install postgresql-15
```
The following will add some additional functions that is required for searching and querying the database.
```bash
# Requires pg_trgm
sudo apt-get install postgresql-contrib

# Starts the service
sudo systemctl start postgresql.service

# psql shell
sudo -i -u postgres
psql

# Change password to "password"
ALTER USER postgres WITH PASSWORD 'password';

# Create custom functions. This will automatically be added
# to the backend when building the database, but it is
# recommended to add the functions manually yourself.
CREATE EXTENSION IF NOT EXISTS "pg_trgm";

create or replace function most_similar(text, text[]) returns double precision
language sql as $$
select max(similarity($1,x)) from unnest($2) f(x)
$$;
curl -fsSL https://bun.sh/install | bash
```
### Redis
For caching purposes, Anify uses Redis to store data temporarily (usually for about a hour).
For caching purposes, Anify uses Redis to store data temporarily (usually for about an hour).
```bash
# Prerequisites
sudo apt install lsb-release
Expand Down
26 changes: 5 additions & 21 deletions anify-auth/.env.example
Original file line number Diff line number Diff line change
@@ -1,25 +1,9 @@
# What port to listen to
PORT="3606"
# Public URL
PUBLIC_URL="http://localhost:3606"
FRONTEND_URL="http://localhost:3000"
# Backend URL
BACKEND_URL="http://localhost:3060"
# API Key
BACKEND_KEY=""
# SQLite database URL
DATABASE_URL="file:./db.sqlite"
# Redis URL
REDIS_URL="redis://localhost:6379"
# How long to cache redis data
REDIS_CACHE_TIME="18000"
PUBLIC_URL=https://auth.anify.tv
FRONTEND_URL=http://localhost:3000
REDIS_URL=redis://localhost:6379
REDIS_CACHE_TIME=3600

# Provider API information
ANILIST_CLIENT_ID=""
ANILIST_CLIENT_SECRET=""

MAL_CLIENT_ID=""
MAL_CLIENT_SECRET=""

SIMKL_CLIENT_ID=""
SIMKL_CLIENT_SECRET=""
MAL_CLIENT_SECRET=""
2 changes: 0 additions & 2 deletions anify-auth/.eslintignore

This file was deleted.

33 changes: 0 additions & 33 deletions anify-auth/.eslintrc

This file was deleted.

Loading
Loading