Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[v2-beta] Move from Neo4j to Postgres, use TypeScript for server. Fixes
#217, #197, #190, #75 (#220) * Disable underscore-dangle rule * Add mongoose package * Use mongodb for auth queries * Move to MongoDB and TypeScript * Init plan * Update steps * Update config * 💥 Move to Postgres from MongoDB * Remove unused * Improve migration scripts * Decrease concurrent connections * Add skip and fix query * Add migration guide * Increase cahr limit to 1023 * Decrease target limit to 1023 * Update migration guid with important note * Update example with new env vars * Update with v2 guides * Fix migrating visit referrers * Add script to delete duplicated visit relationship * Add shortUrl to link response for backward compatibility * Linting * Fix creating anonymous links * Fix IP cooldown not working * Fix deleting links by deleting visits of links first * Fix and improve links migration script * Add fail note to migration * 2.0.0 * Fix main path * Fix limit when getting list of links * Improve table nav buttons clicking and listing * Return countAll as number instead of string * Fix proptype warning * Fix not authenticating user on initial load. Fixes #71
- Loading branch information
Showing
78 changed files
with
5,087 additions
and
2,213 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| { | ||
| "presets": ["next/babel"], | ||
| "presets": ["next/babel", "@zeit/next-typescript/babel"], | ||
| "plugins": [["styled-components", { "ssr": true, "displayName": true, "preprocess": false }]] | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,5 @@ | ||
| .next/ | ||
| flow-typed/ | ||
| node_modules/ | ||
| client/**/__test__/ | ||
| client/**/__test__/ | ||
| production-server |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,35 +1,43 @@ | ||
| { | ||
| "extends": [ | ||
| "airbnb", | ||
| "prettier", | ||
| "prettier/react" | ||
| "eslint:recommended", | ||
| "plugin:@typescript-eslint/recommended", | ||
| "plugin:prettier/recommended" | ||
| ], | ||
| "parser": "babel-eslint", | ||
| "parser": "@typescript-eslint/parser", | ||
| "parserOptions": { | ||
| "project": "./tsconfig.server.json", | ||
| }, | ||
| "plugins": ["@typescript-eslint"], | ||
| "rules": { | ||
| "eqeqeq": ["warn", "always", { "null": "ignore" }], | ||
| "no-useless-return": "warn", | ||
| "no-var": "warn", | ||
| "no-console": "warn", | ||
| "max-len": ["warn", { "comments": 80 }], | ||
| "no-param-reassign": ["warn", { "props": false }], | ||
| "require-atomic-updates": 0, | ||
| "@typescript-eslint/interface-name-prefix": "off", | ||
| "@typescript-eslint/no-unused-vars": "off", // "warn" for production | ||
| "@typescript-eslint/no-explicit-any": "off", // "warn" for production | ||
| "@typescript-eslint/no-var-requires": "off", | ||
| "@typescript-eslint/camelcase": "off", | ||
| "@typescript-eslint/no-object-literal-type-assertion": "off", | ||
| "@typescript-eslint/no-parameter-properties": "off", | ||
| "@typescript-eslint/explicit-function-return-type": "off" | ||
| }, | ||
| "env": { | ||
| "es6": true, | ||
| "browser": true, | ||
| "node": true | ||
| "node": true, | ||
| "mocha": true | ||
| }, | ||
| "rules": { | ||
| "react/jsx-filename-extension": [ | ||
| 1, | ||
| { | ||
| "extensions": [ | ||
| ".js", | ||
| ".jsx" | ||
| ] | ||
| } | ||
| ], | ||
| "prettier/prettier": [ | ||
| "error", | ||
| { | ||
| "trailingComma": "es5", | ||
| "singleQuote": true, | ||
| "printWidth": 100 | ||
| } | ||
| ], | ||
| "consistent-return": "off" | ||
| "globals": { | ||
| "assert": true | ||
| }, | ||
| "settings": { | ||
| "react": { | ||
| "version": "detect" | ||
| } | ||
| }, | ||
| "plugins": [ | ||
| "prettier" | ||
| ] | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| # Migrate database from Neo4j to Postgres | ||
|
|
||
| As explained in issue #197, Kutt is ditching Neo4j in favor of Postgres in version 2. But what happens to old data? Well, I have created migration scripts that you can use to transfer data from your Neo4j database to your new Postgres database. | ||
|
|
||
| ### 🚧 IMPORTANT: v2 is still in beta, proceed carefully!!1! | ||
|
|
||
| ## General recommendations | ||
|
|
||
| - Importing Neo4j data into local Neo4j database and migrate from there would speed things up. | ||
| - Use local Postgres database (where app lives), because using a remote database server will be way slower. If you're doing this locally, you can import data from local database to the remote one after migration has finished. I used this command to move data: | ||
|
|
||
| ## 1. Set up a Postgres database | ||
|
|
||
| Set up a Postgres database, either on your own server or using a SaaS service. | ||
|
|
||
| ## 2. Pull and run Kutt's new version | ||
|
|
||
| Right now version 2 is in beta. Therefore, pull from `v2-beta` branch and create and fill the `.env` file based on `.example.env`. | ||
|
|
||
| **NOTE**: Run the app at least once and let it create and initialize tables in the database. You just need to do `npm run dev` and wait for it to create tables. Then check your database to make sure tables have been created. (If your production database is separate, you need to initialize it too). | ||
|
|
||
| ## 3. Migrate data using scripts | ||
|
|
||
| First, do `npm run build` to build the files. Now if you check `production-server/migration` folder you will fine 4 files. You can now run these scripts one by one. | ||
|
|
||
| **NOTE:** that the order of running the scripts is important. | ||
|
|
||
| **NOTE:** Step 4 is going to take a good chunk of time. | ||
|
|
||
| **NOTE:** If step 4 fails at any stage, you should delete links and visits data from the database and try again. | ||
|
|
||
| ``` | ||
| // 1. Migrate data: Hosts | ||
| node production-server/migration/01_hosts.js | ||
| // 2. Migrate data: Users | ||
| node production-server/migration/02_users.js | ||
| // 3. Migrate data: Domains | ||
| node production-server/migration/03_domains.js | ||
| // 4. Migrate data: Links | ||
| node production-server/migration/04_links.js | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.