Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
1593 lines (1105 sloc) 46.1 KB

Install

Yarn vs NPM

we preffer YARN. treba izbjegavati npm, jer yarn i npm imaju razlicite global foldere

Environment (Windows)

For Windows 10 the best way is to do it through the "Windows Subsystem for Linux" (WSL):

Environment (Mac)

Installing Node.JS

  • https://nodejs.org/en/download/
    • This package will install: • Node.js v12.13.1 to /usr/local/bin/node • npm v6.12.1 to /usr/local/bin/npm
  • IMPORTANT: due to issues with versioning ng/cli, angular, material, typescript, node, etc, we should currently stick to the latest v10 of node.js.
    • NOTE: there are node containers to help you to have various versions of node per different projects
  • after this, you can test if you have installed successfully node and containing npm, by running:
node -v
npm -v
  • tested versions: node: v6.11.2, npm: 3.10.10
  • curently testing versions: node: v10.16.0, npm: 6.9.0

Updating

If old, just download and reinstall newer version

However, cleaner and easier way is to install n package for managing node versions. We will use the n package in order to upgrade node:

npm cache clean -f
npm install -g n
# check available versions
n ls
# installing particular version (10.16.0), latest v10
n 10.16.0
# Under Linux it is installed as node, not nodejs
# under /usr/local/bin/node, not /usr/bin/nodejs

# it is possible to install it in a speciffic folder

Fixing npm-sudo-privileges

explained in https://github.com/Cha-OS/colabo.space-development-gists/blob/master/node.js_npm_yarn/npm-sudo-privileges.md

Yarn

Yarn is a replacement for npm tools. It is faster, caching, reducing space, bandwith and time. We use it here, but you can use npm instead if you insist :)

# to be sure the `~/.bash_profile` file exists
touch  ~/.bash_profile
# install yarn
curl -o- -L https://yarnpkg.com/install.sh | bash
# make `yarn` available in current terminal screen
source ~/.bash_profile

Installing brew

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew update

Installing mongodb server

MongoDB for OSX

Installing

brew update
brew tap mongodb/brew
brew install mongodb-community@4.2
sudo mkdir -p /data/db
mongod --version

Upgrading

if old:

brew upgrade mongodb

Running

To have launchd start mongodb now and restart it at login: brew services start mongodb-community

Or, if you don't want/need a background service you can just run:

mongod --config /etc/mongod.conf

To start it in background (forked):

mongod --config /usr/local/etc/mongod.conf --fork

Checking

ps aux | grep -v grep | grep mongod
# or
sudo netstat -tulpn | grep ':27017'

installing Xcode

We install Xcode because we need the gcc compiler for native npm packages. It might be possible without it, but need to test. Also in that scenario some of the packages (like mongo driver), might be less efficient.

If you do not have XCode installed you should do

  • xcode-select --install

  • choose "install" to install Xcode Command Line Tools

  • or choose "Get XCode" to install Xcode (big)

  • after installing execute:

    • sudo xcode-select -s /Applications/Xcode.app/Contents/Developer

Verify that you’ve successfully installed Xcode Command Line Tools

  • xcode-select -p
  • Just to be certain, verify that gcc is installed
    • gcc --version

Environment (Ubuntu)

Installing Node.JS + NPM

# From https://github.com/nodesource/distributions/blob/master/README.md#deb
# This might change, better check the upper link
curl -sL https://deb.nodesource.com/setup_13.x | sudo -E bash -
sudo apt-get install -y nodejs

After installing Node.JS you will get instructions how to install Yarn:

# From https://github.com/nodesource/distributions/blob/master/README.md#deb
# This might change, better check the upper link
curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt-get update && sudo apt-get install yarn

It is important to change npm to not require sudo for global packages: https://github.com/Cha-OS/colabo.space-development-gists/blob/master/node.js/npm-sudo-privileges.md

MongoDB

https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/

# import the MongoDB public GPG Key from https://www.mongodb.org/static/pgp/server-4.2.asc
wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | sudo apt-key add -
# Install gnupg and its required libraries
sudo apt-get install gnupg
# Once installed, retry importing the key
wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | sudo apt-key add -
# Create the list file /etc/apt/sources.list.d/mongodb-org-4.2.list for your version of Ubuntu
echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.2.list
# reload the local package database
sudo apt-get update
# install the latest stable version
sudo apt-get install -y mongodb-org

sudo mkdir -p /data/db
sudo touch /data/db/log

sudo mongod --dbpath /data/db --logpath /data/db/log
sudo mongod --dbpath /data/db --fork --logpath /data/db/log

# start
sudo service mongod start
sudo service mongod status

Typescript

This is Typescript compiler

# typescript compiler should be global because we use it in angular, backend, etc
# later you run it as a `tsc` command
# CAREFULL! READ ALL OPTIONS FIRST!


# we prefer installing it with yarn:
yarn global add typescript@2.7.2 (maximum for @angular we are using, but ...)
# currently we prefer this
yarn global add typescript@2.8.4 (minimum for @types/jasmine we are using)
yarn global add typescript@3.7.2 (minimum for the @types/chai we are using)

# another option would be installing it with npm
# latest
npm install -g typescript
# or maybe precise version, as ng requires v2.*
npm install -g typescript@2.7.2

SASS support

# this didn't work, ng serve was still complaining: Module build failed: Error: Cannot find module 'node-sass'
yarn global add node-sass
# this works
npm install -g node-sass

CLIs

Angular NG CLI

Angular CLI is a main tool for developing angular applications

Links: https://cli.angular.io/ https://angular.io/cli (this is for the version 6: https://github.com/angular/angular-cli/wiki)

yarn global add @angular/cli

we can also do

(even though not necessary): yarn global add gulp

Colabo.Space CLI

Colabo tools (CLI) is a main tool for maintaining Colabo.Space modular application consisting of colabo puzzles.

Links: https://www.npmjs.com/package/@colabo/cli

If you are planning to install development version of the Colabo.Space CLI please do, otherwise skip it:

yarn global add @colabo/cli
#or
npm install -g @colabo/cli

Summary

after this procedure the yarn's global package.json should look like (with probably different versions)

{
  "dependencies": {
    "@angular/cli": "^8.3.20",
    "@colabo/cli": "^0.4.4",
    "node-sass": "^4.13.0",
    "typescript": "3.7.2"
  }
}

Project related

SHORT - COMPILATION from all install docs

# Create a development folder
mkdir colabo.space
cd colabo.space

# clone the colabo.space repo
git clone https://github.com/Cha-OS/colabo
# if we need to work on another branch, e.g. one named `cf-ng5`,
# change to it;
cd colabo
# in the case you have code in branch, change to that branch by doing:
# git checkout cf-ng5

### Colabo tools ###
# colabo.space/colabo/src/tools
cd src/tools
# install (and build (from TypeScript)) the Colabo Tools:
yarn
# export the `colabo` tool globally:
npm link
# TODO: to replace the procedure to use `yarn link`

### isomorphic ###
cd ../isomorphic
yarn

### Backend ###
cd ../backend 
yarn

### Backend App ###
cd apps/<demo_app>
# e.g: colabo/src/backend/apps/colabo-space
yarn

### Frontend ###
# colabo.space/colabo/src/frontend
cd ../../../frontend/
yarn

### App (Demo-app) ###
cd colabo/src/frontend/apps/<demo_app>
# e.g: cd colabo/src/frontend/apps/psc
yarn

# Running:
ng serve -o

Main - by Sasha

# Create development folder
mkdir <colabo-project-folder>
# clone the colabo.space repo
git clone https://github.com/Cha-OS/colabo
# if we need to work on another branch, e.g. one named `cf-ng5`
# change to it
git checkout cf-ng5
sudo npm install gulp -g
# probably not necessary anymore
# sudo npm i typings -g
# sudo npm install -g bower
# sudo npm install -g typescript
# I assume not necessary 
# sudo npm install marked -g

# it could be necessary to do the following as well
# but do it only if you have privileges issues
cd /usr/local/lib/node_modules
sudo chmod -R o+rx .
sudo chmod g+s .

Install tools

Before installing backend or frontend part of the Colabo.Space ecosystem you need to install Colabo tools as necessary for further installation phases.

The installation is described in the separate document tools/README.md

Install backend

It is described in a separate document backend/INSTALL.md

install frontend

It is described in a separate document frontend/INSTALL.md

Database and collections

MongoChef

MongoChef is a great client for Mongo db.

After installing it, you need to connect, and create new connection, it should be stright forward for the local server. Be sure that connection is with host: localhost, and port 27017.

Creating database and collections

  • Start MongoChef
  • Connect to localhost
  • create database KnAllEdge (File > Add Database ...).
  • download KnAllEdge DB samples
  • import them in Mongo
    • select database in MongoChef
    • import collections (tables in relational DBS are called collections in Mongod Schemaless DBS)
      • Database > Import Collections
      • Click on plus and navigate to your demo KnAllEdge collections
      • select all collections (each exported as a separate json file)
      • If you haven't installed KnAllEdge before and do not have important data then chose option Drop collection first if it allready exists
      • You can also select Validate JSON before import
      • Import
  • Now you can explore collections through the MongoChef interface

Starting for the first time

  • first we start mongo, by
sudo mongod
  • then we start the server, from "backend" folder by typing
npm start
  • at the end, we start the frontend from by typing
npm start

at "fronted" folder. It will start the browser. THen we press CTRL + F, back in the console. This will cause the first full build of the fronted. After its finished, we will probably have to kill the started browser tab, stop frontend from the console with CTRL + C and then restart it "npm start".

With this, our dev environment and CoLaboFramework should be working

Windows Condensed steps for Devs (OBSOLATED)

NOTE: This section is OBSOLATED. If you need Windows, please contact us to update it!

  • cd c:/data/development/Knalledge/

Project

  • create/get-in development folder
  • clone project from github
git clone https://github.com/mprinc/Knalledge

MongoDB

Default MongoDB installation should be good enough. The mongo database must be running before starting the server.

Installing mongodb server

MongoDB for Windows

Get info about windows:

wmic os get caption
wmic os get osarchitecture

Download the community version.

NOTE: These instructions assume that you have installed MongoDB to C:/mongodb folder. Therefore you need to choose custom installation and set the installation path to C:/mongodb.

Data folder

MongoDB requires a data directory to store all data. MongoDB’s default data directory path is /data/db. Create this folder using the following commands from a Command Prompt:

md c:/data/db

Running

Navigate to c:/mongodb/bin/in command prompt and start the server:

c:
cd c:\mongodb\bin\
mongod

When server starts, windows firewall system will ask you for granting access. Allow full access to the server.

Mongod should start on 27017 port.

BACKEND

Install backend

cd c:/data/development/Knalledge/
cd src/backend
npm install
cd modules/topiChat
npm install
cd ../topiChat-knalledge
npm install

Run backend server

cd c:/data/development/Knalledge/
cd src/backend
npm start

When server starts, windows firewall system will ask you for granting access. Allow full access to the server.

KnAllEdge backend server should start on 8888 port and its real-time support (topiChat) should start at 8060 port.

Testing the server

You can go to broweser and type the following command in order to check the connection and database integration:

http://localhost:8888/kmaps/by-participant.json

You should expect something similar to (just and excerpt):

{"data":[{"_id":"5543c546645912db4fee9654","name":"Test Sasha","rootNodeId":"5543c545645912db4fee9653","type":"mcm_map","ideaId":0,"parentMapId":"Test","dataContent":{"mcm":{"authors":"Sasha"}},"__v":0,"updatedAt":"2015-05-01T18:26:14.352Z","createdAt":"2015-05-01T18:26:14.352Z","isPublic":true,"participants":[],"version":1,"activeVersion":1},{"_id":"5543c656645912db4fee9666","name":"SNAC","rootNodeId":"5543c656645912db4fee9665","type":"mcm_map","ideaId":0,"parentMapId":"SNAC","dataContent":{"mcm":{"authors":"Eunseo Choi"}},"__v":0,"updatedAt":"2015-05-01T18:30:46.402Z","createdAt":"2015-05-01T18:30:46.402Z","isPublic":true,"participants":[],"version":1.2,"activeVersion":1},{"_id":"5543cee34a653c345007c839","name":"mcm_changes","rootNodeId":"5543cee34a653c345007c838","type":"mcm_changes","ideaId":0,"parentMapId":"","dataContent":{"mcm":{"authors":"Sasha"}},"__v":0,"updatedAt":"2015-05-01T19:07:15.585Z","createdAt":"2015-05-01T19:07:15.583Z","isPublic":true,"participants":[],"version":1,"activeVersion":1},

and similarly, for:

http://localhost:8888/howAmIs/all/.json

Note: you will have prefix in the returned json: )]}', that helps to avoid browser JS attacks.

This is a positive signal that the backend is installed properly.

Frontend

building tools

npm install gulp -g
npm i typings -g
npm install -g bower

npm install -g typescript
npm install node-gyp -g


# npm install ts-node -g
# npm install typescript-node -g
# npm install node-gyp -g

npm install marked -g

code

cd c:/data/development/Knalledge/
cd src/frontend
npm install
npm run typings install
# or
typings -v
node_modules/typings/dist/bin.js -v

node_modules/typings/dist/bin.js install

Typings issues

  • open src\frontend\typings\globals\angular-protractor\index.d.ts and
  • go to the bottom of the file and comment the line declare var $: cssSelectorHelper; => // declare var $: cssSelectorHelper;

Building Process

The majority of server and client components are built on MEAN stack so the environment necessary for starting tool should be straightforward.

Local machine

fixing-npm-permissions sudo chown -R $(whoami) $(npm config get prefix)/{lib/node_modules,bin,share}

Install tools:

Installing on Server

This section is OBSOLATED

Server is at the moment completely built as node.js environment. Therefore you need node and npm tools installed to run it properly. When you have them installed all you need to do is to install necessary packages with

git clone https://github.com/mprinc/KnAllEdge
mv KnAllEdge knalledge
chmod -R g+ws knalledge
    drwxrwsr-x+  7 mprinc developers
    drwxrwsr-x   7 mprinc developers    4096 Mar 17 09:28 knalledge
# npm install


    # http://stackoverflow.com/questions/7487793/symbolic-link-not-inheriting-permissions
    cd /usr/local/bin
    sudo chmod -h go+rx typings

cd src/backend/
    npm install --production

    # sudo npm cache clean
    # rm -rf node_modules

joe /etc/apache2/sites-enabled/framework.colabo.space
# DocumentRoot /var/www/knalledge/src/frontend
# ->
# DocumentRoot /var/www/knalledge_frontend
apache2ctl restart

cd /var/www/knalledge_frontend/
chmod -R o+r *
ls -al node_modules | grep ng2
chmod -R g-s node_modules/*
chmod -R go+rx node_modules/*
ls -al node_modules | grep ng2
# From here are tasks on the local server:
# go to the folder `frontend/dist/dev`
# and zip it
# `zip dev.zip`
# rename it to show date `dev-YYYY.MM.DD.zip` (e.g `dev-2016.03.18.zip`)

# use SFTP to put this zip at destination `/var/www/knalledge_frontend/dist`

# now, login to server:
# > `ssh user@server`
cd /var/www/knalledge_frontend/dist
ls -al
    resulting in: # drwxrwxr-x  9 mprinc developers    4096 Mar 17 12:39 dev
rm -rf dev
unzip <name_of_the_zip> (e.g. `unzip dev-2016.03.18.zip`)
# check about this, but probably should not do it: `chmod -R g-s dev`
chmod -R go+rx dev

cd /var/www/knalledge_frontend
cp dist/dev/components/collaboPlugins/globalEmitterService.js dist/dev/components/collaboPlugins/GlobalEmitterService.js
cp dist/dev/components/collaboPlugins/globalEmitterServicesArray.js dist/dev/components/collaboPlugins/GlobalEmitterServicesArray.js


joe dist/dev/js/config/config.env.js
#uncomment:    `//var env = envs.localhost;`
#comment:    `var env = envs.server;`

# =======================
cp -r dist/dev
cd ../frontend
    npm install --production

joe /var/www/knalledge_frontend/dist/dev/index.html
# var disableLog = true;

joe /var/www/knalledge_frontend/dist/dev/components/knalledgeMap/css/default.css

# .knalledge_map_middle_bottom{
#     display: none;
#     ...
# }

stop knalledge-b
start knalledge-b
restart knalledge-b

Test as a script:

nodejs /var/www/knalledge/src/backend/KnAllEdgeBackend.js

Test in the service context:

sudo -u www-data /usr/bin/nodejs /var/www/knalledge/src/backend/KnAllEdgeBackend.js 8888

TypeScript

sudo npm install -g ts-node sudo npm install -g typescript rm -r typings npm run postinstall

Precompile tools at one machine tsc copy them to Sinisha's machine

Typings

typings install open --global --save typings install merge-stream --ambient --save-dev

typings install https://github.com/DefinitelyTyped/DefinitelyTyped/blob/ffceea9dd124d277c4597c7bd12930666ec074c5/open/open.d.ts "open": "github:DefinitelyTyped/DefinitelyTyped/open/open.d.ts#ffceea9dd124d277c4597c7bd12930666ec074c5" src/frontend/typings.json

declare module 'open' {
	function open(target: string, app?: string): void;
	export = open;
}
declare module open {
	export function open(target: string, app?: string): void;
}

declare module "open" {
	export = open;
}
// Generated by typings
// Source: https://raw.githubusercontent.com/DefinitelyTyped/DefinitelyTyped/dc2c61c4b295d88befeba7fa542582d0e8ec85dc/merge-stream/merge-stream.d.ts
// Type definitions for merge-stream
// Project: https://github.com/grncdr/merge-stream
// Definitions by: Keita Kagurazaka <https://github.com/k-kagurazaka>
// Definitions: https://github.com/borisyankov/DefinitelyTyped


declare module MergeStream {

    interface IMergedStream extends NodeJS.ReadWriteStream {
        add: (source: NodeJS.ReadableStream) => IMergedStream;
    }

    function merge<T extends NodeJS.ReadableStream>(...streams: T[]): IMergedStream;
    export = merge;
}

declare module "merge-stream" {
	export = MergeStream;
}
npm WARN enoent ENOENT: no such file or directory, open '/Users/sir/Documents/data/Development/KnAllEdge/src/backend/node_modules/v8-debug/package.json'
npm ERR! Darwin 15.3.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "--dev"
npm ERR! node v5.8.0
npm ERR! npm  v3.8.0
npm ERR! path /Users/sir/Documents/data/Development/KnAllEdge/src/backend/node_modules/.staging/npmlog-57f83870
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall rename

npm ERR! enoent ENOENT: no such file or directory, rename '/Users/sir/Documents/data/Development/KnAllEdge/src/backend/node_modules/.staging/npmlog-57f83870' -> '/Users/sir/Documents/data/Development/KnAllEdge/src/backend/node_modules/node-inspector/node_modules/v8-debug/node_modules/node-pre-gyp/node_modules/npmlog'
npm ERR! enoent ENOENT: no such file or directory, rename '/Users/sir/Documents/data/Development/KnAllEdge/src/backend/node_modules/.staging/npmlog-57f83870' -> '/Users/sir/Documents/data/Development/KnAllEdge/src/backend/node_modules/node-inspector/node_modules/v8-debug/node_modules/node-pre-gyp/node_modules/npmlog'
npm ERR! enoent This is most likely not a problem with npm itself
npm ERR! enoent and is related to npm not being able to find a file.
npm ERR! enoent

Colabo Server

users

add mprinc/sir to sudoers

usermod -a -G sudo mprinc
usermod -a -G sudo sir

Node

info

sudo apt-get update
curl -sL https://deb.nodesource.com/setup_5.x | sudo -E bash -
sudo apt-get install -y nodejs

MongoDB

Default MongoDB installation should be good enough. The mongo database must be running before starting the server.

Linux

sudo service mongod stop
sudo service mongod start
sudo service mongod restart

After that you need to restart all services listening to it

restart knalledge-b

TODO: we need to make backend smart enough to detect new/restart/crash of the database

Running the backend

When you want to run the server you need to run specific package.json command: start. You do it by running:

npm start

Server will start, connect to default mongodb port and start listening for client connections on the announced port.

Client

KnAllEdge - frontend class-diagram

Client is implemented at the moment as a ng1/2 hybrid (Angular1/Angular2). Angular

Debugging support

Debugging is handled with the support of the debugpp and debug libraries. Therefore in the browser console you need to enable debugging in order to see it:

This is how you would globally enable debugging:

localStorage.debug = '*';

or just for particular namespace (and all subspaces)

localStorage.debug = 'knalledge.collaboPluginsServices.*';

or just for the semantic-subspace (error in this case) of a for particular namespace:

localStorage.debug = 'knalledge.collaboPluginsServices.*.error';

Development deployment

server

Anything that should be served OUT of the dist/dev folder, should be mapped in the server config to be available: src/frontend/tools/utils/code_change_tools.ts

This is how we have enabled bower_components and images:

let routes:any = {
  ...
  '/bower_components': 'bower_components',
  '/app/images': 'app/images'
};

DefinitelyTyped conflicts

jQuery and Protractor (Seleinum) both use the same global variable: $. There fore TypeScript compiler complains:

⨯ Unable to compile TypeScript

typings/main/ambient/jquery/jquery.d.ts (3212,13): Subsequent variable declarations must have the same type.  Variable '$' must be of type 'cssSelectorHelper', but here has type 'JQueryStatic'. (2403)

At the moment there is no simple solution except remove it from some of the tools, etc. We did remove it manually:

Here are

Installing new machine (problems)

  • get backup of working machine
  • git clone ...
  • copy/overwrite folders/files
    • frontend
      • bower_components
      • node_modules
      • tools/manual_typings
      • typings
      • typings.json
    • backend
      • it should work fine so just
        • cd backend
        • npm install
          • cd modules/topiChat
          • npm install
          • cd modules/topiChat-knalledge
          • npm install
      • or do it faster by copying/overwriting folders/files
        • node_modules
        • modules/topiChat/node_modules
        • modules/topiChat-knalledge/node_modules
  • now you can do symbolic linking

Proxying

system rerouting 80 -> 8088

One time

sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 8088

Continuous

joe /etc/rc.local
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 8088

Enable it sudo chown root /etc/rc.local sudo chmod 755 /etc/rc.local sudo /etc/init.d/rc.local start Check services and ports

Proxying - node-fork

(/etc/init/knalledge-fork.conf)

start/status/stop knalledge-fork

or in a testing mode:

/usr/bin/nodejs /var/www/web_fork/index.js
exec sudo -u www-data /usr/bin/nodejs /var/www/web_fork/index.js

check port listening processes:

sudo netstat -tulpn | grep ':80'
sudo netstat -tulpn | grep ':8088'

more info

Proxying - Nginx

Install the nginx proxy and:

  • remove default from sites-enabled
api.colabo.space

TopiChat

server { listen 80;

alias for two subdomains

server_name topichat.colabo.space;

location / { ​ proxy_pass http://localhost:8002; } }

API

server { listen 80;

alias for two subdomains

server_name api.colabo.space;

location / { ​ proxy_pass http://localhost:8001; } }

server { listen 80;

alias for two subdomains

server_name cf.colabo.space framework.colabo.space map.colabo.space;

root /var/www/domain1;

location / { ​ proxy_pass http://localhost:8000; } }

​```sh
cd /etc/nginx
rm /sites-available/default
joe /sites-enabled/framework.colabo.space
ln -s  ../sites-available/framework.colabo.space framework.colabo.space
# test configuration
sudo nginx -t
sudo service nginx restart
# if necessary
sudo shutdown -h now

Errors

Could not build the server_names_hash, you should increase server_names_hash_bucket_size

joe /etc/nginx/nginx.conf

uncoment and increase

http{
  server_names_hash_bucket_size 256;
}

Big files

How to Find Out Top Directories and Files

du -a | sort -n -r | head -n 5
du -hs * | sort -rh | head -5
# To display the largest folders/files including the sub-directories, run:
du -Sh | sort -rh | head -5
# Top File Sizes Only
find -type f -exec du -Sh {} + | sort -rh | head -n 5

Troubleshooting

brew

NOTE: If brew makes problems later with installing mongodb etc, mentioning problem with folder /usr/local/Frameworks privileges, please do the following

Permission denied @ dir_s_mkdir - /usr/local/Frameworks #30652

sudo mkdir /usr/local/Frameworks
sudo chown $USER /usr/local/Frameworks/

MongoDB

Info

official install

  • log: /var/log/mongodb/*
  • upstart config: /etc/mongod.conf
  • mongod config: etc/init/mongod.conf
  • data: ``
  • service: /etc/init.d/mongod

relevant files:

/var/log/upstart/
/data/db/mongod.lock
/etc/init.d/.#mongod
/usr/bin/mongod
/var/lib/mongodb
/var/lib/mongodb/EarthCube.0
/var/log/mongodb

running services:

chkconfig --list
runlevel

Upstart logs your service execution in a log file by the same name in /var/log/upstart/your-service-name.log. It should be helpful.

repair

rm /data/db/mongod.lock
rm /var/lib/mongodb/mongod.lock
mongod --repair

Upgrading MongoDB (OSX)

https://docs.mongodb.com/manual/release-notes/4.0-upgrade-standalone/ https://docs.mongodb.com/manual/release-notes/3.6/#upgrade-procedures https://stackoverflow.com/questions/30379127/how-to-install-earlier-version-of-mongodb-with-homebrew

brew search mongodb

# 3.2 -> 3.4
brew install mongodb@3.4
# instead of 3.4.17 in 2 following rows, you might need to use appropriate version
sudo  /usr/local/Cellar/mongodb@3.4/3.4.17/bin/mongod
#start:
/usr/local/Cellar/mongodb@3.4/3.4.17/bin/mongo
# and execute in the client the following commands:
    db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )
    db.adminCommand( { setFeatureCompatibilityVersion: "3.4" } )
    db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )

# 3.4 -> 3.6
brew install mongodb@3.6
sudo  /usr/local/Cellar/mongodb@3.6/3.6.8/bin/mongod
#start:
/usr/local/Cellar/mongodb@3.6/3.6.8/bin/mongo
# and execute in the client the following commands:
    db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )
    db.adminCommand( { setFeatureCompatibilityVersion: "3.6" } )
    db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )

brew remove mongodb@3.4
brew remove mongodb@3.6

# 3.6 -> 4
brew install mongodb --with-openssl
sudo mongod
mongo
    db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )
    db.adminCommand( { setFeatureCompatibilityVersion: "4.0" } )

XCode (gcc)

http://railsapps.github.io/xcode-command-line-tools.html

If you had the full XCode installed before and want to downgrade to Xcode Command Line Tools (smaller) then you should do

  • uninstall the XCode
  • set the CLT path xcode-select --switch /Library/Developer/CommandLineTools/

To check if gcc is properly working, you can create simple C++ file test.cpp at any convinient place:

#include <stdio.h>

int main(){
printf("Hello world!\n");
}

Then you can compile it with the gcc compiler:

gcc test.cpp

and run it:

./a.out

You should get the message: Hello world!

npm

NPM privileges problem

cd /usr/local/lib/node_modules
sudo chmod o+rx npm
cd npm
sudo chmod -R o+r  *
sudo chmod o+rx  node_modules/

This didn't work: solution?

npm with yarn

npm WARN lifecycle The node binary used for scripts is /var/folders/kf/4b568dk10xdbh1hn4c08dpc00000gp/T/yarn--1562450253736-0.8092412955507404/node but npm is using /usr/local/bin/node itself. Use the `--scripts-prepend-node-path` option to include the path for the node binary npm was executed with.

You need to create a .npmrc file together with package.json with the scripts-prepend-node-path variable set to true:

scripts-prepend-node-path = true

Uninstall. Cleaning

Uninstalling apps

inace za bilo koju unix app mozes: which app; i kaze ti putanju

Links

Brew

we should not uninstall it

XCode

removing global packages (yarn & npm)

  • npm & yarn folders are different!
  • imas komande za spisak globalnih

yarn

  • yarn global list listing all global packages

    • @sir's case: ![image-20191207124342084](../../../../Library/Application Support/typora-user-images/image-20191207124342084.png)
  • but this dispaly is not consistent enough to user in yarn remove command. What we do we do:

    • yarn global dir that prints the output of the global installation folder that houses the global node_modules. By default that will be: ~/.config/yarn/global. (e.g. /Users/sir/.config/yarn/global)

    • we go to that folder and open its package.json and there we find package names.

      • for the above @sir's case, it contains

      • {
          "dependencies": {
            "@angular/cli": "^8.3.20",
            "gulp": "^4.0.2",
            "node-sass": "^4.13.0",
            "typescript": "3.7.2"
          }
        
    • for each of the package names we do yarn global remove <package-name>

      • this will remove all the folders from the <global dir>/node_modules (even though there are many of them, they are probably there as dependencies of the few of those listed in the package.json, i.e in yarn global list)
  • for local packages (https://github.com/yarnpkg/yarn/pull/1284#issuecomment-255748517)

    • In the case where I would want to remove all dependencies I would just rm -rf node_modules yarn.lock package.json and I would almost never want to do that.
  • Linked packages

  • Troubleshooting

    • maybe another package is dependent on the one you want to remove. I got a same situation when yarn global remove docsify,then I found docsify-cli was dependent on docsify, it worked when I do yarn global remove docsify-cli.

npm

  • npm installs all its global modules to the same directory. we find it by npm root -g The exact location may vary, but typically it's /usr/local/lib/node_modules

    • @sir's listing:

      ![image-20191207235539130](../../../../Library/Application Support/typora-user-images/image-20191207235539130.png)

  • https://stackoverflow.com/questions/9283472/command-to-remove-all-npm-modules-globally

  • Kai Sternad's solution : The following command removes all global npm modules. Note: this does not work on Windows. For a working Windows version, see Ollie Bennett's Answer.

    npm ls -gp --depth=0 | awk -F/ '/node_modules/ && !/\/npm$/ {print $NF}' | xargs npm -g rm
    • but this command might break like in @sir's upper example with npm ERR! enoent ENOENT: no such file or directory, access '/usr/local/lib/node_modules/colabo-tools' (if it is linked to unexisting folder); so we can just do rm -rf colabo-tools and similar for each of the linked problematic ones

    • @sir's result of this set of commands: removed 956 packages in 27.737s

    • option 2 (Leonid Beschastny's solution)

    • TROUBLESHOOTING !!

      • Yet the upper command fails for the scoped packages, e.g. @angular. This is the command that handles them too: npm ls -gp --depth=0 | awk -F/ '/node_modules\/@/ {print $(NF-1)"/"$NF} /node_modules\/[^@]/ && !/\/npm$/ {print $NF}' | xargs npm -g rm

      • if it breaks for the non-existing folder, we also go inside its parent and remove that symlink (it's that in mosta cases)

      • yet, it also broke with this

        npm ERR! code EEXIST
        npm ERR! path /usr/local/bin/colabo
        npm ERR! Refusing to delete /usr/local/bin/colabo: ../lib/node_modules/@colabo/cli/dist/index.js symlink target is not controlled by npm /usr/local/lib/node_modules/@colabo/cli
        npm ERR! File exists: /usr/local/bin/colabo
        npm ERR! Move it away, and try again.
        
    • I tried Kai Sternad's solution but it seemed imperfect to me. There was a lot of special symbols left after the last awk from the deps tree itself.

    • So, I came up with my own modification of Kai Sternad's solution (with a little help from cashmere's idea):

    npm ls -gp --depth=0 | awk -F/node_modules/ '{print $2}' | grep -vE '^(npm|)$' | xargs -r npm -g rm
    • ​ @BrDaHa just checked it on my ubuntu with npm 1.4.21 and it worked fine. Though, it attempted to remove n node binary manager and node-gyp addon builder, but not npm. – Leonid Beschastny Nov 4 '14 at 18:42
  • you can see the list of all global npm-installed packages: npm list -g --depth=0

uninstall yarn

  • as (as can be seen in the lower INSTALL section) we installed it using CURL, this is the procedure https://github.com/yarnpkg/website/issues/514:
    • All it does is extracts the tarball to ~/.yarn so you can simply delete that directory. Maybe in the future we could have an uninstall script.
    • Oh yeah, it also adds ~/.yarn/bin to the PATH. Feel free to submit a pull request adding an uninstallation script if you like.
    • brew uninstall --force yarn cannot uninstall it in our case

uninstall npm & node

  • we can either use this manual approach https://stackoverflow.com/a/11178106/4742336

    • To completely uninstall node + npm is to do the following:

      1. go to /usr/local/lib and delete any node and node_modules
      2. go to /usr/local/include and delete any node and node_modules directory
        1. @sir: btw, this contains npm installation as I remember
      3. THIS DOESN'T WORK IN OUR CASE: if you installed with brew install node, then run brew uninstall node in your terminal
      4. check your Home directory for any local or lib or include folders, and delete any node or node_modules from there
        1. @sir, yet I (as others also suggested) have seen and deleted rm -rf /Users/[homedir]/.npm
      5. go to /usr/local/bin and delete any node executable
        1. @sir: Here I've also done:
          1. hmm. what? forgot to write it here :)

      You may also need to do:

      sudo rm -rf /opt/local/bin/node /opt/local/include/node /opt/local/lib/node_modules
      sudo rm -rf /usr/local/bin/npm /usr/local/share/man/man1/node.1 /usr/local/lib/dtrace/node.d
      • @sir in /usr/local/share/man/man1 I've also done:

        • rm -rf npm-* because there were a lot of symlinks whose name started with npm-

        • rm npm.1
          rm npx.1
          rm node.1
          
      • @sir in /usr/local/share/man/man5 I've also done:

        • rm -rf npm-*
          rm npmrc.5
          rm package*
          
      • @sir in /usr/local/share/man/man7 I've also done:

        • rm -rf npm-*
          rm removing-npm.7
          rm semver.7
          

      Additionally, NVM modifies the PATH variable in $HOME/.bashrc, which must be reverted manually.

  • OR we can start this script https://github.com/brock/node-reinstall that deletes many things but then reinstalls and installs NVM

  • we installed node+Npm from the .pkg, but it's not in the Mac's Application folder

    • pa ok, pokrenuo si app installer ali je on instalirao samo u foldere, ne u Applications
    • @sir : ![image-20191208201749590](../../../../Library/Application Support/typora-user-images/image-20191208201749590.png)
    • @mprinc's folders node-npm-folders
  • Discussion

    • @mprinc: pa samo sa n instaliraj novi, ali mozes i deinstalirati stari, narano bolje je
    • i onda samo te obrisem? foldere? i tako deinstaliram? npm i node
      • [ December 6, 2019 23:48 ] Sasa Rudan: ja ne bih tako [ December 6, 2019 23:48 ] Sasa Rudan: pojuri nesto iskusnije, neko uputstvo

activities Order

  1. reinstaliraj xcode, instalira se nekako drugacije mislim, minimalna verija
  2. obrisi npm i yarn pakete
    1. obrisem globalne pakete pre nego deinstaliram npm i yarn?
  3. uninstall yarn
  4. uninstall npm i node
  5. OPTIONAL: drugo, instaliraj onaj n, za node.js versioning
    1. али за сада не
    2. tek na kraju ako ovo ne pomongne mozemo vidjeti sta dalje
  • ...

Tips

  • my node-sass (instalirao ga jarn dakle)

0-02-0a-a3dda18f2ddfe18e24c104259439a4e3a9215a5065dbe3535f6c2e0923f0dbd4_1c6d9b03b5544f

Cleanup

  • do at the end
    • (brew doctor)
    • brew cleanup

Troubleshoot during the next INSTALL phase

FSEVENTS -> NODE-SASS CRASH

  • during the frontend: yarn it gets broken

  • it look's like it causes problems also to node-sass

  • we solved it with

    • using npm instead of yarn for node-sass install (in the later install phase)
    • deleting frontend/yarn.lock (including deleting frontend/node_modules)
    • yarn cache clean
    • fronted> yarn
    • we've repeated the same procedure for the frontend app yarn
  • this is the history of other attempts, just for some future need (not needed to be done now)

    • on discussion forums it's said that it gets fixed with the fsevents: ^ 1.2.9

    • we run in frontend: npm ls fsevents to check the versions installed

    • SRs-MacBook-Pro:frontend sir$ npm ls fsevents
      Colabo-frontend@1.0.0 /Users/sir/Documents/data/development/colabo.space/colabo/src/frontend
      ├─┬ @angular-devkit/build-angular@0.7.0
      │ └─┬ @angular-devkit/core@0.7.0
      │   └─┬ chokidar@2.0.4
      │     └── UNMET OPTIONAL DEPENDENCY fsevents@^1.2.2
      ├─┬ @angular/compiler-cli@6.1.0
      │ └─┬ chokidar@1.7.0
      │   └── UNMET OPTIONAL DEPENDENCY fsevents@^1.0.0
      └─┬ karma@1.7.1
        └─┬ chokidar@1.7.0
          └── UNMET OPTIONAL DEPENDENCY fsevents@^1.0.0
      
    • I've changed yarn.lock

      • changed its dependencies to "^1.2.9"

      • and also changed fsevents entry from 1.2.4 to resolved "https://registry.npmjs.org/fsevents/-/fsevents-1.2.9.tgz#3f5ed66583ccd6f400b5a00db6f7e861363e388f"

      • then I've deleted frontend/node_modules and reran yarn

      • during it I've got warnings

        warning @angular-devkit/build-angular > @angular-devkit/core > chokidar > fsevents@1.2.9: One of your dependencies needs to upgrade to fsevents v2: 1) Proper nodejs v10+ support 2) No more fetching binaries from AWS, smaller package size
        warning @angular/compiler-cli > chokidar > fsevents@1.2.9: One of your dependencies needs to upgrade to fsevents v2: 1) Proper nodejs v10+ support 2) No more fetching binaries from AWS, smaller package size
        
      • but still the same errors

      • then I changed to resolved "https://registry.npmjs.org/fsevents/-/fsevents-2.1.2.tgz#4c0a1fb34bc68e543b4b82a9ec392bfbda840805"

      • and I've done yarn cache clean and rm -rf node_modules/

      • after that, no fsevent errors, but only

        warning Error running install script for optional dependency: "/Users/sir/Documents/data/development/colabo.space/colabo/src/frontend/node_modules/node-sass: Command failed.
        Exit code: 1
        
      • but now npm ls fsevents gives the ERROR:

        Colabo-frontend@1.0.0 /Users/sir/Documents/data/development/colabo.space/colabo/src/frontend
        ├─┬ @angular-devkit/build-angular@0.7.0
        │ └─┬ @angular-devkit/core@0.7.0
        │   └─┬ chokidar@2.0.4
        │     └── UNMET OPTIONAL DEPENDENCY fsevents@^1.2.2
        ├─┬ @angular/compiler-cli@6.1.0
        │ └─┬ chokidar@1.7.0
        │   └── UNMET OPTIONAL DEPENDENCY fsevents@^1.0.0
        ├── fsevents@2.1.2  extraneous
        └─┬ karma@1.7.1
          └─┬ chokidar@1.7.0
            └── UNMET OPTIONAL DEPENDENCY fsevents@^1.0.0
        

Misc Links

Old (should not be necessary anymore)

installing node-gyp

  • https://github.com/nodejs/node-gyp

  • node-gyp is a cross-platform command-line tool written in Node.js for compiling native addon modules for Node.js

  • npm install -g node-gyp
  • tested version v3.6.2

installing node-inspector

installing v8-profiler

You can’t perform that action at this time.