Deploy preferences or config error

[umutucer]

I followed the steps outlined in the readme file but were unable to build, and I'm unsure if there are any missing preferences or configuration settings.

I try,
You need to have Maven and Docker installed.
1 Clone the repository git clone https://github.com/opexdev/core.git
2 Run cd core
3 Run mvn clean install command.
4 Run docker-compose up --build.
5 Run docker ps to see if every service is running.
result;
Tree container is exited; filebeat_1, elasticsearch_1,postgres-opex_1.

postgres error log:
Error: Database is uninitialized and superuser password is not specified.
You must specify POSTGRES_PASSWORD to a non-empty value for the
superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
connections without a password. This is not recommended.
See PostgreSQL documentation about "trust":
https://www.postgresql.org/docs/current/auth-trust.html

***but I am not change anything in docker-compose file so it has got environment:
- POSTGRES_USER=${DB_USER:-opex}

filebeat error log:
Exiting: error initializing publisher: missing field accessing 'output.elasticsearch.api_key' (source:'filebeat.yml')

***but again I am not change anything in docker-compose file so environment:
- FILEBEAT_API_KEY=$FILEBEAT_API_KEY
and filebeat.yml is not chaned,

output.elasticsearch:
hosts: [ "elasticsearch:9200" ]
protocol: https
api_key: ${FILEBEAT_API_KEY}

elasticsearch error log:
ERROR: Elasticsearch exited unexpectedly

I am not aware of the missing file, missing information, or API resources that I need to obtain from different components. I am not sure if I am receiving errors related to this, but I need your help.

[Marchosiax]

In case of postgres-opex_1, there is a bug in the docker-compose.yml configuration which causes this container to exit. But since this container is not being used by any of the services, It doesn't matter to the system whether this container is running or not. So just ignore this until we fix it. It won't affect the system.

For filebeat and elasticsearch, make sure you have set all of the environment variables that are listed below (you can create a .env file in the root directory of the project and copy all of these values there):

APP_NAME=Opex
APP_BASE_URL=http://localhost
PANEL_PASS=admin
BACKEND_USER=admin
SMTP_PASS=x
OPEX_ADMIN_KEYCLOAK_CLIENT_SECRET=x
API_KEY_CLIENT_SECRET=06ca1af7-5765-4e23-83cd-1bc04306d629
KEYCLOAK_FRONTEND_URL=http://localhost:8083/auth
KEYCLOAK_ADMIN_URL=http://localhost:8083/auth
KEYCLOAK_VERIFY_REDIRECT_URL=http://localhost:8080/verify
KEYCLOAK_FORGOT_REDIRECT_URL=http://localhost:8080/forgot
PREFERENCES=preferences-dev.yml
KIBANA_PUBLIC_URL=http://localhost:9090 
FILEBEAT_API_KEY=aaa

Setting KIBANA_PUBLIC_URL and FILEBEAT_API_KEY should resolve the issue

[umutucer]

Hi, thanks yours help. All containers are running but only filebeat container is restarting. Because file permission. Im not fix it because of Container is restarting. it is not accesseble.

Exiting: error loading config file: config file ("filebeat.yml") can only be writable by the owner but the permissions are "-rwxrwxrwx" (to fix the permissions use: 'chmod go-w /usr/share/filebeat/filebeat.yml')

[Marchosiax]

That looks like a problem related to docker and your host OS. Please refer to this link for a possible solution and let me know if you need any help

[Marchosiax]

By the way, it's not absolutely necessary to run filebeat, elastic, and kibana. The system works without these containers. They're just for monitoring and still under development

[umutucer]

Thank you Marchosiax. I checked all docker-compose file but I did not find exposed port. Example vault:9200, web-app 80 port. Can you share web-app and admin-panel packages env files?

[umutucer]

[Marchosiax]

Thank you Marchosiax. I checked all docker-compose file but I did not find exposed port. Example vault:9200, web-app 80 port. Can you share web-app and admin-panel packages env files?

For security reasons, we don't expose ports in configs. You can modify the docker-compose.yml and expose required ports

[Marchosiax]

Web app

REACT_APP_ENV: "development",
REACT_APP_API_BASE_URL: "http://localhost:3000",
REACT_APP_CLIENT_ID: "web-app",
REACT_APP_CLIENT_SECRET: "263d27a3-0ed7-4a81-b0cd-0973acafd878",
REACT_APP_TITLE: "OPEX",
REACT_APP_DESCRIPTION_CONTENT: "Open Source Exchange",
REACT_APP_LOGIN_CLIENT_ID: "admin-cli",
REACT_APP_MULTI_LANGS_SUPPORT : "TRUE",
REACT_APP_PRELOAD_LANG : "en",
REACT_APP_LANGS_SUPPORT : "[en]",
REACT_APP_DEFAULT_THEME : "DARK",
REACT_APP_SYSTEM_EMAIL_ADDRESS : "for.demo.purpose.only@opex.dev",
REACT_APP_REFERENCE_FIAT_CURRENCY : "TUSDT",
REACT_APP_CALENDAR_TYPE : "Jalali"

Admin panel

REACT_APP_DEFAULT_DIRECTION:"LTR",
REACT_APP_BRAND_NAME:"Opex",
REACT_APP_API_BASE_URL:"https://api-dev.opex.dev",
REACT_APP_SUPERSET_URL:"https://dashbrd-dev.opex.dev",
REACT_APP_FRONT_URL:'http://localhost:3001',
REACT_APP_CLIENT_ID:'web-app',
REACT_APP_CLIENT_SECRET:'d3d927e4-6ad6-4545-9f6f-0725a79b2d9e',
REACT_APP_CALENDAR_TYPE : "Jalali"

[umutucer]

Thank you peyman, js-api-client is necessary module ? How am I up this package? And again could you help me about js-api env file.

[umutucer]

Hi Peyman.
I have two question about run demo.
1- Could you help me about this error "POST http://localhost:8080/auth/realms/opex/protocol/openid-connect/token net::ERR_CONNECTION_REFUSED". Maybe ENV file parameter error, maybe wrong port number.

2- You said me about exposed ports;

For security reasons, we don't expose ports in configs. You can modify the docker-compose.yml and expose required ports

Can you tell me which ports required? Which container exposeable any port?

[umutucer]

Hi, I change "localhost" with my ip adress on env file. And this error change the diffrent error;
"Access to XMLHttpRequest at 'http://161.35.229.74:8080/wallet/admin/withdraw?offset=0&size=10' from origin 'http://161.35.229.74:81' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.". How am I ignore and run demo?

[umutucer]

web-app;

admin-panel

[Marchosiax]

Thank you peyman, js-api-client is necessary module ? How am I up this package? And again could you help me about js-api env file.

I'll mention front-end team here to answer this question:
@Hossein-ab99
@Benyamin001

[Marchosiax]

You are clearly having difficulty with CORS configuration. We've configured CORS with Nginx (which is available here btw), so every request goes through there and as far as I see, you're not running Nginx.

2- You said me about exposed ports;

For security reasons, we don't expose ports in configs. You can modify the docker-compose.yml and expose required ports

Can you tell me which ports required? Which container exposeable any port?

If you want to access each container individually on your localhost, you need to expose a port mapping to that container. here's an example docker-compose.local.yml:

version: '3.8'
services:
  vault:
    ports:
      - "8200:8200"
  consul:
    ports:
      - "8500:8500"
      - "8300:8300"
      - "8600:8600"
  postgres-accountant:
    ports:
      - "5432:5432"
  postgres-eventlog:
    ports:
      - "5433:5432"
  postgres-auth:
    ports:
      - "5434:5432"
  postgres-wallet:
    ports:
      - "5435:5432"
  postgres-api:
    ports:
      - "5436:5432"
  postgres-bc-gateway:
    ports:
      - "5437:5432"
  postgres-market:
    ports:
      - "5438:5432"
  accountant:
    ports:
      - "127.0.0.1:8089:8080"
      - "127.0.0.1:1045:5005"
  matching-engine:
    ports:
      - "127.0.0.1:8092:8080"
      - "127.0.0.1:1046:5005"
  matching-gateway:
    ports:
      - "127.0.0.1:8093:8080"
      - "127.0.0.1:1047:5005"
  auth:
    ports:
      - "127.0.0.1:8083:8080"
      - "127.0.0.1:1048:5005"
  wallet:
    ports:
      - "127.0.0.1:8091:8080"
      - "127.0.0.1:1049:5005"
  api:
    ports:
      - "127.0.0.1:8094:8080"
      - "127.0.0.1:1050:5005"
  websocket:
    ports:
      - "127.0.0.1:8097:8080"
      - "127.0.0.1:1054:5005"
  bc-gateway:
    ports:
      - "0.0.0.0:8095:8080"
      - "127.0.0.1:1052:5005"
  market:
    ports:
      - "127.0.0.1:8099:8080"
      - "127.0.0.1:1056:5005"

You can run with this configuration alongside the main docker-compose.yml file like below:

docker-compose -f docker-compose.yml -f docker-compose.local.yml up -d

[umutucer]

Thank you peyman. I will use to NGINX. Lastly, Could you send me .env file for proxy-manager.

[Marchosiax]

Thank you peyman. I will use to NGINX. Lastly, Could you send me .env file for proxy-manager.

EXPOSED_PORT=443
SERVER_NAME_DASHBOARD=dashbrd-dev.opex.dev
SERVER_NAME_ADMIN_PANEL=adm-dev.opex.dev
SERVER_NAME_WEB_APP=dev.opex.dev
SERVER_NAME_AUTH=auth-dev.opex.dev
SERVER_NAME_API=api-dev.opex.dev
SERVER_NAME_KIBANA=kibana-dev.opex.dev

Remember, you also need a opex.dev.crt and a private.pem file for SSL configuration. If you look here, we use these files for SSL config. You can change the file name if you want, as long as you change their names in docker-compose.yml file accordingly.

[Marchosiax]

If you wish to not have SSL, you need to modify this file and remove the part which is responsible for SSL configuration

[umutucer]

Ok. Thanks peyman. I will try.

[Marchosiax]

You should connect to the Nginx. For example, the value of SERVER_NAME_API which you've set for Nginx, correlates to REACT_APP_API_BASE_URL env in web-app. Just change url envs in web-app and admin-panel to connect to the Nginx.

[umutucer]

Peyman, thank you for your supports. Keycloak, Vault, Reverse proxy and finally admin-panel and web-app are okey. its time to chain-scanner. Lastly, Could you send me .env file for chain-scanner.

[umutucer]

Hi peyman. I'm getting an error "Captcha entered incorrectly!" when registering.

When I right-click / inspect / network,

I see that I can't access a path on Keycloak (user-management/user).

Can you help me with this?

[umutucer]

This is my captcha containers log:
at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
2023-01-25 18:14:16.994 DEBUG 7 --- [ single-1] PayloadBufferServerHttpResponseDecorator : 50a90e9a-0194-4048-9c98-9e46ddf03fee-response: : {"error":"BadRequest","code":1002,"message":"Bad request","timestamp":"2023-01-25T18:14:16.922+00:00"}
2023-01-25 18:14:16.996 INFO 7 --- [or-http-epoll-2] co.nilin.opex.utility.log.RequestFilter : 50a90e9a-0194-4048-9c98-9e46ddf03fee-Response Time：0.111 s

[Marchosiax]

Hi peyman. I'm getting an error "Captcha entered incorrectly!" when registering. When I right-click / inspect / network, I see that I can't access a path on Keycloak (user-management/user).

Can you help me with this?

That's a bug in captcha module which we are already aware of but still haven't had the time to fix it yet. As for keycloak, It looks like there's an error in the module. Could you provide the logs of keycloak?

[vanodevium]

@umutucer @Marchosiax

Guys, it looks like a very big drawback in the documentation.

Judging by how many questions and how few answers, it was very cool to collect together to the tutorial how to raise the entire system locally.
At a minimum, add env.example in the repository (with set up variables for local running), as well as a brief instruction to launch each microservice with already prepared docker-compose.local.yml (with already exposed ports or any required settings).

---

As for me I have problems with core, web-app and proxy-reverse too.
If this project is really open source and friendly to each other, it have to be really simple for starting and understanding.

[maryarm]

@vanodevium thanks for your interest in the project. We do agree that there is a huge lack of documentation but we are a small team and still have a bunch of issues to fix. Documentation is in the plan but it is not part of the roadmap of Q1/Q2. however we are Open Source and we are absolutely open to PRs and would warmly welcome every one who like to contribute, specially in documentation. :)
And for your current issues, please post your problems with proper description and logs and what ever that could help us understand it better and we'll be back to you.

[vanodevium]

@maryarm
Can I post my problems to this issue or it is better to create issues in every repo?

As opensourcer, I can help you and your team to collect info about friendly start and write short brief as readme part :)

[maryarm]

Please open a sperate one so it will be easier to trace and follow unless you feel that you have the same problems.

Thanks, it sounds Cool, we and the other users would appreciate it.

[vanodevium]

@umutucer
May I ask you to share your .env files for local start?

[maryarm]

@umutucer May I ask you to share your .env files for local start?

Could you please tell us the failing modules and Have you already tried with .envs that @Marchosiax has shared before?

[Marchosiax]

@umutucer May I ask you to share your .env files for local start?

Could you please tell us the failing modules and Have you already tried with .envs that @Marchosiax has shared before?

core:

APP_NAME=Opex
APP_BASE_URL=http://dev.opex.dev
PANEL_PASS=admin
BACKEND_USER=admin
SMTP_PASS=your-smtp-password
OPEX_ADMIN_KEYCLOAK_CLIENT_SECRET=get-opex-admin-client-secrent-in-keycloak
API_KEY_CLIENT_SECRET=get-api-key-client-secrent-in-keycloak
KEYCLOAK_FRONTEND_URL=http://localhost:8083/auth
KEYCLOAK_ADMIN_URL=http://localhost:8083/auth
KEYCLOAK_VERIFY_REDIRECT_URL=http://localhost:8080/verify
KEYCLOAK_FORGOT_REDIRECT_URL=http://localhost:8080/forgot
PREFERENCES=preferences-dev.yml
KIBANA_PUBLIC_URL=http://localhost:9090
FILEBEAT_API_KEY=x
TAG=debug

chain-scan:

BITCOIN_RPC_ENDPOINT=https://btc.getblock.io/mainnet
TEST_BITCOIN_RPC_ENDPOINT=https://btc.getblock.io/testnet
BITCOIN_REST_ENDPOINT=https://btc.getblock.io/mainnet/rest
TEST_BITCOIN_REST_ENDPOINT=https://btc.getblock.io/testnet/rest
BITCOIN_API_KEY=your-api-key
ETH_RPC_ENDPOINT=https://mainnet.infura.io/v3/your-api-key
TEST_ETH_RPC_ENDPOINT=https://ropsten.infura.io/v3/your-api-key
BSC_RPC_ENDPOINT=https://bsc.getblock.io/mainnet/?api_key=your-api-key
TEST_BSC_RPC_ENDPOINT=https://bsc.getblock.io/testnet/?api_key=your-api-key
TRON_REST_ENDPOINT=https://api.trongrid.io
TRON_API_KEY=your-api-key
TEST_BITCOIN_API_KEY=your-api-key
PANEL_PASS=1234
BACKEND_USER=admin
PREFERENCES=preferences.yml
PREFERENCES_IDENTIFIER=dev

web-app:

REACT_APP_ENV=development
REACT_APP_API_BASE_URL=https://nginx-host
REACT_APP_CLIENT_ID=web-app
REACT_APP_CLIENT_SECRET=get-web-app-client-secrent-in-keycloak
REACT_APP_TITLE=OPEX
REACT_APP_DESCRIPTION_CONTENT=Open Source Exchange
REACT_APP_LOGIN_CLIENT_ID=admin-cli
REACT_APP_MULTI_LANGS_SUPPORT=TRUE
REACT_APP_PRELOAD_LANG=en
REACT_APP_LANGS_SUPPORT=[en]
REACT_APP_DEFAULT_THEME=DARK
REACT_APP_SYSTEM_EMAIL_ADDRESS=youremailaddress@yourhost.com
REACT_APP_REFERENCE_FIAT_CURRENCY=TUSDT
REACT_APP_CALENDAR_TYPE =Jalali

[eliudrojas]

hi i would like to write some project documentation, but first i will need configure in order to run with default docker-compose files include them in the project

could you share with me the default env. values, default docker files values and so on in order to executed locally (using de default images and compiling locally, both of them) i would like to configure using default docker images and compiling locally and execute them

may be you could share with me your local environment by google drive in a zip file, in order to avoid waste time checking by my side all the possibles configuration errors

if you could help me with that, i could have initial documentation may be in two weeks , i would like document module by module

thanks in advance

[eliudrojas]

Hi

I would like to install opex locally without using docker images, but i can't find some source code in order to build my own image

do you have a documentation to create my own images?

thanks
e.g
ghcr.io/opexdev/postgres-opex
ghcr.io/opexdev/kafka
ghcr.io/opexdev/vault-opex
ghcr.io/opexdev/auth

[Marchosiax]

Hey @eliudrojas
We do not have documentation to create your own images. I don't recommend building locally without docker. It is possible you encounter unexpected problems.

Nonetheless, you can find configurations for all of the images in docker-compose files. For some custom images like the ones you mentioned, you'll find them in docker-images directory