Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to run docker image? thx! #57

Closed
mingrui opened this issue Apr 29, 2018 · 12 comments

Comments

@mingrui
Copy link

commented Apr 29, 2018

Hi total noob here, spent an hour trying to figure out how to run this database.

I tried go, but could not build because of this error:

$ make configure
dep ensure
Solving failure: unable to deduce repository and source type for "code.cloudfoundry.org/bytefmt": unable to read metadata: go-import metadata not found
Makefile:17: recipe for target 'configure' failed
make: *** [configure] Error 1

I tried docker too:

$ docker run alpacamarkets/marketstore:v2.1.1
chown: /project: No such file or directory
go fmt ./...
go vet ./...
go test ./...
ok  	github.com/alpacahq/marketstore/SQLParser	3.473s
?   	github.com/alpacahq/marketstore/SQLParser/parser	[no test files]
ok  	github.com/alpacahq/marketstore/catalog	0.087s
?   	github.com/alpacahq/marketstore/cmd/marketstore	[no test files]
?   	github.com/alpacahq/marketstore/cmd/tools/integritycheck	[no test files]
?   	github.com/alpacahq/marketstore/cmd/tools/mkts	[no test files]
?   	github.com/alpacahq/marketstore/cmd/tools/mkts/csvreader	[no test files]
?   	github.com/alpacahq/marketstore/cmd/tools/walDebugger	[no test files]
?   	github.com/alpacahq/marketstore/contrib/candler	[no test files]
ok  	github.com/alpacahq/marketstore/contrib/candler/candlecandler	0.697s
ok  	github.com/alpacahq/marketstore/contrib/candler/tickcandler	0.095s
ok  	github.com/alpacahq/marketstore/contrib/gdaxfeeder	0.011s
?   	github.com/alpacahq/marketstore/contrib/ondiskagg	[no test files]
ok  	github.com/alpacahq/marketstore/contrib/ondiskagg/aggtrigger	0.100s
ok  	github.com/alpacahq/marketstore/contrib/ondiskagg/calendar	0.040s
ok  	github.com/alpacahq/marketstore/contrib/slait	0.045s
ok  	github.com/alpacahq/marketstore/executor	5.266s
ok  	github.com/alpacahq/marketstore/executor/buffile	0.027s
?   	github.com/alpacahq/marketstore/executor/readhint	[no test files]
ok  	github.com/alpacahq/marketstore/frontend	2.122s
?   	github.com/alpacahq/marketstore/frontend/client	[no test files]
ok  	github.com/alpacahq/marketstore/planner	0.070s
ok  	github.com/alpacahq/marketstore/plugins	1.575s
?   	github.com/alpacahq/marketstore/plugins/bgworker	[no test files]
ok  	github.com/alpacahq/marketstore/plugins/trigger	0.045s
?   	github.com/alpacahq/marketstore/uda	[no test files]
?   	github.com/alpacahq/marketstore/uda/avg	[no test files]
?   	github.com/alpacahq/marketstore/uda/count	[no test files]
?   	github.com/alpacahq/marketstore/uda/max	[no test files]
?   	github.com/alpacahq/marketstore/uda/min	[no test files]
ok  	github.com/alpacahq/marketstore/utils	0.072s
ok  	github.com/alpacahq/marketstore/utils/functions	0.059s
ok  	github.com/alpacahq/marketstore/utils/io	0.008s
?   	github.com/alpacahq/marketstore/utils/log	[no test files]
ok  	github.com/alpacahq/marketstore/utils/rpc/msgpack2	0.005s
?   	github.com/alpacahq/marketstore/utils/stats	[no test files]
?   	github.com/alpacahq/marketstore/utils/test	[no test files]

Is there documentation on how to run?
Thanks for your time!
I really want to try this database out, but so far it's so difficult to setup. (Mainly because I'm a noob at docker and go...)

@umitanuki

This comment has been minimized.

Copy link
Contributor

commented Apr 30, 2018

Thanks for trying it out. Turns out the docker image needs some improvement so you can run that way, but here is a short example to start with.

$ curl  https://raw.githubusercontent.com/alpacahq/marketstore/master/mkts.yml > /tmp/mkts.yml
docker run -v /tmp/mktsdb:/project/data/mktsdb -v /tmp/mkts.yml:/tmp/mkts.yml alpacamarkets/marketstore:v2.1.1 marketstore -config /tmp/mkts.yml

Then you will see something like this.

I0430 05:54:56.091770       1 log.go:14] Disabling "enable_last_known" feature until it is fixed...
I0430 05:54:56.092200       1 log.go:14] Initializing MarketStore...
I0430 05:54:56.092236       1 log.go:14] WAL Setup: initCatalog true, initWALCache true, backgroundSync true, WALBypass false:
I0430 05:54:56.092340       1 log.go:14] Root Directory: /project/data/mktsdb
I0430 05:54:56.097066       1 log.go:14] My WALFILE: WALFile.1525067696092950500.walfile
I0430 05:54:56.097104       1 log.go:14] Found a WALFILE: WALFile.1525067686432055600.walfile, entering replay...
I0430 05:54:56.100352       1 log.go:14] Beginning WAL Replay
I0430 05:54:56.100725       1 log.go:14] Partial Read
I0430 05:54:56.100746       1 log.go:14] Entering replay of TGData
I0430 05:54:56.100762       1 log.go:14] Replay of WAL file /project/data/mktsdb/WALFile.1525067686432055600.walfile finished
I0430 05:54:56.101506       1 log.go:14] Finished replay of TGData
I0430 05:54:56.109380       1 plugins.go:14] InitializeTriggers
I0430 05:54:56.110664       1 plugins.go:42] InitializeBgWorkers
I0430 05:54:56.110742       1 log.go:14] Launching rpc data server...
I0430 05:54:56.110800       1 log.go:14] Launching heartbeat service...
I0430 05:54:56.110822       1 log.go:14] Enabling Query Access...
I0430 05:54:56.110844       1 log.go:14] Launching tcp listener for all services...

From here, you can edit /tmp/mkts.yml for data ingestion and so on. Hope this help. In the mean time, we'll adjust Dockerfile to improve the usability. Again, thanks for the report!

@mingrui

This comment has been minimized.

Copy link
Author

commented May 8, 2018

thanks!

@mingrui mingrui closed this May 8, 2018

@umitanuki

This comment has been minimized.

Copy link
Contributor

commented May 8, 2018

A blog post for the tutorial is coming soon.

@umitanuki

This comment has been minimized.

Copy link
Contributor

commented May 8, 2018

https://hackernoon.com/how-to-setup-bitcoin-historical-price-data-for-algo-trading-in-five-minutes-485cd5a0b3ed "How to Setup Bitcoin Historical Price Data for Algo Trading in Five Minutes"

@umitanuki

This comment has been minimized.

Copy link
Contributor

commented May 8, 2018

I believe we still need to fix Dockerfile CMD so it doesn't run unit test by default

@globophobe

This comment has been minimized.

Copy link

commented Jun 25, 2018

Error Creating WAL File, so docker image doesn't run. Directories exist. Edited for brevity:

I0625 00:11:59.591689       1 log.go:14] Initializing MarketStore...
I0625 00:11:59.591811       1 log.go:14] WAL Setup: initCatalog true, initWALCache true, backgroundSync true, WALBypass false: 
I0625 00:11:59.591840       1 log.go:14] Root Directory: $PWD/data/mktsdb
E0625 00:11:59.592055       1 log.go:22] /usr/local/go/src/fmt/print.go:597:CreateFile/go/src/github.com/alpacahq/marketstore/executor/wal.go:87open $PWD/data/mktsdb/WALFile.1529885519591959000.walfile: no such file or directory: Error Creating WAL File
@umitanuki

This comment has been minimized.

Copy link
Contributor

commented Jun 25, 2018

Your configuration looks like a bit broken. It shouldn't say "$PWD" if it's correctly configured. Seems like you wrote it in the yml file? Try full path without $PWD.

@globophobe

This comment has been minimized.

Copy link

commented Jun 25, 2018

Thank you. Actually, I edited the full path out of my comment.

The command I tried was:

docker run -p 5993:5993 alpacamarkets/marketstore:v2.1.2

The essence of the error I got was:

no such file or directory: Error Creating WAL File

@umitanuki

This comment has been minimized.

Copy link
Contributor

commented Jun 25, 2018

What does your mkts.yml file look like?

@globophobe

This comment has been minimized.

Copy link

commented Jun 25, 2018

Do I need to make one? At first, I thought I'd try the docker file with the default mkts.yml, as per the readme...

docker run -p 5993:5993 alpacamarkets/marketstore:v2.1.2
chown: /project: No such file or directory
I0625 07:43:27.271635      11 log.go:14] Disabling "enable_last_known" feature until it is fixed...
I0625 07:43:27.272894      11 log.go:14] Initializing MarketStore...
I0625 07:43:27.273130      11 log.go:14] WAL Setup: initCatalog true, initWALCache true, backgroundSync true, WALBypass false: 
I0625 07:43:27.273439      11 log.go:14] Root Directory: /project/data/mktsdb
E0625 07:43:27.274048      11 log.go:22] /usr/local/go/src/fmt/print.go:597:CreateFile/go/src/github.com/alpacahq/marketstore/executor/wal.go:87open /project/data/mktsdb/WALFile.1529912607273962500.walfile: no such file or directory: Error Creating WAL File

Pardon, it fails here first:

chown: /project: No such file or directory

@globophobe

This comment has been minimized.

Copy link

commented Jun 25, 2018

I added sudo mkdir /project to entrypoint.sh and rebuilt the docker container. After that it seems to run.

@rocketbitz

This comment has been minimized.

Copy link
Contributor

commented Jun 25, 2018

You’re supposed to mount wherever you want your DB to /project with -v in your docker command... you don’t need to edit the file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.