several commands to help automating miner operation #65
Conversation
| @@ -35,7 +35,7 @@ require ( | |||
| github.com/stretchr/objx v0.2.0 // indirect | |||
| github.com/stretchr/testify v1.7.0 | |||
| github.com/syndtr/goleveldb v1.0.1-0.20210305035536-64b5b1c73954 // indirect | |||
| github.com/textileio/cli v1.0.0 | |||
| github.com/textileio/cli v1.0.2 | |||
There was a problem hiding this comment.
to use the log-plaintext flag
| @@ -49,6 +50,8 @@ func createMux(service Service) *http.ServeMux { | |||
| mux := http.NewServeMux() | |||
| mux.HandleFunc("/health", getOnly(healthHandler)) | |||
| mux.HandleFunc("/id", getOnly(idHandler(service))) | |||
| mux.HandleFunc("/pause", putOnly(pauseHandler(service))) | |||
| mux.HandleFunc("/resume", putOnly(resumeHandler(service))) | |||
There was a problem hiding this comment.
figured http PUT is the correct verb.
| _ = v.ReadInConfig() | ||
| err := v.ReadInConfig() | ||
| if err != nil { | ||
| log.Errorf("loading config from %s: %v", v.ConfigFileUsed(), err) |
There was a problem hiding this comment.
print error when config file format is invalid as a result of manual editing. currently the log message is very confusing, something like 2021/10/13 21:14:39 --storage-provider-id is required. See ‘bidbot help init’ for instructions.
don't use cli.CheckErrf because we need to allow bidbot init to run, at which point the config file doesn't exist.
| Restart=on-failure | ||
| RestartSec=10 | ||
|
|
||
| ExecStart=%s daemon --log-plaintext |
There was a problem hiding this comment.
Even with this, syslog on my Ubuntu16 still replaces tab with #011 which makes the output messy, but at least we tried. maybe later versions don't have the problem?
There was a problem hiding this comment.
I'm not entirely sure about the problem, but found this. Dunno if worth fixing, just sharing here :)
There was a problem hiding this comment.
sorry I should've mentioned I checked this exact SO thread. Just don't think we should mess with rsyslog.conf.
| RestartSec=10 | ||
|
|
||
| ExecStart=%s daemon --log-plaintext | ||
| StandardOutput=journal |
There was a problem hiding this comment.
I tried using append:/var/log/bidbot.log, but Ubuntu16 doesn't recognize it - too old a systemctl version, so I resorted to this. Anyway, the user can configure syslog conf to divert bidbot log to a separate file if they want. Also this is meant to be a start point to which the miner can adjust by themselves.
There was a problem hiding this comment.
Yeah, totally agree. Miners are "operators" by nature, probably they're expert in systemd configs and will tune it to fit their architecture of logging and similars.
| _, err = user.Lookup(u) | ||
| cli.CheckErrf("looking up user: %v", err) | ||
| _, err = user.LookupGroup(g) | ||
| cli.CheckErrf("looking up group: %v", err) |
There was a problem hiding this comment.
make sure the user and group exists. thought about just use the current user, but feels too clever.
| @@ -571,20 +571,6 @@ func (s *Store) enqueueDataURI(txn ds.Txn, b *Bid) error { | |||
| func (s *Store) fetchWorker(num int) { | |||
| defer s.wg.Done() | |||
|
|
|||
| fail := func(b *Bid, err error) (status BidStatus) { | |||
There was a problem hiding this comment.
extracted methods to avoid fetchWorker becoming unmanageable.
Signed-off-by: Merlin Ran <merlinran@gmail.com>
Signed-off-by: Merlin Ran <merlinran@gmail.com>
merlinran
force-pushed
the
merlin/automation-commands
branch
from
573c853
to
0598609
Compare
main.go
Outdated
| if runtime.GOOS != "linux" { | ||
| log.Fatal("only available in Linux") | ||
| } |
There was a problem hiding this comment.
Maybe we should add that we're assuming systemd?
Is definitely the safest bet we can do, but I think other distros might not have it?
There was a problem hiding this comment.
good point. will add both here and the command description.
| _, err = user.Lookup(u) | ||
| cli.CheckErrf("looking up user: %v", err) | ||
| _, err = user.LookupGroup(g) | ||
| cli.CheckErrf("looking up group: %v", err) |
| Restart=on-failure | ||
| RestartSec=10 | ||
|
|
||
| ExecStart=%s daemon --log-plaintext |
There was a problem hiding this comment.
I'm not entirely sure about the problem, but found this. Dunno if worth fixing, just sharing here :)
| RestartSec=10 | ||
|
|
||
| ExecStart=%s daemon --log-plaintext | ||
| StandardOutput=journal |
There was a problem hiding this comment.
Yeah, totally agree. Miners are "operators" by nature, probably they're expert in systemd configs and will tune it to fit their architecture of logging and similars.
| Restart=on-failure | ||
| RestartSec=10 | ||
|
|
||
| ExecStart=%s daemon --log-plaintext |
There was a problem hiding this comment.
So bidbot should be on PATH on the executing user?
(Kind of an open question to see if that might be relevant to clarify somewhere)
There was a problem hiding this comment.
This will be the full path of bidbot extracted from os.Executable(), so no PATH is involved here.
There was a problem hiding this comment.
Oh right, is the template. Dismiss
| @@ -154,6 +155,7 @@ type Service struct { | |||
| lc lotusclient.LotusClient | |||
| store *bidstore.Store | |||
|
|
|||
| paused int32 // atomic. 1 == paused; 0 == not paused | |||
There was a problem hiding this comment.
You definitely like atomics :P
There was a problem hiding this comment.
bad habit formed when frequently need to have thousands of checks per second, making mutex too expensive. it saves a few lines of code too, so I'm kinda still like it.
| // AuctionsHandler implements MessageHandler. | ||
| func (s *Service) AuctionsHandler(from core.ID, a *pb.Auction) error { | ||
| if s.isPaused() { | ||
| log.Info("not bidding when bidbot is paused") |
There was a problem hiding this comment.
Nice, this will be useful for sure.
Signed-off-by: Merlin Ran <merlinran@gmail.com>
It does following:
pauseandresumecommand, to be used in scripts. Tested manually viamake upto make sure it doesn't bid when being paused.install-servicecommand, to add systemd service. Tested on a Ubuntu 16.04 VPS.