tools: update block-generator to use conduit binary #5306
Conversation
| @@ -312,7 +312,9 @@ func (g *generator) WriteGenesis(output io.Writer) error { | |||
| FeeSink: g.feeSink.String(), | |||
| Timestamp: g.timestamp, | |||
| } | |||
| return json.NewEncoder(output).Encode(gen) | |||
|
|
|||
| _, err := output.Write(protocol.EncodeJSON(gen)) | |||
There was a problem hiding this comment.
genesis response needs to be msgp encoded.
| output.Write(block) | ||
| g.finishRound(numTxnForBlock) | ||
| return nil | ||
| } |
There was a problem hiding this comment.
postgresql exporter adds blocks starting from round 0.
| if err != nil { | ||
| return err | ||
| } | ||
| g.ledger.WaitForCommit(basics.Round(g.round)) | ||
| output.Write(block) |
There was a problem hiding this comment.
sdk requests the raw block.
| RunnerCmd.Flags().StringVarP(&runnerArgs.PostgresConnectionString, "postgres-connection-string", "c", "", "Postgres connection string.") | ||
| RunnerCmd.Flags().DurationVarP(&runnerArgs.RunDuration, "test-duration", "d", 5*time.Minute, "Duration to use for each scenario.") | ||
| RunnerCmd.Flags().StringVarP(&runnerArgs.ReportDirectory, "report-directory", "r", "", "Location to place test reports.") | ||
| RunnerCmd.Flags().StringVarP(&runnerArgs.LogLevel, "log-level", "l", "error", "LogLevel to use when starting Indexer. [error, warn, info, debug, trace]") | ||
| RunnerCmd.Flags().StringVarP(&runnerArgs.LogLevel, "log-level", "l", "ERROR", "LogLevel to use when starting Indexer. [PANIC, FATAL, ERROR, WARN, INFO, DEBUG, TRACE]") |
There was a problem hiding this comment.
updating these to match log-level supported by conduit.
There was a problem hiding this comment.
Does conduit fail if the casing is wrong?
There was a problem hiding this comment.
it looks like lower case also works. I'll revert this change.
| @@ -17,7 +17,7 @@ help() { | |||
| echo " -r|--report-dir directory where the report should be written." | |||
| echo " -d|--duration test duration." | |||
| echo " -l|--level log level to pass to Indexer." | |||
| echo " -g|--generator use a different indexer binary to run the generator." | |||
There was a problem hiding this comment.
Since it's no longer part of the indexer binary, this should be required. Mark as required on line 79
| --algod-net localhost:11111 \ | ||
| --algod-token security-is-our-number-one-priority \ | ||
| --metrics-mode VERBOSE \ | ||
| -P "host=localhost user=algorand password=algorand dbname=generator_db port=15432 sslmode=disable" |
There was a problem hiding this comment.
for this script to work for conduit, we would need to set up a configuration conduit config file. I think we can remove this script since run_runner and run_tests are running the same setup using runner.
There was a problem hiding this comment.
I'd be ok with deleting the other scripts too.
| // check /metrics endpoint is available before running the test | ||
| var resp *http.Response | ||
| for retry := 0; retry < 10; retry++ { | ||
| resp, err = http.Get(fmt.Sprintf("http://%s/metrics", metricsNet)) |
There was a problem hiding this comment.
I noticed that '/metrics' isn't always available by the time runTest starts. this causes block-generator to exit with error problem collecting metrics (1 / 3.003180017s): unable to read metrics url 'http://localhost:9999/metrics'.
I'm adding these lines to make sure /metrics is available before running tests.
|
In a followup PR could you do a search/replace for |
| // create config file in the right data directory | ||
| f, err := os.Create(path.Join(dataDir, "conduit.yml")) | ||
| if err != nil { | ||
| return fmt.Errorf("creating conduit.yml: %v", err) |
There was a problem hiding this comment.
| return fmt.Errorf("creating conduit.yml: %v", err) | |
| return fmt.Errorf("creating conduit.yml: %w", err) |
nit for consistency with other error wraps here
|
|
||
| err = t.Execute(f, conduitConfig) | ||
| if err != nil { | ||
| return fmt.Errorf("execute template file: %v", err) |
There was a problem hiding this comment.
| return fmt.Errorf("execute template file: %v", err) | |
| return fmt.Errorf("execute template file: %w", err) |
I'm doing this for the README. Stay tuned. |
#2869. This PR replaces the indexer binary with conduit binary in the runner. It also includes some encoding fixes in the generator.