Lag when Waiting for Query Results

[jortiz16]

I noticed there being a lag of a few seconds right after executing a query on myria-web. For example for running a simple TwitterK query (which takes ~739ms to run), it takes about to 11 seconds to see the result on the Results panel. I also noticed that it seems to take longer on (@service.myria) than in (@demo.myria).

I'm going to guess this because there are more relations to query over in the catalog over (@service.myria) than in (@demo.myria)? Will this get worse over time in the future if more relations are added?

[domoritz]

I looked into it. Seems like somebody is ingesting huge amounts of data right now. Check htop on any of the machines.

[jhyrkas]

Not quite an ingest; Leila is processing the ngrams data right now, which means doing a lot of crunching in postgres.