Running multiple times aggregation queries on the main branch eventually generate memory limit errors

[fmassot]

On the github archive dataset, if I run several aggregations, the first runs will succeed but after several attempts, it generates the following errors:

Exception: ('Error while querying', '{\n  "message": "Internal error: `(Internal error: `Aborting aggregation because memory limit was exceeded. Limit: 1.00 GB, Current: 1.00 GB`., split_id: 01H26SCHJ2AR36YAXATXMSW4FW), (Internal error: `Aborting aggregation because memory limit was exceeded. Limit: 1.00 GB, Current: 1.00 GB`., split_id: 01H26SKYQVE5SQV08XSVDCK3XY), (Internal error: `Aborting aggregation because memory limit was exceeded. Limit: 1.00 GB, Current: 1.00 GB`., split_id: 01H26SV71BZGPT45364VPYEPY8), (Internal error: `Aborting aggregation because memory limit was exceeded. Limit: 1.00 GB, Current: 1.00 GB`., split_id: 01H26T2GZT54S7H4Q05XHM0NPN), (Internal error: `Aborting aggregation because memory limit was exceeded. Limit: 1.00 GB, Current: 1.00 GB`., split_id: 01H26T9Z9QNSQBQVNB066ZBT7Q), (Internal error: `Aborting aggregation because memory limit was exceeded. Limit: 1.00 GB, Current: 1.00 GB`., split_id: 01H26THAEMY8JS08A6ZNDZSJDG)`."\n}')

If I restart, the requests first succeed and then fail again.

Here are the queries I run:

{"tags":["search"], "query":{"query":"payload.description:the", "sort_by_field": "-created_at"}}
{"tags":["search", "timestamp_filter"], "query":{"query":"payload.description:the", "start_timestamp": 1421276400, "end_timestamp": 1421708400}}
{"tags":["search", "union"], "query":{"query":"payload.description:the OR payload.comment.body:the OR payload.issue.body:the"}}
{"tags":["search", "union", "timestamp_filter"], "query":{"query":"payload.description:the OR payload.comment.body:the OR payload.issue.body:the", "start_timestamp": 1421276400, "end_timestamp": 1421708400}}
{"tags":["search"],  "query":{"query":"payload.description:javascript OR payload.comment.body:javascript OR payload.issue.body:javascript"}}
{"tags":["search", "union"], "query":{"query":"payload.description:javascript OR payload.comment.body:javascript OR payload.issue.body:javascript"}}
{"tags":["search"], "query":{"query":"payload.description:rust"}}
{"tags":["search", "intersection"], "query":{"query":"payload.issue.body:rust AND payload.issue.body:bug"}}
{"tags":["search", "intersection", "timstamp_filter"], "query":{"query":"payload.issue.body:rust AND payload.issue.body:bug", "start_timestamp": 1421276400, "end_timestamp": 1421708400}}
{"tags":["search", "union"], "query":{"query":"payload.issue.body:error OR payload.issue.body:bug"}}
{"tags":["search", "union"], "query":{"query":"payload.description:rust OR payload.comment.body:rust OR payload.issue.body:rust"}}
{"tags":["search", "union", "timestamp_filter"], "query":{"query":"payload.description:rust OR payload.comment.body:rust OR payload.issue.body:rust", "start_timestamp": 1421276400, "end_timestamp": 1421708400}}
{"tags": ["aggregation"], "query":{"query":"*","max_hits":0,"aggs":{"top_repositories":{"terms":{"size":10,"field":"repo.name","order":{"_count":"desc"}}}}}}
{"tags": ["search", "aggregation"], "query":{"query":"payload.description:error","max_hits":0,"aggs":{"top_repositories":{"terms":{"size":10,"field":"repo.name","order":{"_count":"desc"}}}}}}
{"tags": ["aggregation", "timestamp_filter"], "query":{"query":"*","max_hits":0,"aggs":{"top_repositories":{"terms":{"size":10,"field":"repo.name","order":{"_count":"desc"}}}}, "start_timestamp": 1421276400, "end_timestamp": 1421708400}}
{"tags": ["aggregation", "date_histogram"], "query":{"query":"*","max_hits":0,"aggs":{"events":{"date_histogram":{"field":"created_at","fixed_interval":"1d"},"aggs":{"top_repositories":{"terms":{"size":10,"field":"repo.name","order":{"_count":"desc"}}}}}}}}
{"tags": ["search", "aggregation", "date_histogram"], "query":{"query":"payload.description:error","max_hits":0,"aggs":{"events":{"date_histogram":{"field":"created_at","fixed_interval":"1d"},"aggs":{"top_repositories":{"terms":{"size":10,"field":"repo.name","order":{"_count":"desc"}}}}}}}}

[fulmicoton]

Dataset
https://quickwit-datasets-public.s3.amazonaws.com/benchmarks/gharchive.json.gz

version: 0.5
index_id: gharchive

doc_mapping:
  mode: dynamic
  dynamic_mapping:
    fast: true
  field_mappings:
    - name: created_at
      type: datetime
      precision: seconds
      fast: true
  timestamp_field: created_at

indexing_settings:
  split_num_docs_target: 2000000