perf: create DocumentArray index map lazily

[alaeddine-13]

closes https://github.com/jina-ai/internal-tasks/issues/279

results:

from jina import Document
from jina.logging.profile import TimeContext

doc = Document()
with TimeContext('appending chunks'):
    for _ in range(10000):
        doc.chunks.append(
            Document(
                text='hey',
                offset=0,
                weight=1.0 ,
                location=[1, 2],
            )
        )

on master:

appending chunks ...	appending chunks takes 31 seconds (31.40s)

on feature branch:

appending chunks ...	appending chunks takes 0 seconds (0.64s)

[codecov]

Codecov Report

Merging #3944 (c3ee85b) into master (3957743) will increase coverage by 0.04%.
The diff coverage is 96.15%.

@@            Coverage Diff             @@
##           master    #3944      +/-   ##
==========================================
+ Coverage   89.04%   89.08%   +0.04%     
==========================================
  Files         180      180              
  Lines       12629    12642      +13     
==========================================
+ Hits        11245    11262      +17     
+ Misses       1384     1380       -4     

Flag	Coverage Δ
daemon	43.41% <15.38%> (-0.03%)	⬇️
jina	87.32% <96.15%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
jina/types/arrays/document.py	92.48% <94.73%> (-0.27%)	⬇️
jina/types/document/graph.py	92.45% <100.00%> (ø)
jina/math/evaluation.py	87.50% <0.00%> (-5.36%)	⬇️
jina/peapods/zmq/__init__.py	89.06% <0.00%> (-1.40%)	⬇️
jina/peapods/runtimes/zmq/zed.py	91.32% <0.00%> (-0.46%)	⬇️
jina/helper.py	83.27% <0.00%> (+0.17%)	⬆️
jina/peapods/peas/__init__.py	87.23% <0.00%> (+2.12%)	⬆️
jina/types/arrays/mixins/evaluation.py	82.35% <0.00%> (+2.35%)	⬆️
jina/peapods/pods/compound.py	90.05% <0.00%> (+3.31%)	⬆️
jina/peapods/stream/client.py	90.90% <0.00%> (+12.12%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b4f6d5c...c3ee85b. Read the comment docs.

[github-actions]

Latency summary

Current PR yields:

• 🐎🐎🐎🐎 index QPS at 1467, delta to last 2 avg.: +26%
• 🐎🐎🐎🐎 query QPS at 65, delta to last 2 avg.: +26%
• 🐎🐎🐎🐎 dam extend QPS at 42484, delta to last 2 avg.: +31%
• 🐎🐎🐎🐎 avg flow time within 1.4928 seconds, delta to last 2 avg.: +0%
• 🐢🐢 import jina within 0.387 seconds, delta to last 2 avg.: -17%

Breakdown

Version	Index QPS	Query QPS	DAM Extend QPS	Avg Flow Time (s)	Import Time (s)
current	1467	65	42484	1.4928	0.387
2.4.5	1224	53	33468	1.5396	0.4513
2.4.4	1094	48	31193	1.4183	0.4912

Backed by latency-tracking. Further commits will update this comment.

[JoanFM]

make it provate

[alaeddine-13]

it's used in jina/types/document/graph.py so it can't be private (IDE used to show a warning)

[hanxiao]

lets still make it private ignore graphDoc for now. graphDoc has been removed from our docs and will highly likely to be deprecated in 3.0

[alaeddine-13]

changed

[bwanglzu]

LGTM! i also think we can safely ignore graph document at the moment

[JoanFM]

Add a test in the types than make sure that a DocumentArray has _id_to_index to None at the beginning?

[alaeddine-13]

Add a test in the types than make sure that a DocumentArray has _id_to_index to None at the beginning?

added

[JoanFM]

I would change the test to show that the _index_map is not None after accessing by id

[alaeddine-13]

done

[JoanFM]

LGTM! Great job!