The query time is very long, 7 seconds on average. How to optimize it

[tiankonghewo]

dgrapg version is 1.0.15
import stage:
rdf data size is 400G,
node memory size is 256GB
Originally I wanted to use the new version v1.1.0, but when importing data, the new version always reported an OOM error at map stage. So I had to use the old version 1.0.x

when run up, my cluster server configuration is:
3 nodes:16core64G
each data:170G,380G,440G

second, my query is very simple, use pydgraph==1.2.0,my dgraph version is 1.0.15
but the query time is very long, 7 seconds on average

query = """query all($a: string, $value: string) {
all(func: eq(type, $a))@filter(eq(value, $value)) {
uid
type
value
}
}"""
variables = {'$a': 'PERSON', '$value': value}
t0 = time.time()
res = client.txn(read_only=True).query(query, variables=variables)
ppl = json.loads(res.json)

Finally, I would like to ask, for TB data size,
What is the recommended cluster size and node configuration?
Is there any production case for reference?
If I need to add nodes, how many should I add? 3 or 6

[MichelDiz]

You might need more memory to do the first ingestion. 64GB It's too little for the dataset size.

Can you share the query? at least a similar one. And an idea of how much data would come from it. Also your cluster config context.

[tiankonghewo]

You might need more memory to do the first ingestion. 64GB It's too little for the dataset size.

Can you share the query? at least a similar one. And an idea of how much data would come from it. Also your cluster config context.

first, I have three node,each node memory is 64GB, So the total memory is 192GB
Is this enough memory?
second, my query is very simple, use pydgraph==1.2.0,my dgraph version is 1.0.15

query = """query all($a: string, $value: string) {
all(func: eq(type, $a))@filter(eq(value, $value)) {
uid
type
value
}
}"""
variables = {'$a': 'PERSON', '$value': value}
t0 = time.time()
res = client.txn(read_only=True).query(query, variables=variables)
ppl = json.loads(res.json)

Finally, I would like to ask, for TB data size,
What is the recommended cluster size and node configuration?
Is there any production case for reference?
If I need to add nodes, how many should I add? 3 or 6

[tiankonghewo]

You might need more memory to do the first ingestion. 64GB It's too little for the dataset size.

when import data, we use 256GB memory not 64GB, but dgraph V1.1.0 will OOM at map stage,
but v1.0.15 run normally

[MichelDiz]

Would you mind to rerun the test using a build from master?
You can just clone the repo, go to /dgraph/dgraph and run Go build.

Is your issue the same as this? #3918

Is see that you're talking about multiple issues. We should treat step by step.
The query thing may be related to the version. There are old issues with performance in old versions. Your query is very simple and not so wide. So I can't see why this is happening.

So, test with master and lets see. Logs of the bulkloading would also help.

UPDATE:
This also may be related #3017 #3095

[MichelDiz]

Closing, due related issues already exist and long time no interaction. If the issue persists, we can reopen it.

Cheers.