Replication does not work on 3 freshly started VMs with memgraph 2.15.0-1 #1774
Comments
gitbuda
added
community
|
I get a similar error on the replica, but the initial replication seemed to work: I have been using a setup with docker compose on one machine: version: "3"
services:
memgraph-main:
image: memgraph/memgraph-mage:1.15-memgraph-2.15
volumes:
- mg_lib:/var/lib/memgraph
- mg_log_main:/var/log/memgraph
- mg_etc_main:/etc/memgraph
container_name: memgraph-main
ports:
- "7687:7687"
- "7444:7444"
command: ["--replication-restore-state-on-startup=true"]
memgraph-public:
image: memgraph/memgraph-mage:1.15-memgraph-2.15
volumes:
- mg_log_public:/var/log/memgraph
- mg_etc_public:/etc/memgraph
container_name: memgraph-public
ports:
- "7689:7687"
- "7445:7444"
depends_on:
- memgraph-main
command: ["--replication-restore-state-on-startup=true"]
lab:
image: memgraph/lab:2.12.0
container_name: memgraph-lab
ports:
- "3000:3000"
depends_on:
- memgraph-main
environment:
- QUICK_CONNECT_MG_HOST=memgraph-main
- QUICK_CONNECT_MG_PORT=7687
volumes:
mg_lib:
mg_log_main:
mg_log_public:
mg_etc_main:
mg_etc_public:memgraph-main is the main node and memgraph-public was setup as REPLICA in SYNC mode. The error above has been showing up after a restart of the docker containers. EDIT: After the initial import and a restart, it is no longer possible to create nodes on the main. The error in the log reads: and the query response displayed is: |
|
Hi @nils-stefan-weiher, I managed to reproduce your issue, but it seems a bit different than reported above. I created a new issue from your comment, so you can track the progress there. Thank you for reporting this 🙏 |
|
After having read @bradacina's issue once again, I think these two are actually the same reports. @bradacina can you confirm if you ran the queries after you restarted instances? (replication is not working after restart) |
|
Following @bradacina and @nils-stefan-weiher comments, I managed to reproduce the issue. To Reproduce
Expected behavior - Replication continues working after the restart. Logs On MAIN: On REPLICA: |
katarinasupe
added
Frequency - EveryTime
|
@bradacina |
|
@andrejtonev That's why I thought these issues were not related (and they might not be; that is, solving one might not solve the other). Still, I reproduced the issue in in-memory transactional mode, just like @nils-stefan-weiher reported. |
|
@katarinasupe @nils-stefan-weiher |
|
@bradacina hi, question for you also: Since you are using Can you check for logs which can provide us with more info on whether there is a bug on our side: On MAIN: If |
|
@andrejtonev it seems that |
|
@katarinasupe |
|
@andrejtonev correct me if I am wrong but I checked the code and @katarinasupe Also, and additional problem is that we are still having in our codebase To conclude, |
Thanks for trying. I'm only back in office since Monday. Will try if this works in our case. |
|
Using the parameters and after deleting the container volumes I could enable replication (SYNC with one replica) again. I only have one question about the upgrade process. While upgrading the containers from 2.15.2 to 2.16.0 I had to REGISTER the replica again and set the correct ROLE. Is this because I don't have a persistent volume for the data in the /var/lib folder in the replica? |
Yeah, the persistency is in |
|
Related to #2061 |
antepusic
added
Effort - High
Memgraph version
memgraph/now 2.15.0-1 amd64
NOT USING ENTERPRISE LICENSE
Environment
3 Azure VMs, in the same subnet, Ubuntu 22.04.4, x86_64
One of the VMs is the MAIN, the other 2 VMs are Replicas
In my memgraph.conf for all 3 VMs I have:
--log-level=TRACE
--storage-mode=ON_DISK_TRANSACTIONAL
--replication-restore-state-on-startup=true
I have restarted the memgraph service on all 3 VMs
I have demoted 2 VMs to Replicas
I have registered the replicas on the MAIN VM
SHOW REPLICAS;displays the 2 registered replicasI have restarted the memgraph service on all 3 VMs AGAIN
Describe the bug
I've followed the https://memgraph.com/docs/configuration/replication#set-up-a-replication-cluster to setup a cluster but replication doesn't actually happen. Creating a node on the MAIN does not replicated the data on the Replicas.
To Reproduce
Steps to reproduce the behavior:
create (c:Customer {customerNumber:1}) return c;match (c:Customer) return c;Expected behavior
The node should be replicated on the Replica
Logs
On MAIN:
On Replica:
Verification Environment
Once we fix it, what do you need to verify the fix?
Do you need:
Thank you for your work/time!
The text was updated successfully, but these errors were encountered: