A tick missed to fire. Node blocks too long

[AJAviJain]

If you suspect this could be a bug, follow the template.

• What version of Dgraph are you using?
  v1.0.7

• Have you tried reproducing the issue with latest release?
  Yes

• What is the hardware spec (RAM, OS)?
  16 GB windows

• Steps to reproduce the issue (command/config used to run Dgraph).
  shutdown server after uploading some data, and trying to restart the server again for the same data.

• Expected behavior and actual result.
  actual result >> A tick missed to fire. Node blocks too long!
  This warning is coming continuously.

Expecting a proper start of server with previous data uploaded.

I am using windows platform, so it generates 3 different folders for the server and zero >> P, W and ZW.

So i just want to take database backup without using export from browser. and the copying of folders is not working every time.

[MichelDiz]

You can use https://docs.dgraph.io/deploy/#export-database

Check this on the docs https://docs.dgraph.io/deploy/#upgrade-database

Thanks for taking an interest in Dgraph. It would be interesting to move questions to our channel in the Discuss or Slack. This makes Issue Tracker more focused on troubleshooting technical issues. Thank you for understanding.

https://discuss.dgraph.io/

Cheers

[manishrjain]

This issue indicates a bug in the system -- node blocks too long. @danielmai is looking to see if we can reproduce it at our end.

[danielmai]

shutdown server after uploading some data, and trying to restart the server again for the same data.

@AJAviJain, are you following the same steps as Deploy - Upgrade Database? That is, are you exporting the data, bringing down the cluster, and using bulk loader to restore the data to a new Dgraph instance?

[danielmai]

I'm not able to reproduce the "tick missed to fire" problem with the details given in this issue. I can try again when we have more details the exact steps to reproduce the issue.

I tested with the data set from the Getting Started docs or the 1million.rdf.gz data set.

Here's what I did with Dgraph version v1.0.7.

$ dgraph version

Dgraph version   : v1.0.7
Commit SHA-1     : f1803442
Commit timestamp : 2018-08-10 13:00:21 -0700
Branch           : HEAD

With the data set from the Getting Started docs:

1. Create a new ~/dgraph directory.
2. Start Dgraph Zero and Dgraph Server with ~/dgraph as working directory.
3. Run mutation to add the data and schema like the docs.
4. Shut down Dgraph Server (either Ctrl+C the Server process or curl the endpoint localhost:8080/admin/shutdown).
5. Run Dgraph Server again with same data directories. "A tick missed to fire" does not show up.
6. Force shut down Server (Ctrl+C consecutively three times).
7. Run Dgraph Server again with same data directories. "A tick missed to fire" does not show up.
8. Shut down Dgraph Zero and Dgraph Server processes.
9. Start Dgraph Zero and Dgraph Server with same data directories. "A tick missed to fire" does not show up.

With the 1million.rdf.gz movie data set:

1. Create a new ~/dgraph directory.
2. Start Dgraph Zero and Dgraph Server with ~/dgraph as working directory.
3. Either bulk load or live load the data set into Dgraph.
4. Shut down Dgraph Server (either Ctrl+C the Server process or curl the endpoint localhost:8080/admin/shutdown).
5. Run Dgraph Server again with same data directories. "A tick missed to fire" does not show up.
6. Force shut down Server (Ctrl+C consecutively three times).
7. Run Dgraph Server again with same data directories. "A tick missed to fire" does not show up.
8. Shut down Dgraph Zero and Dgraph Server processes.
9. Start Dgraph Zero and Dgraph Server with same data directories. "A tick missed to fire" does not show up.

[danielmai]

I didn't have any luck reproducing the problem with the 21 million movie data set either. Followed Steps 3-9 above after bulk loading the data.

[AJAviJain]

@danielmai
yes i have followed the tutorial given on https://docs.dgraph.io .

I have tried it by exporting even, but my requirement is a bit different so export thing won't work for me.

But it will work if i will simply take backup of folders, P and W but it sometime works and sometimes it doesn't. like on my recent work, i have uploaded large data, but after stopping the server i couldn't start with the same data.

I am using windows platform, and for windows version of dgraph, which has nothing but just two .exe files.

[AJAviJain]

@MichelDiz , thanks for this, but i have already followed that. This issue is a bit different.

[AJAviJain]

It makes up four files in the folder after i start the server.
.vlog, .sst, Lock and Manifest

I want to know the use of every file and exactly where my data gets stored. and can i take backup of my data from these files or folder. and why there are two folders for dgraph, P and W what exactly the structure is.

I am sorry, i haven't found these answers anywhere in the docs.

Until i understand the complete structure, i won't be able to get the reason of getting this error. and couldn't solve it.

[AJAviJain]

sometimes when it starts the server normally, it gives below error while trying to fetch the data

: Invalid value pointer offset: 1037 greater than current offset: 0

[MichelDiz]

In fact, what you are trying to reproduce is not supported. So I gave the example of "Export" mentioned in the documentation. The "Backup" feature is still being developed. That is, Backups are not officially supported. Therefore, Export is the only type of backup available.

What you are doing may work, but it is not recommended.

The errors "tick missed to fire", ": Invalid value pointer offset: 1037 greater than current offset: 0" may be happening just because you are reusing files generated by Dgraph in different versions or another situation.

As Backup feature is on the Roadmap see #1966 There is not much to do, just wait for it.

Cheers

[AJAviJain]

Simply stopping the server and starting again should not give error. that's what i am trying right now.

[danielmai]

Are you able to reproduce the issue consistently from a clean cluster setup?