New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source file system gets modified | OP: UNCONFIRMED #27

Closed
cwynd opened this Issue Apr 13, 2017 · 29 comments

Comments

Projects
None yet
3 participants
@cwynd

cwynd commented Apr 13, 2017

Hi, I really appreciate cronopete, and use it to archive about 1.2T across 3 disks.

Most of the time this works fine, but occasionally it appears to get "confused" and write fragments of the source disk file system back on to the source disk. This may include illegal characters leading to non-printing characters in file/folder names. This has not resulted in fs corruption yet so far as I can tell, but requires careful manual cleanup, and obviously makes me concerned about the veracity of the backup images created.

Has anyone else experienced this? Any suggestions to investigate further?

Thanks

OS: debian 8.2 / Xfce
cronopete: 3.21.0

Edit: typo

@rastersoft

This comment has been minimized.

Owner

rastersoft commented Apr 13, 2017

That's odd... Can you put an example?

@cwynd

This comment has been minimized.

cwynd commented Apr 13, 2017

I don't have one currently (they're all cleaned up), but as soon as I have a new one, yes certainly I will post details here, thanks!

May be a day or two - or of course since we're now watching it could be...

@cwynd

This comment has been minimized.

cwynd commented Apr 19, 2017

No news - just confirming that I am watching for an "incident". Will report back as soon as I have something.

@cwynd cwynd changed the title from Source file system gets modified to Source file system gets modified | OP: UNCONFIRMED Apr 27, 2017

@cwynd

This comment has been minimized.

cwynd commented Apr 27, 2017

Hello All, despite watching diligently for over a week now, I have seen no more instances of this behaviour, so I am unable to confirm.

Since if this was confirmed it would raise a potentially serious issue, I have marked the subject line correspondingly. I will continue to watch, and update if necessary. Apologies for the distraction.

cw

@sebmate

This comment has been minimized.

sebmate commented May 1, 2017

Hi!

I think I also came across this issue. Today, I noticed a strange folder in my home directory, which I'm backing up with Cronopete. I was unable to delete it, neither in KDE nor the Terminal, but was finally lucky with Midnight Commander. The folder had no contents.

Please find attached a screenshot, taken from one of the target directories of Cronopete (it was backuped again). Note that it was created at 5:21 am, a time when I was asleep. ;-) Also note that there was a backup created at 3:21 am (right side), which makes me believe that this directory was created by Cronopete.

Anyway, keep up the good work - Cronopete is GREAT!

Sebastian

screenshot_20170501_155802

@cwynd

This comment has been minimized.

cwynd commented May 1, 2017

Yes, this is similar to what I saw. The first time I saw it, I immediately rebooted and did a fsck, suspecting the fs was corrupted (was fine). I have always been able to delete either from thunar (file explorer in xfce) or from bash.

But yes, this is what I saw..

cw

@sebmate

This comment has been minimized.

sebmate commented May 1, 2017

And another one I've just noticed: There's also a "home/sebmate/h" directory structure (in /home/sebmate) without any contents, created on April 27, 0:40 (see the "home" folder on the screenshot). I can't remember creating this, but I can't accredit this to Conopete for sure either.

cw, thanks for your comment. I also ran fsck, but the file system was probably fine (did not see the result, it was done behind the nice graphical Kubuntu boot screen), and the strange directory was still there.

I've switched Cronopete off for the moment. I think that this needs further investigation.

My system: Kubuntu 17.04 with pre-compiled Cronopete installed.

Sebastian

@cwynd

This comment has been minimized.

cwynd commented May 1, 2017

My install is the debian jessie package: cronopete 3.21.0 amd64 (just for the record)

cw

@rastersoft

This comment has been minimized.

Owner

rastersoft commented May 1, 2017

Can you check if the folders are created after a "restore" operation?

Now that I see the kind of folder name (thanks sebmate), I remember to also have that once in my system about two years ago, but never again. Also remember to check the source code to see if there was a bug in cronopete, but was unable to find any line where that folder could be created.

@cwynd

This comment has been minimized.

cwynd commented May 1, 2017

I have only ever used Cronopete to (very gratefully) manually restore individual CAD files that I accidentally messed up - I have never used the restore UI. When I saw folders created was after a backup operation.

In case it helps any, I have in the past seen cases where the extraneous folder name appears to be literally a substring of an existing source folder or file name - e.g. sebmate's "h" folder may be an instance of a truncated "home" - and I have seen longer than one-character fragments as well.

One other piece of info - I backup to an internal drive, not automounted in fstab, since if it automounts, Cronopete does not (I think?) recognize it as a suitable target drive.

cw

@sebmate

This comment has been minimized.

sebmate commented May 1, 2017

So far I did not use the "restore" function. The timestamps indicate that the strange folders (has anyone also seen files being created?) are created during the backup.

cwynd, I'm not sure if this are simply substrings of already existing file paths. My example (with the messed-up file name) indicates that the names come from binary data. But the other one supports your idea.

I think this could come from an array overflow / memory leak, where the program starts to interpret binary memory content as file names and then creates these files/directories. I'm just speculating/guessing, because I have no idea how Cronopete works internally. I would have to look at the source code. Maybe it's also related to Vala.

Sebastian

@cwynd

This comment has been minimized.

cwynd commented May 1, 2017

sebmate, that's a really good thought - it prompted me to check dates, and about 1 month ago, I rearranged my disks - previously I had an almost full 1TB /home/user/ on a separate drive, and so I added another 1TB and moved quite a bit of content over there, and linked it from the original /home/user/X location. I have not seen an 'incident' since. It's likely unrelated, but I wonder if maybe I stopped bumping in to the array overflow you guess at as a result, if it relates to a single physical device limit...

@sebmate

This comment has been minimized.

sebmate commented May 1, 2017

Well, I'm almost certain that it has nothing to do with the file system. I also don't think it's a problem in Sergio's program code, as I'm sure he's taking proper care when processing arrays and such. I would also guess that the Vala programming language would normally throw error messages if he wouldn't. But I have never programmed in Vala yet (I actually didn't know that it existed until I came across this tool).

If you do a search for "Vala memory leak" you can find many websites. But I didn't follow this further. I can imagine that Sergio's Vala compiler may have a bug that is causing this trouble (remember, we're using his binaries). Maybe just recompiling the source code would fix this issue, but at the moment I don't want to test this on my "production" machine :-/

@cwynd

This comment has been minimized.

cwynd commented May 16, 2017

Hello All, OP here, just saw my first 'incident' in quite a while. In bash I only see ????? as the foldername, whereas in the GUI browser (Thunar) I see:
���� (invalid encoding)
The folder is empty.

Attached are screenshots of the folder browser and also from bash:
invalid-folder-bash

invalid-folder-screen-snippet

.In case it hints at anything, I was moving several fairly large (1-2GB) files around via my local machine today, and deleting each as soon as I was done with it locally.

@rastersoft

This comment has been minimized.

Owner

rastersoft commented May 17, 2017

Mmmm... Maybe it has to do with deleting a file while it is being backuped...

I'll do some tests...

@cwynd

This comment has been minimized.

cwynd commented Jun 11, 2017

Hello All, another incident to report for the record - I am not aware of any large file moves going on this time. Screen snippet below:
folder-name-invalid-coding

@rastersoft

This comment has been minimized.

Owner

rastersoft commented Jun 16, 2017

Running cronopete with Valgrind I found that there is some kind of memory problem when calling g_file_copy (the GLib's call to copy a file). All the params are fine, so maybe this is a bug in GLib... I'll try to remove all GLib calls for file management and use only POSIX ones.

copiando: /home/raster/workspace/cronopete/src/ipc.vala
==5549== Thread 6 Backup thread:
==5549== Syscall param ioctl(generic) points to unaddressable byte(s)
==5549== at 0x8825E07: ioctl (syscall-template.S:84)
==5549== by 0x7B578C2: btrfs_reflink_with_progress (gfile.c:3012)
==5549== by 0x7B578C2: file_copy_fallback (gfile.c:3186)
==5549== by 0x7B578C2: g_file_copy (gfile.c:3394)
==5549== by 0x15F52C: usbhd_backend_real_copy_file (usbhd_backend.vala:503)
==5549== by 0x1174C1: backends_copy_file (backup.vala:48)
==5549== by 0x11BDFD: nanockup_copy_dir (backup.vala:534)
==5549== by 0x11A4F7: nanockup_do_backup (backup.vala:388)
==5549== by 0x12D231: cp_callback_do_backup (cronopete.vala:603)
==5549== by 0x12A543: _cp_callback_do_backup_gthread_func (cronopete.c:1183)
==5549== by 0x81633D4: g_thread_proxy (gthread.c:784)
==5549== by 0x8F0D493: start_thread (pthread_create.c:333)
==5549== by 0x882DAFE: clone (clone.S:97)
==5549== Address 0x10 is not stack'd, malloc'd or (recently) free'd
==5549==
Copiado

@cwynd

This comment has been minimized.

cwynd commented Jun 16, 2017

Well found! If you need a beta tester at some point just let me know (may be premature I know).

@rastersoft

This comment has been minimized.

Owner

rastersoft commented Jun 17, 2017

While doing the changes in another branch, found some code related to a make_directory_with_parents that, in some very odd cases can fail, so maybe there is the problem (maybe it calls make_directory_with_parents to an uninitialized object, so the piece with the filename contains garbage, and since it doesn't start with '/', it will be created in the user's folder). I fixed it and uploaded to the main branch. Before replacing all the GLib.File functions with Posix ones, I think these changes should be tested (just to avoid inserting new bugs).

@rastersoft

This comment has been minimized.

Owner

rastersoft commented Jun 17, 2017

There are new packages available in my homepage.

@cwynd

This comment has been minimized.

cwynd commented Jun 19, 2017

Now running the Debian 64 version of this new package, 3.26
There's a new dependency I had to install: libgsl2 (just to mention).

Thank you very much! Will report back to the thread as necessary.

cw

@rastersoft

This comment has been minimized.

Owner

rastersoft commented Jun 26, 2017

Ok, yesterday a folder with that mysterious coding was created in my system, so definitely my last bugfix didn't fix this. This will require a more in-deep revision.

@cwynd

This comment has been minimized.

cwynd commented Aug 31, 2017

Still getting these:
cronopete-glitch

cw

@rastersoft

This comment has been minimized.

Owner

rastersoft commented Mar 11, 2018

I'm doing a big refactorization work, and the new code will use "rsync", so these problems should go away with that version.

@cwynd

This comment has been minimized.

cwynd commented Mar 12, 2018

Sounds great! Happy to help with beta testing if you need it.

@rastersoft

This comment has been minimized.

Owner

rastersoft commented Mar 12, 2018

The current beta is in the "rsync" branch, in case you wan to test it. It works (in fact, I'm using it daily), but I still want to add more things.

@cwynd

This comment has been minimized.

cwynd commented Mar 15, 2018

Quick question about the "rsync" branch: is it backwards compatible? i.e. can I safely replace the current version with the rsync beta one, and point it at the same backup disk (which has all my cronopete history on it of course). Thanks!

@rastersoft

This comment has been minimized.

Owner

rastersoft commented Mar 15, 2018

Yes, it must be compatible. You should have no problems. But, of course, remember that it is still a development version, so in case you find a bug, please, report it ASAP.

@rastersoft

This comment has been minimized.

Owner

rastersoft commented Apr 27, 2018

Ok, I launched version 4.0, which uses RSYNC internally, so this bug should be gone. Closing.

@rastersoft rastersoft closed this Apr 27, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment