outliner folder gets renamed and contents replaced when dragged over other folder, only in upward direction #719
Comments
bartart3d
changed the title
|
I just reïnstalled 0.10.0-1, and there it happens too. |
|
Did you expand the destination folder and look inside? For example when I drag the Chapter 5 folder on top of the Chapter 1 folder then the Chapter 5 folder becomes a child of the Chapter 1 folder.
When I tried the steps you mentioned, the screen shot above is what occurred. Hence I am unable to reproduce the issue you described. |
|
Hi, |
|
I also tried dragging a lower folder and dropping on top of a higher folder with an existing project and was unable to reproduce the error you experienced. If you are able to determine the steps with a new project that cause the issue then that would go a long way towards identifying the problem. |
|
I'm starting to think it is because of a slow, or bad disk. I already mentioned in earlier reports the slow rendering of a folder full of text files as one long text. It could take minutes to appear on the screen. I've been looking inside the manuskript file at what happens when dragging folders and moving them around, and if I understand it correctly, files and folders are actually being renamed or copied, or overwritten, to change the order since there is no database or xml tree involved in the actual document. I only saw an xml file for the plots. I have copied to file to another drive, and tested dragging the folders around, and nothing went wrong. This does sound like a bad drive to me, but I can't be really sure of course. |
|
I have this issue again today, not with folders but with files. Yesterday, while writing I decided to move a series of scenes into a new chapter (folder), so I created the new folder and dragged the scenes into it. Then I read everything through and saved the file. Today I reopen the file and I notice that the files back in the original chapter (folder), but not in the right order. In the new chapter (folder), only a part of the files I yesterday have moved are also present (so they've been duplicated, present in both folders). These are in the right order. Trying to fix things, I dragged one of the files in the original folder up the scene tree to fix the ordering, which caused another file to be renamed, now having two of the same text files, renamed and contents overwritten. Again, when dragging down the tree, nothing went wrong. In a previous comment I thought it was because of a faulty disk, but since I'm now working on an entirely different machine, it is very unlikely. I'm certain there is something fundamental going wrong in the saving process. It certainly is not secure, that is obvious to me. the manuskript file is not saved as a single file, revisions are disabled and autosave is luckily disabled too. I'm using 0.11.0 now, on Ubuntu. I absolutely enjoy this piece of software, but it the project I'm working on is way to valuable for me. I'll move to something else for the time being, but I'll keep the manuskript file for testing purposes if anyone asks me tot try something out to get it finally fixed. |
|
Because it's sometimes difficult to describe issues in words (even when learning to write a novel ;-) ), I thought about gif's :-). When dragging down the tree, nothing goes wrong: |
|
I had the same thing, with a new file I added and then moved up. |
|
I see you are a contributor (and Dutch - 👋). Is there anything you can do yourself to repair this? |
|
I'm hardly a contributor in terms of coding. I did work on some translation to Dutch though. I have coded in the past, in php and I wish I had the time for it, but I have too much to write to code :-). |
|
After a new version being released, 0.12.0 I thought of trying it out again, but the issue remains. Good news though! I think I found a way to reproduce the problem for others to check! Maybe finally a solution can be found! Step by step guide on how I encountered the issue on a new file. I had started writing a new project in Joplin, so I used that text to copy paste. Of course, choose you own text:
I hope others can reproduce it this way |
|
@bartart3d Thanks for the detailed description. I will try to reproduce but I already have an idea how the code could look like causing the issue. So this can probably be solved. |
|
Hmm... nevermind I can't reproduce the issue. At the last point I just get Scene 1_2 before Scene 1 and the text in both scenes is different and as expected from the regarding split. |
|
@bartart3d Which operating system do you use? Maybe it's related to the filesystem or something... do you use the single-file mode for a project? |
|
Hey,
This is a really weird and annoying bug. Now I´m running ubuntustudio
20.10, filesystem is ext4. It happens both in single file mode and without
single file mode.
This is going on for years now and happens on both of my computers, both
linux. So weird it´s so hard to reproduce by others while it happens here
all the time on two computers.
Op di 4 mei 2021 15:40 schreef Tobias Frisch ***@***.***>:
… @bartart3d <https://github.com/bartart3d> Which operating system do you
use? Maybe it's related to the filesystem or something... do you use the
single-file mode for a project?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#719 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAVYKTW25J2LUDJOQMPYCZLTL72LNANCNFSM4KLEG3QA>
.
|
Mhmm, okay I am using Archlinux with ext4 on my system. I'll try a VM to reproduce, maybe it's caused by some inconsistent versions of dependencies since Arch uses newest stable releases and Ubuntu does usually wait longer to update these. |
|
I am on Ubuntu 20.04 - and the corresponding older version of Manuskript - still 0.10.0. Here it happens too, but not consistently. But then I haven't been experimenting with it. |
|
Okay, so I tried to reproduce it in UbuntuStudio 20.04 now because I didn't find a download for 20.10 on the website. But I still can't reproduce the issue with 0.10.0 which can be installed from repository. Actually I wanted to reproduce the issue with 0.12.0 but I can't run manuskript from source because I get an unhandled exception from Qt somehow: It's very weird that the package from repository just works though. Also I can install and run the build .deb package from 0.12.0 release and don't get an issue while startup. I can not reproduce this issue here as well. So I'm really clueless what's the actual problem. I recommend updating your system overall maybe it will fix this issue but I'm unsure (I assume you already tried that?). |
|
Hi, It was the reason why I strongly recommended to disable automatic saving by default, and I see it's enabled by default again This bug, combined with automatic saving caused me to loose al of data in the past, which was for me a reason to abandon Manuskript altogether. Again, I strongly advise to disable automatic saving because of this bug. Speaking of which, single file mode was enabled by default, I had to disable it when the new file was created. I guess you asked about it because it is thought to cause problems, why is it on by default then? Shouldn't it be safer to be off? People can't disable it without a file open. I read this latest version has a new logging system, can it be used to track the cause of a problem like this? grtz, |
The single-file-mode is mostly for compatibility to older files, I think. Storing a project in multiple files is much better for versioning, editing with other tools, arranging your project, getting an overview and it's less probable to get issues though to no need to use any compression or extraction. I am currently working on refactoring the whole IO system to make loading faster, safer and reduce the codebase as well. I will probably also change the default to opt in for single-file-mode. Also I plan to practically not use the single-file-mode at all (the new system will just extract all files from the one of the project to a new temporary directory... everything else will act the same way as using multiple files in the background but saving will still overwrite the one archive).
You can get to your log file by pressing Shift+F1 or via the Help-menu. After closing Manuskript the log-file should be complete and consist all logged commands but it's probably still a problem that many functions in the application don't get logged. Once to not flood the system with log-files and second because it's rather new functionality. So maybe something can be read from those but I can't promise anything. It's definitely better to post your log if the issue appears as long as I can't reproduce it. I mean technically the inconsistency is a hint how it could get caused. I fixed some multi-threading related bugs in the past which only appear randomly. I suspect this here is something similar or maybe related to the auto-saving or another timed functionality... so whenever the timing fits, the issue becomes possible. I just can only speculate which function is the problem. ^^' |
|
Hey, the log is still present from the test I ran for my first post today. Getting too tired to see if it shows where it goes wrong, I pinpointed the actions I described though, copy pasting them here in case someone here can see what goes wrong. |
|
Okay, I think the error is caught on the log because "Scene 1_2 has been renamed" appears two times in a row which marks and re-marks the same scene for moving. I assume this causes the removal of "0-Scene-1.md" in the end which is the critical data loss. In comparison this is a log from me trying to reproduce but not getting an issue: So we probably have to adjust Manuskript that it either cancels renaming if it causes overwrite or that it cancels the first renaming if a file gets renamed twice in a row. From looking at the log this is pretty much inconsistent asynchronous IO handling which causes the issue... fortunately this is something I wanted to refactor anyway. So I think it makes more sense to get the whole refactoring done than trying to fix this issue inside of the old code. |
|
Hey,
that is great news! Now we're getting somewhere :-D !
Thank you for looking into it, I'm really looking forward to the refactored
version :) If you need someone for testing.. just shout!
grtz,
Bart
https://esmiltania.be
On Twitter <https://twitter.com/Bart_Issimo>
On Google+ <https://plus.google.com/u/0/b/116379400376517483499/>
Op di 4 mei 2021 om 20:22 schreef Tobias Frisch ***@***.***>:
… Okay, I think the error is caught on the log because "Scene 1_2 has been
renamed" appears two times in a row which marks and re-marks the same scene
for moving. I assume this causes the removal of "0-Scene-1.md" in the end
which is the critical data loss.
In comparison this is a log from me trying to reproduce but not getting an
issue:
2021-05-04 20:05:17,037 - manuskript.mainWindow - INFO - Project Bug.msk saved.
2021-05-04 20:05:53,734 - manuskript.load_save.version_1 - INFO - Saving to: folder
2021-05-04 20:05:53,735 - manuskript.load_save.version_1 - DEBUG - Scene 1_2 has been renamed (outline/00-Chapter_1/0-Scene_1.md → outline/00-Chapter_1/0-Scene_1-2.md)
2021-05-04 20:05:53,735 - manuskript.load_save.version_1 - DEBUG - → We mark for moving: outline/00-Chapter_1/0-Scene_1.md
2021-05-04 20:05:53,735 - manuskript.load_save.version_1 - DEBUG - Scene 1 has been renamed (outline/00-Chapter_1/0-Scene_1.md → outline/00-Chapter_1/1-Scene_1.md)
2021-05-04 20:05:53,735 - manuskript.load_save.version_1 - DEBUG - → We mark for moving: outline/00-Chapter_1/0-Scene_1.md
2021-05-04 20:05:53,739 - manuskript.load_save.version_1 - DEBUG - Saving to folder Bug
2021-05-04 20:05:53,739 - manuskript.load_save.version_1 - DEBUG - * Renaming/moving outline/00-Chapter_1/0-Scene_1.md to outline/00-Chapter_1/0-Scene_1-2.md
2021-05-04 20:05:53,739 - manuskript.load_save.version_1 - DEBUG - * Updating cache: outline/00-Chapter_1/0-Scene_1.md, outline/00-Chapter_1/0-Scene_1-2.md
2021-05-04 20:05:53,740 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/folder.txt (different)
2021-05-04 20:05:53,740 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/0-Scene_1-2.md (different)
2021-05-04 20:05:53,740 - manuskript.load_save.version_1 - DEBUG - * Writing file outline/00-Chapter_1/1-Scene_1.md (not in cache)
2021-05-04 20:05:53,742 - manuskript.mainWindow - INFO - Project Bug.msk saved.
So we probably have to adjust Manuskript that it either cancels renaming
if it causes overwrite or that it cancels the first renaming if a file gets
renamed twice in a row. From looking at the log this is pretty much
inconsistent asynchronous IO handling which causes the issue... fortunately
this is something I wanted to refactor anyway. So I think it makes more
sense to get the whole refactoring done than trying to fix this issue
inside of the old code.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#719 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAVYKTX5HHTLO34AATWNNUDTMA3MXANCNFSM4KLEG3QA>
.
|
|
I'm not sure if this would be helpful, but I'm experiencing the same issue on both 0.11 and 0.12, both set for single file saving. I'm running Manjaro 5.10, and the version of Manuskript was built from the AUR through pamac. I'm seeing it moving individual text pages, not just folders, but only in larger/older files. I couldn't recreate it with a new file. I'll include what looks like the relavent lines from the log, I can include more if requested. I moved a single text page around a few times until it happened, and it did seem to only occur when moving the page upwards in the tree.
Please let me know if I can help by providing any more information. I really enjoy using Manuskript so I am happy to help improve it for others any way I can. |
|
Hello, just to let know that this issue is not solved yet in 0.13.1! Any ideas on when it can be solved? It seems like quite a critical bug though.... I guess I'll have to stick with novelWriter, also a great creative writing application! |
|
For a possible work around this issue did not occur for me if I always drag chapters/scenes down never up. |
|
@gedakc : This is at most 'avoiding the bug to happen' and your suggestion is already suggested by myself in the title. I don't work with software that harms my data, period. |
|
@bartart3d, can you run manuskript from source, and if so could you check #1002? It might be a fix. |
|
@olivierkes, hey, I tried it as soon as I could :-) The issue is not solved though, as soon as I dragged up, the file got renamed and contents overwritten. |
|
Thanks for the feedback @bartart3d . Could you try to:
|
|
@olivierkes So, these were my steps now:
I looked up which files had which ID's, so I looked inside the folders. The ID's that had been fixed, were not those of the documents I dragged up. The fixed ID's were in documents of later chapters (folders), empty documents. |
|
Thanks @bartart3d. In my tests, I was able to reproduce the bug only on items with duplicate IDs. But clearly that's not the issue here. Could you send me your file to see if I can reproduce the bug with it on my end? |
|
@olivierkes : I've created a reduced version of the file, the problem is still happening here, consistently when I drag Scene 1_2 up, to above Scene 1 in Chapter 2. I tried the same thing in chapter 3, it happens there too when dragging Scene 3 up to above Scene 2. Those don't even contain any text. I zipped the project, because I did not use a single-file option, revisions should be disabled, just extract and you should have the project in the state as it is on my drive. |
|
@olivierkes : it has been a while since I sent the file in the comment above. Maybe you didn't get a notification because I had forgotten to mention you in it, changed that in an edit. |
|
I've now merged the PR to potentially fix it because I think it should address a huge portion. If there's still a problem, I will try to fix it. But I want to make a new release in the next week which will hopefully get rid of the issue since we have multiple people reporting issues with ID conflicts. |
|
@TheJackiMonster : Thanks for merging, but the issue I reported does not seem fixed here... too bad. I installed the newest version, 0.14.0 from .deb, having python 3.9.7, PyQT 5.15.4 Qt 5.15.2 . Using the zipped document in attachment here , Scene 1 in Chapter 2 ends up being renamed as Scene 1_2 when dragged up to before Scene 1, and it's contents is also replaced. If fixed, this would not be the case. I'd suggest to reopen the report. |
|
Hmm, that's unfortunate. My thinking was that the partial fix would at least help with multiple issues and I currently don't have enough time to check into it myself. I think we still need to replace the back-end in a way that we can be sure at all times there's no state with ID collisions. Also manual changes in files from outside the application should be handled properly and we probably need a versioning/revision tool for that like GIT or SVN. I don't think it makes sense to work-around single issues when the whole data storage is still not as reliable as it should be and we don't even have the option to process partial changes in files to gain better performance. Anyway the issue is reopened then. |
Hi all,
I've got a weird, but serious issue with the latest manuskript edition, 0.11, but I can't reproduce it with a newly created document.
In the outliner, when I drag a folder up, so, more to the beginning of the document tree, the first folder it is being dragged over ends up being renamed to the same name as the folder that I am dragging. The contents is being replaced too. Wen dragging a folder down the document tree, further away from the beginning, this does not happen. Perhaps this info can pin down the cause of it.
First I had automatic saving on when exiting. Luckily I had a very recent backup. From now on I uncheck 'save on project close' for every new file I make. Over the years I have encountered way too many bugs causing data loss. I strongly suggest this to become the default setting for the time being.
The text was updated successfully, but these errors were encountered: