Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing of new version 1.8.1 #623

Closed
dpradov opened this issue Nov 13, 2023 · 103 comments
Closed

Testing of new version 1.8.1 #623

dpradov opened this issue Nov 13, 2023 · 103 comments
Milestone

Comments

@dpradov
Copy link
Owner

dpradov commented Nov 13, 2023

Hello everyone
There is a new beta version, 1.8.1 Beta 1.


Update. New version: 1.8.1 Beta 6


It includes an important improvement in image management, and several other corrections.
Please, look at 'history.txt' for a list of changes. In 'Changes in 1.8.1 Beta1.txt' there is more information of each modificacion,
and a very detailed description of the new image management.

Please try it and tell me if you find any bugs or anything to improve.
Dont forget to make and conserve backup of your files, as always, in case you use it with your normal KNT files.

Once I can confirm that it does not have any major bugs, I will be able to release it as a new version.
I will generate new versions (Beta 2, ...) according to what you can find.

Regards
Daniel

history.txt
Changes in 1.8.1 Beta1.txt
Changes in 1.8.1 Beta2.txt
Changes in 1.8.1 Beta3.txt
Changes in 1.8.1 Beta4.txt
Changes in 1.8.1 Beta5.txt
Changes in 1.8.1 Beta6.txt

KeyNote_1.8.1 Beta6.zip

@dpradov dpradov added this to the 1.8.1 milestone Nov 13, 2023
@dpradov
Copy link
Owner Author

dpradov commented Nov 13, 2023

Some screenshots:

imagen

imagen

imagen

imagen

imagen

imagen

@mpradovelasco
Copy link

Thank you for this new version of KeyNote. I will begin to use now! I will notify you any issue that could appear.

@luc0x61
Copy link

luc0x61 commented Nov 14, 2023

Will you release a complete install package anytime soon...?

@dpradov
Copy link
Owner Author

dpradov commented Nov 14, 2023

Once these main changes have been made, the next thing I would want to do is, in fact, prepare a complete installation mechanism as well as include a proper help file.

@dpradov
Copy link
Owner Author

dpradov commented Nov 14, 2023

Note:
I have replaced the .zip file with another one, with the same name (also 1.8.1 Beta 1), in which I have corrected a problem that has been pointed out to me and that can occur on fast computers when cutting to the clipboard (an exception "Cannot open clipboard: Access Denied" could occasionally be raised). It is not problematic, the operation is performed ok, but it is annoying.
I've left it as 1.8.1 Beta 1, but you can see it in the About form, that the new version corresponds to today (14/nov/23)

imagen

@dpradov
Copy link
Owner Author

dpradov commented Nov 14, 2023

It is important to indicate that this new version 1.8.1 Beta 1 resolves one problem (described in history.txt as "* Improvement/correction to the RemoveKNTHIddenCharactersInRTF procedure")
that in some cases it has been able to cause the application to hang, perhaps in Find All.

It never happened to me, but as I have been told, it has happened, and it is surely due to the error that I corrected in that method, where indeed, in a certain case, it could get stuck.

@dpradov
Copy link
Owner Author

dpradov commented Nov 15, 2023

Today I have been using this latest version in a real situation, at work, taking notes in a training session. I have interspersed notes (text) with over 90 screenshots of several sizes and it has worked perfectly. I initially started with the usual mode (EmbeddedRTF), with a few images, then I changed to EmbeddedKNT and finally I continued with External mode, using a local folder as storage.
In this way, the KNT file that I have been saving has not exceeded 20Kb, and I think the images in the folder together add up to around 1.3 MB. Note that each time I save the file, only the .knt and the new added images are saved to disk.

One thing I've of course been doing, for safety, is making some backups from time to time. I want to point out that, in addition to the usual backup mechanism, it is also very easy to make copies using the File | Copy To... The copies generated in this way may or may not include the images depending on the following configurable option: Storage mode on Export: EmbeddedRTF, EmbeddedKNT, No export images.
In my case on this occasion I have used EmbeddedRTF. The file is generated and saved very quickly and its size is much smaller than usual since the screenshots have been previously converted to PNG or JPG. The size of the resulting file have been around 3 MB
(It is also possible to export all or some notes to KNT format using File | Export..., applying that same option)

I say the sizes off the top of my head, but they were in that order, very little considering that multiple screenshots captured parts of an application that showed a background photograph. But in those cases the application has automatically been selecting the JPG format, since although I have selected PNG in the "Default format from clipboard option", I also have a value <> 0 set in the option "Ratio Size" (1.5)

@dpradov
Copy link
Owner Author

dpradov commented Nov 15, 2023

Of course, there will surely be some issues that will arise and that I will have to correct, but for now I have a very good feeling. Until now I shied away from using images in my notes, due to the impractical file sizes, but now that has all changed.

@dpradov
Copy link
Owner Author

dpradov commented Nov 15, 2023

WARNING:
Doing some tests I saw something strange when running File | CopyTo... on a file with a note configured as plain text.
I'm reviewing it. I advise not using this version for now with this type of notes (plain text) until I see it clearly and solve it.
Of course, it can be used for testing.

@dpradov dpradov changed the title Testing of new version 1.8.1 Beta 1 Testing of new version 1.8.1 Nov 16, 2023
@dpradov
Copy link
Owner Author

dpradov commented Nov 16, 2023

I have uploaded a new version that corrects the problem I found yesterday. It could lead to exceptions on exporting (File | Copy To... or File | Export... with KNT format ) when including plaintext notes

@Stefanoko
Copy link

Stefanoko commented Nov 19, 2023

Excited about the new image handling. But cannot get it to work.
Beta2 on Win11.
Storage mode = external, folder.

When I drag a jpg into the content area of my multi-level rtf file, the insert dialog pops up. But all I get is a new blank paragraph. No link or image visible. A subfolder with the name of the knt-file is created. But no image ends up inside.
What am I missing here?

@dpradov
Copy link
Owner Author

dpradov commented Nov 19, 2023

While I guide you to see what may be happening, can you try other options? For example, try pasting a screenshot. And also try EmbeddedKNT mode.
I will help you what may be happening. I have tried W10 and W11, in many ways and I have not had any problems so far.

@dpradov
Copy link
Owner Author

dpradov commented Nov 19, 2023

Try also with different files and formats (gif, png, tif, other jpg, etc.). The idea: get more info to try to narrow down the problem.

@Stefanoko
Copy link

Trying to inch closer:
1st_Attempt.pdf

@Stefanoko
Copy link

More testing:
2nd_Attempt.pdf

@dpradov
Copy link
Owner Author

dpradov commented Nov 20, 2023

Hello @Stefanoko, I'm glad to hear from you and that you find what this new one offers interesting (another thing is the problems you may have encountered, which I'm going to look at now and respond to). I was surprised that there were hardly any comments on this version. This is a very big change for which I have spent (and continue to spend) many hours, as I have had to review many aspects of the application.
Personally, for me it is a before and after in the use of images with KNT. In previous versions I avoided its use due to the enormous size that the file could reach, but this is no longer the case.

@dpradov
Copy link
Owner Author

dpradov commented Nov 20, 2023

You began with these options:
imagen
but you should keep in mind that the first two are the default values for new files.
If we want to use all the possibilities of the new version we have to choose a storage format other than EmbeddedRTF, through the properties of the current file:
imagen
In already existing files the format is the same as always: EmbeddedRTF I call it here.

I'll surely have to take a look at the names, hints and layout of those options to make it clearer.

Having read your notes, I realize I have to save the knt-files to see images incorporated. This is what I will do: Press save after each step (without explicitly mentioning it anymore

The changes are not persisted until the file is saved (as has always been the case) and this means that, if the images are saved to external storage, the corresponding files will only be created at that time. But of course, as you work with the KNT file, all the images that are inserted must be viewed.
One can add and delete images, and only those that remain at the time of saving will be added. It is possible to delete an image, but if the changes are not saved, it is perfectly possible to perform Undo and recover it, apart from the fact that we can have several instances of the same image. And as long as we have at least one instance of an image it will not be deleted from its storage when saving.

I keep checking your comments

@dpradov
Copy link
Owner Author

dpradov commented Nov 20, 2023

  1. I drag a picture "Groundhog-Day.jpg" into the editor. Nothing happens
  2. I drag the same file once again, but this time — out of curiosity — I enable the link-option on the dialog ... And it does work

Do you drag and drop exactly the same way, from the same folder? It's strange

imagen

No, it isn't. And not happened to me. I'll check what can be happening.

My file "Groundhog-Day.jpg" was renamed to "5_Image_19Novjpg". Might be intended as well?

KNT requires the name of the files in the storage to be unique, and to get it, it gives a name that starts by the ID of the image. The rest of the name comes from the source (clipboard, clipcap, etc), but if the file has been inserted (File | Insert Picture...) or dropped from the explorer, it should be the name of that file. I will check it

imagen

The field offered in the KNT options is the one that will be used by default unless otherwise indicated. If the image is inserted using File | Insert Picture... then it will necessarily correspond to what is established there, since nothing can be indicated about it from that screen. But if you drag a file, what KNT does is pre-check the Linked box if the option is set, since it is the default option, the preferred one. Of course, this does not prevent you from changing the criteria when dragging specific images, you just have to choose the desired option.

@dpradov
Copy link
Owner Author

dpradov commented Nov 20, 2023

Config Options > Image tab
"Default Link Mode ('not owned')" was not clear to me right away. Especially since I do not plan on using image links, but image insertions, always visible. Imo, an easier caption would be something like:
Link to originals by default (no copies)

I'll keep it in mind. My native language is not English, and I defend myself as best I can. Surely my text was not the most accurate ;)

Especially since I do not plan on using image links, but image insertions, always visible.

All images can be shown or hidden at will and does not depend on whether they have been linked or not in the KNT file. The only implication of using linked files is that they are completely independent, and KNT does not incorporate them into the file's image storage, nor does it delete it if it is no longer used. Apart from that the treatment is identical to the rest.

What also threw me off: Upon selecting "External", I wondered where they get stored, but did not worry about it, assuming that I would be asked on first drag+drop, or that they would end up in a subfolder automatically. Which is the case for new knt-files, as it turns out. But when selecting "External" with an existing file, you have to perform an additional step on "File > Properties". With makes sense, ofc, but did not occur to me right away

This is what I indicated at the beginning, the storage to be used is a characteristic of the file that is established in 'File properties', in the same way that the file format (native, compressed or encrypted) is indicated there.
I'll put some clarification in the Images tab of the options screen to make it clear.
Once you enter File properties, it is clear, logically, where the images will be saved.

@dpradov
Copy link
Owner Author

dpradov commented Nov 20, 2023

File > Properties
If you do not activate "change" at this step, the image files generated will miss the dot-separator between file name and extension. A problem I encountered and described in my previous post "1st_Attempt".

The default path can be ok and in that case is normal not to click on Change. I hadn't realized the problem with the missing dot separator. It has to be something very simple that I will of course correct.

  1. Back on the top node:
    I delete the first image „dog_to_vet_2.jpg“ I save the file. It looks good at first sight, on both nodes, but when I come back to the top node, the deleted „dog_to_vet_2.jpg“ is there again, now sporting the dimension of the
    second image:

Not good.
Easy to spot with photographs. But when this happens with graphs and screenshots of manuals I prepare, where dimensions are always the same and there is little difference between step-by-step graphics, this may go unnoticed for a long time. Even with multiple backup-copies it will be hard to detect at what time something went wrong or even that the graphics show the wrong content at all

Of course, but we are talking about the beginning of thorough use of the new version. Surely there are some things that will show up soon and will be corrected.
In this case I see that the two images were completely glued to each other. How did you delete the first one, to play it? Did you select it and press DELETE, or did you delete it by pressing BACKSPACE?

What has happened here is that although the image has been deleted, the hidden mark that identifies it has not, and that is what must have caused the problem. I also see that the image was (it seems) right at the exact beginning of the note, in position 0. Surely that case was not well handled and that is what caused the problem. It is easy to reproduce and correct.

@dpradov
Copy link
Owner Author

dpradov commented Nov 20, 2023

BTW, how do I make the image titles show up in the editor? I dbl-click an image and assign the title „Step 1: Start here“ Where do I get to see this title?

Right now the title of the images is shown in the viewer and in the corresponding link when the image is hidden
imagen

imagen

If we paste an image from a browser and it includes a title, that title is automatically incorporated into the image and as a link:

https://www.diariodesevilla.es/deporte/cumplen-anos-gol-Iniesta-Final-Mundial-video_0_1372062940.html :
imagen

And also:
How would I go about resetting/ restoring the proportions of an inserted image to its original height/width ratio?
Say if I inadvertently dragged the wrong size handle? It were nice to have s.th. like „Restore original proportions“ in the context menu

I'm still including more options in this version, and some of them are along those lines. One of the options that I am going to incorporate is to allow, through the button that allows you to show or hide the images, if the CTRL key is held down, to allow you to choose how to show all the images of that node, or of the current selection perhaps , selecting a value equivalent to 'Max.Auto Widht on Insert', set in the options.

This will come in handy, for example, in the following situation: as I have indicated, I am using the application while I am following a training course. I'm keeping the application open in a small part of the window (using the recent function CTRL+Shift+F11...). In that situation I was interested in setting the 'Max.Auto Widht on Insert' property to -1, so that the images would fit in the small width visible while taking notes. But as soon as the course ends it will reset the size of all the images. Depending on my interest, I will place them at their actual size or limited to a maximum width, but much larger than the one I used while capturing them.

@dpradov
Copy link
Owner Author

dpradov commented Nov 20, 2023

Checking up on it in my file manager, I expect a new subfolder to be created, but nope. All images are in the same subfolder that has nothing to do with the real names of the notes.
This makes me question the purpose of the subfolders. Why not simply put all images into one and same location per knt-file? It would make looking for them easier when I want to edit an image

My intention is that the images are added to a subfolder corresponding to the name of the note at the time it is saved. That same image can be copied or moved to other notes and will not be copied back to storage. Renaming the folder cannot, logically, cause us to lose access to it, but it does not seem reasonable to have to rename the folder every time we rename the note. There would be many implications that would complicate a situation that would normally be simpler. The normal thing will be that the note exists and we add images to it. If we have a file with multiple tabs, it is very likely that this classification is not arbitrary... it will respond to some criteria: projects, themes, etc. With that objective I have distributed the images in folders. I also considered the possibility of leaving it as something configurable, but at the moment I have not seen it as important and I have had quite a bit of work with what I have already done.
But hey, I'll see if I add that configuration option

What you have tried I had not contemplated, you have added the image to a new note ("New Note" at that time), and then you have saved both.
( I create a second note and insert „dog_to_vet_1.jpg“ in its top node. I rename this note into „More Testing“. )
However, the note is registered and associated with the name of the note at the time it is added, not when it is saved. Typically, the note already exists and is saved when the image is added. I will try to control this case, checking the paths of the new images in case the note is renamed.

@dpradov
Copy link
Owner Author

dpradov commented Nov 20, 2023

I also take a look inside „_LOG.txt“ and I notice original image locations being saved there. What is the purpose of storing the original paths? It is highly unlikely that a „screenshot.jpg“ or „photo.jpg“ will point to the same content weeks later or on a different machine. Will this log file become bigger all the time? I tend to do heavy editing in my knt-files, on a daily basis. Might the log files slow me down after a few years?

You can delete that LOG file at any time. It is just a help to know what is happening, in case of doubts or problems.

Internally KNT takes note of the original path, as additional information that could be of interest, when inserting images from existing files (linked or not), never when the image comes from a screenshot

imagen

imagen

imagen

It's a problem? I can always create an option configurable only from the INI file that controls whether or not to collect that additional information

@dpradov
Copy link
Owner Author

dpradov commented Nov 20, 2023

Note that some of my captures corresponds to a version still in progress, that soon I will upload:

imagen

imagen

@Stefanoko
Copy link

Stefanoko commented Nov 20, 2023

Thanks a ton for taking the time to explain and clarify.
Must have had a bunch of blackouts, brain drain. It all seems very clear now, in hindsight.

I forgot that Config>Options refers to program settings. So ofc, it makes total sense that the actual storage method used, is a decision made per file, and will be set via file properties. Whereas the first 2 entries on the Options dialog define the default for NEW files in general.

Same applies to the DEFAULT link method. I tried it a few more times, and it works flawless, brings up the insert-dialog with its link-checkbox enabled or disabled.

The term "link mode" threw me off track. It seemed to indicate something else, especially in combination with "not owned". It all makes sense now.

File > Properties
The default path can be ok and in that case is normal not to click on Change.

Reassuring to know that I can simply click OK after changing the mode from embedded to external.
And yet, enabling the Change button seems to have a different effect (no missing dot-separator) as opposed to leaving the Change-button alone (missing dot-separator), I just tried it again by converting a new file back and forth between embedded and external.

I see that the two images were completely glued to each other.

Not really. I press the space bar, sometimes twice, before inserting the next image next to it.
Selecting each one individually is easy: I just click on it. Only one gets selected, not both.

Did you select it and press DELETE, or did you delete it by pressing BACKSPACE?

Neither. I right-clicked the image and picked Delete from the context menu.

the title of the images is shown .... in the corresponding link when the image is hidden

Got it.
I still have issues with those title links. Will report back when I can pinpoint it more precisely. In some instances it created both, a visible image AND a title link. When I dragged the image away from the title link, the internal viewer would no longer open. Will try to reproduce it and create screenshots.

it does not seem reasonable to have to rename the folder every time we rename the note.

Agree. Not necessary.
I still don't see the advantage of creating subfolders for each tab/note. The image names are unique to the knt, not to note names, which will continually change/rename. At least in my case. I work with knt-files every day, and there is a lot of re-structuring and moving across tabs/notes. Keynote is my THINKING Tool, many of my knt-files are a work-in-progress. Not just a storage place with pre-defined pigeonholes where data gets buried.

But hey, I'll see if I add that configuration option

Probably overkill and not necessary, since the app keeps track of the original tabname/ subfolder storage by its own — hopefully without too much overhead computing going on the background.
To edit an image or make annotations to it, I will open it/ pass it to my external viewer, right out of Keynote, and take it from there. Instead of selecting it from the file system.

You can delete that LOG file at any time. It is just a help to know what is happening, in case of doubts or problems.

Wow. Did not expect that. Very convenient.

It's a problem? I can always create an option configurable only from the INI file that controls whether or not to collect that additional information

No problem at all. I was just concerned it might create problems in the future if an original path becomes obsolete or no longer reflects the same content.
No ini-option needed.

@dpradov
Copy link
Owner Author

dpradov commented Dec 12, 2023

Hello @Ulfff
I understand that you have "only" lost today's changes (which is not small) and not the entire file. Among other things, because it would make no sense for you not to save a copy of your file somewhere else, especially if you have it compressed. It could be corrupted by a physical failure, for example. You should always have more than one copy that is not touched.
Furthermore, if you have the 'Backup at regular intervals' option activated, you must at least preserve the status of your file before today's first modification.

If what you wrote today is important and you need to recover it, do not delete the corrupt file. If you can open it, it is always possible to change the format to native mode and try to recover the information. I can tell you how to locate the text you wrote today (if it has not really been lost)

That said, having a 75MB file doesn't seem like a good idea to me at all. At the very least, it should be taking unnecessarily longer than necessary to load and save. I recommend that you have multiple files with different tabs. I have had files for more than 20 years but with very smaller sizes. I use multiple files, which I also save in compressed format (the encrypted format does not compress, it only encrypts), so the size is reduced.

At work I always have two instances of KeyNote NF open, usually one with a general file and then another instance from which I open more specific files, for different projects.

I tell you this because I think you should tend to it. Additionally, I plan to facilitate the use of multiple files in future versions, which can be opened from the same instance.
Since the latest versions of KN it is easy to create new .knt files from a given one, exporting one or several selected notes (e.g.) with knt format

In a 75MB file, it would not be at all unreasonable that some physical error could occur that causes problems when opening it, or that it cannot even be opened. And it could be something not attributable to the application.

You mentioned that with version 1.8.1 Beta 5 all worked ok. The fact is that between that version and the next the changes made are very minor and do not justify that problem:

  • Improvements for profiles management | Profiles\Default | Profiles macros
  • Images. Added new ini option: ImgViewerPath
  • Image viewer: added shortcuts for zoom and scrolling
  • Fixed: Can be difficult to open images inside tables with the internal or an external viewer

Whatever happened should not be anything specific to 1.8.1 .06 but perhaps it has been present long before and is showing up in sporadic situations that are not clear at the moment.

To try to narrow down the problem, do you use plain text notes or only RTF notes?

I wasn't able to switch to a note with a larger table (error message: "Failed to load stream"). Also, the last 80% of the table were missing.

In many cases this data has not been lost. Something has been recorded incorrectly and is not allowing the node to be read correctly, but the data is there (of course, if it is saved again the it is definitively lost, because couldn't be loaded)

It is essential to know what may be causing these problems, to solve it. Any details you can give me could help. If you recorded a few minutes before it was corrupted (and that backup was correct), perhaps you may remember what note/node you were modifying, what to insert that could have caused the failure.

If you can save copies of the files, before and after corruption, in native, unencrypted format, you could compare the two to see what difference there is that could be causing the problem. For example, there should be no #0 characters in text notes. Check if you see anything strange.

@dpradov
Copy link
Owner Author

dpradov commented Dec 12, 2023

In version 1.8.1 Beta 5 I fixed an issue that could have been problematic, but would have only arisen when dealing with certain images, and as you indicate, your file contains only text, isn't it?

  • Fixed: Image could be duplicated on storage (with new ID) after pasting an image previously copied or cut to clipboard
    The reason for this duplicity is explained here: Testing of new version 1.8.1 #623 (comment)
    Despite the latest corrections made, this problem could still occur when pasting an already registered image from
    the clipboard, previously copied or cut.
    • It was not controlled that the size of the RTF string constructed by the application from the image stream could be
      greater than that available from the clipboard, because the RichEdit control will have eliminated the following
      information if it was present: Embedded color profile, EXIF metadata, IPTC metadata
    • Among the consequences: the hidden mark was not being added, so it would end up being interpreted as another image.
      Besides, data could be being copied beyond the reserved buffer (In debug mode a Range check error exception is generated...), which it could be causing other errors that are more difficult to detect.

It is much safer to use version 1.8.1 Beta 5 or later than any version 1.8.1 before them.

@Stefanoko
Copy link

@Ulfff,
I want to try and see if I can reproduce the error. I also have a bunch of big files (not 65MB as yours, but 15-20 MB, text only), created with rather old versions of Keynote.
Maybe I can inject tables into them to see if that makes them break.

Can you help me with more details? What kind of tables, from where do they originate? Microsoft Excel?
How many rows and columns (just give me a rough idea what size of table I have to create).
Did you format the table cells, with BG-colors and possibly character-and/or paragraph styles?

@Stefanoko
Copy link

@dpradov,
You are right. The statement "If a different .ini file is passed (e.g. "keynote_F9.ini")..."
does in no way contradict the instruction that custom config files "... should also be created within the "Profiles" folder. Ex: "Profiles\F9"
I found it confusing, but that's probably just me. I did not occur to me that I can not only put different .ini files into profiles\subfolders, but even rename my custom .ini files.

Btw, I also have a "keynote.fvr_relative"
No idea, could it be a file you renamed in the past?

Now that you mention it, I probably did rename it myself. To keep a copy of keynote.fvr that I edited manually, to get rid of the hard-wired drive-letters and make my favorite links portable.

Went ahead and deleted settings.ini. Who knows where the heck it came from. Everything works fine without it. Good riddance.

I am interested in making changes and improvements that are much more interesting and important (I think) than correcting that plugin at this time.

No objections at all. Let's forget about plugins for the time being.
Same for timeformats.txt and dateformats.txt.
These things came up as I was trying to match all files in my existing keynote folder to their correct locations in the 1.8.1 distribution. And so I asked about it. No hurry. No worries.

@dpradov
Copy link
Owner Author

dpradov commented Dec 12, 2023

@Stefanoko

These things came up as I was trying to match all files in my existing keynote folder to their correct locations in the 1.8.1 distribution. And so I asked about it. No hurry. No worries.

You are meticulous and that is good and helps me. To begin, I am reviewing some texts based on what you have told me. For example:

Profiles.txt
plugins/Readme.txt

I will also modify the setup program to correctly locate "dateformats.txt" and "timeformats.txt", since right now KNT it will be ignoring the ones placed in the profile folders.

@Ulfff
Copy link

Ulfff commented Dec 13, 2023

Thanks for taking the time for the comprehensive answer, Daniel! have lost only a few hours work from today and rewritten already half of it. That wasn’t very funny, but I was aware oft he risk I took. I have of course many copies oft he file on different media and in different places and do backups on a daily basis on different drives, but only once a day. I do like the internal backup-system of keynote and changed the settings yesterday to save the last 8 backupfiles (in ordert o be as cautious as I can), but these automatic backup files from today already seemed to have the problem, so they were useless.

I started using Keynote like you recommend with many smaller files, but about 10 years ago I reduced them (for different reasons, e.g. to enter a password for encrypted files only once) to only three files, one main file, one with a different application and a third which I use as an advanced scratch for the main file, where I clone notes from the main files (or the whole file) to be able to see different notes (from the same file) at once. That would also be a nice addition : being able to open two windows of the same file.

I never had a single problem with the large main file all the time and was very impressed by the stability of the file, especially when taking into account that it is an encrypted file.

Back to the problem with my file: I did some experiments in order to replicate the problem and was at least 20 times successful with it. Like I assumed, it has to do with my tables, which I create (and then copy and paste) in MS Word, before I fill in the text in Keynote.

Eventually I was able to provoke the problem (=file does not open with the mentioned error message) with a newly created (uncompressed and unencrypted) file with just one note (with tree) in it and a table with some text in it. What I did was mostly adding some random text somewhere in the table, saving the file and reopen it again. At first (and with a table with a lot oft ext in it) I need only about 5 trials to provoke the the problem, but I wasn’t able to replicate the problem with the same or similar entered text the next time. It felt somehow random. Then I deleted more and more text in the original table with the result, that I needed more trials to provoke the problem.

In my last trial I used a calendar-like table from a word-file that I can attach here. I pasted the table in the note, then I selected the whole note (STRG+A), pasted the table from the clipboard again, entered a few letters in one random cell, saved the file and closed Keynote. When I tried to open it again I got the error message. I will attach also the broken keynote file, this should give you a good I idea of what might have happened.

Are you (or Stefanoko: thanks for your offer!) able to replicate it? It may be that you have to do something else with the file (entering text, pasting the table again, etc.) and always save the file, close keynote and reopen everything, up to maybe 20 times. My range to get the error varied from 1 to 30 trials.

Interesting: before uploading the knt-file here I tested it again with an older Keynote version ( 1.7.9.9) on another computer and was able to open it without an error message, but not with 1.8.0. beta 6, 1.8.1 beta 3 and 6 for example. Even better: with this older version I was able to open my broken work-file from today! :) Before, I tried to open it only with 1.8.0 beta 6, but not with 1.7.9.9.

table.docx
testfile_7.zip

@DrownedStarfish
Copy link

Hi, everyone!

I just downloaded beta 6, and everything seems to be working fine. I also tested it with my old primitive method for linking images, a link starting with "file:///", and that seems to work fine, too.

Thanks again.
DS.

@Stefanoko
Copy link

@Ulfff,
your testfile_7.knt does not open here, either. Win11, RichEdit 7.5
Also ran a bunch of trials with your Word-table. So far, I have not been able to break the file, but I keep trying, also with my own tables, pasted from different source programs.

@dpradov
Copy link
Owner Author

dpradov commented Dec 13, 2023

Hello @DrownedStarfish
Why didn't you try the latest version, 1.8.1. 07? There is little difference with beta 6, but it would be best if you tried that version, which is the latest.

@dpradov
Copy link
Owner Author

dpradov commented Dec 13, 2023

have lost only a few hours work from today and rewritten already half of it. That wasn’t very funny, but I was aware oft he risk I took. I have of course many copies oft he file on different media and in different places and do backups on a daily basis on different drives, but only once a day.

I'm glad to know it. You take a weight off my shoulders.

In my last trial I used a calendar-like table from a word-file that I can attach here. I pasted the table in the note, then I selected the whole note (STRG+A), pasted the table from the clipboard again, entered a few letters in one random cell, saved the file and closed Keynote. When I tried to open it again I got the error message. I will attach also the broken keynote file, this should give you a good I idea of what might have happened.

Great, that will help me see what's going on.

Interesting: before uploading the knt-file here I tested it again with an older Keynote version ( 1.7.9.9) on another computer and was able to open it without an error message, but not with 1.8.0. beta 6, 1.8.1 beta 3 and 6 for example. Even better: with this older version I was able to open my broken work-file from today! :) Before, I tried to open it only with 1.8.0 beta 6, but not with 1.7.9.9.

It is a useful information. Certainly from version 1.7.9.9 to 1.8.0 there was a very important jump, since I had to make a complete migration of the application from Delphi 2006 to Embarcadero Delphi CE 11.3, to a version with an important change related to text strings , so I had to review absolutely everything.

your testfile_7.knt does not open here, either. Win11, RichEdit 7.5
Also ran a bunch of trials with your Word-table. So far, I have not been able to break the file, but I keep trying, also with my own tables, pasted from different source programs.

@Stefanoko, thank you for your help in trying to locate the problem.
As soon as I try the files that @Ulfff uploaded, I think I will be able to locate and solve the problem.

@dpradov
Copy link
Owner Author

dpradov commented Dec 13, 2023

From what I'm seeing, the problem is related to this other issue: #610

On that occasion there was a problem with the conversion to UTF8 from Delphi 2006, which was resolved in Delphi CE, but I also detected that the following happened with the new version:

There are some texts, codified in UTF8, that can include some incorrect character conversions, that with the old method worked ok and only ignored the incorrect symbols, and now raises exception

All of this stemmed from changes to the RxRichEdit library, which I have used since version 1.8.0. It is an evolution of the original library that I used at the time, with a series of improvements. It seems that the library is not as fault tolerant as the previous one, and is throwing an exception when it detects something strange, which the previous one probably ignored and continued.
On the previous library I made a series of changes and improvements to be able to adapt it to Unicode, but I preferred to replace that one with this other one because I think that the migration made here is better, more correct and is already adapted to the version of the compiler that I use it (the changes I made worked on Delphi 2006, and now I am on Delphi CE 11.3)

I have to review and compare both libraries in the method that is generating the exception.
When I dealt with that issue I made a change to properly manage the possible exception that I had identified. On that occasion it was happening with plain text and it was resolved. Now it is happening with an RTF string, and the solution I proposed would not be applicable, and I cannot act by interpreting that it is not really a correct RTF and displaying it as plain text, because it would look like something that doesn't make sense.

In this specific case I can verify that if I ignore the exception and make the application continue, the contents of the table appear to be displayed correctly:

imagen

But that doesn't always have to be the case, it could be incorrect and show nothing or display the content poorly.
I'm going to look into the problem a little more to see if there is a way to make the RichEdit control not throw the exception, or at least I'm going to see how to deal with it in a more appropriate way.

The application has always, when it detects an exception during the file loading process, communicates the exception and closes the file, to avoid greater evils. Why hasn't shown the error on some occasions and yes on others? The problem does not occur when loading and processing the content of the nodes, but when trying to display the content of the problematic node in the editor. If at the time of saving the file the node selected in the note is not the problematic one, nothing will happen when you open it, it will not give any error. It will be given at the time of selecting it.

At first (and with a table with a lot oft ext in it) I need only about 5 trials to provoke the the problem, but I wasn’t able to replicate the problem with the same or similar entered text the next time. It felt somehow random. Then I deleted more and more text in the original table with the result, that I needed more trials to provoke the problem.

Although I'll try to do some checking to determine this, I think it's not KeyNote NF's fault that the RTF content of that note with that table is (at least partially) incorrect/problematic. It's probably a bug in the RichEdit control. What happens is that with the previous version of KeyNote NF (and with WordPad itself, where I have tried to open the content you have passed) that error was being ignored and now it is not.
Hence the seemingly random behavior. If it is bug in RichEdit control, the causes that lead to it can be quite complex to reproduce systematically.

Even better: with this older version I was able to open my broken work-file from today! :) Before, I tried to open it only with 1.8.0 beta 6, but not with 1.7.9.9.

One thing you should keep in mind about all this: even if a file could not be opened with a previous version, this does not necessarily mean that the content is irretrievably lost. It is possible that something is causing an uncontrolled failure that prevents it from opening, but its contents could will be almost completely fine. Therefore, you should not delete it lightly. At least not if you can lose a lot of information and work done.

@dpradov
Copy link
Owner Author

dpradov commented Dec 13, 2023

In this specific case I can verify that if I ignore the exception and make the application continue, the contents of the table appear to be displayed correctly. But that doesn't always have to be the case, it could be incorrect and show nothing or display the content poorly.

Is not correct. If I ignore the error, the table is shown, but truncating rows from the end.
The curious thing is that I save the truncated file with a different name and compare it with the original. Apart from the truncated rows, there is a minimal difference: two spaces are added (which in principle should have no effect). If I add all the rows from the previous one to the file thus obtained (with the two spaces), so that the content of both is identical, with the only difference being those two spaces, then it loads it correctly, showing all rows.

imagen

It's very weird. And certainly from 1.7.9.9 the initial file opens correctly (without the two spaces). I have to see with a magnifying glass what difference there is, because I don't understand it.

@DrownedStarfish
Copy link

Hello @DrownedStarfish Why didn't you try the latest version, 1.8.1. 07? There is little difference with beta 6, but it would be best if you tried that version, which is the latest.

@dpradov

I just tried installing kntSetup_1.8.1.7.2.exe . It did absolutely nothing. Didn't react at all. I'm running Win11 Pro as follows

OS Name Microsoft Windows 11 Pro
Version 10.0.22621 Build 22621
Other OS Description Not Available
Platform Role Desktop
Secure Boot State Off
Locale United States
Hardware Abstraction Layer Version = "10.0.22621.2506"
User Name MC-66KIUCQZJWDF\KidsAdmin
Time Zone Jerusalem Standard Time

and was in an administrator account at the time.

If it's relevant: I have two different versions of KN in two different folders.

Thanks.
DS.

@Stefanoko
Copy link

I just tried installing kntSetup_1.8.1.7.2.exe . It did absolutely nothing. Didn't react at all.

Are you saying the setup-file did not react at all?
I am also on Win11 Pro, slightly different build, though. Here is my "installed" (unpacked) version as a zip-package. The way it ended up on my machine after the setup-file had run its course.

Don't forget to copy your keynote.ini into the subfolder \Profiles\default before you start it up. Along with other ini- and config-files you may have (toolbar.ini, keyboard.ini ...)

KeyNote_NF_18172.zip

@dpradov
Copy link
Owner Author

dpradov commented Dec 14, 2023

I just tried installing kntSetup_1.8.1.7.2.exe . It did absolutely nothing. Didn't react at all

Well, honestly, I don't understand it, unless you have an antivirus or similar that is not allowing it to run.
Or the file is not downloaded correctly, have you checked the .sh256 or .sig files?

For the installation I have relied on a very stable program that is used as a setup program for many applications: Inno Setup (by Jordan Russell). I could have made a mistake in configuring what should be installed, which I don't think, but the program should start. Of course, first the user account control will ask confirmation. And depending on the antivirus you have, another confirmation requests can previously appear. For example, with COMODO, having HIPS (Host Intrusion Prevention System) activated, wich monitors everything in maximum detail, it asks me for confirmation with actions as the following (this appears to me with the installation of most applications)

imagen

Finally the setup begins:
imagen

@Ulfff
Copy link

Ulfff commented Dec 14, 2023

In this specific case I can verify that if I ignore the exception and make the application continue, the contents of the table appear to be displayed correctly:
But that doesn't always have to be the case, it could be incorrect and show nothing or display the content poorly. I'm going to look into the problem a little more to see if there is a way to make the RichEdit control not throw the exception, or at least I'm going to see how to deal with it in a more appropriate way.
The application has always, when it detects an exception during the file loading process, communicates the exception and closes the file, to avoid greater evils. Why hasn't shown the error on some occasions and yes on others? The problem does not occur when loading and processing the content of the nodes, but when trying to display the content of the problematic node in the editor. If at the time of saving the file the node selected in the note is not the problematic one, nothing will happen when you open it, it will not give any error. It will be given at the time of selecting it.

Exactly my experience: when I close the file while the table is visible and I somehow produced an error in it, the file won't open the next time. When I switch to another note after (unknowingly and without notice from the program) producing the error in the table, the file will open the next time without an error-message and in the non-table note where I left it; but as soon as I switch to the note with the table, I will get the error message and I can also see that the table is displayed incomplete.

What I find really strange is the randomness of the error and the difficulty to reproduce it with the exact same actions (although I might have missed so far what the essential part of the error-producing action actually constitutes).

One thing you should keep in mind about all this: even if a file could not be opened with a previous version, this does not necessarily mean that the content is irretrievably lost. It is possible that something is causing an uncontrolled failure that prevents it from opening, but its contents could will be almost completely fine. Therefore, you should not delete it lightly. At least not if you can lose a lot of information and work done.

I keep that in mind, thanks!

(I forgot and remember now that a few years back I had a partly and silently corrupted file, where hundreds of notes got lost or where overwritten with text from others notes; the file as a whole worked fine and I detected the error only months after it must have occured, because it concerned only notes I seldom used. Luckily I had backups from the time just before the error must have happened; nonetheless it was a lot of work to find the missing notes and replace them without replacing newer notes; since then I find it even more important to keep old backups.)

@DrownedStarfish
Copy link

@dpradov ,

Well, honestly, I don't understand it, unless you have an antivirus or similar that is not allowing it to run. Or the file is not downloaded correctly, have you checked the .sh256 or .sig files?

I just tried it with my AV paused. No change. I'll try checking the hash file. Are you sure that it has nothing to do with my having two different knt program folders on my machine?

I just looked at the Windows Event Viewer. Absolutely nothing there.

Thanks.
DS.

dpradov added a commit that referenced this issue Dec 14, 2023
…ing part of the node content [IMPORTANT]

Related to issues #662 (#622 ) and the problem described
in #623 (comment)

I have confirmed the error and its resolution in the second issue. And seeing what was happening,
it most likely must have been the cause of what was indicated in the first one.
It could affect RTF and plain text nodes

It was caused by a bug in the new RxRichEdit library used by KeyNote NF since version 1.8.0
@DrownedStarfish
Copy link

@dpradov ,

Success!, sort of.
I just tried installing it from a command line, using administrator rights, and it installed just fine, though it didn't notice either of the old installations.

As far as I can tell from a quick glance, the program itself runs fine.

Remember you said something about my having a strange codepage, or something like that?

Thanks.
DS

@dpradov
Copy link
Owner Author

dpradov commented Dec 14, 2023

Hello @Ulfff
Thanks to the file you uploaded I was able to identify and solve the problem.

I think that it can explain also the issue #662 (#622 ), apart from yours (#623 (comment))

I have confirmed the problem and its resolution with your file, and seeing what was happening, it most likely must have been the cause of what was indicated in the first one.

I have discovered that there was a bug in the new RxRichEdit library used by KeyNote NF since version 1.8.0
It was causing seemingly random errors that could truncate the contents of the affected nodes/notes
And all because of the insertion of zeros where line breaks should have been added. It happened sporadically,
usually only in nodes with a lot of text (I suppose).

That library uses certain procedures from the Delphi code (from Vcl.ComCtrls), but an instruction was missing in one of the methods. Instruction that was not used in previous versions of Delphi.

The error appeared in the execution of the procedure that adjusts the line break characters. Despite this, seeing the difficulties in reproducing the problem, it must be something that has happened on a few occasions, mainly in very long nodes.
In the example pointed out by you, the note (that was correctly saved as RTF), of 505180 bytes, gave error on opening due to the appearance of (only) two incorrect replacements at the process of loading into the RichEdit editor, in which instead of 0D 0A, 00 0A was being written.

@dpradov
Copy link
Owner Author

dpradov commented Dec 14, 2023

I will generate a new version with the fix.

Apart from that, I will also review the way in which these exceptions are treated when opening files (which I hope do not occur or occur as little as possible). At a minimum, apart from the error, I want to show some message that guides what to do in that situation. Maybe, instead of aborting the loading, let it open but warn of the problem and suggest that the native format be established (if it is not there) and save it without making modifications to another file (with Save As) in case was necessary and possible to recover the information.

@dpradov
Copy link
Owner Author

dpradov commented Dec 14, 2023

Success!,

Great, thanks

Remember you said something about my having a strange codepage, or something like that?

I think that you used UTF8 as system codepage, if I remember. But it seems strange to me that this could affect the launch of an executable.

@dpradov
Copy link
Owner Author

dpradov commented Dec 14, 2023

I have released a new version with the correction to the problem reported by Ulfff: 1.8.1 .08

Regards
Daniel

@Ulfff
Copy link

Ulfff commented Dec 15, 2023

I have released a new version with the correction to the problem reported by Ulfff: 1.8.1 .08

Your fix seems to work very good! I tried for an hour to provoke the error again with a file that took me only 10s to get a corrupted file with the last version - and failed. I also had saved about 10 more corrupted files like the testfile_7 I send you, but with different and more text. I was not able to open them in the last betas, but they open and display now without an error and not truncated. Great! :)

dpradov added a commit that referenced this issue Dec 25, 2023
…ode (or note)

By default, navigation toolbar buttons will show previous or next image instance in the node (or note) where the selected image is inserted
If click with Ctrl, then it will show image with previous or next ID (in the whole .knt file)

As described here: #623 (comment)
@dpradov dpradov closed this as completed Dec 29, 2023
@dpradov dpradov unpinned this issue Dec 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants