Linked files with relative path not always found (unlike in older JabRef version)

[AEgit]

JabRef version

JabRef 5.7--2022-07-18--be4a7c5

Operating system

Windows

Details on version and operating system

Windows 10 21H2

Checked with the latest development build

• I made a backup of my libraries before testing the latest development version.
• I have tested the latest development version and the problem persists

Steps to reproduce the behaviour

While using the wonderful "Copy linked files to folder..." function of JabRef 5 I noticed a behaviour of linked files that differs a lot from previous versions (namely JabRef 3.8.2) and which can have serious consequences. When adding linked files to JabRef (at least this happened in the older version) you would sometimes end up with files with
(a) a "full relative path" (don't know the proper term), i. e. you had specified your main file directory in the preferences, and the file would be shown in the "File" field of the editor with just its name, e.g., Baez2004.pdf
and
(b) a "partial relative path" (again, don't know the proper term), i. e. you had specified the same main file directory as in (a), but the file would be shown in the "File" field of the editor with its file name + the directory the file was stored in, e.g., Literature_database/Baez2008.pdf ("Literature_database" is the directory, "Baez2008.pdf" is the actual file)
and
(c) the absolute path, e.g. C:/Users/Literature_database/Baez2012.pdf

Let's forget about (c) since I have not seen the issue so far appear with (c) (in fact I don't even know, whether the newer JabRef versions support absolute file paths).

Now the problem is as follows:

1. Open a linked file from within JabRef (current developer version 5.7) that has a "full relative path" (Baez2004.pdf). We assume you have set the correct main file directory in your preferences (e.g., C:\Users\Literature_database)
2. The linked file is opened (all is good!)
3. Now try to open a linked file from within JabRef that has a "partial relative path" (Literature_database/Baez2008.pdf)
4. The file will not open and instead the file manger of the operating system (e.g., Windows Explorer) is opened
5. You can get around the issue of (4) if you change your main file directory to the directory that contains the file directory, e.g. C:\Users
6. Now files from (4) will open, but files from (2) will no longer open (instead you get again the file manager)

This new behaviour is completely different from JabRef 3.8.2, where - if the correct main file directory was set - it would not matter, whether the file had the "full relative path" or the "partial relative path". The files would always be opened. Below I attach some screenshots of the file editor view of files with "full" and "partial" relative paths for both JabRef 5.7 (current developer version) and JabRef 3.8.2 (both use the same (!) main file directory). Note, that only JabRef 3.8.2 would open always open the files, as long as the correct main file directory was set.

This behaviour also has implications for downstream functions in JabRef, e.g., the "Copy linked files to folder..." will not copy files that it does not find (as in (4)) and it will not even tell you, that it did not copy these files.

Also note: Manually changing the paths is not feasible (?) if you are dealing with a database that contains thousands of entries.

Appendix

JabRef 5.7: Full relative path - file will be opened in (2)

JabRef 5.7: Partial relative path - file will NOT be opened in (4)

JabRef 3.8.2: Full relative path - file will be opened in (2)

JabRef 3.8.2: Partial relative path - file will be opened in (4)

[ThiloteE]

Also note: Manually changing the paths is not feasible (?) if you are dealing with a database that contains thousands of entries.

Workaround: https://discourse.jabref.org/t/move-folder-with-pdfs-and-have-jabref-reconnect-links-to-them-automatically/2285/13

[AEgit]

Thanks for the workaround - yes, that could be an option, but I still would prefer the actual problem to be solved, since:
(1) If you are not aware of the problem and use "Copy linked files to folder..." function, PDFs will just not be copied - and JabRef will not even tell you about this.
(2) It worked in older versions of JabRef so longterm users might be used to the previous behaviour
(3) You generally want to avoid (if possible) making these type of changes to massive databases where you do not know/remember whether something down the line will break because of this change (= you are potentially changing the linked file paths for thousands of files). And you still might miss some files, which leads to the problem described in (1)

[Siedlerchr]

Hi, thanks for the report. I am trying to understand your case:

I have the following setup now:

1. Main File directory is /Users/....JabrefTemp/Files
2. Add a file from a subdirectory /Users/....JabrefTemp/Files/TESt_SingleFile/2020_StDenis_et_al_FilteringDisasterTwitterForLocal_ISCRAM.pdf
3. Bib entry has:
   file = {:TESt_SingleFile/2020_StDenis_et_al_FilteringDisasterTwitterForLocal_ISCRAM.pdf:PDF},
   Opening works because TESt_SingleFile is a subdirectory of the main file directory.

Now your case is that you somehow ended up with:
file = {:Files/TESt_SingleFile/2020_StDenis_et_al_FilteringDisasterTwitterForLocal_ISCRAM.pdf:PDF},

Logically, JabRef will try to find the file in the following directory then:
/Users/....JabrefTemp/Files/Files/TESt_SingleFile

From my point of view the behavior in 3.8.2 was faulty. JabRef 5.x behaves correctly.
I maybe have a solution for this but need to make sure that it does not break with subdirectories that have the same name

[AEgit]

Your description somewhat fits the problem, but not completely.

Files which would open ("full relative path"), are linked as follows in JabRef 5:

file = {Baez2004.pdf:Baez2004.pdf:PDF},

Files which would not open ("partial relative path"), are linked as follows in JabRef 5:

file = {Baez2008.pdf:Literature_database\\Baez2008.pdf:PDF},

Note, that in both cases the main file directory is the same, namely "C:/Users/Literature_database/". But for whatever reason JabRef 3.8.2 would sometimes store files with the "full relative path" and sometimes with the "partial relative path", despite the fact that both files were actually residing in exactly the same folder. This is not something that the user did manually (at least not willingly!) and it might be a result of how even older versions of JabRef stored information on linked files in the database (I have been using JabRef since 2013 or maybe even before). These paths were then kept as such in all subsequent JabRef versions - and there was never a problem in opening files. Only the most recent version has these issues - I can only speculate, but presumably the older version would ALWAYS check whether the file could be found in the main file directory. If that was the case, the "Literature_database" of the "partial relative path" would be ignored and the file would still open.

[Siedlerchr]

@AEgit Would it be possible for you to test this version? https://builds.jabref.org/pull/9046/merge/ if the files can be opened. I added a check if the "full relative path" is included in the configured file directory in JabRef.
You should make a copy of your bib file before.

[AEgit]

JabRef 5.8-PullRequest9046.21--2022-08-12--52b5f6f
Windows 10 10.0 amd64
Java 18.0.2
JavaFX 18.0.2+2

@Siedlerchr Thank you very much! Indeed, as far as I can tell, this build solves the problem. Well done!

[AEgit]

Update: Actually, the problem is not fully solved. I now found an example in my database, that also contains the full absolute path (Example (c) in the first post) to the main file directory (the main file directory is always the same in these examples):

file = {Boyd2012.pdf:C\:\\Users\\Literature_database\\Boyd2012.pdf:PDF},

This file will not open, even with the new build:

JabRef 5.8-PullRequest9046.21--2022-08-12--52b5f6f
Windows 10 10.0 amd64
Java 18.0.2
JavaFX 18.0.2+2

[Siedlerchr]

Normally absolute paths should not be a problem, but I think there's something wrong with the colon in between the fie path that JabRef chuckles on that. I have no Windows machine at hand, so I can't test this

[AEgit]

With version 3.8.2 the exact same file path works and JabRef opens the linked file. So it has to be something that was introduced in the new version.

[Siedlerchr]

@koppor could reproduce this as well with the absolute paths
TODO: Check FileFieldParser also FileHelper#find

[AEgit]

JabRef 5.8--2022-09-03--416ba16
Windows 10 10.0 amd64
Java 18.0.2.1
JavaFX 18.0.2+2

I can confirm, this issue appears to be fixed in the latest dev version. Well done!