Replace jdatapath dependency with Java 8 user home directory finding

[tfmorris]

We currently use an obscure dependency, jdatapath, to find the user's home directory on Windows. Since Java 8 was taught how to do that correctly itself, we should switch to that and drop the dependency.

System.getProperty("user.home") or org.apache.commons.lang3.SystemUtils.getUserHome()

https://stackoverflow.com/questions/585534/what-is-the-best-way-to-find-the-users-home-directory-in-java

[StoltHD]

How about you let the users select where they want things to be themself?

Not everyone wish to have their data stored on the C-drive on Windows, and not everyone knows how to change the path to their home folder for the system. And not everyone knows how to create sym-links to redirect a file or folder location.

Projects and workload areas should always be allowed to be chosen by the user...

I.e. some people may have 2 really fast nvme ssd's in their system, one relatively small for the OS (C: drive on Windows) and a larger one for workloads, cache folders and other active projects, while they save long time data on a larger HDD.

It would be great to be able to select the active work folder and cache folder without digging into system paths and manually setting java settings...

Just a suggestion...

[thadguidry]

Hi @StoltHD You can set the workspace folder location refine.data_dir ... in the refine.ini file using JAVA_OPTIONS. Take a look!

# Some sample configurations. These have no defaults.
#JAVA_HOME=C:\Program Files\Java\jdk1.8.0_151
# Use a single JAVA_OPTIONS that includes any JVM options you need upon OpenRefine startup
#JAVA_OPTIONS=-XX:+UseParallelGC -verbose:gc -Drefine.headless=true -Drefine.data_dir=C:\Users\user\AppData\Roaming\OpenRefine

Or you can specify a JAVA_OPTIONS along with refine.bat just with a Windows desktop shortcut as you would with any CMD line program on Windows.

Thought: For even more convenience (even for making your shortcuts to the refine.bat file with options, we could add a new refine.bat option such as /f to specify the OpenRefine data folder if you think that would make things easier?

[StoltHD]

Thanks for answering, I did find it after some digging and try and fail, I use the "exe" file and had to set it in the "openrefine.l4j.ini" file... not very logical when most Windows applications use a "appname.ini", "settings.ini" or "config.ini" file.
... but my problem at the end was that the parameters in the different files are somehow different...

Du you really need to have 3-4-5 different ways to set initial configurations?

And one more thing, is it somehow possible to check that Openrefine actually utilize the 4-8-10-16-24-32GB (or more) of ram I set it to use, before I run any cleanup/addition on a large dataset and it crash?

There are a few settings and a few system (java) calls that would be really great to be able to "just use" without digging down and using lots of time reading all types of forums and comments on github...
I am lucky, I actually know where to look, and I am not affraid asking stupid questions, but I know a few people that stopped using Openrefine even before they started because they din't figure out how to do simple pre-use configurations..

Thank you for answering so fast, its over a year now since I last used Openrefing because I gave up trying to figure out the settings I needed.

I really appreciate you took your time to answer.

[thadguidry]

@StoltHD You will find that in the Windows console when OpenRefine runs that the beginning part of the log will confirm what amount of memory you are using:

...
17:28:43.504 [            refine_server] Starting Server bound to '127.0.0.1:3333' (0ms)
17:28:43.505 [            refine_server] refine.memory size: 4000M JVM Max heap: 4194304000 (1ms)
17:28:43.518 [            refine_server] Initializing context: '/' from 'E:\GitHub Repos\OpenRefine\main\webapp' (13ms)
SLF4J: Class path contains multiple SLF4J bindings.
...

I agree that we could certainly add that somewhere on the main OpenRefine screen upon first startup to make this easier to see.

[StoltHD]

Thanks, That was great information, but I do not get any info like that... do do however get a python error I do use the 3.4 beta 2 version Maybe I should create a seperate issue for this 8:19:58.286 [ refine_server] Starting Server bound to ' 127.0.0.1:3333' (0ms) 08:19:58.293 [ refine_server] Initializing context: '/' from 'R:\Applikasjon - Programmer\openrefine\webapp' (7ms) SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/R:/Applikasjon%20-%20Programmer/openrefine/server/target/lib/slf4j-log4j12-1.7.18.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/R:/Applikasjon%20-%20Programmer/openrefine/webapp/WEB-INF/lib/slf4j-log4j12-1.7.18.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 08:19:58.816 [ refine] Starting OpenRefine 3.4-beta2 [c67e13b]... (523ms) 08:19:58.816 [ refine] initializing FileProjectManager with dir (0ms) 08:19:58.816 [ refine] R:\Applikasjon - Databaser\OpenRefine Projects (0ms) WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.python.core.PySystemState (file:/R:/Applikasjon%20-%20Programmer/openrefine/webapp/extensions/jython/module/MOD-INF/lib/jython-standalone-2.7.1.jar) to method java.io.Console.encoding() WARNING: Please consider reporting this to the maintainers of org.python.core.PySystemState WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release lør. 8. aug. 2020 kl. 03:05 skrev Thad Guidry <notifications@github.com>:

…

@StoltHD <https://github.com/StoltHD> You will find that in the Windows console when OpenRefine runs that the beginning part of the log will confirm what amount of memory you are using: ... 17:28:43.504 [ refine_server] Starting Server bound to '127.0.0.1:3333' (0ms) 17:28:43.505 [ refine_server] refine.memory size: 4000M JVM Max heap: 4194304000 (1ms) 17:28:43.518 [ refine_server] Initializing context: '/' from 'E:\GitHub Repos\OpenRefine\main\webapp' (13ms) SLF4J: Class path contains multiple SLF4J bindings. ... I agree that we could certainly add that somewhere on the main OpenRefine screen upon first startup to make this easier to see. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2961 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGWEGB7DMYCOPYEXWWL2S63R7SQHBANCNFSM4PAEU5IA> .

[tfmorris]

Maybe I should create a seperate issue for this

Yes, please do. Also if you'd like to suggest that the data directory be split from the location of the core metadata, feel free to create an issue for that.

Let's focus the discussion in this issue on the topic of replacing the jdatapath dependency and move all other discussion elsewhere. Thank you.