Journal storage does not work with "A required privilege is not held by the client".

[hrntsm]

Expected behavior

When I tried to use journal storage with Optuna 3.1.0b0, it did not work with the error "A required privilege is not held by the client".
How can I use it?

Environment

• Optuna version: 3.1.0b0
• Python version: 3.9.13
• OS: Windows-10-10.0.22000-SP0

Error messages, stack traces, or logs

c:\Users\hrntsm\Desktop\optuna-test\journal.py:9: ExperimentalWarning: JournalStorage is experimental (supported from v3.1.0). The interface can change in the future.
  storage = optuna.storages.JournalStorage(
Traceback (most recent call last):
  File "c:\Users\hrntsm\Desktop\optuna-test\journal.py", line 13, in <module>
    study = optuna.create_study(storage=storage, study_name="test")
  File "C:\Users\hrntsm\Desktop\optuna-test\venv\lib\site-packages\optuna\_convert_positional_args.py", line 63, in converter_wrapper
    return func(**kwargs)
  File "C:\Users\hrntsm\Desktop\optuna-test\venv\lib\site-packages\optuna\study\study.py", line 1158, in create_study
    study_id = storage.create_new_study(study_name)
  File "C:\Users\hrntsm\Desktop\optuna-test\venv\lib\site-packages\optuna\storages\_journal\storage.py", line 142, in create_new_study
    self._write_log(JournalOperation.CREATE_STUDY, {"study_name": study_name})
  File "C:\Users\hiroa\Desktop\optuna-test\venv\lib\site-packages\optuna\storages\_journal\storage.py", line 132, in _write_log
    self._backend.append_logs([{"op_code": op_code, "worker_id": worker_id, **extra_fields}])
  File "C:\Users\hrntsm\Desktop\optuna-test\venv\lib\site-packages\optuna\storages\_journal\file.py", line 191, in append_logs
    with get_lock_file(self._lock):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\contextlib.py", line 119, in __enter__   
    return next(self.gen)
  File "C:\Users\hrntsm\Desktop\optuna-test\venv\lib\site-packages\optuna\storages\_journal\file.py", line 138, in get_lock_file
    lock_obj.acquire()
  File "C:\Users\hrntsm\Desktop\optuna-test\venv\lib\site-packages\optuna\storages\_journal\file.py", line 65, in acquire
    raise err
  File "C:\Users\hrntsm\Desktop\optuna-test\venv\lib\site-packages\optuna\storages\_journal\file.py", line 58, in acquire
    os.symlink(self._lock_target_file, self._lock_file)
OSError: [WinError 1314] A required privilege is not held by the client: './journal.log' -> './journal.log.lock'

Steps to reproduce

1. run pip install optuna==3.1.0b0
2. run below code

import optuna


def objective(trial):
    x = trial.suggest_float("x", -10, 10)
    return x**2


storage = optuna.storages.JournalStorage(
    optuna.storages.JournalFileStorage("./journal.log"),
)

study = optuna.create_study(storage=storage, study_name="test")
study.optimize(objective, n_trials=100)

Additional context (optional)

When I executed the same code in the following environment, it worked fine. Is it necessary to do something different in Windows?

• Optuna version:3.1.0b0
• Python version:3.9.12
• OS:macOS-13.1-arm64-arm-64bit

[nzw0301]

It sounds like a windows-specific problem, ref https://answers.microsoft.com/en-us/windows/forum/all/how-to-fix-problem-a-required-privilege-is-not/d206b4b3-b6c2-4b0b-8629-5d4dfcb8f8d0 (so I believe that this is not the bug). According to #4151 (review), using developer mode might resolve the issue.

[hrntsm]

Thank you for comment.
I looked at the link and it looked like a permissions issue, so I ran it as admin and that solved it.
Therefore, this was not a bug.

However, not everyone can change permissions such as when the PC is managed by an organization, so if there is any way to improve this, please consider it.

If necessary, I will create a PR with this added to the JournalStorage documentation.

[nzw0301]

@toshihikoyanase @c-bata Do you have any opinion on #4298 (comment)?

[c-bata]

@hrntsm Thank you for using the beta release of Optuna 3.1!

If necessary, I will create a PR with this added to the JournalStorage documentation.

Sounds great!

I think @toshihikoyanase also ran into the similar permission issue and used the developer mode. As another workaround, you may be able to use JournalFileOpenLock like the following since it doesn't rely on os.symlink. I haven't tested yet though (sorry, I don't have a Windows machine).

import optuna
from optuna.storages import JournalStorage, JournalFileStorage, JournalFileOpenLock

file_path = "./journal.log"

lock_obj = JournalFileOpenLock(file_path)
storage =JournalStorage(JournalFileStorage(file_path, lock_obj=lock_obj))

Could you try the above code?

[hrntsm]

@c-bata
Thanks for the sample code.
I tried this code and it worked without any permission changes in my windows environment.
I will include the sample of the code here and reflect it in the document.

[c-bata]

Thank you for your investigation. It would be great!

[c-bata]

Let me close this issue since #4308 was merged. Thank you again for the report!