Can't write first entry in directory journal when directory doesn't exist

[micahellison]

Diagnostic output

jrnl: v4.1
Python: 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]
OS: Windows 10

Current Behavior

FileNotFoundError when writing the first entry of a directory journal whose directory does not yet exist. Not sure if this is a Windows-specific bug.

Expected Behavior

jrnl should create any missing directories, whether it's the root directory of the directory journal, or the subdirectories for years and months.

Repro Steps

• In your config file, point the default journal's path to a new folder, like so: default: c:\temp\testing\newfolder\
• Ensure that the new folder doesn't exist: rmdir /s c:\temp\testing\newfolder\
• Run jrnl and create a new entry: jrnl here is my new entry

Debug output

[22:14:40] DEBUG    Logging start                                                                                           main.py:32
           DEBUG    Parsed args:                                                                                            main.py:42
                    Namespace(debug=True, preconfig_cmd=None, postconfig_cmd=None, filename=None, template=None,
                    on_date=None, today_in_history=False, month=None, day=None, year=None, start_date=None, end_date=None,
                    contains=None, strict=False, starred=False, tagged=False, limit=None, excluded=[], edit=False,
                    delete=False, change_time=None, export=False, tags=False, short=False, config_override=[],
                    config_file_path='', text=['here', 'is', 'my', 'new', 'entry'], exclude_starred=False,
                    exclude_tagged=False)
           DEBUG    Reading configuration from file C:\Users\micah\.config\jrnl\jrnl.yaml                                install.py:91
           DEBUG    Using configuration:                                                                                install.py:117
                    "{
                        'colors': {
                            'body': 'none',
                            'date': 'none',
                            'tags': 'none',
                            'title': 'none'
                        },
                        'default_hour': 9,
                        'default_minute': 0,
                        'editor': '',
                        'encrypt': False,
                        'extension': '.md',
                        'highlight': True,
                        'indent_character': '|',
                        'journals': {'default': 'c:\\temp\\testing\\newfolder\\'},
                        'linewrap': 80,
                        'tagsymbols': '@',
                        'template': False,
                        'timeformat': '%Y-%m-%d %H:%M',
                        'version': 'v4.1'
                    }"
           DEBUG    Using journal name: default                                                                          config.py:206
           DEBUG    Scoped config:                                                                                       config.py:120
                    {
                        'colors': {
                            'body': 'none',
                            'date': 'none',
                            'tags': 'none',
                            'title': 'none'
                        },
                        'default_hour': 9,
                        'default_minute': 0,
                        'editor': '',
                        'encrypt': False,
                        'extension': '.md',
                        'highlight': True,
                        'indent_character': '|',
                        'journals': {'default': 'c:\\temp\\testing\\newfolder\\'},
                        'linewrap': 80,
                        'tagsymbols': '@',
                        'template': False,
                        'timeformat': '%Y-%m-%d %H:%M',
                        'version': 'v4.1',
                        'journal': 'c:\\temp\\testing\\newfolder\\'
                    }
           DEBUG    open_journal 'default'                                                                              Journal.py:473
           DEBUG    Append mode: starting                                                                            controller.py:142
           DEBUG    Get template:                                                                                    controller.py:183
                    --template: None
                    from config: False
           DEBUG    Append mode: cli text detected: ['here', 'is', 'my', 'new', 'entry']                             controller.py:147
           DEBUG    Append mode: appending raw text to journal 'default': here is my new entry                       controller.py:165
  File "C:\Users\micah\.local\pipx\venvs\jrnl\Lib\site-packages\jrnl\main.py", line 44, in run
    status_code = controller.run(args)
                  ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\micah\.local\pipx\venvs\jrnl\Lib\site-packages\jrnl\controller.py", line 79, in run
    append_mode(**kwargs)
  File "C:\Users\micah\.local\pipx\venvs\jrnl\Lib\site-packages\jrnl\controller.py", line 177, in append_mode
    journal.write()
  File "C:\Users\micah\.local\pipx\venvs\jrnl\Lib\site-packages\jrnl\journals\FolderJournal.py", line 71, in write
    os.makedirs(dirname)
  File "<frozen os>", line 215, in makedirs
  File "<frozen os>", line 225, in makedirs
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Users\micah\.local\pipx\venvs\jrnl\Lib\site-packages\jrnl\main.py:44 in run                   │
│                                                                                                  │
│   43 │   │                                                                                       │
│ ❱ 44 │   │   status_code = controller.run(args)                                                  │
│   45                                                                                             │
│                                                                                                  │
│ C:\Users\micah\.local\pipx\venvs\jrnl\Lib\site-packages\jrnl\controller.py:79 in run             │
│                                                                                                  │
│    78 │   if _is_append_mode(**kwargs):                                                          │
│ ❱  79 │   │   append_mode(**kwargs)                                                              │
│    80 │   │   return                                                                             │
│                                                                                                  │
│ C:\Users\micah\.local\pipx\venvs\jrnl\Lib\site-packages\jrnl\controller.py:177 in append_mode    │
│                                                                                                  │
│   176 │   │   )                                                                                  │
│ ❱ 177 │   journal.write()                                                                        │
│   178 │   logging.debug("Append mode: completed journal.write()")                                │
│                                                                                                  │
│ C:\Users\micah\.local\pipx\venvs\jrnl\Lib\site-packages\jrnl\journals\FolderJournal.py:71 in     │
│ write                                                                                            │
│                                                                                                  │
│    70 │   │   │   if not os.path.exists(dirname):                                                │
│ ❱  71 │   │   │   │   os.makedirs(dirname)                                                       │
│    72 │   │   │   for e in self.entries:                                                         │
│ in makedirs:215                                                                                  │
│ in makedirs:225                                                                                  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'c:\\temp\\testing\\newfolder\\2024'
┏━ Error ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃  FileNotFoundError                                                                             ┃
┃  [WinError 3] The system cannot find the path specified: 'c:\\temp\\testing\\newfolder\\2024'  ┃
┃                                                                                                ┃
┃  This is probably a bug. Please file an issue at:                                              ┃
┃  https://github.com/jrnl-org/jrnl/issues/new/choose                                            ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

Other Information

As a user, you can get past this bug by creating the missing directory first, then running jrnl.

Discovered while reviewing #1873. Not sure how long this bug has existed.

[micahellison]

Looks like this is Windows-only, since a related CI test is failing only on Windows.

I also couldn't replicate this locally on WSL/Ubuntu. jrnl --diagnostic output:

jrnl: v4.1
Python: 3.11.6 (main, Oct  2 2023, 13:45:54) [GCC 11.4.0]
OS: Linux 5.15.146.1-microsoft-standard-WSL2

[micahellison]

I'm mystified by this bug. os.makedirs is supposed to create all directories in the path that don't exist, so why is Windows kicking out an error about it not existing? Furthermore, we have a passing BDD test in file_storage.feature for exactly this problem.

I was able to reproduce this locally for a little while. I also reproduced it in the interactive Python 3.12 shell with os.makedirs. Then I did something--I'm not sure what--and now it just works. What changed? Maybe a permissions issue on one of those temp subfolders? But why would that have existed in the first place? And why does this appear in GitHub Actions, but only for a new PR?

I searched around for others with issue, but most of the resolutions I see also don't seem to make much sense, and I think they're not actually solving the problem so much as drawing post hoc conclusions. I don't see anyone writing code to fix this problem.

Whatever the case, that brief test that reproduced this problem in interactive Python showed that it's a problem with Windows (or Python on Windows), rather than a problem with jrnl. Python on Windows is failing to deliver what the os.makedirs documentation promises, or, at the least, it's failing to accurately report the reason it can't meet that promise, so I'm closing this as wont-fix.

If you're a jrnl user and you have this problem, please let us know here and maybe we can narrow it down some more, but without reports of anyone else running into this, I don't think it's worth the effort to keep investigating.

[comkieffer]

I've found the root cause for this and it's expected behaviour (though unintuitive behaviour).

From the docs, Jrnl detects that a journal is a Folder journal if

• a folder with the name of the journal before running jrnl.
• the journal path in the config ends with the path separator

In the empty_folder config, the journal path is specified without a path separator at the end. To fix the test failures we could add the path separator to the path in the config but then two configs would be needed (one for Posix, one for

A proper fix would be to use pathlib instead of os.path since that translates Posix paths into windows paths when required.

The fact that the tests work on Linux is the actual bug. Running

jrnl --cf tests/data/configs/empty_folder.yaml --debug "this is my jrnl entry"

shows that, on Linux, it creates a FileJournal and happily chugs along.