Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

Save conversations check for parent directory #4218

Merged
merged 5 commits into from Dec 2, 2021
Merged

Save conversations check for parent directory #4218

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f8748da
check dir function added.
mojtaba-komeili Nov 30, 2021
06f202a
path manager
mojtaba-komeili Dec 1, 2021
023882f
passing unit tests
mojtaba-komeili Dec 1, 2021
18356a1
removed extra files.
mojtaba-komeili Dec 1, 2021
b6a434b
removed the extra condition
mojtaba-komeili Dec 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
9 changes: 9 additions & 0 deletions parlai/utils/conversations.py
View file
Open in desktop
Expand Up @@ -276,6 +276,14 @@ def _get_path(datapath):
fle, _ = os.path.splitext(datapath)
return fle + '.jsonl'

@staticmethod
def _check_parent_dir_exits(datapath):
parent_dir = os.path.dirname(datapath)
if os.path.exists(parent_dir) and os.path.isdir(parent_dir):
return
logging.info(f'Parent directory ({parent_dir}) did not exist and was created.')
os.makedirs(parent_dir)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs to be migrated to pathmanager, thx

@classmethod
def save_conversations(
cls,
Expand All @@ -294,6 +302,7 @@ def save_conversations(
each of which is comprised of a list of act pairs (i.e. a list dictionaries
returned from one parley)
"""
cls._check_parent_dir_exits(datapath)
to_save = cls._get_path(datapath)

context_ids = context_ids.strip().split(',')
Expand Down