Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chained conversation support #40

Merged
merged 12 commits into from
Apr 25, 2023
Merged

Chained conversation support #40

merged 12 commits into from
Apr 25, 2023

Conversation

psinger
Copy link
Collaborator

@psinger psinger commented Apr 24, 2023

This PR addresses a few issues.

First, and foremost, it closes #17
It is now possible to add a parent_column which maps to a separate id column in the data and then chains the samples.

There are also two additional augmentation settings:
skip_parent_probability randomly skips chained samples in training
random_parent_probability randomly prepends any random sample to a record

For inference, full hierarchy is prepended, or nothing if not available

New data preparation was added to support this functionality.

Additionally, this PR fixes the following issues identified while implementing this PR:

  • Neptune logging did not work while re-saving the cfg
  • force_embedding_gradients was buggy and should be fixed now

It adds also another new add_prompt_answer_tokens setting to manually specify whether to add separator tokens to the tokenizer.

@psinger psinger marked this pull request as ready for review April 25, 2023 09:33
train.py Show resolved Hide resolved
Copy link
Contributor

@maxjeblick maxjeblick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for implementing this nice feature!
I've left some suggestions to rename variables.

@psinger psinger merged commit a53804f into main Apr 25, 2023
@psinger psinger deleted the psi/treestructure branch April 25, 2023 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] Support nested tree conversation data
2 participants