[Epic] Data Augmentation Issue Overview and Discussions

[tianjianjiang]

@tianjianjiang (Mike) hopes that this issue ticket will act like an Epic issue that covers potential tasks of data augmentation.
Please kindly note that Mike is merely the first member who tries to work on data augmentation and by no means an authority of this topic. Feel free to participate in any possible way.

Issues (PRs)

• define data augmentation API | Streamlit → produce variations of prompt and/or input #10 (feat: draft augmentation API with back-translation strategy #10 #19)
• parse or annotate Jinja template | before augmentation → augmenters know where the off-limit variables are #49

Request for Proposal

Although the section title is https://en.wikipedia.org/wiki/Request_for_proposal but this section isn't anywhere near a formal RFP yet. Please don't hesitate to edit or comment.

Purpose

Diversify our prompts and generalize our models.
(For example, see this comment: #52 (comment).)

Scopes

1. Prompt template
2. (Possibly in the future) perturbation or more complex approaches for training/fine-tuning, e.g., papers of prompt/prefix tuning.

Implementation Plans

Prompt template

Expected input/output in Jinja

Presumably world politics, sports, business, and science and technology must remain intact.

• Input

{{text}} 
Is this a piece of news regarding world politics, sports, business, or science and technology? ||| 
{{ ["World politics", "Sports", "Business", "Science and technology"][label] }}

• Output

{{text}} 
Is this news about world politics, sports, business, or science and technology? ||| 
{{ ["World politics", "Sports", "Business", "Science and technology"][label] }}

TBD

• Input/output formats for a shared API
  • Jinja compatibility; on-going discussions (Mike was hoping that we can use https://docs.github.com/en/discussions but no strong preference here):
    1. parse or annotate Jinja template | before augmentation → augmenters know where the off-limit variables are #49 (comment)
    2. parse or annotate Jinja template | before augmentation → augmenters know where the off-limit variables are #49 (comment)
    3. parse or annotate Jinja template | before augmentation → augmenters know where the off-limit variables are #49 (comment)
  • Streamlit UI behavior
  • Template PR flow
• Different strategies (e.g, back-translation, summarization, https://github.com/makcedward/nlpaug, https://github.com/QData/TextAttack, etc.)
  • Integration test

[VictorSanh]

this plan looks good to me! thanks @tianjianjiang

Since we are in agreement that protecting text with jinja is a good solution (#49 (comment)) let's go with that.

we can discuss directly on issues/pull requests. I'll take a closer look at your PR later today.

(note that we'll start the prompt sourcing sprint independently of the augmentation pipeline since it's almost two orthogonal and parallelizable questions)

[stephenbach]

Closing due to inactivity. Feel free to reopen if you want to revisit this!