Task Summary
Add a dedicated QaRankingCodegenSpec.scala that pins the payload/parse Python-snippet contract of QaRankingCodegen — the Hugging Face codegen for the question-answering / ranking task family. Mirror the existing TextGenCodegenSpec (same package) as the template.
Background
QaRankingCodegen (common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/QaRankingCodegen.scala) is an object extends TaskCodegen. It produces two Python source snippets — payloadPython(ctx) and parsePython(ctx) — that are spliced into the generated inference operator. It handles five HF pipeline tasks and, like every codegen, must reference self.* attributes rather than inlining raw user strings from the CodegenContext.
override val task: String = "question-answering"
override val tasks: Set[String] = Set(
"question-answering", "table-question-answering",
"zero-shot-classification", "sentence-similarity", "text-ranking"
)
Behavior to pin
| Member |
Contract |
task |
equals "question-answering" |
tasks |
equals exactly the 5-element set above |
payloadPython |
opens with if task == "question-answering":; has an elif branch for table-QA, zero-shot, sentence-similarity and text-ranking; ends with the else: payload = {"inputs": prompt_value} fallback |
parsePython |
the QA / table-QA branches read body.get("answer", ...); zero-shot / sentence-similarity / text-ranking return json.dumps(body) |
| no raw-value leakage |
with sentinel CodegenContext string fields (e.g. contextColumn / candidateLabels / sentencesColumn set to distinctive "MARKER_…" values), neither snippet contains the sentinel — the code references self.CONTEXT_COLUMN / self.CANDIDATE_LABELS / self.SENTENCES_COLUMN instead |
Build the CodegenContext with a small makeCtx helper exactly like TextGenCodegenSpec (set task = "question-answering" plus the QA-specific columns). Assert on snippet structure / marker substrings, not on exact whitespace.
Scope
- New spec:
QaRankingCodegenSpec.scala under common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/codegen/.
- No production-code changes.
Task Type
Task Summary
Add a dedicated
QaRankingCodegenSpec.scalathat pins the payload/parse Python-snippet contract ofQaRankingCodegen— the Hugging Face codegen for the question-answering / ranking task family. Mirror the existingTextGenCodegenSpec(same package) as the template.Background
QaRankingCodegen(common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/QaRankingCodegen.scala) is anobject extends TaskCodegen. It produces two Python source snippets —payloadPython(ctx)andparsePython(ctx)— that are spliced into the generated inference operator. It handles five HF pipeline tasks and, like every codegen, must referenceself.*attributes rather than inlining raw user strings from theCodegenContext.Behavior to pin
task"question-answering"taskspayloadPythonif task == "question-answering":; has anelifbranch for table-QA, zero-shot, sentence-similarity and text-ranking; ends with theelse: payload = {"inputs": prompt_value}fallbackparsePythonbody.get("answer", ...); zero-shot / sentence-similarity / text-ranking returnjson.dumps(body)CodegenContextstring fields (e.g.contextColumn/candidateLabels/sentencesColumnset to distinctive"MARKER_…"values), neither snippet contains the sentinel — the code referencesself.CONTEXT_COLUMN/self.CANDIDATE_LABELS/self.SENTENCES_COLUMNinsteadBuild the
CodegenContextwith a smallmakeCtxhelper exactly likeTextGenCodegenSpec(settask = "question-answering"plus the QA-specific columns). Assert on snippet structure / marker substrings, not on exact whitespace.Scope
QaRankingCodegenSpec.scalaundercommon/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/codegen/.Task Type