fix: OpenAI Prompt output for reasoning models

[ranadeepsingh]

Related Issues/PRs

#xxx

What changes are proposed in this pull request?

The OpenAIResponses (which runs when OpenAIPrompt is set to use responses API) was not reading the right part of the response content.

• Correct output: response.output[-1].content[0].text
• Incorrect beign read: response.output[0] -> which had reasoning content and therefore Null as OpenAI models don't share reasonsing

How is this patch tested?

• I have written tests (not required for typo or doc fix) and confirmed the proposed feature/bug-fix/change works.

Does this PR change any dependencies?

• No. You can skip this section.
• Yes. Make sure the dependencies are resolved correctly, and list changes here.

Does this PR add a new feature? If so, have you added samples on website?

• No. You can skip this section.
• Yes. Make sure you have added samples following below steps.

1. Find the corresponding markdown file for your new feature in website/docs/documentation folder.
   Make sure you choose the correct class estimators/transformers and namespace.
2. Follow the pattern in markdown file and add another section for your new API, including pyspark, scala (and .NET potentially) samples.
3. Make sure the DocTable points to correct API link.
4. Navigate to website folder, and run yarn run start to make sure the website renders correctly.
5. Don't forget to add <!--pytest-codeblocks:cont--> before each python code blocks to enable auto-tests for python samples.
6. Make sure the WebsiteSamplesTests job pass in the pipeline.

[github-actions]

Hey @ranadeepsingh 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

• fix: Fix LightGBM crashes with empty partitions
• feat: Make HTTP on Spark back-offs configurable
• docs: Update Spark Serving usage
• build: Add codecov support
• perf: improve LightGBM memory usage
• refactor: make python code generation rely on classes
• style: Remove nulls from CNTKModel
• test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

[ranadeepsingh]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Copilot]

Pull request overview

Fixes how SynapseML’s OpenAI “Responses API” integration extracts the user-visible output text, particularly for reasoning-capable models where the first output item may contain reasoning metadata instead of message text.

Changes:

• Update OpenAIResponses to extract text from the last output entry (and update content-filter detection accordingly).
• Add/extend Scala tests validating output extraction and filtering behavior for Responses API (including reasoning-first outputs and multiple message outputs).
• Adjust sbt build/codegen packaging behavior around conda activation and Python wheel discovery/installation.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
project/build.scala	Avoids nesting conda run when already inside the target conda env.
project/CodegenPlugin.scala	Adds wheel auto-discovery and cleans old wheels before building/installing/publishing.
cognitive/src/main/scala/.../OpenAIResponses.scala	Changes output text extraction to use the last output entry; revises filtering logic.
cognitive/src/main/scala/.../OpenAIPrompt.scala	Validates unsupported Responses+AI Foundry combinations; improves AI Foundry endpoint detection; maps model→deployment for Responses.
cognitive/src/test/scala/.../OpenAIResponsesSuite.scala	Adds unit tests for last-output text extraction and filtering behavior.
cognitive/src/test/scala/.../OpenAIPromptSuite.scala	Adds coverage for Responses API output on gpt-4.1/gpt-5 and payload shaping for gpt-5.

[ranadeepsingh]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[ranadeepsingh]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[ranadeepsingh]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[codecov-commenter]

Codecov Report

❌ Patch coverage is 96.96970% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 84.61%. Comparing base (5d4a3ef) to head (0b1b816).

Files with missing lines	Patch %	Lines
...zure/synapse/ml/services/openai/OpenAIPrompt.scala	95.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2510      +/-   ##
==========================================
- Coverage   84.61%   84.61%   -0.01%     
==========================================
  Files         334      334              
  Lines       17669    17688      +19     
  Branches     1625     1628       +3     
==========================================
+ Hits        14951    14966      +15     
- Misses       2718     2722       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ranadeepsingh]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[ranadeepsingh]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).