Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add TextAnalyze transformer to add support for Text Analytics /analyze endpoint #1267

Merged

Conversation

stuartleeks
Copy link
Contributor

@stuartleeks stuartleeks commented Nov 22, 2021

The Text Analytics /analyze endpoint provides a way to perform multiple tasks in a single API call (good for high-throughput scenarios)

  • Add TextAnalyze
  • Restructure results to be by document, then by task
  • Add options for defining which tasks to perform
  • Add sample notebook

@stuartleeks stuartleeks changed the title WIP - Add support for Text Analytics /analyze endpoint WIP - feat: Add TextAnalyze transformer to add support for Text Analytics /analyze endpoint Nov 23, 2021
@stuartleeks stuartleeks force-pushed the sl/text-analytics-analyze branch 2 times, most recently from 26dcc7e to c0da2a4 Compare November 24, 2021 07:35
@stuartleeks stuartleeks changed the title WIP - feat: Add TextAnalyze transformer to add support for Text Analytics /analyze endpoint feat: Add TextAnalyze transformer to add support for Text Analytics /analyze endpoint Nov 24, 2021
@stuartleeks stuartleeks marked this pull request as ready for review November 24, 2021 08:55
@stuartleeks
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@codecov-commenter
Copy link

codecov-commenter commented Nov 24, 2021

Codecov Report

Merging #1267 (2e9c4a9) into master (3898ad9) will increase coverage by 0.02%.
The diff coverage is 92.53%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1267      +/-   ##
==========================================
+ Coverage   83.28%   83.30%   +0.02%     
==========================================
  Files         300      300              
  Lines       13720    13828     +108     
  Branches      670      672       +2     
==========================================
+ Hits        11427    11520      +93     
- Misses       2293     2308      +15     
Impacted Files Coverage Δ
...re/synapse/ml/cognitive/CognitiveServiceBase.scala 75.96% <ø> (-0.37%) ⬇️
...oft/azure/synapse/ml/io/http/HTTPTransformer.scala 93.61% <85.71%> (+0.28%) ⬆️
...oft/azure/synapse/ml/cognitive/TextAnalytics.scala 88.84% <92.62%> (+2.51%) ⬆️
...ft/azure/synapse/ml/cognitive/ComputerVision.scala 78.57% <100.00%> (ø)
...re/synapse/ml/cognitive/TextAnalyticsSchemas.scala 100.00% <100.00%> (ø)
...crosoft/azure/synapse/ml/io/http/HTTPClients.scala 73.33% <0.00%> (-10.00%) ⬇️
...zure/synapse/ml/stages/PartitionConsolidator.scala 93.61% <0.00%> (-2.13%) ⬇️
...e/src/main/python/synapse/ml/cognitive/__init__.py 100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3898ad9...2e9c4a9. Read the comment docs.

@stuartleeks
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@stuartleeks stuartleeks force-pushed the sl/text-analytics-analyze branch 3 times, most recently from f1b1192 to 44fb0db Compare November 24, 2021 11:10
@stuartleeks
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@stuartleeks
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Commenter does not have sufficient privileges for PR 1267 in repo microsoft/SynapseML

@stuartleeks
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@stuartleeks
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

sentimentAnalysis: Option[Seq[TAAnalyzeResultTaskResults[SentimentScoredDocumentV3]]])

object TAAnalyzeResults extends SparkBindings[TAAnalyzeResult]

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This file has a bit of extra whitespace in a few areas

@mhamilton723
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants