Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

External recommender fails when CAS contains control characters #1511

Closed
jcklie opened this issue Nov 12, 2019 · 2 comments
Closed

External recommender fails when CAS contains control characters #1511

jcklie opened this issue Nov 12, 2019 · 2 comments
Assignees
@reckart
Labels
🐛Bug Something isn't working Module: Recommender
Projects
Kanban
Milestone
28.2

Comments

@jcklie
Copy link
Contributor

jcklie commented Nov 12, 2019

Describe the bug
External recommender fails when CAS contains control characters.

To Reproduce
Steps to reproduce the behavior:

  1. Create a document with content 第四卷第一四二八页。 　�
  2. Configure an external recommender for that
  3. See error

Expected behavior
No error and nice predictions.

Screenshots
`Caused by: org.xml.sax.SAXParseException: Trying to serialize non-XML 1.0 character: 0x14 at offset 975 in string starting with 毛主席语录

Fix

Do the same as in dkpro/dkpro-core#1426
@jcklie jcklie added 🐛Bug Something isn't working Module: Recommender labels Nov 12, 2019
@jcklie jcklie added this to the Bug backlog milestone Nov 12, 2019
@jcklie jcklie self-assigned this Nov 12, 2019
@jcklie jcklie modified the milestones: Bug backlog, 0.16.0 Jan 14, 2020
@jcklie
Copy link
Contributor Author

jcklie commented Mar 16, 2020

@reckart I think that we fixed that in dkpro

@reckart
Copy link
Member

reckart commented Mar 16, 2020

I added support for XML 1.1 to the DKPro Core XmiWriter, but I don't think we use the XmiWriter in the external recommender. I expect we use similar code and the code in the external recommender should be adjusted in the same way as the DKPro Core code.

@reckart reckart modified the milestones: 0.16.0, 0.17.0 May 4, 2020
@jcklie jcklie modified the milestones: 0.17.0, Bug backlog Jul 21, 2020
@reckart reckart modified the milestones: 🦟 Bug backlog, 28.2 May 30, 2023
@reckart reckart added this to 🔖 To do in Kanban via automation May 30, 2023
@reckart reckart assigned reckart and unassigned jcklie May 30, 2023
reckart added a commit that referenced this issue May 30, 2023
@reckart
#1511 - External recommender fails when CAS contains control characters
d900012 
- Sanitize data before passing it to the XMI serializer - replacing problematic characters with spaces in text nodes and attributes
- Added unit test
reckart added a commit that referenced this issue May 30, 2023
@reckart
#1511 - External recommender fails when CAS contains control characters
745bd41 
- Sanitize data before passing it to the XMI serializer - replacing problematic characters with spaces in text nodes and attributes
- Added unit test
reckart added a commit that referenced this issue May 30, 2023
@reckart
Merge pull request #4045 from inception-project/bugfix/1511-External-…
16289c6 
…recommender-fails-when-CAS-contains-control-characters

#1511 - External recommender fails when CAS contains control characters
@reckart reckart closed this as completed May 30, 2023
Kanban automation moved this from 🔖 To do to 🍹 Done May 30, 2023
reckart added a commit that referenced this issue May 30, 2023
@reckart
Merge branch 'release/28.x' into bugfix/1496-Some-spans-are-missing-b…
aff80b7 
…egin-offset-field

* release/28.x:
  #1511 - External recommender fails when CAS contains control characters
reckart added a commit that referenced this issue May 30, 2023
@reckart
Merge branch 'release/28.x'
4800b99 
* release/28.x:
  #1496 - Some spans are missing begin offset field
  #1511 - External recommender fails when CAS contains control characters
  #1496 - Some spans are missing begin offset field
reckart added a commit that referenced this issue Jun 13, 2023
@reckart
Merge branch 'main' into refactoring/Upgrade-to-Spring-Boot-3
14d38be 
* main: (189 commits)
  No issue. Minor additions to BioC format description
  #4062 - ViewportTracker should focus on block-like elements
  #4032 - Allow using externalized strings from backend code
  #4060 - Clean up redundant code in annotation handlers
  #4026: Support for error tracking with Sentry
  #3673 - Update dependencies
  update dead link to the new file
  #4055 - Editor scrolls up when left sidebar is opened/closed
  [maven-release-plugin] prepare for next development iteration
  [maven-release-plugin] prepare release inception-28.2
  #4052 - Admins no longer see all projects in the project overview
  #3673 - Update dependencies
  #4048 - Document navigation options not visible to manager when viewing other users document
  #3673 - Update dependencies
  #3673 - Update dependencies
  #1496 - Some spans are missing begin offset field
  #1511 - External recommender fails when CAS contains control characters
  #1496 - Some spans are missing begin offset field
  #4040 - Ability to store preferences from client-side code
  #1066 - Recommender status info
  ...

% Conflicts:
%	inception/inception-api-annotation/src/main/java/de/tudarmstadt/ukp/clarin/webanno/api/annotation/page/AnnotationPageBase.java
%	inception/inception-brat-editor/src/main/java/de/tudarmstadt/ukp/clarin/webanno/brat/annotation/BratAnnotationEditor.java
%	inception/inception-diam/src/main/java/de/tudarmstadt/ukp/inception/diam/service/DiamWebsocketController.java
%	inception/inception-documents/src/test/java/de/tudarmstadt/ukp/inception/documents/DocumentServiceImplConcurrencyTest.java
%	inception/inception-external-search-solr/pom.xml
%	inception/inception-html-editor/src/main/java/de/tudarmstadt/ukp/inception/htmleditor/docview/HtmlDocumentViewControllerImpl.java
%	inception/inception-html-editor/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports
%	inception/inception-preferences/src/main/java/de/tudarmstadt/ukp/inception/preferences/config/PreferencesServiceAutoConfig.java
%	inception/inception-recommendation/src/main/java/de/tudarmstadt/ukp/inception/recommendation/service/LearningRecordServiceImpl.java
%	inception/inception-recommendation/src/main/java/de/tudarmstadt/ukp/inception/recommendation/service/RecommendationServiceImpl.java
%	inception/inception-recommendation/src/main/java/de/tudarmstadt/ukp/inception/recommendation/tasks/TrainingTask.java
%	inception/inception-recommendation/src/test/java/de/tudarmstadt/ukp/inception/recommendation/footer/RecommendationEventWebsocketControllerImplTest.java
%	inception/inception-support/pom.xml
%	inception/inception-ui-core/pom.xml
%	inception/inception-websocket/pom.xml
%	inception/pom.xml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@reckart reckart

Labels
🐛Bug Something isn't working Module: Recommender
Projects
Kanban
  
🍹 Done
Milestone
28.2
Development

No branches or pull requests
2 participants
@jcklie @reckart