syntax highlighting bug #16

Closed
NelleV opened this Issue Sep 29, 2010 · 4 comments

Comments

Projects
None yet
3 participants
@NelleV
Member

NelleV commented Sep 29, 2010

Trying to load a file in the grader view we get the error message: "Could not render this file in the code viewer. Click on the Download button to download this file instead."

After clicking OK, we get the message "redundant UTF-8 sequence" and the Loading Code... wheel continue to show.

Clicking on Download leads to the trace given below.

Checking out the file through svn and checking the file type shows "ISO-8859 English text, with CRLF line terminators" There are non-ASCII characters in the file.

ActiveRecord::RecordNotFound in ResultsController#download

Couldn't find SubmissionFile without an ID

RAILS_ROOT: /data/markus/instance/csc108-2009-09 Application Trace | Framework Trace | Full Trace

/u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/base.rb:1567:in find_from_ids' /u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/base.rb:616:infind' /data/markus/instance/csc108-2009-09/app/controllers/results_controller.rb:116:in `download'

/u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/base.rb:1567:in find_from_ids' /u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/base.rb:616:infind' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/base.rb:1331:in send' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/base.rb:1331:inperform_action_without_filters' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/filters.rb:617:in call_filters' /u/drprj/.gem/ruby/1.8/gems/actionpack-
2.3.4/lib/action_controller/filters.rb:610:inperform_action_without_benchmark' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/benchmarking.rb:68:in perform_action_without_rescue' /u/drprj/.gem/ruby/1.8/gems/activesupport-2.3.4/lib/active_support/core_ext/benchmark.rb:17:inms' /usr/lib/ruby/1.8/benchmark.rb:308:in realtime' /u/drprj/.gem/ruby/1.8/gems/activesupport-2.3.4/lib/active_support/core_ext/benchmark.rb:17:inms' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/benchmarking.rb:68:in perform_action_without_rescue' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/rescue.rb:160:inperform_action_without_flash' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/flash.rb:146:in perform_action' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/base.rb:532:insend' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/base.rb:532:in process_without_filters' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/filters.rb:606:inprocess' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/base.rb:391:in process' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/base.rb:386:incall' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/routing/route_set.rb:437:in call' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/dispatcher.rb:87:indispatch' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/dispatcher.rb:121:in _call' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/dispatcher.rb:130:inbuild_middleware_stack' /u/drprj/.gem/ruby/1.8/gems/rack-1.0.0/lib/rack/head.rb:9:in call' /u/drprj/.gem/ruby/1.8/gems/rack-1.0.0/lib/rack/head.rb:9:incall' /u/drprj/.gem/ruby/1.8/gems/rack-1.0.0/lib/rack/methodoverride.rb:24:in call' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/params_parser.rb:15:incall' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/session/abstract_store.rb:122:in call' /u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/query_cache.rb:29:incall' /u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/connection_adapters/abstract/query_cache.rb:34:in cache' /u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/query_cache.rb:9:incache' /u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/query_cache.rb:28:in call' /u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:361:incall' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/failsafe.rb:26:in call' /u/drprj/.gem/ruby/1.8/gems/rack-1.0.0/lib/rack/lock.rb:11:incall' /u/drprj/.gem/ruby/1.8/gems/rack-1.0.0/lib/rack/lock.rb:11:in synchronize' /u/drprj/.gem/ruby/1.8/gems/rack-1.0.0/lib/rack/lock.rb:11:incall' /u/drprj/.gem/ruby/1.8/gems/actionpack-
2.3.4/lib/action_controller/dispatcher.rb:106:in call' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/cgi_process.rb:44:indispatch_cgi' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/dispatcher.rb:101:in dispatch_cgi' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/dispatcher.rb:27:indispatch' /usr/lib/ruby/1.8/mongrel/rails.rb:76:in process' /usr/lib/ruby/1.8/mongrel/rails.rb:74:insynchronize' /usr/lib/ruby/1.8/mongrel/rails.rb:74:in process' /usr/lib/ruby/1.8/mongrel.rb:159:inprocess_client' /usr/lib/ruby/1.8/mongrel.rb:158:in each' /usr/lib/ruby/1.8/mongrel.rb:158:inprocess_client' /usr/lib/ruby/1.8/mongrel.rb:285:in run' /usr/lib/ruby/1.8/mongrel.rb:285:ininitialize' /usr/lib/ruby/1.8/mongrel.rb:285:in new' /usr/lib/ruby/1.8/mongrel.rb:285:inrun' /usr/lib/ruby/1.8/mongrel.rb:268:in initialize' /usr/lib/ruby/1.8/mongrel.rb:268:innew' /usr/lib/ruby/1.8/mongrel.rb:268:in run' /usr/lib/ruby/1.8/mongrel/configurator.rb:282:inrun' /usr/lib/ruby/1.8/mongrel/configurator.rb:281:in each' /usr/lib/ruby/1.8/mongrel/configurator.rb:281:inrun' /usr/lib/ruby/1.8/mongrel/command.rb:212:in `run'

/u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/base.rb:1567:in find_from_ids' /u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/base.rb:616:infind' /data/markus/instance/csc108-2009-09/app/controllers/results_controller.rb:116:in download' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/base.rb:1331:insend' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/base.rb:1331:in perform_action_without_filters' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/filters.rb:617:incall_filters' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/filters.rb:610:in perform_action_without_benchmark' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/benchmarking.rb:68:inperform_action_without_rescue' /u/drprj/.gem/ruby/1.8/gems/activesupport-2.3.4/lib/active_support/core_ext/benchmark.rb:17:in ms' /usr/lib/ruby/1.8/benchmark.rb:308:inrealtime' /u/drprj/.gem/ruby/1.8/gems/activesupport-2.3.4/lib/active_support/core_ext/benchmark.rb:17:in ms' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/benchmarking.rb:68:inperform_action_without_rescue' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/rescue.rb:160:in perform_action_without_flash' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/flash.rb:146:inperform_action' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/base.rb:532:in send' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/base.rb:532:inprocess_without_filters' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/filters.rb:606:in process' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/base.rb:391:inprocess' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/base.rb:386:in call' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/routing/route_set.rb:437:incall' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/dispatcher.rb:87:in dispatch' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/dispatcher.rb:121:incall' /u/drprj/.gem/ruby/1.8/gems/actionpack-
2.3.4/lib/actioncontroller/dispatcher.rb:130:in build_middleware_stack' /u/drprj/.gem/ruby/1.8/gems/rack-1.0.0/lib/rack/head.rb:9:incall' /u/drprj/.gem/ruby/1.8/gems/rack-1.0.0/lib/rack/head.rb:9:in call' /u/drprj/.gem/ruby/1.8/gems/rack-1.0.0/lib/rack/methodoverride.rb:24:incall' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/params_parser.rb:15:in call' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/session/abstract_store.rb:122:incall' /u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/query_cache.rb:29:in call' /u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/connection_adapters/abstract/query_cache.rb:34:incache' /u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/query_cache.rb:9:in cache' /u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/query_cache.rb:28:incall' /u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:361:in call' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/failsafe.rb:26:incall' /u/drprj/.gem/ruby/1.8/gems/rack-1.0.0/lib/rack/lock.rb:11:in call' /u/drprj/.gem/ruby/1.8/gems/rack-1.0.0/lib/rack/lock.rb:11:insynchronize' /u/drprj/.gem/ruby/1.8/gems/rack-1.0.0/lib/rack/lock.rb:11:in call' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/dispatcher.rb:106:incall' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/cgi_process.rb:44:in dispatch_cgi' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/dispatcher.rb:101:indispatch_cgi' /u/drprj/.gem/ruby/1.8/gems/actionpack-2.3.4/lib/action_controller/dispatcher.rb:27:in dispatch' /usr/lib/ruby/1.8/mongrel/rails.rb:76:inprocess' /usr/lib/ruby/1.8/mongrel/rails.rb:74:in synchronize' /usr/lib/ruby/1.8/mongrel/rails.rb:74:inprocess' /usr/lib/ruby/1.8/mongrel.rb:159:in process_client' /usr/lib/ruby/1.8/mongrel.rb:158:ineach' /usr/lib/ruby/1.8/mongrel.rb:158:in process_client' /usr/lib/ruby/1.8/mongrel.rb:285:inrun' /usr/lib/ruby/1.8/mongrel.rb:285:in initialize' /usr/lib/ruby/1.8/mongrel.rb:285:innew' /usr/lib/ruby/1.8/mongrel.rb:285:in run' /usr/lib/ruby/1.8/mongrel.rb:268:ininitialize' /usr/lib/ruby/1.8/mongrel.rb:268:in new' /usr/lib/ruby/1.8/mongrel.rb:268:inrun' /usr/lib/ruby/1.8/mongrel/configurator.rb:282:in run' /usr/lib/ruby/1.8/mongrel/configurator.rb:281:ineach' /usr/lib/ruby/1.8/mongrel/configurator.rb:281:in run' /usr/bin/mongrel_rails:129:inrun' /usr/lib/ruby/1.8/mongrel/command.rb:212:in `run' /usr/bin/mongrel_rails:282

Request

Parameters:
{"commit"=>"Download", "authenticity_token"=>"HqzHOx3y4CQj1JdVTwxzvwbteaI/UD7RN0bAX/77xuI="}

Show session dump

Response

Headers:

{"Content-Type"=>"", "Cache-Control"=>"no-cache"}

Modified by c6conley 5 months ago:

Seems to be a Rails issue - see: https://rails.lighthouseapp.com/projects/8994/tickets/1112-redundant-utf-8-sequence-in-stringto_json

@jerboaa

This comment has been minimized.

Show comment
Hide comment
@jerboaa

jerboaa Apr 17, 2011

Member

Does anybody know what steps are required to reproduce this?

Member

jerboaa commented Apr 17, 2011

Does anybody know what steps are required to reproduce this?

@jerboaa

This comment has been minimized.

Show comment
Hide comment
@jerboaa

jerboaa May 7, 2011

Member

I don't think this is a rails issue. It's more likely that some file got uploaded and the encoding got messed up. Then, when we try to render the files, problems start to surface. Here's what I'd propose:

  1. Use the cmess gem in order to make an educated guess what encoding an uploaded file uses
  2. Based on the guessed encoding from step 1, convert it to UTF-8 using Iconv.iconv

We may not be able to catch all errors, but may be able to catch more encoding errors than we currently do. Have a look at [1],[2] for more info.

[1] https://groups.google.com/group/rubyonrails-talk/browse_thread/thread/4c262908624f4083?hl=en
[2] http://prometheus.rubyforge.org/cmess/

Thoughts?

Member

jerboaa commented May 7, 2011

I don't think this is a rails issue. It's more likely that some file got uploaded and the encoding got messed up. Then, when we try to render the files, problems start to surface. Here's what I'd propose:

  1. Use the cmess gem in order to make an educated guess what encoding an uploaded file uses
  2. Based on the guessed encoding from step 1, convert it to UTF-8 using Iconv.iconv

We may not be able to catch all errors, but may be able to catch more encoding errors than we currently do. Have a look at [1],[2] for more info.

[1] https://groups.google.com/group/rubyonrails-talk/browse_thread/thread/4c262908624f4083?hl=en
[2] http://prometheus.rubyforge.org/cmess/

Thoughts?

@jerboaa

This comment has been minimized.

Show comment
Hide comment
@jerboaa

jerboaa May 7, 2011

Member

I've created #338 to implement this.

Member

jerboaa commented May 7, 2011

I've created #338 to implement this.

@jerboaa

This comment has been minimized.

Show comment
Hide comment
@jerboaa

jerboaa May 8, 2011

Member

I don't think this will make it into 0.10.0. Once #338 is implemented, this should be more or less fixed.

Member

jerboaa commented May 8, 2011

I don't think this will make it into 0.10.0. Once #338 is implemented, this should be more or less fixed.

@reidka reidka closed this Jan 19, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment