2.2.19 breaks on tables with non-ASCII chars in column names

[psy-q]

As reported via Google Group: http://groups.google.com/group/rails-sqlserver-adapter/browse_thread/thread/b8037769e73a391c

Thanks :)

RegexpError (premature end of regular expression: /([\w-]*)\s+as\s
+AnlassLektionPeriodizit t/):
/usr/lib/ruby/gems/1.8/gems/activerecord-sqlserver-adapter-2.2.19/
lib/active_record/connection_adapters/sqlserver_adapter.rb:1004:in
views_real_column_name' /usr/lib/ruby/gems/1.8/gems/activerecord-sqlserver-adapter-2.2.19/ lib/active_record/connection_adapters/sqlserver_adapter.rb:1076:in column_definitions'
/usr/lib/ruby/gems/1.8/gems/activerecord-sqlserver-adapter-2.2.19/
lib/active_record/connection_adapters/sqlserver_adapter.rb:1062:in
collect' /usr/lib/ruby/gems/1.8/gems/activerecord-sqlserver-adapter-2.2.19/ lib/active_record/connection_adapters/sqlserver_adapter.rb:1062:in column_definitions'
/usr/lib/ruby/gems/1.8/gems/activerecord-sqlserver-adapter-2.2.19/
lib/active_record/connection_adapters/sqlserver_adapter.rb:618:in
columns' /usr/lib/ruby/gems/1.8/gems/activerecord-sqlserver-adapter-2.2.19/ lib/active_record/connection_adapters/sqlserver_adapter.rb:1109:in special_columns'
/usr/lib/ruby/gems/1.8/gems/activerecord-sqlserver-adapter-2.2.19/
lib/active_record/connection_adapters/sqlserver_adapter.rb:1113:in
repair_special_columns' /usr/lib/ruby/gems/1.8/gems/activerecord-sqlserver-adapter-2.2.19/ lib/active_record/connection_adapters/sqlserver_adapter.rb:815:in select'
app/models/evento/sync.rb:10:in write_course' app/models/course.rb:90:insync_write'
app/controllers/courses_controller.rb:80:in `update'

Is there any way to debug this further or is there anything else that
would help you solve this or narrow down where the problem is? We are
using:

UNIXODBC 2.2.11
FreeTDS 0.82
Debian GNU/Linux 5
Rails and AR 2.3.2

[metaskills]

Just getting to looking at bugs. Have you defined a regex that addresses this?

[psy-q]

We couldn't figure out exactly why and where it failed (on the UTF-8 characters? On some UTF-8 settings?) so we simply excluded that field from the list of fields passed to the adapter. It's only a workaround, but it works for now.

I'm sorry we're not more useful in this :(

[metaskills]

I'll keep this open and hopefully someone can post more info

[metaskills]

FYI, I tried last night to add a test for this and if failed badly. Maybe someone can pickup up where I left off and debug this? Here is what I did. First I changed the columns method to create a SQLServerColumn with a name that was correctly encoded. See below.

def columns(table_name, name = nil)
#...
sqlserver_options = ci.except(:name,:default_value,:type,:null)
column_name = SQLServerColumn.string_to_utf8_encoding(ci[:name])
SQLServerColumn.new column_name, ci[:default_value], ci[:type], ci[:null], sqlserver_options

Then in unicode_test_sqlserver.rb I created a new context "Testing unicode column names" and added a column to the our unicode test table like so.

SqlServerUnicode.connection.add_column :sql_server_unicodes, 'arcångel', :string

Then started writing some assertions. I found that I was making headway and getting the column definition for the name back in unicode, but all other insert/update SQL statements were failing due to some 8BIT-ASCII/UTF-8 mix match. Perhaps someone can pick this up and dig around more?

[KDGundermann]

Hi Ken,

Perhaps someone can pick this up and dig around more?
Yes I am willing to do so, BUT currently I have some problems to run
rake test on Windows...

[KDGundermann]

FYI
http://yehudakatz.com/2010/05/17/encodings-unabridged/

"The problem of incompatible encodings is likely to happen in Western applications only when combining ISO-8859-* data with Unicode data"

The vast majority of encoding bugs to date have resulted from outdated drivers that returned BINARY data instead of Strings with proper encoding tags.

[metaskills]

Old and closing