Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

possibility to set up manual connection charset on connect #9

Open
EugenyPavlenko opened this issue May 28, 2012 · 31 comments

Comments

Projects
None yet
5 participants
@EugenyPavlenko
Copy link

commented May 28, 2012

Hello

if there any possibility to setup manual connection CHARSET on connect?
It seems by default it is UTF8, that doesn't go for some projects...

best wishes!

@xdenser

This comment has been minimized.

Copy link
Owner

commented May 28, 2012

As for now it is not possible.
What for do you need other CHARSET? You will not be able to work correctly with strings in JavaScript.
libfbclient makes transcoding for you.
Other connetion charset support will require to return node buffers for all char, varchar fields.

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented May 28, 2012

Client's default FB database charset is set to WIN1251 (cyrilic),
Also columns in tables have win1251 charset, and the collate also :)

But, the fact, that data in DB is in win1257 (baltic) charset.

(to change the charset of all DB, on all tables in DB is NOT possible because it is big and they use it like that, I mean their software is done to work with win1257 charset...)

So, I have to deal with 3 charsets:

  1. win1251 - the default charset of DB,
  2. win1257 - the charset of data stored in that DB,
  3. utf8 - the charset of my website, that is meant to get data from nodejs and show them :).
    (nodejs uses your module to get data from FB DB)

In PHP, to work all fine, I do the following:

  1. connecting to DB using win1251
  2. converting received data from win1257 to utf8
  3. showing the data on the page!
    It works fine!

After long hour of investigation, I found out, that libfbclient module using utf8 by default.
In fact, it means, that by the moment nodejs server sends data taken from FB — it is ALREADY broken (because of "transcoding" and/or utf8).
So in my page I see invalid characters.

I found the sting nr. 53 in your software:
char lc_type = "WIN1251"; / UTF8 */

Changed utf8 to win1251 with hope that it will help me.
It didn't help. I suppose because all your other code is written with utf8 in mind :)

I also tried "iconv" module to convert data taken from FB with your module, but now I thing it was no sense because at the moment it was get by module — it was broken by transcoding, because of difference of charsets of DB and Data (or maybe simply because and is not utf8).

P.S: iconv also didn't help.

In my situation, may you give some advice how to deal with charset...?

Thank you very much!

@xdenser

This comment has been minimized.

Copy link
Owner

commented May 28, 2012

Well to read non utf8 charsets fetch, fetchSync methods should return Buffers not strings.
That need major changes in fb-bindings-fbresult.cc.

For example for varchar field type line 392-394 may be changed to

vary2 = (PARAMVARY*) var->sqldata;
node::Buffer *slowBuffer = node::Buffer::New(vary2->vary_length);
memcpy(node::Buffer::Data(slowBuffer), (const char*)(vary2->vary_string), vary2->vary_length);
v8::Local<v8::Object> globalObj = v8::Context::GetCurrent()->Global();
v8::Local<v8::Function> bufferConstructor = v8::Local<v8::Function>::Cast(globalObj->Get(v8::String::New("Buffer")));
v8::Handle<v8::Value> constructorArgs[3] = { slowBuffer->handle_, v8::Integer::New(vary2->vary_length), v8::Integer::New(0) };
js_field = bufferConstructor->NewInstance(3, constructorArgs);

that should return you buffers for varchar fields. You may apply iconv to buffers then. For char() fields something similar should be done. But things become more complex when you need updates. Your only option is to use prepared statements and to patch FBResult::set_params method to use buffers instead of strings.

But anyway it will be custom patch - It will never go into master branch, Better way is to provide whole record buffer to JavaScript code and parse it in javascript. But this requires many changes to API - it will not be done fast.

And offcourse all this needs string nr. 53 to be patched too
char lc_type = "WIN1251"; / UTF8 */

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented May 29, 2012

Hello,

I changed the code like so:
case SQL_VARYING:
/*
vary2 = (PARAMVARY_) var->sqldata;
vary2->vary_string[vary2->vary_length] = '\0';
js_field = String::New((const char_)(vary2->vary_string));
/
vary2 = (PARAMVARY
) var->sqldata;
node::Buffer slowBuffer = node::Buffer::New(vary2->vary_length);
memcpy(node::Buffer::Data(slowBuffer), (const char
)(vary2->vary_string), vary2->vary_length);
v8::Localv8::Object globalObj = v8::Context::GetCurrent()->Global();
v8::Localv8::Function bufferConstructor = v8::Localv8::Function::Cast(globalObj->Get(v8::String::New("Buffer")));
v8::Handlev8::Value constructorArgs[3] = { slowBuffer->handle_, v8::Integer::New(vary2->vary_length), v8::Integer::New(0) };
js_field = bufferConstructor->NewInstance(3, constructorArgs);
break;

        case SQL_SHORT:

Old code 392-394 commented, and a patch-code added after, but while compiling it gaves error:

web@webcaurlaides:/var/www/protected/nodejs/node_modules/firebird$ node-waf configure build
Checking for program g++ or c++ : /usr/bin/g++
Checking for program cpp : /usr/bin/cpp
Checking for program ar : /usr/bin/ar
Checking for program ranlib : /usr/bin/ranlib
Checking for g++ : ok
Checking for node path : not found
Checking for node prefix : ok /usr
Checking for program fb_config : not found
'configure' finished successfully (0.130s)
Waf: Entering directory /var/www/protected/nodejs/node_modules/firebird/build' [5/8] cxx: src/fb-bindings-fbresult.cc -> build/Release/src/fb-bindings-fbresult_1.o ../src/fb-bindings-fbresult.cc: In static member function āstatic v8::Local<v8::Value> FBResult::GetFieldValue(XSQLVAR*, Connection*)ā: ../src/fb-bindings-fbresult.cc:406:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:27: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:29: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:27: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:19: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:407:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:27: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:29: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:27: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:19: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:408:11: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:27: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:29: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:27: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:19: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:450:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:27: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:29: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:27: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:19: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:454:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:27: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:29: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:27: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:19: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:458:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:27: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:29: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:27: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:19: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:480:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:27: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:29: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:27: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:19: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:516:17: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:27: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:29: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:27: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:19: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:539:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:27: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:29: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:27: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:19: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:540:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:27: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:29: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:27: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:19: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc: In destructor āvirtual FBResult::~FBResult()ā: ../src/fb-bindings-fbresult.cc:886:131: warning: format ā%dā expects argument of type āintā, but argument 3 has type āISC_STATUS {aka long int}ā [-Wformat] ../src/fb-bindings-fbresult.cc:886:131: warning: format ā%dā expects argument of type āintā, but argument 4 has type ālong intā [-Wformat] ../src/fb-bindings-fbresult.cc:891:17: warning: converting to non-pointer type āisc_stmt_handle {aka unsigned int}ā from NULL [-Wconversion-null] Waf: Leaving directory/var/www/protected/nodejs/node_modules/firebird/build'
Build failed: -> task failed (err #1):
{task: cxx fb-bindings-fbresult.cc -> fb-bindings-fbresult_1.o}
web@webcaurlaides:/var/www/protected/nodejs/node_modules/firebird$

Unfortunately my C++ knowledge is poor to fix that error...

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented May 29, 2012

But things become more complex when you need updates.

For the luck, in my node server I need only SELECT statements, so if the custom patch will finally be compilated, it would be a great save for me))

@xdenser

This comment has been minimized.

Copy link
Owner

commented May 29, 2012

Try w/o namespaces

vary2 = (PARAMVARY*) var->sqldata;
Buffer *slowBuffer = Buffer::New(vary2->vary_length);
memcpy(Buffer::Data(slowBuffer), (const char*)(vary2->vary_string), vary2->vary_length);
Local<Object> globalObj = Context::GetCurrent()->Global();
Local<Function> bufferConstructor = Local<Function>::Cast(globalObj->Get(String::New("Buffer")));
Handle<Value> constructorArgs[3] = { slowBuffer->handle_, Integer::New(vary2->vary_length), Integer::New(0) };
js_field = bufferConstructor->NewInstance(3, constructorArgs);
@xdenser

This comment has been minimized.

Copy link
Owner

commented May 29, 2012

Ok I'll try to make the changes, In saparate branch.

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented May 29, 2012

New code:

case SQL_VARYING:
/*
vary2 = (PARAMVARY_) var->sqldata;
vary2->vary_string[vary2->vary_length] = '\0';
js_field = String::New((const char_)(vary2->vary_string));
/
vary2 = (PARAMVARY
) var->sqldata;
Buffer slowBuffer = Buffer::New(vary2->vary_length);
memcpy(Buffer::Data(slowBuffer), (const char
)(vary2->vary_string), vary2->vary_length);
Local globalObj = Context::GetCurrent()->Global();
Local bufferConstructor = Local::Cast(globalObj->Get(String::New("Buffer")));
Handle constructorArgs[3] = { slowBuffer->handle_, Integer::New(vary2->vary_length), Integer::New(0) };
js_field = bufferConstructor->NewInstance(3, constructorArgs);
break;

Gave the following on compilation:

node-waf configure build
Checking for program g++ or c++ : /usr/bin/g++
Checking for program cpp : /usr/bin/cpp
Checking for program ar : /usr/bin/ar
Checking for program ranlib : /usr/bin/ranlib
Checking for g++ : ok
Checking for node path : not found
Checking for node prefix : ok /usr
Checking for program fb_config : not found
'configure' finished successfully (1.234s)
Waf: Entering directory /var/www/protected/nodejs/node_modules/firebird/build' [5/8] cxx: src/fb-bindings-fbresult.cc -> build/Release/src/fb-bindings-fbresult_1.o ../src/fb-bindings-fbresult.cc: In static member function āstatic v8::Local<v8::Value> FBResult::GetFieldValue(XSQLVAR*, Connection*)ā: ../src/fb-bindings-fbresult.cc:406:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:19: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:21: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:19: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:13: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:407:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:19: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:21: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:19: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:13: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:408:11: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:19: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:21: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:19: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:13: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:450:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:19: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:21: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:19: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:13: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:454:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:19: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:21: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:19: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:13: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:458:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:19: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:21: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:19: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:13: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:480:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:19: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:21: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:19: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:13: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:516:17: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:19: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:21: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:19: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:13: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:539:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:19: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:21: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:19: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:13: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc:540:18: error: jump to case label [-fpermissive] ../src/fb-bindings-fbresult.cc:402:19: error: crosses initialization of āv8::Handle<v8::Value> constructorArgs [3]ā ../src/fb-bindings-fbresult.cc:401:21: error: crosses initialization of āv8::Local<v8::Function> bufferConstructorā ../src/fb-bindings-fbresult.cc:400:19: error: crosses initialization of āv8::Local<v8::Object> globalObjā ../src/fb-bindings-fbresult.cc:398:13: error: crosses initialization of ānode::Buffer* slowBufferā ../src/fb-bindings-fbresult.cc: In destructor āvirtual FBResult::~FBResult()ā: ../src/fb-bindings-fbresult.cc:886:131: warning: format ā%dā expects argument of type āintā, but argument 3 has type āISC_STATUS {aka long int}ā [-Wformat] ../src/fb-bindings-fbresult.cc:886:131: warning: format ā%dā expects argument of type āintā, but argument 4 has type ālong intā [-Wformat] ../src/fb-bindings-fbresult.cc:891:17: warning: converting to non-pointer type āisc_stmt_handle {aka unsigned int}ā from NULL [-Wconversion-null] Waf: Leaving directory/var/www/protected/nodejs/node_modules/firebird/build'
Build failed: -> task failed (err #1):
{task: cxx fb-bindings-fbresult.cc -> fb-bindings-fbresult_1.o}
web@webcaurlaides:/var/www/protected/nodejs/node_modules/firebird$

I compared some peace of that output just to find out if it is different from previous (just to make sure I made changes in source, and the compiler doe not cache anything) — they are different.

@xdenser

This comment has been minimized.

Copy link
Owner

commented May 29, 2012

Try this commit

fc6a54b

it compiles for me.

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented May 30, 2012

Thank you very much, it was compiled for me also (with some "warnings")!
Now it works, so that symbols are normally converted to utf8 AND on the clients it is desired results!!

But unfortunately, every time, after 1-2 min. of work — nodejs fails with "Segmentation fault", after that patch...

What I do in my server?

  1. Doing THE SAME select query to FB server, that returns an object (FBResult?) every 3 seconds
  2. Converting pObj.FFIRST and pObj.FLAST fields from win1257 to utf8 with my func "myConv"

function myConv( pObj, from, to ) {

var obj = {};

obj.dir = from + ' -> ' + to;

try {
    var iconv = new Iconv(from, to);

    //var body = new Buffer(objCH.FFIRST);
    var FFIRST = pObj.FFIRST;
    obj.name = iconv.convert(FFIRST).toString();

    var FLAST = pObj.FLAST;
    obj.surname = iconv.convert(FLAST).toString();

} catch (err) {
    console.log( "myConv error:" + err.message );
    console.log( "myConv direc:" + from + ' -> ' + to );
}

return obj;

}

setInterval(function() {
var chData = getCardHoldersData(5); // it does SQL query by Id=5 to FB and returns object (FBResult?) as result
var myVar = 'Это тест русского!';

arr.push( myConv( chData, 'CP1257', 'UTF-8' ) );

//... broadcasting to clients after all work done
broadcast({
channel: REDIS_CHANNEL,
code: 300,
act: 'RefreshBlock',
name: chData.FFIRST,
surname: chData.FLAST,
poRusski: myVar,
convertations: arr
});

}, 3000);

All works fine instead of "Segmentation fault".
Maybe this will help:
http://stackoverflow.com/questions/9762616/why-does-this-crash-nodejs

Also, I can check (test) your new patches, if needed.

@xdenser

This comment has been minimized.

Copy link
Owner

commented May 30, 2012

You may try to remove/comment HandleScope declarations.

HandleScope scope1;

lines 392,412
in fb-bindings-fbresult.cc

I am unsure if I am using them correctly.

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented May 30, 2012

I tried that, but the same "Seg fault" appears.

Also, I removed the call on iconv, just to be sure it is not a fault of iconv.

The logs are empty also...

@xdenser

This comment has been minimized.

Copy link
Owner

commented May 30, 2012

try this commit
530c7e1

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented Jun 1, 2012

Started my server yesterday in the evening — still working!

thank you very much!

Can you please leave that issue for users who will have the same need as me (not to have "utf8" charset at connection) ? They will use that code, branch for patching...

@xdenser

This comment has been minimized.

Copy link
Owner

commented Jun 1, 2012

You also may now try a very perspective pure JS solution from here

https://github.com/hgourvest/node-firebird

thought it has no synchronous versions of methods and possibly should be patched too for your case.

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented Jun 5, 2012

hello!

testing a new patch on my server, it is find out that it also fails with "Segmentation fault" after 20-30 min of work... :(

@xdenser

This comment has been minimized.

Copy link
Owner

commented Jun 5, 2012

Please show your db related code as gist. It has syntax high lighting. From code I have seen in other issue - you are creating connections constantlym but not colsing them. Either use single cnnection and transactions or connect every time with new connection, but close it after. If you do not close connections - you have memory leak.

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented Jun 5, 2012

https://gist.github.com/2876204
Starts from latest line: "fbGetAndPushEvents();" (253)

My new code establish 1 connection (actually 2, because I have to take data from 2 different DB).
And in every function I do before SQL:
if(!conn.inTransaction)
conn.startTransactionSync();

And after quering I do:
conn.commitSync();

What is interesting, before that changes (connected with starting and committing transactions), I was leaving server for 1 day and night and it was totally fine:
"dzuchara commented 4 days ago
Started my server yesterday in the evening — still working!"

So, it seams seg fault started after "transaction changes"...

Maybe I should:

  1. Create 2 connections (1 for 1 DB)
  2. Do all QUERY stuff
  3. close all 2 connections calling:
    fbConn.disconnect();
    fbConnOuter.disconnect();
    fbConn = null;
    fbConnOuter = null;

And on the next iteration (I mean: "setTimeout(fbGetAndPushEvents, fbGetAndPushEventsTimeout);") — I will do a new connections again and so on?
In that case I should not deal with "transactions", right ?

Is it critical to create a 2 new connections every 900 ms and close them after some 100 ms??

If creating the connection to FB does not take much time and CPU, I will try, because have something to do with seg fault...

@xdenser

This comment has been minimized.

Copy link
Owner

commented Jun 5, 2012

меня смущает этот комментарий

// если не делать эту проверку то при комите выдает ошибку чтото типа "cannot handle transaction..."

очень странно

а все понял
рекомендую делать
conn.commitSync();

сразу после

var rows = rs.fetchSync("all", true);

там же внутри цикла перебора идут еще обращения к базе.
хотя нет

совсем правильно начать транзакцию в начале вызова большой функции
и закончить в конце. а все промежуточные вызовы должны идти в рамках этой же транзакции
но на SegFault это все равно влиять не должно - это скорее на целостность данных в БД влияет.
т.е. где то так

странно что падает с SegFault
у меня есть место где похожий код на одном коннекте висит несколько месяцев и запросы идут в течение нескольких миллисекунд.
скорее это связано с патчем

попробую повторить у себя

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented Jun 5, 2012

Там есть СКЛ которые выполняются когда новый клиент присоединяется к серверу — это одно.

А есть общий цикл который каждые Х секунд выполняет некоторые СКЛ по разным таблицам — это другое (об этом цикле вы писали).

Я сейчас сделал следующее — оставил только создание и комит транзакций в главном цикле, и оставил сервер крутится на ночь, посмотрим, что будет на утро.

@xdenser

This comment has been minimized.

Copy link
Owner

commented Jun 5, 2012

Ничего не будет
у меня такой код

var conn = fb.createConnection();
conn.connectSync(cfg.db, cfg.user, cfg.password, cfg.role);

var cnt = 0;
function doIt(){

if(!conn.inTransaction) conn.startTransactionSync();
var res = conn.querySync("select * from test")

var rows = res.fetchSync("all",true);
conn.commitSync();
console.log(rows);
console.log(cnt++);

setTimeout(doIt,10);

}

doIt();

тоже дает SegFault
иногда после 100 итераций
иногда после 10000+ итераций
пока не могу отловить где
так что рекомендую использовать forever

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented Jun 6, 2012

Только что проверил, да, меня тоже вылетел с Segmentation fault...

Попробую сейчас сделать с conn.disconnect();

так что рекомендую использовать forever

А что значит "forever" ?

@xdenser

This comment has been minimized.

Copy link
Owner

commented Jun 6, 2012

Модуль для "демонизации", заодно рестарт приложения при падении

npm install forever -g
forever start server.js

https://github.com/nodejitsu/forever

а с SEGFAULT проблема однозначно в патче, потому что мастер ветка не падает
работу с буфером вроде сделал правильно... пока не понимаю в чем проблема, запуск под дебаггером ситуацию не прояснил, тут какие-то тонкости работы с v8.

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented Jun 6, 2012

В варианте без транзакций и с conn.disconnect(); он проработал 4 часа и потом вылетел с SEGFAULT, то есть по моему мнению стабильней чем с транзакциями и постоянным открытым соединением.. запустил еще раз..

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented Jun 6, 2012

Да, падать в дисконект-варианте он стал немного поменьше, но все равно падает.

xdenser added a commit that referenced this issue Jun 6, 2012

@xdenser

This comment has been minimized.

Copy link
Owner

commented Jun 6, 2012

Все это работает нормально. Более 100 тыс. итераций ни одного падения. Спасибо Nathan Rajlich.

@EugenyPavlenko

This comment has been minimized.

Copy link
Author

commented Jun 7, 2012

Спасибо Вам большое Денис! За последнюю ночь и за сегоднешний день еще ни разу не вылетел мой сервер. На всякий случай, для надежности запущу его под forever в продакшене!

Удачного дня!

@kakaki

This comment has been minimized.

Copy link

commented Jan 13, 2017

We have similar problem, trying to run Interbase with library from firebird, it's working in UTF8 but ours database is in WIN1250, and there are errors on field that are in Polish chars (about field to long).

We have tried to run branch cp1251 - but it's not working with current version of node , is there any chance to update this branch to current version of node ?

@mariuz

This comment has been minimized.

Copy link
Contributor

commented Jan 13, 2017

best way is to migrate database to UTF-8 with fbclone https://github.com/zedalaye/fbclone , also this lib is for Firebird

@Claud

This comment has been minimized.

Copy link

commented Oct 17, 2017

Подскажите как собрать код из ветки cp1251, на node.js 4.6.2x32? Сборщик за пару лет изменился на node-gyp пробовал изменить сбоку, по аналогии с тем что в последнем релизе модуля, но не помогает.

@xdenser

This comment has been minimized.

Copy link
Owner

commented Oct 23, 2017

Sorry for late answer

Check out
332f1cb
change in fb-bindings-connection.h

char *lc_type = const_cast<char *>("UTF8");
bool isUTF8lctype = true;

to

char *lc_type = const_cast<char *>("WIN1251");
bool isUTF8lctype = false;

and rebuild.
I will add JS api to change this later

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.