Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could not find how to perform per-thread initialization e.g. for sqlite #194

Open
cpbotha opened this issue Apr 8, 2019 · 7 comments

Comments

Projects
None yet
3 participants
@cpbotha
Copy link

commented Apr 8, 2019

In sqlite3's MULTITHREAD mode, one has to use a separate database connection for each thread.

I've gone through the jester and httpbeast source, but could not find where / how to perform per-thread init, for example to open a sqlite3 connection for each thread.

Unless I have misunderstood, I need a { .threadVar. } db connection variable, but then have each thread init that separately. I don't see any places where I could easily hook this up, but I would love to be corrected!

(alternatively, one could opt to open the sqlite3 database in SERIALIZED mode to share the same connection option over threads, but the nim sqlite3 wrapper does not yet support the pre-requisite function calls (I might try to PR this at some point). Read more about sqlite3 threading modes here: https://www.sqlite.org/threadsafe.html

The default mode depends on sqlite3 compilation options. Here on my dev mac, it was compiled with SQLITE_CONFIG_THREADSAFE=2, which means a default of MULTITHREAD and not SERIALIZED. I would have to configure at runtime with either sqlite3_config() or preferably sqlite3_open_v2())

@Araq

This comment has been minimized.

Copy link
Contributor

commented Apr 9, 2019

Is Jester even creating threads under the hood? That's news to me. If it doesn't, you're free to setup the threads on your own and setting up the SQlite connections. Right?

@cpbotha

This comment has been minimized.

Copy link
Author

commented Apr 9, 2019

When the jester app is compiled with --threads:on, httpbeast's run() will spawn countProcessor() threads, see https://github.com/dom96/httpbeast/blob/master/src/httpbeast.nim#L415

It would be useful to be able to perform some sort of per-thread init from the jester side, e.g. creating a sqlite connection, amongst others.

@dom96

This comment has been minimized.

Copy link
Owner

commented Apr 9, 2019

It would be useful to perform per-thread init from the Nim side (I've said this multiple times now too). Otherwise every library will need to provide hooks into the thread startup which will be a PITA.

@dom96

This comment has been minimized.

Copy link
Owner

commented Apr 9, 2019

Anyway, you can create a proc that initializes these thread vars for you if they are nil. Just create a threadvar for your connection and a proc like:

proc getSqliteConnection() =
  if sqliteConnection.isNil:
    # Initialise
  return sqliteConnection
@Araq

This comment has been minimized.

Copy link
Contributor

commented Apr 10, 2019

I've said this multiple times now too

And I've said it multiple times now too, been there, done that, it's more complex than it looks and encourages data races.

@dom96

This comment has been minimized.

Copy link
Owner

commented Apr 10, 2019

And I've said it multiple times now too, been there, done that, it's more complex than it looks and encourages data races.

Can you elaborate and explain how it encourages data races and why it is complex?

@Araq

This comment has been minimized.

Copy link
Contributor

commented Apr 10, 2019

For example, threadpools creates the worker threads at "init" time, then later modules are initialized that call something like atThreadCreation(...) and by then it's too late, the threads are started and cannot see that they should have run the registered callbacks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.