Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial Setup #1

Merged
merged 101 commits into from
Feb 15, 2024
Merged

Initial Setup #1

merged 101 commits into from
Feb 15, 2024

Conversation

elcritch
Copy link
Contributor

This makes an async compatible queue, see queues.nim. Then jobs.nim builds on this with a type and macro for submitting jobs to taskpools and getting a result back via an async future. Finally tasks.nim automates the jobs wrapping and adds some type extra type checking.

Here's an example of the transformations for an "asyncTasklet":

proc addNums(a, b: float): float {.asyncTask.} =
  os.sleep(100)
  echo "adding: ", a, " + ", b
  return a + b

becomes:

proc addNumsTasklet(a, b: float): float {.nimcall.} =
  os.sleep(100)
  echo "adding: ", a, " + ", b
  return a + b

proc addNums(jobResult: JobResult[float]; a, b: float) {.nimcall.} =
  let val {.inject.} = addNumsTasklet(checkParamType(a), checkParamType(b))
  discard jobResult.queue.send((jobResult.id, val))

Calling addNums:

    let res = await jobs.submit(addNums(1.0, 2.0,))

becomes:

let (jobRes_503318661, fut_503318662) = createFuture(jobs, "addNums(1.0, 2.0)")
jobs.taskpool.spawn(addNums(jobRes_503318661, 1.0, 2.0))
fut_503318662

@elcritch
Copy link
Contributor Author

Hey @tbekas finally got a PR up. It's fairly complete but there's a few corner cases to figure out.

The openArray seem to work, but will need to verify the GC won't free the main GC data.

@elcritch
Copy link
Contributor Author

This will need more tests, stress testing to verify the threading. It'd be nice to do some basic benchmarking as well.

Also the API's likely need some changes. In particular the jobs newJobQueue should probably return a result type.

Copy link

@tbekas tbekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall a great job 👍 This PR would use some cleanup of the commented and unused code as well as some echos.

apatheia.nimble Outdated Show resolved Hide resolved
src/apatheia/jobs.nim Outdated Show resolved Hide resolved
@elcritch elcritch merged commit 5c3b379 into main Feb 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants