Workshop

[NickNaso]

Hi everyone I'm opening this issue with the intention to collect materials and idea for a N-API workshop that we can propose at NodeConf EU, Node Summit and Node+JS Interactive.

Below I report all materials that we have from the workshop that we did last year:

• Slides

• Tutorials:

  • Starting a new add-on module from scratch
  • Starting a new add-on module from scratch with ObjectWrap
  • Migrating an existing NAN module
  • Async Worker
  • Node pre-gyp

List of candidate modules to port available at goo.gl/9bksyw

[jschlight]

Title

Next Generation N-API: A hands-on workshop

Description

The world of Node native add-on development continues to rapidly evolve. After widespread adoption of N-API starting last year, the Node native add-on ecosystem is gaining momentum as the underlying technology and tools continue to advance.

This workshop, offered by experts from the N-API team, describes important new features essential to the performance of your native add-on. These new features include asynchronous operations which permit you to safely and reliably spread your work across multiple threads. Context awareness permits your add-on to be loaded multiple times without the multiple copies interfering with each other. Finally, adopting thread-safe functions insures your functions run reliably regardless of the thread from which they are called or in which they run.

Whether you are just starting out with native add-ons or are an old pro looking for an update on N-API, bring your project or work through the workshop tutorials under the helpful guidance of the N-API experts. You’ll leave this workshop with a solid understanding of these important new technologies and how they apply to your own projects. And perhaps, with a bit of luck, working code!

New Tutorials

• Asynchronous Operations
• Context Awareness
• Thread-safe Functions
• How to use reference to Function or Object

[NickNaso]

Maybe I prefer talking of asynchronous operations instead of AsyncWorker, because the AsyncWorker is not the only way to do this kind of task.

[jschlight]

Good point, @NickNaso. I've updated the Proposed Abstract comment.

[mhdawson]

So that we don't scare off people just starting out maybe add something like

Wether you just want to start out with native addons or are an old pro looking for an update on N-API, bring your project or work through ....

Otherwise looks great to me.

[jschlight]

2019 Conferences

• NodeConf 2019 – Kilkenny, Ireland – November 10-13, 2019 ☹️
  This workshop proposal has been declined by the conference organizers.
  • Workshop proposal submitted.
  • Includes Michael, Gabriel, Nicola, and Jim.
• Node+JS Interactive 2019 – Montreal, Canada – December 11-12, 2019 😊
  The workshop proposal has been accepted by the conference organizers.
  The birds of a feather proposal has been declined by the conference organizers.
  • Accepting both Workshops (2-4 hours) and Birds of a Feather (30-40 minutes)
  • Both a workshop and birds of a feather proposals have been submitted.
  • Submissions are open until June 14 extended to June 21.
  • Results will be announced July 31.

[jschlight]

I've updated the comment above to reflect the responses we've received from the conference organizers.

[mhdawson]

Node+JS Interactive - Workshop was accepted !
NodeConfEU - Declined.

Jim will start working on the new ones this week.

[NickNaso]

Hi @jschlight,
I post a little example that could be a base for a tutorial of how to use reference to Function or Object
ref.zip

[jschlight]

Workshop Promotion

Proposed OpenJS Foundation Release

Are you working on a Node native add-on? Or are you planning to? You should be aware of significant improvements to the N-API infrastructure including asynchronous operations, thread-safe functions, and context awareness. Gabriel Schulhof has a good overview of these improvements here. These topics and more will be covered in the Next Generation N-API Hands-on Workshop being offered by the N-API team at Node+JS Interactive 2019 in Montreal, Canada (December 11-12, 2019). In the meantime, be sure to check out the N-API Resource for the current set of N-API tutorials and best practices.

[mhdawson]

@jschlight sounds good. If you want to tweet that out I can then ask the Node.js account to retweet

[mhdawson]

@jschlight, @gabrielschulhof I notice that the workshop only lists Nicola and myself in the title/speakers. Is that because you've not accepted the speaker role yet? I have other talks so we should make sure you are listed as speakers (even if that means removing me) so that you get the free ticket...

[NickNaso]

I sent the email to the organiser and I left them the email of @jschlight in case they need to ask you your bio and other things.

[NickNaso]

@mhdawson @jschlight Now the workshop lists me and @jschlight .

[mhdawson]

:) @jschlight hope that helps you being able to make it.

[mhdawson]

@NickNaso will create slides for the workshop, starting with those we used the last time.

[mhdawson]

Stuff to cover in intro deck:

• Thread safe functions
• Worker progress
• Context awareness
• AsyncWorker improvements
• Function and Object references
• Building (pre-build, cmake etc.)

Call to join/get involved.

[mhdawson]

@NickNaso will paste in link for new deck, Gabriel to add slide for Context awareness.

[juanarbol]

I have two suggestions while the workshop is happening:

1. Explain that boilerplate required for running C/C++ code -it's more like and setup, no just boilerplate- (like binding.gyp, bindings package, etc) before jumping into Yeoman, It's good to have a context.

2. Instead of recommending attatch debugger to see what is happening inside, you could do it and explain a little bit what is going on.

[NickNaso]

Hi everyone,
I started working on the slides the work is not complete, but I provide to post the link here:
https://docs.google.com/presentation/d/1NNKhULBFOqJspy_VMZKP2W-kJTpw4DGsoOTKOEkKQts
I have question that I want to discuss today:

• Is it a good idea add section to explain how to organise a project for native add-on?
  
• Is it a good idea add an introductory part explaining what is a native add-on?

[juanarbol]

@NickNaso I think both are very nice ideas, I think also would be nice to explain binding.gyp file, explain all of that before jumping to Yeoman generator.

[jschlight]

@NickNaso We should keep the information you include for #367 (comment) in sync with https://napi.inspiredware.com/getting-started/first.html#Project-structure which is how the Yeoman generator creates a project.

I can add an introductory part explaining what is a native add-on to this page: https://napi.inspiredware.com/about/what.html#Node

[NickNaso]

@NickNaso We should keep the information you include for #367 (comment) in sync with https://napi.inspiredware.com/getting-started/first.html#Project-structure which is how the Yeoman generator creates a project.

I can add an introductory part explaining what is a native add-on to this page: https://napi.inspiredware.com/about/what.html#Node

I added slide n. 9 where I reported what native add-on is and in slide n. 10 I want explain how the all the things work in high level way. It's a good idea to write something about native add-on on the tutorial.

I will adapt slides n. 29 and 30 to be consistent with the output produced by the generator.

[NickNaso]

Hi everyone,
I provided to update the information about the list of native add-ons to use in the workshop. I underlined in red the add-ons that already have a porting to N-API.
Here the link: https://docs.google.com/spreadsheets/d/1HopFSLMs5gj3G6_eBEZM2ET7oXKdokAF7VU2XZnzYGk
Feel free to add other add-ons to the list.

[rolftimmermans]

FYI regarding https://github.com/zeromq/zeromq.js, there is a pull request (by me) that implements the library in N-API. zeromq/zeromq.js#343

[NickNaso]

FYI regarding https://github.com/zeromq/zeromq.js, there is a pull request (by me) that implements the library in N-API. zeromq/zeromq.js#343

Thanks to report I missed it, sorry. I updated now.

[jschlight]

@NickNaso and others. Could I please get clarification on what topics should be included in the new Asynchronous Operations section that differ from those in the existing AsyncWorker section?

[NickNaso]

@NickNaso and others. Could I please get clarification on what topics should be included in the new Asynchronous Operations section that differ from those in the existing AsyncWorker section?

My idea was to create an example for the AsyncContext .

[jschlight]

New Workshop Sections

Section	Status
node-gyp	Ready for review.
CMake.js	Ready for review.
prebuild	Ready for review.
Object and function references	Ready for review.
Thread-safe functions	Ready for review.
Context awareness	Ready for review.

The links above take you directly to the preview site.

Besides reviews of the completed content, I can use help with the outlines of the remaining sections. Specifically, what topics should be covered along with a brief explanation. If I can get some guidance, I can take it from there.

It’s probably best to post comments here for everyone to see.

[NickNaso]

@jschlight I try to give you some opinions:

node-gyp

Older versions of node-gyp require Python 2.7. More recent verisons of node-gyp can also use Python 3.5+. The specifics can be found here.

It's not only Python 2.7 or 3.5+ that is requested to work with node-gyp, all depends on which operating system you are so I want be more generic and say that to use node-gyp developer needs to satisfy some requirements and link to the official documentation.

CMake.js

In addition to the entries required for any CMake build, additional entries are required when building natove modules.

Typo natove should be native.

When building a native module based on N-API, it is important to decalre the miniumen N-API version against which your modeule is designed to work. For CMake.js, this is accomplished by adding a line like this to the CMakeLists.txt file:

Typo modeule should be module.

Here is a complete CMakeLists.txt file for an N-API native module built on node-addon-api.:

I think that you need to remove the dot before the final colon.

[NickNaso]

Hi @jschlight,
I read the new section about the object reference and maybe it seems good.
I want change the Weak section https://preview.napi.inspiredware.com/special-topics/object-function-refs.html#Weak-Reference in something like this:

• Weak reference is a particular reference on which the internal reference count is initialised to 0. Weak reference are used to minimise the number of unnecessary objects in memory.

• Weak references my be used to solve the problem of circular references. Reference counting mechanism cannot collect cyclic reference because only one object may be collected at a time. Groups of mutually referencing objects which are not directly referenced by other objects could generate memory leaks.

@gabrielschulhof I'm not sure about the second point.

[KevinEady]

Hi @jschlight ,

I started working on the tsfn page. It is still in draft state... https://gist.github.com/KevinEady/112edab9a57e82bead0412a65a6c74ac Is there some way I can move this to some shared Markdown site we have / maybe edit that TSFN page directly?

Todo:

• Example: I don't know what best example to give, as there are so many different use cases and scenarios. For example, see around here: https://github.com/nodejs/node-addon-api/blob/24d75dd82f0206f4298588af063dc740cc297919/test/threadsafe_function/threadsafe_function_sum.js#L15-L26 Does anyone have a preference on which one to make an example of...? Which one would be most useful? Maybe the third one testAcquire can be example-ified
• i'd like to create a sequence diagram of the flow of TSFN creation or calling, since calling a TSFN can involve two different Napi::Functions and one native C++ method
• general cleanup and tidiness

[jschlight]

Thank you @KevinEady.

I've posted the contents of your gist to the preview tutorial site which you can see here: https://preview.napi.inspiredware.com/special-topics/thread-safe-functions.html

If it works for you, you can continue your work inside your gist and I can transfer it to the tutorial site when it's ready.

Up to this point, I've been maintaining the tutorial site myself. It's based on hexo. But now that we have more collaborators I'd like to explore the possibility of transferring the contents of the tutorial site to the auspices of the Node.js Foundation.

Perhaps this is something we can discuss after the workshop.

[mhdawson]

@KevinEady from the discussion in the N-API team meeting today, we figured the best example we already have: https://github.com/nodejs/node/tree/master/test/node-api/test_threadsafe_function
and the equivalent in node-addon-api (https://github.com/nodejs/node-addon-api/blob/master/test/threadsafe_function/threadsafe_function.cc).

We discussed that we should probably copy that test to create an example, but for now you can look at the test code (likely stripped down).

[KevinEady]

Hi @mhdawson ,

Sounds good! I'll work on that this week.

Kevin

[mhdawson]

@KevinEady thanks!

[KevinEady]

@mhdawson / @gabrielschulhof The provided examples use static data eg. https://github.com/nodejs/node-addon-api/blob/master/test/threadsafe_function/threadsafe_function.cc#L12-L13 ... Is this okay for the example? Otherwise I want to use ObjectWrap to encapsulate this data so it's no longer static.

[mhdawson]

It would probably be better to avoid static data unless it makes the result too complicated.

[NickNaso]

I ended to update the list of native modules to use for porting activities at workshop. Here the link: https://docs.google.com/spreadsheets/d/1HopFSLMs5gj3G6_eBEZM2ET7oXKdokAF7VU2XZnzYGk we have 45 add-ons.

[KevinEady]

Hi @jschlight I updated my gist with the example. Haven't touched the text yet though... Can the team review the example?

[jschlight]

Thank you @KevinEady.

I've posted your gist to the preview tutorial site.

Let's plan to ask for reviews during tomorrow's weekly call.

[NickNaso]

Preview of new design for the slides:
presentazione-opt.pdf

[mhdawson]

I like the new design. I think we should add another page at the end which has the links for the workshop tutorials so that we can put that up to get people started.

Then maybe another page which has the link to the page where you have the modules that need to be ported so that we can put that up later on in the workshop?

[NickNaso]

@mhdawson Ok i will provide to do that.
@jschlight will the final link for the workshop be https://napi.inspiredware.com/?

[NickNaso]

Hi everyone,
I just shared the new format for the slides.
@mhdawson I added the slides that you asked (slide number 50).
@gabrielschulhof May I ask you to take a look at image on slide number 26, it could be good have the text in white.
Let me know if you want change something or you can do by yourself.

[jschlight]

@mhdawson @gabrielschulhof @NickNaso @KevinEady

Hi all --

The last two remaining tutorials are now complete and ready for your review. Please post any suggestions here and we can discuss any remaining issues on Monday.

Here's a tweet that could go out Friday to get participants ready for the workshop:

Are you planning to attend the N-API Workshop at Node+JS Interactive next week? Ready to jump in now? Get a head start with the tutorials we'll be using: https://bit.ly/2PhWSz5 The workshop promises to be a good one with experts from the N-API team to guide you. See you there!

Heads up: The link points to the preview tutorials which I'll make live for the workshop after our review.

[NickNaso]

Hi @jschlight,
I set the links and QR code on the slides number 50.

• This is the link for the tutorials http://bit.ly/2P7HfdK that points to https://napi.inspiredware.com/
• This is the link for the list of modules to port http://bit.ly/2P80VxT that points to https://docs.google.com/spreadsheets/d/1HopFSLMs5gj3G6_eBEZM2ET7oXKdokAF7VU2XZnzYGk
  We need to change the link on the slides or in the twitter message.
  I provided to upload the slides on the web site of Node+JSInteractive as they requested.

[mhdawson]

Latest version of the presentation for the workshop: https://docs.google.com/presentation/d/1kGzkj_4RqoE4YJ4I4XPdMLPcaiCHiJqLVL0nRBSUPhQ/edit

[jschlight]

The update tutorial pages are now live at https://napi.inspiredware.com/

[mhdawson]

Workshop delivered we can probably close the issue now.