Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add contributing quickstart guide #3496

Merged
merged 7 commits into from
Oct 2, 2023
Merged

Add contributing quickstart guide #3496

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
fd542a2
Add contributing quickstart guide
tanishiking Sep 21, 2023
a82a1ee
Add URL to sbt publish from "publish locally"
tanishiking Sep 28, 2023
b17677f
Update quick start guide
tanishiking Sep 28, 2023
fcb9df6
Mention MyScalaNativePlugin and scriptedBufferLog
tanishiking Sep 28, 2023
1e57fea
Add disclaimer for test-all
tanishiking Sep 28, 2023
5ce4b22
Remove test-all disclaimer
tanishiking Sep 29, 2023
57b7c44
Add built date to quickstart guide
tanishiking Sep 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
97 changes: 2 additions & 95 deletions docs/contrib/build.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,66 +8,8 @@ This section gives some basic information and tips about the build system. The
components of Scala Native. The ``build.sbt`` file is at the root of the project
along with the sub-projects that make up the system.

Common sbt commands
-------------------
Once you have cloned Scala Native from git, ``cd`` into the base directory and
run ``sbt`` to launch the sbt build. Inside the sbt shell, the most common
commands are the following:

- ``sandbox/run`` -- run the main method of the `sandbox` project
- ``tests/test`` -- run the unit tests
- ``tools/test`` -- run the unit tests of the tools, aka the linker
- ``sbtScalaNative/scripted`` -- run the integration tests of the sbt plugin
(this takes a while)
- ``clean`` -- delete all generated sources, compiled artifacts, intermediate
products, and generally all build-produced files
- ``reload`` -- reload the build, to take into account changes to the sbt plugin
and its transitive dependencies

If you want to run all the tests and benchmarks, which takes a while, you can
run the ``test-all`` command, ideally after ``reload`` and ``clean``.

Normal development workflow
---------------------------
Let us suppose that you wish to work on the ``javalib`` project to add some code
or fix a bug. Once you make a change to the code, run the following command
at the sbt prompt to compile the code and run the tests:

.. code-block:: text

> tests/test

You can run only the test of interest by using one of the following commands:

.. code-block:: text

> tests/testOnly java.lang.StringSuite
> tests/testOnly *StringSuite

Scripted tests are used when you need to interact with the file system,
networking, or the build system that cannot be done with a unit test. They
are located in the `scripted-tests` directory.

Run all the scripted tests or just one test using the following examples respectively.
To run an individual test substitute the test to run for `native-code-include`:

.. code-block:: text

> sbtScalaNative/scripted
> sbtScalaNative/scripted run/native-code-include

Some additional tips are as follows.

- If you modify the ``nscplugin``, you will need to ``clean`` the project that
you want to rebuild with its new version (typically ``sandbox/clean`` or
``tests/clean``). For a full rebuild, use the global ``clean`` command.

- If you modify the sbt plugin or any of its transitive dependencies
(``sbt-scala-native``, ``nir``, ``util``, ``tools``, ``test-runner``), you
will need to ``reload`` for your changes to take effect with most test
commands (except with the ``scripted`` tests).

- For a completely clean build, from scratch, run ``reload`` *and* ``clean``.
- ``project/Build.scala`` defines the sub-projects
- ``project/Commands.scala`` defines the custom commands such as ``test-all``

Build settings via environment variables
--------------------------------------------------
Expand Down Expand Up @@ -134,22 +76,6 @@ The following shows how to set ``nativeGC`` on all the projects.

The same process above will work for setting `nativeMode`.

Locally publish to test in other builds
---------------------------------------
If you need to test your copy of Scala Native in the larger context of a
separate build, you will need to locally publish all the artifacts of Scala
Native.

Use the custom sbt command to publish all projects for a specific Scala version (`x,y,z`):

.. code-block:: text

> publish-local-dev x.y.z

Afterwards, set the version of `sbt-scala-native` in the target project's
`project/plugins.sbt` to the current SNAPSHOT version of Scala Native, and use
normally.

Organization of the build
-------------------------
The build has roughly five groups of sub-projects as follows:
Expand Down Expand Up @@ -260,25 +186,6 @@ if it's not the present script will try to use directory with corresponding Scal
or it would try to use Scala epoch version or `overrides` directory. If none of these directories exists it will fail.
It is also possible to define explicitly overrides directory to use by passing it as the third argument to the script.


Locally publish docs
---------------------------------------
Follow the steps after cloning the `scalanative <https://github.com/scala-native/scala-native>`_ repo and changing to `scala-native` directory.

1. First time building the docs. This command will setup & build the docs.

.. code-block:: text

$ bash scripts/makedocs setup

2. If setup is already done. This command will only build the docs assuming setup is already done.

.. code-block:: text

$ bash scripts/makedocs

3. Navigate to ``docs/_build/html`` directory and open ``index.html`` file in your browser.

The next section has more build and development information for those wanting
to work on :ref:`compiler`.

Expand Down
9 changes: 6 additions & 3 deletions docs/contrib/ides.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
.. _ides:

IDE setup
=========

Metals
======
------
Metals import should work out of the box for most of the modules, it's the recommended IDE.
To speed up indexing and prevent Bloop-related issues by default we export only 1 version of ``MultiScalaProject``, otherwise it would need to cross-compile sources for all binary Scala versions on each source-change.
By default IDE would target Scala 3 projects, to change this behavior modify ``project/MyScalaNativePlugin.scala`` and modify ``ideScalaVersion``. This change would be only required when developing Scala 2 compiler plugins, sbt plugins or Scala 2 specific sources.


IntelliJ IDEA
=============
-------------

* Select "Create project from existing sources" and choose the ``build.sbt`` file. When prompted, select "Open as project". Make sure you select the "Use sbt shell" for both import and build.

Expand All @@ -21,7 +24,7 @@ IntelliJ IDEA
The above is not an exhaustive list, but it is the bare minimum to have the build working. Please keep in mind that you will have to repeat the above steps, in case you reload (re-import) the SBT build. This will need to happen if you change some SBT-related file (e.g. ``build.sbt``).

Setup for clangd
================
----------------

`clangd` is a Language Server Protocol (LSP) for C and C++. Your IDE of choice can connect to `clangd` to help development using C and C++.

Expand Down
1 change: 1 addition & 0 deletions docs/contrib/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Contributor's Guide
.. toctree::
:maxdepth: 2

quickstart
contributing
build
compiler
Expand Down
133 changes: 133 additions & 0 deletions docs/contrib/quickstart.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
.. _quickstart:

Quick Start Guide
=================

Requirements
------------

- Java 8 or newer
- LLVM/Clang 15 or newer
- sbt

Project Structure Overview
--------------------------

See :ref:`build`

Project suffix
--------------

Most projects in ScalaNative cross-build against Scala ``2.12``, ``2.13`` and ``3``, and these projects have a suffix like ``2_12``, ``2_13`` or ``3`` to differentiate the Scala version.
For example, ``sandbox`` has ``sandbox2_12``, ``sandbox2_13`` and ``sandbox3``.

In the following we will use suffix ``3``, but remember that you can build and test for different versions using different suffixes.

Build / Manual Testing on Sandbox
---------------------------------

``sandbox3/run`` to compile, link and run the main method of the sandbox project defined in ``sandbox/src/main/scala/Test.scala``.

It's convenient to run the ``sandbox`` project to verify the build works as expected.

Test
----

**Common Test Commands**

- ``tests3/test`` - run the unit tests for libraries on native build
- ``tests3/testOnly org.scalanative.testsuite.javalib.util.RandomTest`` - run only the test of interest
- ``tests3/testOnly *.RandomTest`` - run only the test of interest using wildcard
- ``testsExt3/test`` - run the unit tests on native build, this module contains tests that requires dummy javalib implementation defined in ``javalibExtDummies``.
- ``nirJVM3/test`` - run the unit tests for NIR
- ``toolsJVM3/test`` - run the unit tests of the tools: ScalaNative backend
- ``sbtScalaNative/scripted`` - run all `scripted tests <https://www.scala-sbt.org/1.x/docs/Testing-sbt-plugins.html>`_ of the sbt plugin (this takes a while).
- ``sbtScalaNative/scripted <test directory to run>`` - run specific scripted tests of the sbt plugin. e.g. ``sbtScalaNative/scripted run/backtrace``
- Scripted tests are used when you need to interact with the file system, networking, or the build system that cannot be done with a unit test.
- ``set ThisBuild / scriptedBufferLog := false`` disables buffer log in scripted test and get more verbose output

**Other Test Commands**

- ``testsJVM3/test`` - run ``tests3/test`` on JVM
- ``testsExtJVM3/test`` - run ``testsExt3/test`` on JVM
- ``test-all`` - run all tests, ideally after ``reload`` and ``clean``
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing I have noticed is that reloading will reset the version of Scala you might have configured when you started sbt.

% sbt "++3.3.0; shell"
...
sbt:scala-native> scalaVersion
[info] 3.3.0
sbt:scala-native> clean; reload
...
sbt:scala-native> scalaVersion
[info] 2.12.18

I might be doing something wrong but that definitely was a surprise for me. Maybe it deserves a mention in this guide.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's more like sbt usage?

  • You don't need to decide the Scala version to use when you launch sbt shell
    • You can just run $ sbt to launch sbt shell, no need for ++x.y.z and shell
  • You can change the scala version in sbt shell whenever you want
$ sbt
...
sbt:scala-native> scalaVersion
[info] 2.12.18
sbt:scala-native> ++3.3.0
...
sbt:scala-native> scalaVersion
[info] 3.3.0

I'm not sure we should add explanation for that here.
And do we want to document test-all, since it takes so long and I personally never used in my local development.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's more like sbt usage?

OK, I understand.

I think I would still add something like "remember that reload will cause the session to be restarted fresh, meaning that your configurations (e.g., ++3.3.0) will be reset as well."

And thanks for the explanations, by the way ;)

And do we want to document test-all, since it takes so long and I personally never used in my local development.

I think we should. FWIW, the first thing I always try to do when I get started with an open source project is to try running the most comprehensive test suite possible.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is better to point to something a little less comprehensive to start like tests3/test or the tools test. The testsJVM will run tests against the VM so the is good to mention for javalib. You might want to mention the 2_13 and 2_12 versions as well. We can't assume we have sbt experts.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hear you but my two cents is that I still believe that there should be a way for someone cloning SN for the first time to make sure everything works as expected, even if test-all is a command that's almost never run in the "standard development workflow".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct but this is very much a WIP and we may have failures especially with new versions of software or different platforms - what we are trying to do is hard and 0.5.0 has not really settled down 100%.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I understand.

If we can agree that the goal is to eventually be able to run test-all on supported platforms, why not keep a brief record of this discussion in the docs for the next me? I would suggest something like this:

Once you've installed all dependencies, run the `test-all` command in `sbt` to confirm that your environment is properly configured. It will run every test defined in this project, which might take a while.

> Note: it is possible that some tests ran by `test-all` might not pass or compile on your system. We are currently working on stabilizing this test suite for our supported platforms. As of this writing (<insert date>), we expect all tests to pass on Linux <insert distribution> X1.Y1 with LLVM X2.Y2, `sbt` X3.Y3, <insert other known dependencies>.
>
> Should `test-all` fail on your machine, you may try to run smaller test suites one after the other, specifically `test-runtime`, `test-tools`, <insert other test commands>.
> Known issues are <insert known issues>.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just added a disclaimer that test-all may fail 1e57fea
I hope someone who know more about test-all can update the sentence later.

- Note: it is possible that some tests ran by `test-all` might not pass or compile on your system. We are currently working on stabilizing this test suite for our supported platforms
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend deleting this section. Two recently merged PR have made test-all more robust.


**Some additional tips**

- If you modify the ``nscplugin``, you will need to ``clean`` the project that
you want to rebuild with its new version (typically ``sandbox/clean`` or
``tests/clean``). For a full rebuild, use the global ``clean`` command.

- If you modify the sbt plugin or any of its transitive dependencies
(``sbt-scala-native``, ``nir``, ``util``, ``tools``, ``test-runner``), you
will need to ``reload`` for your changes to take effect with most test
commands (except with the ``scripted`` tests).

- For a completely clean build, from scratch, run ``reload`` *and* ``clean``.

Formatting
----------

- ``./scripts/scalafmt`` - format all Scala codes
- ``./scripts/clangfmt`` - format all C/C++ codes

`Publish Locally <https://www.scala-sbt.org/1.x/docs/Publishing.html>`_
-----------------------------------------------------------------------

``publish-local-dev x.y.z`` publishes the ScalaNative artifact and sbt plugin for specified scala version locally.
For example, ``publish-local-dev 3.3.1``,

You will see, the log message like the following, which means you have successfully published locally for the version ``0.5.0-SNAPSHOT``.

.. code-block:: text

[info] published tools_native0.5.0-SNAPSHOT_3 to ...
[info] published ivy to ...tools_native0.5.0-SNAPSHOT_3/0.5.0-SNAPSHOT/ivys/ivy.xml

Then you'll be able to use locally published version in other projects.

.. code-block:: text

# project/plugins.sbt
addSbtPlugin("org.scala-native" % "sbt-scala-native" % "0.5.0-SNAPSHOT")

# build.sbt
scalaVersion := "3.3.1" # set to locally published version
enablePlugins(ScalaNativePlugin)

Locally build docs
--------------------

1. First time building the docs. This command will setup & build the docs.

.. code-block:: text

$ bash scripts/makedocs setup

2. If setup is already done. This command will only build the docs assuming setup is already done.

.. code-block:: text

$ bash scripts/makedocs

3. Navigate to ``docs/_build/html`` directory and open ``index.html`` file in your browser.

Configure Native Build
----------------------

To configure the native build in this project, you can edit ``project/MyScalaNativePlugin.scala`` instead of ``project/Build.scala``.

``MyScalaNativePlugin`` is a custom sbt plugin that extends ``ScalaNativePlugin`` and overrides some of its settings for this project.


Further Information
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add parapgrah about modifing default nativeConfig used within the Scala Native project. The contributors should modify project/MyScalaNativePlugin instead of Build.scala. This way they would be able to quickly switch some tested options like usage of lto, release mode, etc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added fcb9df6

-------------------

- How to make a commit and PR :ref:`contributing`
- More detailed build setting explanation :ref:`build`
- Scala Native Internal
- :ref:`compiler`
- :ref:`nir`
- :ref:`name_mangling`
- How to setup IDEs :ref:`ides`