Skip to content
Permalink
Branch: master
Commits on Jul 29, 2019
  1. Commix: use named semaphores (#1658)

    valdisxp1 authored and densh committed Jul 29, 2019
    Using named semaphores only for comparability with MacOs see #1654.
    Needs changes from #1657 to compile on Mac.
Commits on Jul 25, 2019
  1. Fix a couple of typos and improve wording of the interop docs (#1649)

    ekrich authored and densh committed Jul 25, 2019
  2. Fix #1663: Dockerfile no longer needs cre2 (#1668)

    LeeTibbert authored and densh committed Jul 25, 2019
    * This PR fixes Issue "Dockerfile no longer needs cre2".
    
      The library libre2-dev is no longer installed. As of SN 0.4.0,
      SN uses re2s instead of cre2.
    
    Documentation:
    
    * None needed: not a user facing change & relevant devos notified.
    
    Testing:
    
    * I know of know way to test this on my system.  Attempting a
      crash & burn test on Travis CI.
Commits on Jul 22, 2019
  1. Commix: use macos-friendly way to get cpu count (#1657)

    valdisxp1 authored and densh committed Jul 22, 2019
Commits on Jul 18, 2019
  1. Update uppercase lowercase to use UnicodeData.txt instead of CaseFold…

    ekrich authored and densh committed Jul 18, 2019
    …ing.txt (#1611)
    
    * Update uppercase lowercase to use UnicodeData.txt vs CaseFolding.txt
    
    * Separate and update tests and simplify lookup code for new encoding
    
    * Does not use SpecialCasing.txt
    
    * Full parity with JDK8 using Unicode 6.3.0
    
    * Remove test case above Unicode 6.3 upper/lower case range
Commits on Jul 17, 2019
  1. Commix: parallel mark and concurrent sweep GC (#1423)

    valdisxp1 authored and densh committed Jul 17, 2019
    Features:
    - New `SCALANATIVE_GC` option  `commix` to enable the new GC changes.
    - Concurrent, parallel sweep. Reduced GC pause times by utilizing additional processor cores during sweeping.
    - Parallel mark. Reduces GC pause times by utilizing additional processor cores during marking.
    - Configurable number of GC threads with `SCALANATIVE_GC_THREADS` environment variable. Set to number of available processors - 1 up to 8, but no less than 1.
    - Increasing heap size until mark is <=5% of total time. This can be adjusted with `SCALANATIVE_TIME_RATIO`. I.e. `export SCALANATIVE_TIME_RATIO=0.03` for <= 3%.
    - Added an option the change the minimum free block ratio `SCALANATIVE_FREE_RATIO`. If there are not enough free blocks after a garbage collection it will increase the heap size.
    - The minimum heap size setting renamed from `SCALANATIVE_MIN_HEAP_SIZE` to a shorter `SCALANATIVE_MIN_SIZE`. Updated Immix to match.
    - The maxium heap size setting renamed from `SCALANATIVE_MAX_HEAP_SIZE` to a shorter `SCALANATIVE_MAX_SIZE`. Updated Immix to match.
    ### Changes to GC Statistics
    - Stat collection disabled by default to reduce code size. They can be enabled by compiler arguments. See example and table below.
    - Stats changed to event-based CSV format.
    - Stats now also saves events for collection start and end.
    - Stats includes `start_ns` column - time in nanoseconds since application was started.
    - Stats is compatible with the new concurrent sweep events.
     Multiple verbosity levels of enabled for stats with compiler arguments. For example:
    ```scala
    nativeCompileOptions += "-DENABLE_GC_STATS_BATCHES"
    ```
    
    > scala-native-benchmarks flags as per suggested changes in scala-native/scala-native-benchmarks#3
    > results size for default tests when doing a single run `--runs 1` instead of the default which is 20.
    
    Internals:
    - thread-safe `BlockList`s
    - thread-safe `BlockRange`s
    - thread-safe block counts
    - thread-safe `GreyList`s
    - new block flags in `BlockMeta`
    - removed marking `Stack`s
    
    Debugging :
    - asserts in Immix can now be enabled by setting `DEBUG_ASSERT` flag
    ```scala
    nativeCompileOptions += "-DDEBUG_ASSERT"
    ```
    This similar to the way you could enable trace level printing and debug symbols.
    ```scala
    // trace level printing in Immix
    nativeCompileOptions += "-DDEBUG_PRINT"
    // enable debug symbols for C code
    nativeCompileOptions += "-g"
    ```
    - More integrity check asserts for sweeping.
    ## Refactoring:
    - Moved `Block_Sweep` to `Allocator_Sweep` in order to match `LargeAllocator_Sweep`.
    - Moved `Heap_AllocSmall` and `HeapAllocLarge` to their respective allocators
    - Renamed `block_superblock_middle` to ` block_superblock_tail` in order to avoid confusion. This block flag is used for both middle and last blocks
Commits on Jul 6, 2019
  1. Fix #1606: add printf vararg helpers (#1636)

    rwhaling authored and densh committed Jul 6, 2019
    * added libc package object with printf etc vararg wrappers
    
    * ran scalafmt
Commits on Jun 5, 2019
  1. Bump to 0.4.0-SNAPSHOT (#1620)

    densh committed Jun 5, 2019
  2. Fix #1612: remove code that is hiding the true exception (#1617)

    ekrich authored and densh committed Jun 5, 2019
  3. Fix #1616: provide posix.time strptime declaration (#1618)

    LeeTibbert authored and densh committed Jun 5, 2019
    * This pull request was motivated by Issue #1616
        "posix.time strptime method is missing".
    
        That issue is now fixed and four unit tests were added to the
        existing TimeSuite to exercise the declaration and method.
    
      * strptime is the converse of strftime. Scala Native has supported
        the latter for a year or so. Ubuntu "man strptime" describes
        strptime as compliant with several posix standard iterations
        but it may actually first appear in the posix follow on XOPEN
        standards.  Because of the usefulness of the method, I did
        not do too march archaeology. Most current developers would expect
        it to be supported, particularly if strftime was supported.
    
      * I believe the current standard for the posix sub-project is that
        clib functions which are basically straight pass-through do not
        require @name annotations.  For that reason, I did not add one
        for strptime.  Since strftime and strptime are so closely related
        I removed the existing @name annotation from the former.  This
        means that both use the same convention.
    
      * To focus attention on the current change, I did not do a sweep
        through and check if time.scala matches the current OpenGroup
        description for time.h.  Such a sweep should happen, as time
        allows.
    
    Documentation:
    
        * The standard changelog entry is requested.
    
        * I checked the SN 0.4.0-M2 ReadTheDocs for posixlib and
          no change is necessary. This PR introduces a method inside
          an already listed file.
    
    Testing:
    
      * Built and tested ("test-all") in debug mode using sbt 1.2.8 on
        X86_64 only . All tests pass.
Commits on May 24, 2019
  1. Fix latest version in sbt docs for 0.4.0-M2 (#1608)

    densh committed May 24, 2019
  2. Release 0.4.0-M2 (#1607)

    densh committed May 24, 2019
  3. Release 0.4.0-M1 (#1603)

    densh committed May 24, 2019
    * Release 0.4.0-M1
    
    * Drop scaladoc for scalanative.regex
  4. Update documentation for Metals and other build changes (#1604)

    ekrich authored and densh committed May 24, 2019
  5. Add qsort and bsearch to stdlib (#1605)

    ekrich authored and densh committed May 24, 2019
Commits on May 23, 2019
  1. Fix #1197: rename scalanative.native into scalanative.unsafe (#1602)

    densh committed May 23, 2019
    * Rename scalanative.native to scalanative.unsafe
    
    * Split out scalanative.annotation
    
    * Split out scalanative.unsigned
    
    * Fix scripted test
    
    * Fix formatting
    
    * Fix docs
  2. Fix #1573, #1544: re2s code and documentation improvements (#1600)

    LeeTibbert authored and densh committed May 23, 2019
    * Fix #1573, #1544: re2s code and documentation improvements.
    
      * This pull request was motivated by Issue #1573 "re2s needs reduced code
        size". It quickly became evident that Issue #1544 "javalib docs need
        accurate & current description of regex & re2s differences."
    
        Issue #1544 is fixed. The major differences have been described.
        Remaining differences will be added as they are discovered.
    
        The one-dimensional array issue described in #1573 is fixed but
        additional passes are needed to address other opportunities for
        improvement.
    
        Documentation and implementation should now match.
    
      * This pull request does not address #1542 "Unicode tables should be
        unified for consistency & minimal binary size". I spent a number
        of days addressing that issue and backed off to create this
        checkpoint. I intend to take another run or 10 on that issue.
    
        One of the issues is that re2s is based on Unicode ?.?, possibly
        6.? where ? can range from 0 to 3.  I tried to discover the
        version several times, until it became evident that further time
        invested would be neither economic nor prudent.
    
        Thankfully, Scala Native Character.scala quite plainly documents its
        Unicode version as 7.0.  Java 8 shipped with Unicode 6.2.0.
    
      * Methods in CharClass.scala and Matcher.scala with return statements,
        especially with those returning values, in the middle of the method
        were converted to return only at the end/bottom of the method.
        I am told, on good authority, that returns from the middle of methods
        are currently expensive on Scala Native.
    
      * The test Suite for j.u.r.Matcher contains a test case to verify
        that Issue #1506 "j.u.regex returns wrong group" if fixed.
        This change comes under the heading of understanding and documenting
        what works and what does not.
    
        That file was also edited to remove confusing redundant and misplaced
        code.
    
    Documentation:
    
        * A one line entry in the changelog is needed, probably referring
          to this commit message for details.
    
        * The file docs/lib/javalib.rst was updated to my best understanding
          of what is and is not supported and implemented in
          java.util.regex and re2s.
    
          This was a merge of information from the GitHub re2j and re2s
          repositories and personal investigation and experimentation.
          Despite best efforts, errors may remain.
    
          That file now documents a change possibly requiring changes in regex
          code when upgrading from Scala Native 0.3.n (n >= 8),
          cre2 based Scala Native 0.3.8 required POSIX patterns of the
          form '[[:alpha:]]'. re2s based 0.4.0 requires the Java
          standard form '\p{Alpha}' and rejects the cre2 form.
    
    Testing:
    
      * Built and tested ("test-all") in debug mode using sbt 1.2.8 on
        X86_64 only . All tests pass.
    
      * Built and tested ("test-all") in release mode using sbt 1.2.8 on
        X86_64 only . All tests pass.
    
    * Implement review suggestion for describing exceptions
    
    Thank you @ekrich
    
    * Renamed package from scalanative.re2s to scalanative.regex.
    
    Per request by @densh.  To suit the needs of Scala Native,
    scalanative.regex has diverged from the original re2s. It will diverge
    further in the near future.
Commits on May 22, 2019
  1. Fix interop docs typo (#1601)

    densh committed May 22, 2019
  2. Add support for boxed pointers (#1426)

    densh committed May 22, 2019
    * Add support for boxed pointers
    * Add pointer boxing test suite
    * Make stackalloc truly polymorphic over native.Tag
    * Don't rely on runtime tags for c-style varargs
    * Drop support for C-style casts
    * Improve interflow handling of boxed pointers
    * Fix ptr to long conversion
    * Fix #1516: make sure unsigned types are boxed in arrays
    * Make sure sizeof struct is constant folded
    * Change Nat encoding to make it easier to optimize away
    * Even better handling of boxed pointers in interflow
    * Add support for boxed CArray
    * Add support for boxed CStruct
    * Automatically dereference structs and arrays
    * Make Ptr, CStruct and CArray constructors private
    * Make equals and hashCode alwaysinline on box classes
    * Box return type is nullable when boxing pointers
    * Add support for atN methods for CStructN
    * Make sure java.lang.Integer is stateless
    * Rename FuncPtr to CFuncPtr
    * Fix CFuncPtr boxing semantics
    * Move interop unit tests to scalanative.native
    * Restructure interop tests
    * Add more tests for CStruct
    * Don't use c varargs in fcntl bindings
    * Add support for CVarArgList
    * Update interop docs
Commits on May 10, 2019
  1. Fix #1588: make nir dumping opt-in (#1596)

    densh committed May 10, 2019
    * Fix #1588: make nir dumping opt-in
    
    Similarly to #1595, we put dumping of intermediate NIR after linking and optimization behind a flag.
    
    * Fix tools tests
    
    * Dump nir in parallel
Commits on May 9, 2019
  1. Fix #1589: make nir checking opt-in (#1595)

    densh committed May 9, 2019
    This commits adds a setting to turn on/off NIR checking with off being a default.
  2. Fix #1445: link-order test fails to compile with sbt 1.x (#1446)

    LeeTibbert authored and densh committed May 9, 2019
    * Commit_msg_PR_scripted_link-order.utf-8
    
    * Use a bigger hammer: full import statement
    
    * Explicitly convert sbt logger to ProcessLogger.
    
    * Minor edit to minimize diffs & ease review
  3. Fix #1592: j.t.DateFormatSymbols does not memoize correctly (#1593)

    LeeTibbert authored and densh committed May 9, 2019
    * Fix #1592: j.t.DateFormatSymbols does not memoize correctly
    
      * Issue #1592 reported that two methods, getShortWeekdays &
        getShortMonths, in java.text.DateFormatSymbols.scala
        did not properly store the first calculation of a local
        variable for subsequent use (memoize).
    
        The results of the two methods were correct in the flawed
        implementation and remain so in this PR.
    
        The issue is now fixed.
    
      * My thanks to @ekrich for originally detecting the flaw and
        for several rounds of making a solution readable.
    
      * A test for each of the affected methods  was added to
        DateFormatSymbolsSuite to check that the object returned on a second
        call is the same (strictly object equal) as that from the first call.
    
        All DateFormatSymbols tests, new & old, run and pass.
    
    Documentation:
    
        None needed other than the title in the change log. No
        change in user facing behavior.
    
    Testing:
    
      * Built and tested ("test-all") on X86_64 using sbt 1.2.8 in
        release-fast mode. All tests expected to pass do so.
    
    * Incorporate review suggestion from @ekrich to use .eq
    
    Using .eq in two memoization tests rather than === makes it
    clearer to an overburdened reader that object equality is required.
Commits on May 8, 2019
  1. Fix #1359: add test to make sure it stays fixed (#1591)

    densh committed May 8, 2019
    The issue was fixed at some point. This is just a test to make
    sure it stays fixed.
  2. Test release-fast mode on travis (#1475)

    densh committed May 8, 2019
  3. Fix #1551: provide minimal, US-only DateFormatSymbols (#1556)

    LeeTibbert authored and densh committed May 8, 2019
    * Issue #1551 reported an issue were the sandbox project in
        the SN build itself would not link due to missing DateFormatSymbols
        methods.
    
        This was an unintended consequence of PR #1522 which converted
        a number of routines from "???" to using the stub annotation.
    
        The Issue itself suggested adding nativeLinkStubs := true to
        the sandbox project in the SN project build.sbt.
    
        @densh suggested the better proposal of doing a minimal implementation.
        This PR takes that approach.
    
        The issue is now fixed.
    
      * A DateFormatSymbolsSuite was created and populated.
        All tests run and pass.
    
      * The implementation in this file almost complete for the only
        Locale it supports: Locale.US.
    
        The 'almost' comes from the method getZoneStrings(). That method
        is described in the Java 8 documentation as 'discouraged'.
        To save binary size, the method implemented returns results
        which are structurally proper according to the specification
        but semantically meaningless. To wit, five empty strings.
    
        The preferred TimeZone.getDisplayName(), which is, I think,
        currently unimplemented in SN, is the proper way to get this
        information.  Better that whatever effort is available be
        spent implementing that routine & kin.
    
      * This PR is one of a series needed in order to get unit-tests
        to link without nativeLinkStubs being true.
    
      * This PR does add some binary size. I will be creating a Issue
        to track what seems to be the underlying problem:  Something
        is eagerly loading code for methods which are never used.
        I say 'never used' because they never failed when they were
        '???'.
    
    Documentation:
    
        None needed other than the title in the change log.
    
    Testing:
    
      * Built and tested ("test-all") on X86_64 using sbt 1.2.6 in
        debug mode. All tests expected to pass do so.
  4. Fix #1460: split release mode into release-fast and release-full (#1545)

    densh committed May 8, 2019
    * Fix #1460: split release mode into release-fast and release-full
    
    * Update docs on release modes
Commits on May 7, 2019
  1. Fix #1578: fix numeric extension in SocketInputStream.read (#1583)

    LeeTibbert authored and densh committed May 7, 2019
    wrongly returned end-of-file (EOF, OxFFFFFFFF) after reading a
        character 255.
    
        This PR fixes the reported issue.
    
    Documentation:
    
      * None needed beyond an entry in the changelog.
    
    Testing:
    
      * Because this is socket code no simple unit-test is evident.
    
      * In my private development environment I built and tested ("test-all")
        in debug mode with sbt 0.13.18 using the existing 254
        unit-test Suites, ten additional Suites, and a private version
        of DataInputStream.scala (DIS).
    
        That private version of DIS is a preliminary version of a fix
        to #1579.  After a series of calls its readFully() calls the SIS#read()
        of this PR.
    
        The 265 unit-tests executed as expected and without regression.
        That would only happen if those edits were effective.
    
        PR #1582 was submitted because it contains a more performant
        version of DIS#readFully. That version of readFully does not
        directly exercise the edits of this PR.
  2. Fix the implementation of java.io.DataInputStream.readFully (#1582)

    LeeTibbert authored and densh committed May 7, 2019
    * Fix #1008, #1528, partial #1579: unit-tests fails with more than 254 Suites.
    
      * Issue #1008 reported problems with unit-tests when executing
        a large number of Suites.  Issue #1528 renewed concern with
        the problem.  Issue #1579 reported a number of problems
        in java.io.DataInputStream.scala.
    
        This PR fixes issues #1008 and #1528 by implementing a partial
        fix to #1579.
    
        The partial fix removes an erroneous end-of-file (EOF) test and
        the call to the other buggy code identified in #1579 and #1578.
    
        It was simpler to replace readFully with correct and more performant
        code than to create PRs for the entire fault chain.
    
        A full fix for Issue #1579 will follow on after this PR has been
        accepted. This sequencing of edits isolates & highlights the
        effective change in this PR and, I hope, allows for easier review.
    
      * Because of fundamental design issues the 'too many unit-tests'
        experience may re-appear in the future, but not for this reason.
    
        The current protocol sends a list of tasks from the local master
        process to the 'remote' (pseudo) process running the tests.
        The latter re-orders its internal list and replies with the newly
        ordered list.  Is is easy to conceive of a situation where
        these lists exceed the bounds of memory as the number of unit-test
        Suites grows into the thousands.
    
        Tasks (Suites) are executed on at a time, so I see no limit to the
        number of tasks there but each task itself must stay within the
        bounds of available memory.
    
    Documentation:
    
      * None needed. Unit-tests are internal to SN.
    
    Testing:
    
      * In my private development environment I built and tested ("test-all")
        in debug mode with both sbt 0.13.18 & 1.2.8 (1) and the existing 254
        unit-test Suites. All ran & passed (2).
    
      * I then enabled the previously disabled AtomicReferenceSuite.scala,
        giving 255 tests. I ran tests/test again and all 255 ran & passed.
    
      * I added 10 additional Suites, giving 265 Suites. and ran tests/test.
        All tests ran & passed, including Suites 255 through 265.
    
      * After this PR is accepted, I will submit another one which will
        enable AtomicReferenceSuite.scala.  This should provide the
        255th test and give final validation to this PR.  That seems
        to be conceptually simpler & easier all around than my submitting 10,
        or so, dummy Suites with this PR, only to delete them in another PR.
    
      Note 1: I believe it is a known issue, but for completeness
              I report that sbt 1.2.8 builds in release mode but
    	  compiling unit-tests fails after after a half hour or so.
    
      Note 2: Due to a change in sbt between 0.13 and 1.0, link-order
              never passes for me with 1.2.n.  PR #1466 fixes this issue
    	  and is awaiting acceptance.
    
    * Add readFully review corrections. Add readFully test Suite.
  3. Fix #1574: emit all class definitions in the same .ll module (#1587)

    densh committed May 7, 2019
    This a potential fix for #1574. Previously unit tests failed
    in debug mode on macOS on machines with 8 virtual cores.
    This PR changes the partitioning scheme to bundle all definitions
    for a given class together in LLVM .ll files.
    
    I suspect that that the underlying issue could be a bug in LLVM.
    It works as expected if we emit a single LLVM module, or use
    any of the optimization modes other than -O0 or use lto.
  4. Fix #1428: add support for open-ended zones (#1577)

    tiqwab authored and densh committed May 7, 2019
    * Implement Zone methods to allow open-ended zones
    
    * Add Zone#isOpen
    
    * Throw exception on alloc with closed zone
Commits on Apr 26, 2019
  1. Fix #1560: implement AtomicReference getAndUpdate & updateAndGet (#1570)

    LeeTibbert authored and densh committed Apr 26, 2019
    * Fix #1560, part 2: Implement AtomicReference getAndUpdate & updateAndGet
    
      * This pull request is the second of two which address Issue #1560.
        The first in the series was PR #1566 by @ekrich and the 0.3.9
        variant, PR #1567.
    
        The intent was to enable a SN build of scalafmt.
    
        That issue is now fixed.
    
      * Almost all complex code in SN java.util.concurrent.atomic
        relies upon the fact that SN is now single threaded.
        This code makes the same assumption.
    
        All of this code will be a hard to find hazard and hindrance
        once SN supports multithreading.
    
      * A small test Suite to exercise the methods implemented was created.
        The tests in the suite excercise (only) the newly implemented methods and
        check that the expected values are returned. There was no obvious way
        to test concurrency.
    
        When it is run test-only all tests run and pass.
    
        The new Suite seems to drive the testing framwork over its
        (memory?) limit.  In my private build the new Suite runs
        just fine if I delete some existing Suites.  Another manifestation
        of the same issue I encountered whilst implementing regex.
    
        I am submitting the unit-tests but naming the file
        AtomicReferenceSuite.scala.disabled to pass Travis CI
        until the test-runner is fixed. It took me a while to figure
        them out and they may prove useful to a future developer.
    
    Documentation:
    
        A one line entry in the change log is suggested.
    
    Testing:
    
      * Built and tested ("test-all") in debug mode using sbt 1.2.6 on
        X86_64 only . All test expected to pass do so.
    
    * Remove dates in comment as per review request
    
    * Simplify comments in response to review requests & suggestions.
Commits on Apr 25, 2019
  1. Add stubs for java.lang.Thread (#1566)

    ekrich authored and densh committed Apr 25, 2019
  2. Fix #1563: improve linking error diagnostics (#1564)

    densh committed Apr 25, 2019
    This change adds printing of the dependency chain that
    refererred to a missing symbol to aid debugging of linking
    errors.
Commits on Apr 24, 2019
  1. Bump scalafmt version to 1.5.1 (#1561)

    ekrich authored and densh committed Apr 24, 2019
    * Update scalafmt, add version to conf file
    
    * Use getenv to look for env vars
    
    * Code to avoid infinite loop
    
    * Minor version updates
    
    * Revert to use scalafmt 2.11 runner
Older
You can’t perform that action at this time.