Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@
"ghcr.io/devcontainers/features/java:1": {
"version": 24
},
"ghcr.io/devcontainers/features/node:1": {},
"ghcr.io/devcontainers/features/node:1": {},
"ghcr.io/guiyomh/features/just:0" :{}
},
// Attemps to install blas so that native works.
"postCreateCommand": "chmod +x millw & npm install && sudo apt update & wait; sudo apt-get install -y libatlas-base-dev clang & wait; ./millw __.compiledClassesAndSemanticDbFiles"
// Attemps to install blas so that native works.
"postCreateCommand": "chmod +x mill & npm install && sudo apt update & wait; sudo apt-get install -y libatlas-base-dev clang & wait; ./mill __.compiledClassesAndSemanticDbFiles"
// Configure tool-specific properties.
// "customizations": {},
// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
Expand Down
25 changes: 25 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Set default line ending behavior to auto-normalize
* text=auto

# Explicitly declare text files you want to always be normalized and converted
# to native line endings on checkout.
*.scala text
*.sc text
*.sbt text
*.conf text
*.md text
*.yml text
*.yaml text
*.json text
*.xml text
*.properties text

# Declare files that will always have LF line endings on checkout.
*.sh text eol=lf

# Denote all files that are truly binary and should not be modified.
*.jar binary
*.png binary
*.jpg binary
*.gif binary
*.xlsx binary
14 changes: 7 additions & 7 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,16 @@ Answer all questions in the style of a friendly colleague that is an expert in l

## Working Effectively

Mill may be found via it's wrapper script `./millw` in the root of the repository. For example `./millw vecxt.__.compile` will compile the JVM, JS and native targets.
Mill may be found via it's wrapper script `./mill` in the root of the repository. For example `./mill vecxt.__.compile` will compile the JVM, JS and native targets.

Each module contains it's own build definition in the package.mill file in it's module directory.

- BUILDS: Mill cold compilation takes 2 minutes or so. Stay patient!
- Compile specific platforms (e.g. jvm) with `./millw vecxt.jvm.compile` or `./millw vecxt.js.compile` etc.
- Run all tests by following with the same pattern `./millw vecxt.__.test`
- Format code with `./millw mill.scalalib.scalafmt.ScalafmtModule/`. CI will enforce formatting, and will fail if code is not formatted.
- If you see an error like this is JS `[error] @scala.scalajs.js.annotation.internal.JSType is for compiler internal use only. Do not use it yourself.`, run `./millw clean vecxt.js._` to clear the build cache.
- To run a specific main class, use the runMain command and specify the package. `./millw experiments.runMain testCheatsheet` for example.
- Compile specific platforms (e.g. jvm) with `./mill vecxt.jvm.compile` or `./mill vecxt.js.compile` etc.
- Run all tests by following with the same pattern `./mill vecxt.__.test`
- Format code with `./mill mill.scalalib.scalafmt.ScalafmtModule/`. CI will enforce formatting, and will fail if code is not formatted.
- If you see an error like this is JS `[error] @scala.scalajs.js.annotation.internal.JSType is for compiler internal use only. Do not use it yourself.`, run `./mill clean vecxt.js._` to clear the build cache.
- To run a specific main class, use the runMain command and specify the package. `./mill experiments.runMain testCheatsheet` for example.

## Folder structure

Expand All @@ -29,7 +29,7 @@ vecxt/
├── .devcontainer/ # VS Code dev container configuration
├── .vscode/ # VS Code workspace settings
├── build.mill # Mill build configuration (main build file)
├── millw # Mill wrapper script for cross-platform builds
├── mill # Mill wrapper script for cross-platform builds
├── styleguide.md # Coding style guidelines
├── benchmarks/ # Benchmarking code - not published, may be run in CI on request
├── experiments/ # Not published, inlined experiments - use this as a sandbox
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
jvm: temurin@17
# Run benchmark with `go test -bench` and stores the output to a file
- name: Run benchmark
run: ./millw benchmark.runJmh -jvmArgs --add-modules=jdk.incubator.vector -rf json
run: ./mill benchmark.runJmh -jvmArgs --add-modules=jdk.incubator.vector -rf json

- name: Set variables
run: |
Expand Down
40 changes: 25 additions & 15 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,4 @@
# This file was automatically generated by sbt-github-actions using the
# githubWorkflowGenerate task. You should add and commit this file to
# your git repository. It goes without saying that you shouldn't edit
# this file by hand! Instead, if you wish to make changes, you should
# change your sbt build configuration to revise the workflow description
# to meet your needs, then regenerate this file.

name: Continuous Integration

on:
pull_request:
branches: ['**', '!update/**', '!pr/**']
Expand All @@ -16,6 +8,7 @@ on:

env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
FORCE_COLOR: '1'


concurrency:
Expand Down Expand Up @@ -52,7 +45,7 @@ jobs:
cache: npm

- name: formatCheck
run: ./millw mill.scalalib.scalafmt.ScalafmtModule/checkFormatAll __.sources
run: ./mill mill.scalalib.scalafmt.ScalafmtModule/checkFormatAll __.sources

- if: matrix.project == 'js'
run: npm install
Expand All @@ -62,16 +55,33 @@ jobs:

- name: scalaJSLink
if: matrix.project == 'js'
run: ./millw vecxt.${{ matrix.project }}.fastLinkJS
run: ./mill vecxt.${{ matrix.project }}.fastLinkJS

- name: nativeLink
if: matrix.project == 'native'
run: ./millw vecxt.${{ matrix.project }}.test.nativeLink
run: ./mill vecxt.${{ matrix.project }}.test.nativeLink

- name: Test
run: ./millw vecxt.${{ matrix.project }}.test
run: ./mill vecxt.${{ matrix.project }}.test

- name: Laws Test
if: matrix.project == 'jvm'
run: ./mill laws.${{ matrix.project }}.test

- name: Doc Gen
run: ./millw site.siteGen
if: matrix.project == 'jvm'
run: ./mill site.siteGen

- name: Publish Test Report
uses: mikepenz/action-junit-report@v5
if: always()
with:
fail_on_failure: false
include_passed: false
detailed_summary: true
annotate_only: true
require_tests: false
report_paths: 'out/**/test-report.xml'

publish:
if: github.repository == 'Quafadas/vecxt' && contains(github.ref, 'refs/tags/')
Expand All @@ -89,7 +99,7 @@ jobs:
apps: scala-cli

- name: Publish to Maven Central
run: ./millw mill.javalib.SonatypeCentralPublishModule/
run: ./mill mill.javalib.SonatypeCentralPublishModule/
env:
MILL_PGP_PASSPHRASE: ${{ secrets.PGP_PASSPHRASE }}
MILL_PGP_SECRET_BASE64: ${{ secrets.PGP_SECRET }}
Expand Down Expand Up @@ -133,7 +143,7 @@ jobs:
run: mkdir -p site/docs/_assets/benchmarks && cp benchmark_history.json site/docs/_assets/benchmarks/benchmark_history.json

- name: Generate static site
run: ./millw site.siteGen
run: ./mill site.siteGen

- name: Setup Pages
uses: actions/configure-pages@main
Expand Down
8 changes: 5 additions & 3 deletions .github/workflows/copilot-setup-steps.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ name: "Copilot Setup Steps"
on:
workflow_dispatch:
push:
branches:
- main
paths:
- .github/workflows/copilot-setup-steps.yml
pull_request:
Expand All @@ -28,7 +30,7 @@ jobs:
- name: Checkout code
uses: actions/checkout@v5

- name: Setup NodeJS v18 LTS
- name: Setup NodeJS v18 LTS
uses: actions/setup-node@v3
with:
node-version: 20
Expand All @@ -44,8 +46,8 @@ jobs:
java-version: '21'

- name: Verify Java version
run: java --version
run: java --version

# This should resolve a JVM, download dependancies and leave mill ready for use.
- name: setup mill
run: ./millw vecxt.__.compiledClassesAndSemanticDbFiles
run: ./mill vecxt.__.compiledClassesAndSemanticDbFiles
2 changes: 1 addition & 1 deletion .scalafmt.conf
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
version = "3.9.9"
version = "3.10.3"
project.git = true
runner.dialect = scala3
rewrite.scala3.convertToNewSyntax = true
Expand Down
2 changes: 1 addition & 1 deletion .vscode/tasks.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
{
"label": "compiledClassesAndSemanticDbFiles",
"type": "shell",
"command": "./millw __.compiledClassesAndSemanticDbFiles",
"command": "./mill __.compiledClassesAndSemanticDbFiles",
"runOptions": {
"runOn": "folderOpen"
},
Expand Down
73 changes: 36 additions & 37 deletions benchmark/package.mill
Original file line number Diff line number Diff line change
@@ -1,43 +1,42 @@
package build.benchmark

import mill._, scalalib._, publish._
import mill.*, scalalib.*, publish.*
import contrib.jmh.JmhModule

// mill benchmark.runJmh vecxt.benchmark.AndBooleanBenchmark -jvmArgs --add-modules=jdk.incubator.vector -rf json

object `package` extends JmhModule with ScalaModule {
def scalaVersion = build.vecxt.jvm.scalaVersion
def jmhCoreVersion = "1.37"
override def forkArgs: T[Seq[String]] = super.forkArgs() ++ build.vecIncubatorFlag
override def moduleDeps: Seq[JavaModule] = Seq(build.vecxt.jvm)
def enableBsp = false

// override def generateBenchmarkSources = T{
// val dest = T.ctx().dest

// val forkedArgs = forkArgs().toSeq

// val sourcesDir = dest / "jmh_sources"
// val resourcesDir = dest / "jmh_resources"

// os.remove.all(sourcesDir)
// os.makeDir.all(sourcesDir)
// os.remove.all(resourcesDir)
// os.makeDir.all(resourcesDir)

// Jvm.runSubprocess(
// "org.openjdk.jmh.generators.bytecode.JmhBytecodeGenerator",
// (runClasspath() ++ generatorDeps()).map(_.path),
// mainArgs = Seq(
// compile().classes.path.toString,
// sourcesDir.toString,
// resourcesDir.toString,
// "default"
// ),
// jvmArgs = forkedArgs
// )


// (sourcesDir, resourcesDir)
// }
}
object `package` extends JmhModule with ScalaModule:
def scalaVersion = build.vecxt.jvm.scalaVersion
def jmhCoreVersion = "1.37"
override def forkArgs: T[Seq[String]] = super.forkArgs() ++ build.vecIncubatorFlag
override def moduleDeps: Seq[JavaModule] = Seq(build.vecxt.jvm)
def enableBsp = false

// override def generateBenchmarkSources = T{
// val dest = T.ctx().dest

// val forkedArgs = forkArgs().toSeq

// val sourcesDir = dest / "jmh_sources"
// val resourcesDir = dest / "jmh_resources"

// os.remove.all(sourcesDir)
// os.makeDir.all(sourcesDir)
// os.remove.all(resourcesDir)
// os.makeDir.all(resourcesDir)

// Jvm.runSubprocess(
// "org.openjdk.jmh.generators.bytecode.JmhBytecodeGenerator",
// (runClasspath() ++ generatorDeps()).map(_.path),
// mainArgs = Seq(
// compile().classes.path.toString,
// sourcesDir.toString,
// resourcesDir.toString,
// "default"
// ),
// jvmArgs = forkedArgs
// )

// (sourcesDir, resourcesDir)
// }
end `package`
23 changes: 11 additions & 12 deletions benchmark_vs_breeze/package.mill
Original file line number Diff line number Diff line change
@@ -1,18 +1,17 @@
package build.benchmark_vs_breeze

import mill._, scalalib._, publish._
import mill.*, scalalib.*, publish.*
import contrib.jmh.JmhModule

// mill benchmark.runJmh vecxt.benchmark.AndBooleanBenchmark -jvmArgs --add-modules=jdk.incubator.vector -rf json

object `package` extends JmhModule with ScalaModule {
def scalaVersion = build.vecxt.jvm.scalaVersion
def jmhCoreVersion = "1.37"
def enableBsp = false
override def forkArgs: T[Seq[String]] = super.forkArgs() ++ build.vecIncubatorFlag
override def moduleDeps: Seq[JavaModule] = Seq(build.vecxt.jvm)
override def mvnDeps = super.mvnDeps() ++ Seq(
mvn"org.scalanlp::breeze:2.1.0"
)

}
object `package` extends JmhModule with ScalaModule:
def scalaVersion = build.vecxt.jvm.scalaVersion
def jmhCoreVersion = "1.37"
def enableBsp = false
override def forkArgs: T[Seq[String]] = super.forkArgs() ++ build.vecIncubatorFlag
override def moduleDeps: Seq[JavaModule] = Seq(build.vecxt.jvm)
override def mvnDeps = super.mvnDeps() ++ Seq(
mvn"org.scalanlp::breeze:2.1.0"
)
end `package`
4 changes: 2 additions & 2 deletions build.mill
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
//| mill-version: 1.1.0-RC2
//| mill-version: 1.1.0-RC3
//| mill-jvm-version: 24
//| mill-jvm-opts: [ "--add-modules", "jdk.incubator.vector"]
//| mvnDeps:
Expand Down Expand Up @@ -27,7 +27,7 @@ import com.goyeau.mill.scalafix.ScalafixModule
import mill.api.Task.Simple

// def format = mill.scalalib.scalafmt.ScalafmtModule

// mill mill.scalalib.scalafmt/
// mill __.compiledClassesAndSemanticDbFiles
object V:
val spire: Dep = mvn"org.typelevel::spire::0.18.0"
Expand Down
2 changes: 0 additions & 2 deletions experiments/package.mill
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,6 @@ object `package` extends ScalaModule:
override def forkArgs = super.forkArgs() ++ build.vecIncubatorFlag
// override def mainClass = Some("mnist")



override def moduleDeps = Seq(build.vecxt.jvm)
override def mvnDeps = super.mvnDeps() ++ Seq(
mvn"com.lihaoyi::os-lib::0.10.4",
Expand Down
2 changes: 1 addition & 1 deletion experiments/src/inv_test.scala
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
* In [5]: x = A_inv @ b ...: x Out[5]: array([[10.], [10.], [20.], [20.], [10.]])
*/

//./millw experiments.runMain experiments.inv_test
//./mill experiments.runMain experiments.inv_test

@main def inv_test =
import vecxt.all.*
Expand Down
2 changes: 1 addition & 1 deletion experiments/src/mnist.scala
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ import narr.*
end mnist

def dataToCoords(data: Array[Double]): IndexedSeq[(x: Int, y: Int, opacity: Double)] =
for (i <- 0.until(28); j <- 0.until(28)) yield
for i <- 0.until(28); j <- 0.until(28) yield
val value = data(i * 28 + j)
(x = j, y = 28 - i, opacity = value)

Expand Down
9 changes: 4 additions & 5 deletions jsSite/package.mill
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,15 @@ import mill.util.VcsVersion
import build.V
import mill.scalajslib.api.ESModuleImportMapping


object `package` extends SiteJSModule {
object `package` extends SiteJSModule:

override def moduleDeps = Seq(build.vecxt.js, build.vecxtensions.js)
override def scalaVersion = build.vecxt.js.scalaVersion
override def scalaJSVersion = build.vecxt.js.scalaJSVersion

override def scalaJSImportMap = Task{
override def scalaJSImportMap = Task {
Seq(
ESModuleImportMapping.Prefix("@stdlib/blas/base", "https://cdn.jsdelivr.net/npm/@stdlib/blas@0.2.0/base/+esm" )
ESModuleImportMapping.Prefix("@stdlib/blas/base", "https://cdn.jsdelivr.net/npm/@stdlib/blas@0.2.0/base/+esm")
)
}

Expand All @@ -35,4 +34,4 @@ object `package` extends SiteJSModule {
// object test extends ScalaJSTests with CommonTests {
// def moduleKind = ModuleKind.CommonJSModule
// }
}
end `package`
Loading