Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ with TOGSimulator(config_path=...):
- **Docker (recommended):** `docker run -it --ipc=host --name torchsim -w /workspace/PyTorchSim ghcr.io/psal-postech/torchsim-ci:v1.0.1 bash`
- **TOGSim from source:** `cd TOGSim && mkdir -p build && cd build && conan install .. --build=missing && cmake .. && make -j$(nproc)`
- **PyTorchSimDevice (Python package):** `cd PyTorchSimDevice && python -m pip install --no-build-isolation -e .`
- **gem5 / LLVM+MLIR / Spike from source:** `bash scripts/build_from_source.sh` (clones to `/workspace/{gem5,llvm-project,riscv-isa-sim}`)
- **gem5 / LLVM+MLIR / Spike from source:** `bash scripts/build_from_source.sh` (clones to `/workspace/{gem5,llvm-project,riscv-isa-sim}` at the tags pinned in `thirdparty/github-releases.json`, same manifest as the CI docker image).

Conan deps for TOGSim: `boost/1.79.0`, `robin-hood-hashing/3.11.5`, `spdlog/1.11.0`, `yaml-cpp/0.8.0`.

Expand All @@ -137,5 +137,6 @@ Conan deps for TOGSim: `boost/1.79.0`, `robin-hood-hashing/3.11.5`, `spdlog/1.11

## Git workflow (per CONTRIBUTING.md)

- Forkbranch (`feature/<name>`)PR against **`develop`**, not `main`.
- Fork, branch (`feature/<name>`), PR against `develop`, not `main`.
- Commit prefix style observed: `[Frontend] ...`, `[TOGSim] ...`, etc.
- Commit messages: plain text only. No Markdown formatting (no backticks, bold, bullet lists, headings). Avoid Unicode where ASCII works (use `->` not arrows, `--` not em-dashes, straight quotes).
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@ This script builds [Gem5](https://github.com/PSAL-POSTECH/gem5.git), [LLVM](http
```bash
bash scripts/build_from_source.sh
```
The script clones each dep at the tag pinned in [`thirdparty/github-releases.json`](thirdparty/github-releases.json), the same manifest the CI docker image uses, so a from-source build matches the docker env.
### Run Examples
The `tests` directory contains several AI workload examples.
```bash
Expand Down
68 changes: 53 additions & 15 deletions scripts/build_from_source.sh
Original file line number Diff line number Diff line change
@@ -1,22 +1,60 @@
#!/bin/bash
#!/usr/bin/env bash
# Build Gem5 / LLVM+MLIR / Spike from source.
#
# Versions are pinned in thirdparty/github-releases.json - the same manifest
# the CI docker image (ghcr.io/psal-postech/torchsim-ci) is built against.
# Cloning untagged HEADs has caused mlir-opt option-name drift in the past
# (e.g. test-tile-operation-graph's `sample-mode` <-> `tls_mode` rename), so
# always honor the pinned release_tag for a known-good Python<->mlir-opt pair.
set -euo pipefail

ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
MANIFEST="${ROOT}/thirdparty/github-releases.json"
home="/workspace"
cd $home

if [ ! -f "$MANIFEST" ]; then
echo "error: pin manifest not found at $MANIFEST" >&2
exit 1
fi
if ! command -v jq >/dev/null 2>&1; then
echo "jq not found, installing..."
apt -y update && apt -y install jq
fi

read_pin() {
# $1 = key (gem5 / llvm_project / spike), echoes "<repo> <tag>"
jq -r --arg k "$1" '.[$k] | "\(.repository) \(.release_tag)"' "$MANIFEST"
}

read GEM5_REPO GEM5_TAG <<< "$(read_pin gem5)"
read LLVM_REPO LLVM_TAG <<< "$(read_pin llvm_project)"
read SPIKE_REPO SPIKE_TAG <<< "$(read_pin spike)"

echo "Building from source using pins in $MANIFEST:"
echo " gem5 = ${GEM5_REPO} @ ${GEM5_TAG}"
echo " llvm = ${LLVM_REPO} @ ${LLVM_TAG}"
echo " spike = ${SPIKE_REPO} @ ${SPIKE_TAG}"

cd "$home"

# Gem5
apt -y update && apt -y upgrade && apt -y install scons
git clone https://github.com/PSAL-POSTECH/gem5.git
cd gem5 && scons build/RISCV/gem5.opt -j $(nproc)
export GEM5_PATH=$home/gem5/build/RISCV/gem5.opt
cd $home

# LLVM
git clone https://github.com/PSAL-POSTECH/llvm-project.git
cd llvm-project && mkdir build && cd build && \
cmake -DLLVM_ENABLE_PROJECTS=mlir -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/riscv-llvm -DLLVM_TARGETS_TO_BUILD=RISCV -G "Unix Makefiles" ../llvm && \
git clone --depth 1 --branch "$GEM5_TAG" "https://github.com/${GEM5_REPO}.git"
cd gem5 && scons build/RISCV/gem5.opt -j "$(nproc)"
export GEM5_PATH="$home/gem5/build/RISCV/gem5.opt"
cd "$home"

# LLVM + MLIR (RISCV target)
git clone --depth 1 --branch "$LLVM_TAG" "https://github.com/${LLVM_REPO}.git"
cd llvm-project && mkdir -p build && cd build && \
cmake -DLLVM_ENABLE_PROJECTS=mlir -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/riscv-llvm -DLLVM_TARGETS_TO_BUILD=RISCV \
-G "Unix Makefiles" ../llvm && \
make -j && make install
cd $home
cd "$home"

# Spike Simulator
git clone https://github.com/PSAL-POSTECH/riscv-isa-sim.git --branch TorchSim && cd riscv-isa-sim && mkdir build && cd build && \
../configure --prefix=$RISCV && make -j && make install
cd $home
git clone --depth 1 --branch "$SPIKE_TAG" "https://github.com/${SPIKE_REPO}.git"
cd riscv-isa-sim && mkdir -p build && cd build && \
../configure --prefix="$RISCV" && make -j && make install
cd "$home"
Loading