Skip to content
Scripture Forge: Collaborative translation with suggestions and community Scripture checking; all connected to Paratext
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.vscode Remove legacy VS Code tasks May 9, 2019
deploy SF-294 Update vagrant (#33) May 16, 2019
readme_images Update README for new repo (#3) May 3, 2019
scripts Cleanup obsolete identity code Apr 29, 2019
src
test SF-299 Get UserService to return a partial UserResource (#30) May 15, 2019
.buildpath Add beta.qa.scriptureforge.org and README2 Apr 29, 2019
.editorconfig SF-58: Prettier V2 files Apr 29, 2019
.gitattributes Add beta.qa.scriptureforge.org and README2 Apr 29, 2019
.gitignore SF-169 add/edit questions (#552) Apr 29, 2019
.gitmodules Add beta.qa.scriptureforge.org and README2 Apr 29, 2019
.prettierignore SF-58: Prepare Prettier for CI builds Apr 29, 2019
.prettierrc.yml Don't prettier CSHTML (fix bad merge) Apr 29, 2019
.project Add beta.qa.scriptureforge.org and README2 Apr 29, 2019
Directory.Build.targets Add .NET unit test projects Apr 29, 2019
LICENSE.txt Add beta.qa.scriptureforge.org and README2 Apr 29, 2019
NuGet.config Add translate module and editor component Apr 29, 2019
README.md SF-294 Update vagrant (#33) May 16, 2019
after.xForge.sln.targets Add .NET unit test projects Apr 29, 2019
xForge.sln Feature/code generator (#497) Apr 29, 2019

README.md

web-xforge

The existing site details are here. This document is for the Beta sites.

Language Forge and Scripture Forge represent different websites, but have the same code base stored in one repository.

Users

To use Language Forge go to languageforge.org.

To use Scripture Forge go to scriptureforge.org.

User Problems

To report a user issue with the Language Forge application, email "issues @ languageforge dot org".

To report a user issue with the Scripture Forge application, email "issues @ scriptureforge dot org".

Special Thanks To

For end-to-end test automation:

BrowserStack Logo

For error reporting:

Bugsnag logo

Developers

Builds

Status of builds from our continuous integration (CI) server:

Site Master Unit Master E2E QA Live
Scripture Forge v2 Build Status not yet operational Build Status not yet live

Successful builds from our CI server deploy to:

Site QA Live
Scripture Forge v2 beta.qa.scriptureforge.org beta.scriptureforge.org

Gitflow

We use Gitflow with some modifications:

  • Our master branch is the Gitflow develop development branch. All pull requests go against master.
  • Our qa branch is similar to a Gitflow release branch.
  • Our live branch is the the Gitflow master production branch.

If you are working on a site Beta then it looks like normal Gitflow and pull requests go against the relevant site development branch.

We merge from master to the QA testing branch, then ship from the QA branch to the live production branch.

Site Development Branch QA Branch Production Branch
Scripture Forge v2 master sf-qa sf-live

Style Guides

TypeScript follows the Angular Style Guide. This is opinionated not only about things like file name conventions but also file and folder structure.

To this end you'll also want to be familiar with Upgrading from AngularJS particularly the Preparation section.

We use Prettier with a pre-commit hook.

Layout

We use Angular Flex-Layout with Angular MDC including the Material Design Icons.

Recommended Development Environment

Our recommended development environment for web development is Ubuntu 16.04.

  • Vagrant GUI Setup. A Vagrant box with xForge already installed is downloaded and set up on your machine. This is the easiest and cleanest to setup.
  • Local Linux Development Setup. Everything is installed directly on your machine, which needs to be running Ubuntu 16.04. This is the fastest method because development is not done in a virtual machine.

Vagrant GUI Setup

Install VirtualBox, Vagrant, and git. To do this in Linux, run

sudo apt install vagrant virtualbox virtualbox-guest-additions-iso git

Setup git. At least name and email is needed in .gitconfig. You can do this from a terminal by running

git config --global user.name "My Name"
git config --global user.email "me@example.com"

Hardware-assisted virtualization (VT-x or AMD-V) needs to be enabled in your BIOS.

Create a directory to manage the development machine, such as xforge. Checkout the xforge git repository to access (and later receive updates to) the vagrant development machine configuration file:

git clone https://github.com/sillsdev/web-xforge.git
cd web-xforge.git/deploy/vagrant_xenial_gui

Run vagrant up. This will download, initialize, and run the development machine. The machine is about 5GB, so expect the download to take a while.

In the guest development machine, compile Scripture Forge, browse to http://localhost:5000 and use the default login credentials "admin" and "password".

Local Linux Development Setup

Start by installing Git and Ansible:

sudo add-apt-repository ppa:ansible/ansible
sudo apt update
sudo apt install -y git ansible

Now create a directory for installation and clone the repo:

git clone --recurse-submodules https://github.com/sillsdev/web-xforge

The --recurse-submodules is used to fetch many of the Ansible roles used by the Ansible playbooks in the deploy folder. If you've already cloned the repo without --recurse-submodules, run git submodule update --init --recursive to pull and initialize them.

Change the variable mongo_path: /var/lib/mongodb in deploy/vars_palaso.yml. Set it to a location where MongoDB should store its databases.

  • Vagrant VM Setup: uncomment line 6 and comment line 5
  • Local Linux Development Setup: uncomment line 5 and comment line 6 (or whatever is appropriate on your system, its best to have Mongo store databases on your HDD rather than SSD). Make sure the mongodb user has permission to read and write to the path you specify.

Run the following Ansible playbooks to configure Ansible and run both sites.

cd web-xforge/deploy
ansible-playbook -i hosts playbook_create_config.yml --limit localhost -K
ansible-playbook playbook_xenial.yml --limit localhost -K

Development Process

The first task on a job is to create a feature branch. Branch off of the master branch.

git checkout master
git pull
git checkout -b feature/<featureName>

Do some useful work and commit it.

Upload your work:

git push origin feature/<featureName>

Rebase often (at least at the start of the day, and before making a PR). Force pushing to your own branch is fine (even during review).

Make PR's against the master branch. If the master branch has moved on since the feature branch was made, rebase your changes on top of the master branch before making your PR.

Ensure all tests are passing before submitting a PR.

We use Reviewable for GitHub Pull Requests (PRs). When submitting a PR, a This change is Reviewable link is added to the PR description. Remember to click the Publish button after adding comments in Reviewable.

If the person reviewing feels comfortable to approve it they can. However if they want other eyes on it, mention it in a comment on the PR. If you have minor changes to request on a PR you can say 'Make change X and then LGTM'. This means the person making the PR can merge it themselves after the requested change. People merging PRs can and should rebase the completed PR change (default to squash and rebase unless commits have good reason to stay separate).

Delete the PR branch after merge.

Reference

Testing

.NET Unit Testing

To run back end unit tests, from the repo (repository) root

dotnet test

See documentation for running tests and writing tests.

Angular Linting and Prettiering

To check TypeScript for readability, maintainability, and functionality errors, and to check a few other files for proper formatting. From the repo root

cd src/SIL.XForge.Scripture/ClientApp/
npm run prettier
ng lint

Or just use VS Code with this project's recommended extensions.

Angular Unit Testing

To run front end unit tests, make sure ng serve is not running (CTRL-C to end them), then from the repo root

cd src/SIL.XForge.Scripture/ClientApp/
CHROME_BIN=chromium-browser ng test

You can make the environment variable (CHROME_BIN=chromium-browser) permanent by following the instructions here, then you can simply run ng test. The environment variable is already set in the vagrant.

ng test will monitor and run tests in a Chromium browser window. You can also monitor and run tests headlessly from the command line by running

src/SIL.XForge.Scripture/ClientApp/monitor-test-headless.sh

Or just run tests once without monitoring with

src/SIL.XForge.Scripture/ClientApp/test-headless.sh

You can filter the tests to compile and run by passing spec file names as arguments. For example,

src/SIL.XForge.Scripture/ClientApp/monitor-test-headless.sh some.component.spec.ts another.component.spec.ts

Debugging Unit Tests

The best way to debug Angular unit tests is with Chromium.

  • Run npm test (which will include source maps, ng test does not)
  • When the Chromium window appears, press F12
  • Click the Sources tab
  • Files might show up under webpack:// or context/localhost:dddd/src or elsewhere, but you can always press CTRL-P and type the name of a file to get there faster.

This video has a live demo of the process.

Filtering Unit Tests

To run (or not to run) specific tests or fixtures, you can use the prefixes focus and exclude, as in fdescribe or fit to run only the specified functions, or xdescribe and xit to skip running the specified functions (but all functions will still be built). To skip building extra tests, modify the filter in src/SIL.XForge.Scripture/ClientApp/src/test.ts per these instructions.

See documentation for running tests and writing tests.

Angular End-To-End (E2E) Testing

To run E2E tests, make sure you are serving the app. From the repo root

cd src/SIL.XForge.Scripture/
dotnet run --environment "Testing"

In another terminal, from the repo root

cd src/SIL.XForge.Scripture/ClientApp/
./rune2e.sh

Debugging E2E Tests

To debug E2E tests, from the repo root

cd src/SIL.XForge.Scripture/
dotnet run --environment "Testing"

In another terminal, from the repo root

cd src/SIL.XForge.Scripture/ClientApp/
ng serve

Add a new line of debugger; to the *.e2e-spec.ts where you want it to break.

In another terminal, from the repo root

cd src/SIL.XForge.Scripture/ClientApp/
./rune2e.sh debug

Open chrome://inspect/#devices in Chromium and click inspect. This opens an instance of DevTools and immediately breaks the code at the top of the ng module. Click the continue button (or press F8) in your debugger to run your e2e tests, and hit any debugger statements in your code. Close the DevTools window to finish the tests.

PWA Testing

To test the PWA (Progressive Web App), build the app for PWA testing and run the server without ng serve. From the repo root

cd src/SIL.XForge.Scripture/ClientApp/
ng build --configuration=pwaTest

In another terminal, from the repo root

cd src/SIL.XForge.Scripture/
dotnet run --start-ng-serve=no

!!! IMPORTANT !!! When you have finished testing, remove the built app dist folder. From the repo root

rm -rf src/SIL.XForge.Scripture/ClientApp/dist

Backend Development

Normally when you run dotnet run it starts ng serve for you. This works great if you are developing on the front end as it watches for file changes and reloads your browser once it has compiled.

If you are developing on the backend this works better

cd src/SIL.XForge.Scripture/
dotnet watch run --start-ng-serve=listen

In another terminal

cd src/SIL.XForge.Scripture/ClientApp/
ng serve

When files change on the backend it will compile the changes automatically and now ng serve won't re-start every time.

Database

The VS Code extension Azure Cosmos DB can be used to inspect our Mongo DB.

Debugging

In Visual Studio Code, in the debug sidebar, choose Full App (SF) to debug the front-end and back-end at the same time, or Launch Chrome (SF) or .NET Core (SF) to just debug the front-end or back-end.

Code Generator

In Visual Studio Code, ensure dotnet is not running, then click Terminal > Run Task > Generate SF model.

On the command line, change to the /src/SIL.XForge.Scripture/CodeGenerator folder and run dotnet run -s ./sfmodel.json -t ../ClientApp/src/app/core/models/sfdomain-model.generated.ts -m SIL.XForge.Scripture.Models.SFProjectResource -j ../ClientApp/src/app/core/models/sfdomain-model-config.generated.ts

You can’t perform that action at this time.