Feature/0712 model base class #713

bob-carpenter · 2019-07-09T16:11:40Z

Submisison Checklist

Run tests: ./runCmdStanTests.py src/test
Declare copyright holder and open-source license: see below

Summary:

Update the basic command (command.hpp) to instantiate the services with a base model class (stan::model::base_model) and build a reference using a new function defined in the generated model code as of stan-dev/stan PR 2785.

Intended Effect:

Allow the command to be compiled independently of the model to speed up compilation times.

How to Verify:

All functionality should still work as is.

Side Effects:

User-facing behavior is unchanged.

There may be performance issues on some platforms as we've seen one model in our test suite take longer on our old Mac pro but not on other platforms.

Documentation:

There's no user-facing documentation and both code and makefile changes are minimal.

Copyright and Licensing

Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company):

Columbia University

By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses:

Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

…ature/0712-model-base-class

…an-dev/cmdstan into feature/0712-model-base-class

bob-carpenter · 2019-07-09T16:45:04Z

These tests are going to fail until stan-dev/stan PR 2785](stan-dev/stan#2785) is merged, because the submodule won't be pointing to the right place until then.

bob-carpenter · 2019-07-09T16:47:03Z

Sorry, this branch isn't even working on my machine. I'll ping this again when it's working.

…an-dev/cmdstan into feature/0712-model-base-class

…ature/0712-model-base-class

…an-dev/cmdstan into feature/0712-model-base-class

bob-carpenter · 2019-07-09T22:07:31Z

This is ready to review now.

wds15 · 2019-07-10T19:38:35Z

As pointed out on discourse:

"new with this PR (to me) is that now CmdStan picks up stuff which are defined in the Stan makefiles - at least this is how we should handle it. Right now the additional make targets are defined in both repos - which is not good, I think."

mitzimorris · 2019-07-11T10:17:42Z

Right now the additional make targets are defined in both repos - which is not good

pre-existing redundancies between the CmdStan and Stan makefiles should be addressed via a different issue.

in this PR, the files makefile and make/program were changed. otherwise, minimal changes have been made to either CmdStan or Stan makefile in the past 10 months.

wds15 · 2019-07-13T08:55:21Z

Sure, pre-existing redundancies are not part of this PR - it's just that I have the impression that you do introduce additional redundancies with this PR. If that's right then you create technical debt and that should not be the case. Let me have a look into it again and maybe I can suggest something here.

Anyway, right now I am getting this error:

make: *** No rule to make target `stan/src/stan/model/model_header.d', needed by `build'.  Stop.
make: *** Waiting for unfinished jobs....

I have cmdstan at this branch and the stan repository is moved to the branch feature/2759-model-base. Is something wrong with what I have done?

wds15

Please also update the compile-info target in the makefile. This is there to show the commands which are executed for a stan model. This now two commands.

Other than that this is really cool. Compilation times go down from 22s to 5s on my system for the bernoulli example.

wds15 · 2019-07-13T11:35:05Z

make/program

@@ -2,6 +2,7 @@
 # Models (to be passed through stanc)
 ##
 CMDSTAN_MAIN ?= src/cmdstan/main.cpp
+CMDSTAN_MAIN_O ?= src/cmdstan/main.o


Please use the patsubst command to derive from CMDSTAN_MAIN the value for CMDSTAN_MAIN_O.

wds15 · 2019-07-13T11:36:43Z

make/program

 %.hpp : %.stan bin/stanc$(EXE)
 	@echo ''
 	@echo '--- Translating Stan model to C++ code ---'
 	$(WINE) bin/stanc$(EXE) $(STANCFLAGS) --o=$@ $<

+%.d: %.hpp


This definition here overrides the one included in the top makefile which pulls in math/make/dependencies. Could you please document that you do this here on purpose. The alternative is to not include the dependencies from math which we may not need actually for cmdstan?

removing the override.
we need the dependencies rule.

Ok...once you do that, then the model_header.o dependency can be build. The only downside is that then you will build the dependency thing for the stan model hpp file... which wastes 1s of compile time as it anyway gets recreated with any model update...but ok; still a win.

wds15 · 2019-07-13T11:39:12Z

make/program

-	$(LINK.cpp) $(CXXFLAGS_PROGRAM) -include $< $(CMDSTAN_MAIN) $(LDLIBS) $(LIBSUNDIALS) $(MPI_TARGETS) $(OUTPUT_OPTION)
+	@echo '--- Compiling, linking C++ code ---'
+	cp $*.hpp $*.cpp; \
+    $(LINK.cpp) $(CXXFLAGS_PROGRAM) -c -o $*.o $*.cpp ; \


Oh.. this is less ideal, I think. You can make the compiler interpret the hpp file just like a c++ file with the -x c++ option. So I would suggest to replace 38-41 with this:

$(COMPILE.cpp) $(CXXFLAGS_PROGRAM) -x c++ -o $*.o $< $(LINK.cpp) $(LDLIBS) $(LIBSUNDIALS) $(MPI_TARGETS) $(CMDSTAN_MAIN_O) $*.o $(OUTPUT_OPTION)

This way you pickup the right things for compilation (first line) and for linking (second line). Moreover, you do not need to do this copying and removing business.

wds15 · 2019-07-13T11:40:01Z

makefile

@@ -151,7 +151,7 @@ build-mpi: $(MPI_TARGETS)
 	@echo '--- boost mpi bindings built ---'

 .PHONY: build
-build: bin/stanc$(EXE) bin/stansummary$(EXE) bin/print$(EXE) bin/diagnose$(EXE) $(LIBSUNDIALS) $(MPI_TARGETS)
+build: bin/stanc$(EXE) bin/stansummary$(EXE) bin/print$(EXE) bin/diagnose$(EXE) $(LIBSUNDIALS) $(MPI_TARGETS) $(CMDSTAN_MAIN_O) $(STAN)src/stan/model/model_header.d


The model_header.d should just be removed here, I think. At least this caused errors for me and I don't quite see the need for this.

this can stay now once you remove the %.d override, but please verify.

wds15 · 2019-07-13T11:41:16Z

makefile

@@ -182,6 +182,7 @@ clean-manual:

 clean-all: clean clean-deps clean-libraries clean-manual
 	$(RM) -r bin
+	$(RM) -r src/cmdstan/main.o


You want to delete $(CMDSTAN_MAIN_O) here and not directly name the file defined in the variable.

Can this be adjusted?

…ature/0712-model-base-class

mitzimorris · 2019-07-18T02:16:55Z

@syclik or @wds15 - most definitely failing unit tests on linux with mpi - here's the end of the error log:

src/cmdstan/main.o: In function `stan::math::mpi_cluster::~mpi_cluster()':
main.cpp:(.text._ZN4stan4math11mpi_clusterD2Ev[_ZN4stan4math11mpi_clusterD5Ev]+0x44): undefined reference to `boost::mpi::environment::~environment()'
collect2: error: ld returned 1 exit status
make/program:38: recipe for target 'src/test/test-models/print_uninitialized' failed

what needs to be included and/or linked where?

mitzimorris · 2019-07-18T02:25:16Z

this appears to be the command that's failing - complaints are about
stan/lib/stan_math/stan/math/prim/arr/functor/mpi_cluster_inst.o
???

mpicxx.openmpi -std=c++1y  -Wno-sign-compare    -Wno-delete-non-virtual-dtor -O3 -I src -I stan/src -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.3 -I stan/lib/stan_math/lib/boost_1.69.0 -I stan/lib/stan_math/lib/sundials_4.1.0/include -I stan/lib/stan_math/lib/gtest_1.8.1/include -I stan/lib/stan_math/lib/gtest_1.8.1     -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION   -DSTAN_MPI -DGTEST_USE_OWN_TR1_TUPLE       -Wl,-L,"/mnt/nvme1n1/workspace/CmdStan_PR-713/stan/lib/stan_math/lib/boost_1.69.0/stage/lib" -Wl,-rpath,"/mnt/nvme1n1/workspace/CmdStan_PR-713/stan/lib/stan_math/lib/boost_1.69.0/stage/lib"         stan/lib/stan_math/lib/sundials_4.1.0/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_4.1.0/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_4.1.0/lib/libsundials_idas.a stan/lib/stan_math/lib/boost_1.69.0/stage/lib/libboost_serialization.so stan/lib/stan_math/lib/boost_1.69.0/stage/lib/libboost_mpi.so stan/lib/stan_math/stan/math/prim/arr/functor/mpi_cluster_inst.o src/cmdstan/main.o src/test/test-models/proper.o -o src/test/test-models/proper

stan-buildbot · 2019-07-18T02:45:26Z

(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 0.98)
(stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan, 0.94)
(stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan, 1.0)
(stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan, 0.98)
(stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan, 1.02)
(stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan, 0.99)
(stat_comp_benchmarks/benchmarks/arK/arK.stan, 1.01)
(performance.compilation, 1.54)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan, 1.0)
(stat_comp_benchmarks/benchmarks/sir/sir.stan, 1.02)
(stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan, 1.01)
(stat_comp_benchmarks/benchmarks/garch/garch.stan, 1.11)
(stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan, 1.01)
(stat_comp_benchmarks/benchmarks/arma/arma.stan, 0.96)
Result: 1.03819002285
Commit hash: 9404d6f

syclik · 2019-07-18T03:13:47Z

@mitzimorris, I'm sorry -- I don't know. I looked at it, but I wouldn't know where to go for the MPI build stuff; I can read it, but I can't really maintain it. You'll have to get help from @wds15 or someone else that understands exactly how this is being built.

It might be worth checking that GPU also builds independently of this failure.

What should we do about the state of CmdStan and Stan? Should we revert? cc: @bob-carpenter, @seantalts

wds15 · 2019-07-18T07:51:02Z

I looked at this yesterday for a moment, but i wasnˋt able to figure it out of what is going on. I can certainly spend some time on this over the weekend, but today or tomorrow is not possible for me to do that.

Generally you need to dynamically link against the boost stuff and you need to statically link in the mpi cluster inst thing...not sure what is going wrong right now.

seantalts · 2019-07-18T09:05:21Z

Re: reverting Stan - from what I understand, those changes are safe to make without this branch being merged in. The tests seem to be passing in develop too to second that idea.

seantalts · 2019-07-18T09:06:25Z

.circleci/config.yml

-  version: 2
-  build_and_test:
-    jobs:
-      - manual


How are the docs made for CmdStan now, by the way?

updated circleci config to remove dependencies on Stan docs. now circleci checks task make manual - tests passing.

seantalts · 2019-07-18T09:08:28Z

make/program



 ##
-# Dependencies file
+# Dependencies file for precompiled header


I don't think this comment is right - these are dependency files for everything. Line 53 includes the one from the model_header file.

seantalts · 2019-07-18T09:10:34Z

make/program

-	$(LINK.cpp) $(CXXFLAGS_PROGRAM) -include $< $(CMDSTAN_MAIN) $(LDLIBS) $(LIBSUNDIALS) $(MPI_TARGETS) $(OUTPUT_OPTION)
+	@echo '--- Compiling, linking C++ code ---'
+	$(COMPILE.cpp) $(CXXFLAGS_PROGRAM) -x c++ -o $*.o $<
+	$(LINK.cpp) $(LDLIBS) $(LIBSUNDIALS) $(MPI_TARGETS) $(CMDSTAN_MAIN_O) $*.o $(OUTPUT_OPTION)


This sounds dumb, but have you tried swapping the order of the link libraries? Try putting CMDSTAN_MAIN_O back one or even at the beginning of the list. I know link order does matter in weird ways sometimes.

…ature/0712-model-base-class

stan-buildbot · 2019-07-18T14:39:10Z

(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 0.96)
(stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan, 0.96)
(stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan, 1.0)
(stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan, 1.01)
(stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan, 1.0)
(stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan, 1.01)
(stat_comp_benchmarks/benchmarks/arK/arK.stan, 1.02)
(performance.compilation, 1.55)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan, 0.99)
(stat_comp_benchmarks/benchmarks/sir/sir.stan, 0.99)
(stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan, 0.97)
(stat_comp_benchmarks/benchmarks/garch/garch.stan, 1.02)
(stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan, 0.97)
(stat_comp_benchmarks/benchmarks/arma/arma.stan, 1.0)
Result: Regex did not match anything
Commit hash: e0d08d5

wds15 · 2019-07-18T15:06:05Z

+1

stan-buildbot · 2019-07-18T15:11:57Z

(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 1.01)
(stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan, 0.95)
(stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan, 1.0)
(stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan, 1.01)
(stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan, 0.99)
(stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan, 1.03)
(stat_comp_benchmarks/benchmarks/arK/arK.stan, 1.01)
(performance.compilation, 1.55)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan, 0.98)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan, 1.0)
(stat_comp_benchmarks/benchmarks/sir/sir.stan, 1.03)
(stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan, 0.95)
(stat_comp_benchmarks/benchmarks/garch/garch.stan, 1.01)
(stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan, 0.99)
(stat_comp_benchmarks/benchmarks/arma/arma.stan, 0.93)
Result: 1.02884796285
Commit hash: 0e76075

mitzimorris · 2019-07-18T15:48:53Z

@wds15 - got the makefiles working (thanks @seantalts! you were right about link order) -
wanna do another review and/or sign off?

wds15 · 2019-07-18T18:02:11Z

I think all is good... with the latest changes the Jenkins tests are not anymore run. Is that on purpose? I think they should still run.

(This is major hack to ensure things are in place)

mitzimorris · 2019-07-18T18:04:33Z

~~that's really weird and certainly not intended~~

maybe changes just to .circleci/config.yml don't kick off jenkins test? I make a trivial change to the makefile and now circleci, jenkins, and travis are all checking again.

stan-buildbot · 2019-07-18T18:43:17Z

(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan, 0.96)
(stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan, 1.0)
(stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan, 0.98)
(stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan, 0.98)
(stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan, 0.99)
(stat_comp_benchmarks/benchmarks/arK/arK.stan, 1.01)
(performance.compilation, 1.56)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan, 0.99)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan, 1.0)
(stat_comp_benchmarks/benchmarks/sir/sir.stan, 1.01)
(stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan, 0.99)
(stat_comp_benchmarks/benchmarks/garch/garch.stan, 1.01)
(stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan, 1.01)
(stat_comp_benchmarks/benchmarks/arma/arma.stan, 1.0)
Result: 1.03266786656
Commit hash: 655e93f

stan-buildbot · 2019-07-18T18:53:35Z

(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 0.99)
(stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan, 0.98)
(stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan, 1.0)
(stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan, 1.0)
(stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan, 0.98)
(stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan, 1.0)
(stat_comp_benchmarks/benchmarks/arK/arK.stan, 1.01)
(performance.compilation, 1.56)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan, 0.99)
(stat_comp_benchmarks/benchmarks/sir/sir.stan, 0.98)
(stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan, 0.97)
(stat_comp_benchmarks/benchmarks/garch/garch.stan, 1.01)
(stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan, 1.0)
(stat_comp_benchmarks/benchmarks/arma/arma.stan, 0.99)
Result: 1.03137891171
Commit hash: 655e93f

seantalts · 2019-07-19T09:48:51Z

🎉 So glad this made it in! Down to the wire, haha.

Bob Carpenter and others added 14 commits July 5, 2019 16:03

template constructor to base model

523c11f

Merge branch 'develop' of https://github.com/stan-dev/cmdstan into fe…

0e7723b

…ature/0712-model-base-class

remove unused template parameter in command

162ef7b

update includes for command

2590273

include model_base

e9f3f43

updated makefile rules

fc9ef34

checkpointing makefile

de431f8

Merge branch 'feature/0712-model-base-class' of https://github.com/st…

bc7ee52

…an-dev/cmdstan into feature/0712-model-base-class

remove spurious blank line

cde80e7

change link to compile; same definition

1af3eee

model headers at compile, not link

845ec86

cannot make manual on circle ci - docs not in bookdown

9c66a19

revert to develop version of submodule stan

de37126

Merge branch 'feature/0712-model-base-class' of https://github.com/st…

60521aa

…an-dev/cmdstan into feature/0712-model-base-class

mitzimorris and others added 7 commits July 9, 2019 13:36

use makefile variable for precompiled headers

79b133c

Merge branch 'feature/0712-model-base-class' of https://github.com/st…

81de06f

…an-dev/cmdstan into feature/0712-model-base-class

remove .d generation to make executable

6d0dd9f

Merge branch 'develop' of https://github.com/stan-dev/cmdstan into fe…

51f39b8

…ature/0712-model-base-class

Merge branch 'feature/0712-model-base-class' of https://github.com/st…

0f6b9d5

…an-dev/cmdstan into feature/0712-model-base-class

makefile removes main.o as part of clean-all

7e3b9d3

make/program better readiblity; cleans out .cpp and .o files

81f659c

wds15 requested changes Jul 13, 2019

View reviewed changes

mitzimorris added 2 commits July 13, 2019 22:53

use pathsubst for CMDSTAN_MAIN_O

506b25c

Merge branch 'develop' of https://github.com/stan-dev/cmdstan into fe…

7e316f0

…ature/0712-model-base-class

seantalts reviewed Jul 18, 2019

View reviewed changes

mitzimorris added 2 commits July 18, 2019 09:51

Merge branch 'develop' of https://github.com/stan-dev/cmdstan into fe…

4e42237

…ature/0712-model-base-class

fix comment, try different link order

e0d08d5

fix mpi test

0e76075

mitzimorris added 2 commits July 18, 2019 13:42

add back in test for manual

18f23f4

simplify test for manual

2157e92

makefile comment

655e93f

wds15 approved these changes Jul 18, 2019

View reviewed changes

mitzimorris merged commit 3244b1a into develop Jul 18, 2019

mitzimorris mentioned this pull request Jul 18, 2019

use model base class for standalone compilation #712

Closed

serban-nicusor-toptal added this to the 2.20.0 milestone Jul 18, 2019

seantalts deleted the feature/0712-model-base-class branch July 19, 2019 09:49

rok-cesnovar mentioned this pull request Aug 13, 2019

Default Random Seed Doesn't Work As Expected #722

Closed

Feature/0712 model base class #713

Feature/0712 model base class #713

Conversation

bob-carpenter commented Jul 9, 2019

Submisison Checklist

Summary:

Intended Effect:

How to Verify:

Side Effects:

Documentation:

Copyright and Licensing

bob-carpenter commented Jul 9, 2019 • edited

bob-carpenter commented Jul 9, 2019

bob-carpenter commented Jul 9, 2019

wds15 commented Jul 10, 2019

mitzimorris commented Jul 11, 2019

wds15 commented Jul 13, 2019

wds15 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mitzimorris commented Jul 18, 2019

mitzimorris commented Jul 18, 2019

stan-buildbot commented Jul 18, 2019

syclik commented Jul 18, 2019

wds15 commented Jul 18, 2019

seantalts commented Jul 18, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stan-buildbot commented Jul 18, 2019

wds15 commented Jul 18, 2019

stan-buildbot commented Jul 18, 2019

mitzimorris commented Jul 18, 2019

wds15 commented Jul 18, 2019

mitzimorris commented Jul 18, 2019 • edited

stan-buildbot commented Jul 18, 2019

stan-buildbot commented Jul 18, 2019

seantalts commented Jul 19, 2019

bob-carpenter commented Jul 9, 2019 •

edited

mitzimorris commented Jul 18, 2019 •

edited