Fixup documentation links

.github/workflows/main.yml

@@ -67,6 +67,17 @@ jobs:
         # so we instruct git-diff to ignore those when considering changes.
         git --no-pager diff --ignore-cr-at-eol --exit-code
 
+  # Check in-repo markdown links
+  markdown-link-check:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@master
+    - uses: gaurav-nelson/github-action-markdown-link-check@v1
+      with:
+        use-quiet-mode: yes
+        folder-path: './documentation, ./build, ./source, ./test, ./scripts'
+        file-path: './CODE_OF_CONDUCT.md, ./CONTRIBUTING.md, ./README.md, ./SECURITY.md'
+
   build-publish-website:
     name: Build and publish website
     runs-on: ubuntu-20.04
@@ -76,29 +87,20 @@ jobs:
     - name: Set up Python
       uses: actions/setup-python@v2
       with:
-        python-version: '3.x'
+        python-version: '3.7'
     - name: Install apt dependencies
       run: sudo apt-get -y install build-essential libfreetype-dev unixodbc-dev hugo
-    - name: Install MLOS and dependencies
+    - name: Update pip
       run: |
         python -m pip install --upgrade pip
         python -m pip install --upgrade wheel
-        pip install source/Mlos.Python/
-        pip install matplotlib sphinx nbconvert sphinx_rtd_theme numpydoc
-    - name: Generate markdown for hugo
-      run: website/build_site.sh
-    - name: Run hugo
-      run: |
-        cd website
-        hugo
-    - name: Generate API rst files
-      run: website/sphinx/apidoc.sh
-    - name: Run sphinx
-      run: |
-        cd website/sphinx
-        make
-        cp -a _build/html ../python_api
-
+        python -m pip install --upgrade pyyaml
+    - name: Generate Python API docs using sphinx
+      run: make -C website sphinx-site
+    - name: Generate main site from markdown using hugo
+      run: make -C website hugo-site
+    - name: Test the links in the output site
+      run: make -C website link-check
     - name: Deploy to GitHub pages
       if: ${{ github.ref == 'refs/heads/main' }}
       uses: JamesIves/github-pages-deploy-action@3.5.9

.gitignore

@@ -382,6 +382,8 @@ healthchecksdb
 obj/
 objd/
 
+temp/
+
 # Build errors
 *.wrn
 *.err
@@ -419,7 +421,10 @@ source/Mlos.Proto/mlos/Grpc/OptimizerService.proto
 
 # generated website
 website/public
-website/content/documentation/
-website/content/notebooks/
+website/content
+!website/content/menu/index.md
+website/resources
+website/themes
 website/sphinx/_build
 website/sphinx/api
+website/python_api

Dockerfile

@@ -7,7 +7,7 @@
 #   UbuntuVersion=20.04; docker build --build-arg=UbuntuVersion=$UbuntuVersion -t mlos/build:ubuntu-$UbuntuVersion .
 #
 # Optionally, build with a proxy server:
-#   UbuntuVersion=20.04; docker build --build-arg=http_proxy=http://some-proxy-caching-host:3128 -t mlos/build:ubuntu-$UbuntuVersion
+#   UbuntuVersion=20.04; docker build --build-arg=http_proxy=http://some-proxy-caching-host:3128 --build-arg=UbuntuVersion=$UbuntuVersion -t mlos/build:ubuntu-$UbuntuVersion .
 #
 # Run with:
 #   docker run -it --name mlos-build-$UbuntuVersion mlos/build:ubuntu-$UbuntuVersion

README.md

@@ -66,52 +66,57 @@ To achieve this MLOS provides:
 
     Once hooks are created in the target system, iteration on the external agent can be more rapidly developed and deployed.
 
-
 ## Python only installation
 
-Some of the examples require only the installation of the mlos Python library. These examples do not use the shared memory infrastructure.
-
+Some of the examples require only the installation of the `mlos` Python library.
+These are simplified examples for getting started and do not use the shared memory infrastructure for cross language/process interactions.
 
-First, download the `mlos` code using git:
+First, download the MLOS code using git:
 
-```
-$ git clone https://github.com/microsoft/MLOS.git
+```shell
+git clone https://github.com/microsoft/MLOS.git
 ```
 
 For this simplified installation, it's recommended to use the [Anaconda python distribution](https://www.anaconda.com/products/individual).
-For a slimer installation experience, you can also use the [miniconda installer](https://docs.conda.io/en/latest/miniconda.html).
+For a slimmer installation experience, you can also use the [miniconda installer](https://docs.conda.io/en/latest/miniconda.html).
 After installing either anaconda or miniconda, you can create a new environment with all requirements for the examples using
 
+> Note currently MLOS needs Python 3.7, but by default conda packages Python 3.8.
+> For the moment, you may need to search the [installer archives](https://repo.anaconda.com/miniconda/) for a python 3.7 version.
+> e.g. [Miniconda3-py37_4.8.3-Windows-x86_64.exe](https://repo.anaconda.com/miniconda/Miniconda3-py37_4.8.3-Windows-x86_64.exe)
+
+```shell
+conda env create -f MLOS/source/Mlos.Notebooks/environment.yml
 ```
-$ conda env create -f MLOS/source/Mlos.Notebooks/environment.yml
-```
+
 The environment will be called `mlos_python_environment` and you can activate it as follows:
-```
-$ conda activate mlos_python_environment
+
+```shell
+conda activate mlos_python_environment
 ```
 
 Use `pip` to install the Python library:
 
-```
-$ pip install MLOS/source/Mlos.Python/
+```shell
+pip install MLOS/source/Mlos.Python/
 ```
 
 Alternatively, you can also install the package directly without checking out the code:
 
-```
-$ pip install "git+https://github.com/microsoft/MLOS.git#egg=mlos&subdirectory=source/Mlos.Python"
+```shell
+pip install "git+https://github.com/microsoft/MLOS.git#egg=mlos&subdirectory=source/Mlos.Python"
 ```
 
 However, this does not include the examples and requires you to set up your environment manually.
 
 After this installation, you can run any of the Python-only example notebooks. To do so you can:
 
-```
-$ python -m ipykernel install --user --name=mlos_environment
-$ jupyter-notebook --notebook-dir=MLOS/source/Mlos.Notebooks
+```shell
+python -m ipykernel install --user --name=mlos_environment
+jupyter-notebook --notebook-dir=MLOS/source/Mlos.Notebooks
 ```
 
-Jupyter will list a few notebooks. A good place to start is the *BayesianOptimization.ipynb*, which provides an [Introduction to Bayesian Optimization](./notebooks/BayesianOptimization/).
+Jupyter will list a few notebooks. A good place to start is the *BayesianOptimization.ipynb*, which provides an [Introduction to Bayesian Optimization](./source/Mlos.Notebooks/BayesianOptimization.ipynb#mlos-github-tree-view).
 
 ## Full Build (C# and C++ components)
 
@@ -124,8 +129,9 @@ For detailed instructions, please refer to:
 
 ## Examples
 
-Code and documentation for examples of using MLOS to optimize a system are described in the [Notebooks](./notebooks/) section. Additional code is in the  [source/Examples](https://github.com/microsoft/MLOS/tree/main/source/Examples) source directory.
-You can find the source of the notebooks [on github as well](https://github.com/microsoft/MLOS/tree/main/source/Mlos.Notebooks).
+Code and documentation for examples of using MLOS to optimize a system are described in the [Notebooks](https://microsoft.github.io/MLOS/notebooks/) section.
+Additional code is in the  [source/Examples](./source/Examples/#mlos-github-tree-view) source directory.
+You can find the source of the notebooks [on github as well](./source/Mlos.Notebooks/#mlos-github-tree-view).
 
 ## Documentation
 
@@ -145,4 +151,4 @@ For more formal enquiries, you can [contact us](mailto:mlos-maintainers@service.
 
 ## License
 
-- [MIT License](./LICENSE)
+- [MIT License](./LICENSE.txt)