added initial English documents

mmdagent-ex · Dec 22, 2023 · 92ffd47 · 92ffd47
1 parent 197ae9f
commit 92ffd47
Show file tree

Hide file tree

Showing 79 changed files with 6,310 additions and 750 deletions.
diff --git a/content.en/_index.md b/content.en/_index.md
@@ -1,32 +1,35 @@
----
-title: Top
-type: docs
----
-# MMDAgent-EX
-
-{{< hint danger >}}
-**Note**
-This site is currently under construction. Please wait for the official launch.
-{{< /hint >}}
-
-The Voice Interaction and Avatar Communication Toolkit, MMDAgent-EX, is preparing for its release as open-source software around December 2023. MMDAgent-EX is a successor software to [MMDAgent](https://www.mmdagent.jp/), with the following new features.  It works on Windows, macOS, and Linux.
-
-- Extended CG character representation, including PMX compatible support
-- Can integrate Python-based voice recognition, synthesis, dialogue, and LLM modules
-- Full external control via WebSocket communication
-- Bone and morph level external control
-- Easy display of text and images
-- Function to show README
-- Texture animation (APNG support).
-- Web-based content delivery & auto-updates
-- Now on GitHub
-- Documentation
-- Many bug fixes
-
-[The CG-CA character model "Gene" and "Uka"](https://www.slp.nitech.ac.jp/en/avatar/) for the system is also expected to be released with CC license at the same time.
-
-## OK, so what is this site?
-
-We are now working on a renewal of the MMDAgent-EX site for the above release.  The documents in this site are still WIP, may include incorrect information.
-
-We apologize for any inconvenience caused. Please wait for the official launch.
+
+
+---
+title: Home
+type: docs
+---
+
+# MMDAgent-EX
+
+{{< hint danger >}}
+**Attention**
+This site is currently under construction. Please wait for the official release.
+{{< /hint >}}
+
+The voice interaction and avatar communication toolkit MMDAgent-EX is preparing for release as open source software around December 2023. The following points are expected to be updated from the previously released [MMDAgent](https://www.mmdagent.jp/). The supported OS are Windows, macOS, Linux.
+
+- Extension of model expression, including PMX support
+- Integration of Python-based speech recognition, synthesis, dialogue, and LLM modules
+- External control via WebSocket communication
+- External control on a per-bone or per-morph basis
+- Single display of text and images
+- README display, prompt dialog
+- Texture animation
+- Web-based content distribution and automatic updates
+- Transition to GitHub-based
+- Document management and updates
+- Other minor extensions and bug fixes
+
+The [CG-CA character model "Gene" and "Uka")"](https://www.slp.nitech.ac.jp/avatar/) developed for the system is also expected to be released for use at the same time.
+
+## What is this site?
+
+Currently, the MMDAgent-EX site is being renewed in preparation for the above release. The content accessible from the left menu is being updated and is not guaranteed.
+
+We apologize for any inconvenience and ask for your patience until the official release.
diff --git a/content.en/docs/00_build.md b/content.en/docs/00_build.md
@@ -0,0 +1,167 @@
+---
+title: How To Build
+slug: build
+---
+
+# How To Build
+
+MMDAgent-EX runs on macOS, Linux, and Windows. It can also runs on WSL2 environment.  The build operation has been confirmed in the following environments:
+
+- **macOS**: M2 Macbook Air / macOS Ventura 13.5 / Sonoma
+- **Linux**: Ubuntu-22.04, Ubuntu-20.04
+- **Windows**: Windows 11 with Visual Studio 2022
+- **Linux on WSL**: Ubuntu-22.04 on WSL2 (v1.2.5.0) on Windows
+
+## Obtaining the Code
+
+{{< hint danger >}}
+[GIT LFS](https://git-lfs.com/) should be installed beforehand.
+{{< /hint >}}
+
+Before obtaining repository, make sure to have Git LFS installed on your envieonment; if not, install it.
+
+{{< details "Check and Install Git LFS" close >}}
+Check if it's installed
+
+```shell
+git lfs version
+```
+
+Install on macOS using brew:
+
+```shell
+brew install git-lfs
+git lfs install
+```
+
+Install on Linux:
+
+```shell
+sudo apt install git-lfs
+git lfs install
+```
+
+Windows:
+
+Install the LFS extension from the [Git LFS website](https://git-lfs.com/).
+
+{{< /details >}}
+
+Obtain the repository from [GitHub](https://github.com/mmdagent-ex/MMDAgent-EX) using Git LFS:
+
+```shell
+git clone https://github.com/mmdagent-ex/MMDAgent-EX.git
+```
+
+## Build Procedure
+
+### macOS
+
+The following packages are required. Install all before build with `brew install`.
+
+- cmake
+- poco
+- glew
+- libjpeg
+- re2
+- portaudio
+- minizip
+- libsndfile
+- libsamplerate
+- sox
+- rabbitmq-c
+- libomp
+- librdkafka
+
+Sometimes the header file for `libomp` may not be installed correctly. If a build error occurs, perform the following.
+
+```shell
+brew link --force libomp
+```
+
+Build with CMake:
+
+```shell
+cd MMDAgent-EX
+cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release
+cmake --build build
+```
+
+The built executable and plugins will be stored under to the `Release/` directory.
+
+The brew package directory may change depending on your environment, i.e., `/usr/local/` or `/opt/homebrew`.  The build script will try to guess the prefix directory by executing `brew --prefix`.  If it does not work well, specify the brew prefix path with environment variable `HOMEBREW_PREFIX`.
+
+### Linux
+
+The list of required packages is in the `requirements-linux.txt`, so install all the packages listed in it in advance with `apt install`. If you are using `Ubuntu`, you can do all at once as follows:
+
+```shell
+cd MMDAgent-EX
+sudo apt install `cat requirements-linux.txt`
+```
+
+Build with CMake:
+
+```shell
+cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release
+cmake --build build
+```
+
+After a successful build, the necessary executables and plugins are ready at the `Release/` directory.
+
+### Windows
+
+Build with Visual Studio 2022.
+
+{{< hint info >}}
+Building MMDAgent-EX requires C++ development environment and Windows SDK.  During Visual Studio's installation, check "Desktop development with C++" and install MSVC v143 and Windows 11 SDK (confirmed with 10.0.22000.0).
+{{< /hint >}}
+
+1. Open `MMDAgent_vs2022.sln` with Visual Studio 2022
+2. Right-click on `main` in Solution Explorer and set as the startup project
+3. Set the build configuration to `Release`
+4. Execute "Build Solution"
+
+## Built Files
+
+All the files required for execution will be generated under the `Release` folder.
+{{< hint info >}}
+On Windows, an executable file has ".exe" prefix and the plugin files has ".dll" suffix.  On macOS and Linux, there is no ".exe" for executable file, and plugin file has ".so" suffix instead.
+{{< /hint >}}
+
+Under `AppData`, there are various data files required for execution, and `DLLs` includes external DLLs necessary for operation for Windows.
+
+    Release/
+    ├── MMDAgent-EX.exe
+    ├── MMDAgent-EX.mdf
+    ├── AppData/
+    ├── DLLs/
+    └── Plugins/
+        ├── Plugin_AnyScript.dll
+        ├── Plugin_Audio.dll
+        ├── Plugin_Flite_plus_hts_engine.dll
+        ├── Plugin_Julius.dll
+        ├── Plugin_Kafka.dll
+        ├── Plugin_LookAt.dll
+        ├── Plugin_Network.dll
+        ├── Plugin_Open_JTalk.dll
+        ├── Plugin_RabbitMQ.dll
+        ├── Plugin_Remote.dll
+        ├── Plugin_TextArea.dll
+        ├── Plugin_Variables.dll
+        └── Plugin_VIManager.dll
+
+If installation is necessary, copy everything under this `Release` folder, maintaining the directory structure.
+
+## Libraries included in the archive but not used in macOS and Linux
+
+The following bundled libraries are used only on Windows. On macOS / Ubuntu, these bundled files are not used, and the ones from the packages installed on the system are linked instead.
+
+    Library_RE2
+    Library_zlib
+    Library_JPEG
+    Library_libsndfile
+    Library_PortAudio
+    Library_glew
+    Library_Poco
+    Library_librdkafka
diff --git a/content.en/docs/05_run_contents.md b/content.en/docs/05_run_contents.md
@@ -0,0 +1,91 @@
+---
+title: Running Example
+slug: run
+---
+# Running Example
+
+MMDAgent-EX is a software that plays "content".  In this section we explains what the "content" is in MMDAgent-EX, and explain the procedure to launch the example content.
+
+## About the Content
+
+MMDAgent-EX plays back the content. Content refers to a collection of files (assets) that comprise an interactive system, such as 3D models, motions, FST scripts, and audio-visual materials.
+
+The general structure of the content is as follows. In the top directory, there should be always an .mdf file, which serves as both the launch file and the configuration file. Additionally, files such as .fst, .dic, .jconf, .ojt are placed with the same prefix as .mdf. Also, `BUTTON*.txt`, `PACKAGE_DESC.txt` can be placed in the top directory. All other files (models, motions, images etc.) can be placed anywhere under the folder.
+
+    topdir/
+        |- foobar.mdf         Startup / Configuration
+        |- foobar.fst         Dialogue Script
+        |- foobar.dic         Additional dictionary for Julius
+        |- foobar.jconf       Additional Julius jconf file
+        |- foobar.ojt         Open JTalk setting file
+        |- PACKAGE_DESC.txt   Package info for web-based deploy
+        |- README.txt         Readme doc
+        +- (SubDirectories)
+            |- 3-D models (.pmd)
+            |- Motions (.vmd)
+            |- TTS Voice model (.htsvoice)
+            |- Background/Floor (images)
+            |- Sound / Music files (sound files)
+            |- Images, Text files, etc.
+
+## Obtaining the Example Content
+
+The [example content](https://github.com/mmdagent-ex/example) is a repository that includes a minimal script, sample images and motions, the model "mei" for Open JTalk, and the CG models ["Gene"](https://github.com/mmdagent-ex/gene) and ["Uka"](https://github.com/mmdagent-ex/uka).  Get the example content to learn about MMDAgent-EX on this site.
+
+    example/
+        |- main.mdf         Startup / Configuration
+        |- main.fst         Dialogue Script
+        |- main.ojt         Open JTalk setting
+        |- images/
+        |    |- ...
+        |- motions/
+        |    |- ...
+        |- voice/           Voice model "mei" for Open JTalk
+        |    |- mei/
+        |- gene/            CG-CA model "Gene"
+        |    |- Gene.pmd
+        |- uka/             CG-CA model "Uka"
+        |    |- Uka.pmd
+        |- example_motion/     Example files for motion blending
+        +- example_websocket/  Example files for websocket connection
+
+Clone the sample content. Do not forget to use `--recursive` option ad clone, as it loads the CG-CA models as submodules.
+
+```shell
+git clone --recursive https://github.com/mmdagent-ex/example
+```
+
+## Launching Content
+
+To launch content, specify the .mdf file at the top of the content folder as a command line argument for MMDAgent-EX.
+
+macOS / Linux terminal:
+
+```shell
+./Release/MMDAgent-EX ./example/main.mdf
+```
+
+Windows Command shell:
+
+```text
+.\Release\MMDAgent-EX.exe .\example\main.mdf
+```
+
+If you see a screen like the one below, the launch was successful.
+
+<img width="480" alt="example snapshot" src="/images/example_1.png"/>
+
+{{< hint info >}}
+If you set this example as "home content", the content will be start up just by starting the executable.
+
+To set the content being executed as home:
+
+- Press the `/` key on the MMDAgent-EX screen → the menu opens
+- Move to the right menu with →
+- Select "Set current as Home" with the up and down keys and press enter
+
+{{< /hint >}}
+
+### If it fails in WSL
+
+If the screen doesn't appear and the launch fails in WSL, your version of WSL may be outdated. Please use the latest WSL2, which supports GUI apps. For details, [please refer to this site to update your WSL version](https://learn.microsoft.com/windows/wsl/tutorials/gui-apps).
diff --git a/content.en/docs/10_screen.md b/content.en/docs/10_screen.md
@@ -0,0 +1,50 @@
+---
+title: Screen
+slug: screen
+---
+# Screen
+
+Here is a sample screen during content playback:
+
+![Sample Screen](/images/screen.png)
+
+### (1) Status
+
+This part indicates the current status. You can toggle it by `s` key.
+
+- `60.0fps` Display speed (fps)
+- `4x MSAA` Anti-aliasing (MSAA) intensity
+- `[AL number]` AutoLuminous intensity (part luminance)
+- `S` Shadow display method
+  - `S`: projection shadow
+  - `SM`: shadow map rendeing (high graphical cost)
+  - `none`: no shadow
+- `[DF number|number]` Diffusion Effect intensity and range
+
+### (2) Error Messages
+
+If a system error occurs, the details of the error will be displayed here.
+
+### (3) Network Status
+
+Some icons may indicate current connection status when WebSocket communication is enabled.
+
+- ![Net Icon](/images/icon1.png): Now communicating with WebSocket server
+- ![Screen Icon](/images/icon2.png): Transmitting screen capture to the server (not yet implemented)
+- ![Video Icon](/images/icon3.png): Transmitting audio and webcam video to the server (not yet implemented)
+
+### (4) Network Communication Messages
+
+Messages are displayed when the connection status is changed when connected with WebSocket server.
+
+### (5) Help Guide
+
+Help guide text may appear at this area according to the status.
+
+### (6) Tab Bar
+
+- **home**: Opens the home content
+- **readme**: Views the README attached to the content currently playing
+- **bookmark**: Opens bookmarks
+- **history**: Opens history
+- **menu**: Opens the menu