[localization] Fixes for latest master

guide/src/en/SUMMARY.md

@@ -15,13 +15,13 @@
         - [General](format/configuration/general.md)
         - [Preprocessors](format/configuration/preprocessors.md)
         - [Renderers](format/configuration/renderers.md)
+        - [Localization](format/configuration/localization.md)
         - [Environment Variables](format/configuration/environment-variables.md)
     - [Theme](format/theme/README.md)
         - [index.hbs](format/theme/index-hbs.md)
         - [Syntax highlighting](format/theme/syntax-highlighting.md)
         - [Editor](format/theme/editor.md)
     - [MathJax Support](format/mathjax.md)
-    - [Localization](format/localization.md)
     - [mdBook-specific features](format/mdbook.md)
 - [Continuous Integration](continuous-integration.md)
 - [For Developers](for_developers/README.md)

guide/src/en/format/configuration/README.md

@@ -4,11 +4,11 @@ This section details the configuration options available in the ***book.toml***:
 - **[General]** configuration including the `book`, `rust`, `build` sections
 - **[Preprocessor]** configuration for default and custom book preprocessors
 - **[Renderer]** configuration for the HTML, Markdown and custom renderers
-- **[Translations]** configuration for books written in more than one language
+- **[Localization]** configuration for books written in more than one language
 - **[Environment Variable]** configuration for overriding configuration options in your environment
 
 [General]: general.md
 [Preprocessor]: preprocessors.md
 [Renderer]: renderers.md
-[Translations]: translations.md
+[Localization]: localization.md
 [Environment Variable]: environment-variables.md

guide/src/en/format/configuration/localization.md

@@ -1 +1,86 @@
 # Localization
+
+It's possible to write your book in more than one language and bundle all of its
+translations into a single output folder, with the ability for readers to switch
+between each one in the rendered output. The available languages for your book
+are defined in the `[language]` table:
+
+```toml
+[language.en]
+name = "English"
+
+[language.ja]
+name = "日本語"
+title = "本のサンプル"
+description = "この本は実例です。"
+authors = ["Ruin0x11"]
+```
+
+Each language must have a human-readable `name` defined. Also, if the
+`[language]` table is defined, you must define `book.language` to be a key of
+this table, which will indicate the language whose files will be used for
+fallbacks if a page is missing in a translation.
+
+The `title` and `description` fields, if defined, will override the ones set in
+the `[book]` section. This way you can translate the book's title and
+description. `authors` provides a list of this translation's authors.
+
+After defining a new language like `[language.ja]`, add a new subdirectory
+`src/ja` and create your `SUMMARY.md` and other files there.
+
+> **Note:** Whether or not the `[language]` table is defined changes the format
+> of the `src` directory that mdBook expects to see. If there is no `[language]`
+> table, mdBook will treat the `src` directory as a single translation of the
+> book, with `SUMMARY.md` at the root:
+>
+> ```
+> ├── book.toml
+> └── src
+>     ├── chapter
+>     │   ├── 1.md
+>     │   ├── 2.md
+>     │   └── README.md
+>     ├── README.md
+>     └── SUMMARY.md
+> ```
+> 
+> If the `[language]` table is defined, mdBook will instead expect to find
+> subdirectories under `src` named after the keys in the table:
+>
+> ```
+> ├── book.toml
+> └── src
+>     ├── en
+>     │   ├── chapter
+>     │   │   ├── 1.md
+>     │   │   ├── 2.md
+>     │   │   └── README.md
+>     │   ├── README.md
+>     │   └── SUMMARY.md
+>     └── ja
+>         ├── chapter
+>         │   ├── 1.md
+>         │   ├── 2.md
+>         │   └── README.md
+>         ├── README.md
+>         └── SUMMARY.md
+> ```
+
+If the `[language]` table is used, you can pass the `-l <language id>` argument
+to commands like `mdbook build` to build the book for only a single language. In
+this example, `<language id>` can be `en` or `ja`.
+
+Some extra notes on translations:
+
+- In a translation's `SUMMARY.md` or inside Markdown files, you can link to
+  pages, images or other files that don't exist in the current translation, but
+  do exist in the default translation. This is so you can have a fallback in
+  case new pages get added in the default language that haven't been translated
+  yet.
+- Each translation can have its own `SUMMARY.md` with differing content from
+  other translations. Even if the translation's summary goes out of sync with
+  the default language, the links will continue to work so long as the pages
+  exist in either translation.
+- Each translation can have its own pages listed in `SUMMARY.md` that don't
+  exist in the default translation at all, in case extra information specific to
+  that language is needed.

guide/src/en/format/localization.md

@@ -0,0 +1 @@
+# Localization

src/book/book.rs

@@ -61,8 +61,8 @@ fn load_single_book_translation<P: AsRef<Path>>(
     let summary = parse_summary(&summary_content)
         .with_context(|| format!("Summary parsing failed for file={:?}", summary_md))?;
 
-    if cfg.create_missing {
-        create_missing(localized_src_dir, &summary).with_context(|| "Unable to create missing chapters")?;
+    if cfg.build.create_missing {
+        create_missing(&localized_src_dir, &summary).with_context(|| "Unable to create missing chapters")?;
     }
 
     load_book_from_disk(&summary, localized_src_dir, fallback_src_dir, cfg)
@@ -83,17 +83,7 @@ fn create_missing(src_dir: &Path, summary: &Summary) -> Result<()> {
             if let Some(ref location) = link.location {
                 let filename = src_dir.join(location);
                 if !filename.exists() {
-                    if let Some(parent) = filename.parent() {
-                        if !parent.exists() {
-                            fs::create_dir_all(parent)?;
-                        }
-                    }
-                    debug!("Creating missing file {}", filename.display());
-
-                    let mut f = File::create(&filename).with_context(|| {
-                        format!("Unable to create missing file: {}", filename.display())
-                    })?;
-                    writeln!(f, "# {}", link.name)?;
+                    create_missing_link(&filename, link)?;
                 }
             }
 
@@ -104,6 +94,20 @@ fn create_missing(src_dir: &Path, summary: &Summary) -> Result<()> {
     Ok(())
 }
 
+fn create_missing_link(filename: &Path, link: &Link) -> Result<()> {
+    if let Some(parent) = filename.parent() {
+        if !parent.exists() {
+            fs::create_dir_all(parent)?;
+        }
+    }
+    debug!("Creating missing file {}", filename.display());
+
+    let mut f = File::create(&filename)?;
+    writeln!(f, "# {}", link.name)?;
+
+    Ok(())
+}
+
 /// A dumb tree structure representing a book.
 ///
 /// For the moment a book is just a collection of [`BookItems`] which are
@@ -117,6 +121,8 @@ fn create_missing(src_dir: &Path, summary: &Summary) -> Result<()> {
 pub struct Book {
     /// The sections in this book.
     pub sections: Vec<BookItem>,
+    /// Chapter title overrides for this book.
+    pub chapter_titles: HashMap<PathBuf, String>,
     __non_exhaustive: (),
 }
 
@@ -360,6 +366,7 @@ pub(crate) fn load_book_from_disk<P: AsRef<Path>>(
 
     Ok(Book {
         sections: chapters,
+        chapter_titles: HashMap::new(),
         __non_exhaustive: (),
     })
 }
@@ -410,8 +417,8 @@ fn load_chapter<P: AsRef<Path>>(
             );
         }
         if !location.exists() && cfg.build.create_missing {
-            create_missing(&location, &link)
-                .with_context(|| "Unable to create missing chapters")?;
+            create_missing_link(&location, &link)
+                .with_context(|| "Unable to create missing link reference")?;
         }
 
         let mut f = File::open(&location)
@@ -565,6 +572,7 @@ more text.
     #[test]
     fn load_a_single_chapter_with_utf8_bom_from_disk() {
         let temp_dir = TempFileBuilder::new().prefix("book").tempdir().unwrap();
+        let cfg = Config::default();
 
         let chapter_path = temp_dir.path().join("chapter_1.md");
         File::create(&chapter_path)
@@ -581,7 +589,7 @@ more text.
             Vec::new(),
         );
 
-        let got = load_chapter(&link, temp_dir.path(), Vec::new()).unwrap();
+        let got = load_chapter(&link, temp_dir.path(), temp_dir.path(), Vec::new(), &cfg).unwrap();
         assert_eq!(got, should_be);
     }
 
@@ -832,6 +840,7 @@ more text.
                 name: String::from("Chapter 1"),
                 content: String::from(DUMMY_SRC),
                 path: Some(PathBuf::from("chapter_1.md")),
+                source_path: Some(PathBuf::from("chapter_1.md")),
                 ..Default::default()
             })],
             ..Default::default()

src/book/mod.rs

@@ -213,6 +213,25 @@ impl MDBook {
         Ok(())
     }
 
+    fn preprocess(
+        &self,
+        preprocess_ctx: &PreprocessorContext,
+        renderer: &dyn Renderer,
+        book: Book,
+    ) -> Result<Book> {
+        let mut preprocessed_book = book;
+        for preprocessor in &self.preprocessors {
+            if preprocessor_should_run(&**preprocessor, renderer, &self.config) {
+                debug!("Running the {} preprocessor.", preprocessor.name());
+                preprocessed_book = preprocessor.run(&preprocess_ctx, preprocessed_book)?;
+            }
+        }
+        preprocessed_book
+            .chapter_titles
+            .extend(preprocess_ctx.chapter_titles.borrow_mut().drain());
+        Ok(preprocessed_book)
+    }
+
     /// Run the entire build process for a particular [`Renderer`].
     pub fn execute_build_process(&self, renderer: &dyn Renderer) -> Result<()> {
         let preprocessed_books = match &self.book {
@@ -248,19 +267,20 @@ impl MDBook {
             }
         };
 
+        self.render(&preprocessed_books, renderer)
+    }
+
+    fn render(&self, preprocessed_books: &LoadedBook, renderer: &dyn Renderer) -> Result<()> {
         let name = renderer.name();
         let build_dir = self.build_dir_for(name);
 
-        let mut render_context = RenderContext::new(
+        let render_context = RenderContext::new(
             self.root.clone(),
             preprocessed_books.clone(),
             self.build_opts.clone(),
             self.config.clone(),
             build_dir,
         );
-        render_context
-            .chapter_titles
-            .extend(preprocess_ctx.chapter_titles.borrow_mut().drain());
 
         info!("Running the {} backend", renderer.name());
         renderer
@@ -297,6 +317,7 @@ impl MDBook {
             self.config.clone(),
             "test".to_string(),
         );
+
         let book = LinkPreprocessor::new().run(&preprocess_context, book.clone())?;
         // Index Preprocessor is disabled so that chapter paths continue to point to the
         // actual markdown files.
@@ -354,6 +375,30 @@ impl MDBook {
         Ok(())
     }
 
+    /// Run `rustdoc` tests on the book, linking against the provided libraries.
+    pub fn test(&self, library_paths: Vec<&str>) -> Result<()> {
+        let library_args: Vec<&str> = (0..library_paths.len())
+            .map(|_| "-L")
+            .zip(library_paths.into_iter())
+            .flat_map(|x| vec![x.0, x.1])
+            .collect();
+
+        let temp_dir = TempFileBuilder::new().prefix("mdbook-").tempdir()?;
+
+        match self.book {
+            LoadedBook::Localized(ref books) => {
+                for (language_ident, book) in books.0.iter() {
+                    self.test_book(book, &temp_dir, &library_args, Some(language_ident.clone()))?;
+                }
+            }
+            LoadedBook::Single(ref book) => {
+                self.test_book(&book, &temp_dir, &library_args, None)?
+            }
+        }
+
+        Ok(())
+    }
+
     /// The logic for determining where a backend should put its build
     /// artefacts.
     ///