Skip to content

Commit

Permalink
v1.0 initial release
Browse files Browse the repository at this point in the history
  • Loading branch information
dakusan committed Dec 8, 2023
0 parents commit 1dacd65
Show file tree
Hide file tree
Showing 42 changed files with 5,782 additions and 0 deletions.
13 changes: 13 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
root = true

[*]
indent_style = tab
indent_size = 4
tab_width = 4
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = false
end_of_line = lf

[*.go]
insert_final_newline = true
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
settings-gol10n.json
.idea/workspace.xml
.idea/usage.statistics.xml
.idea/shelf
.idea/dictionaries
gol10n.exe
gol10n
6 changes: 6 additions & 0 deletions .idea/encodings.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions .idea/inspectionProfiles/Project_Default.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 18 additions & 0 deletions .idea/js-inspections-ea-ultimate.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions .idea/markdown.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 14 additions & 0 deletions .idea/misc.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions .idea/modules.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 16 additions & 0 deletions .idea/project.dic

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

31 changes: 31 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
BSD 3-Clause License

Copyright (c) 2023, Dakusan aka Jeffrey Riaboy [dakusan/gol10n]
Copyright (c) 2012 Alex Ogier [pflag/LICENSE]
Copyright (c) fsnotify Authors [fsnotify/fsnotify]
Copyright (c) 2012 The Go Authors [Golang]

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
181 changes: 181 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
# Description
This is a highly space and memory optimized l10n (localization) library for Go (GoLang) pronounced “Goal Ten”.<br>
[Translation strings](docs/definitions.md#Translation-strings) are held, per language, in [text files](docs/translation_files.md) (either [YAML](docs/translation_files.md#YAML-files) or [JSON](docs/translation_files.md#JSON-files)), and compile into [.gtr](docs/definitions.md#Compiled-binary-translation-files) or .gtr.gz (gzip compressed) files.

Translations can be [referenced in Go code](docs/using_in_go.md#Using-translations-in-Go) either by an index, or a [namespace](docs/definitions.md#Namespaces) and [translation ID](docs/definitions.md#Translation-IDs).
Referencing by index is the fastest, most efficient, and what this library was built for. Indexes are stored as constants in [generated Go dictionary files](docs/using_in_go.md#generated-go-dictionary-files) by [namespace](docs/definitions.md#Namespaces), and are also held in [the dictionary](docs/definitions.md#The-dictionary).

Features:
* Translations are created in [YAML](docs/translation_files.md#YAML-files) or [JSON](docs/translation_files.md#JSON-files) [config files](docs/translation_files.md)
* Translations are compiled into [optimized binary files](docs/definitions.md#Compiled-binary-translation-files) for super-fast and space-efficient loading and use
* [Go [enum]](docs/using_in_go.md#generated-go-dictionary-files) [dictionary](docs/definitions.md#The-dictionary) files are created so translations can be accessed by constant index within [namespaces](docs/definitions.md#Namespaces)
* [Command line interface](#Command-line-interface) and [golang library level access](docs/using_in_go.md#Automatically-saving-and-loading-the-language-files) are both available
* [Translations](docs/definitions.md#Translation-IDs) can be separated into [namespaces](docs/definitions.md#Namespaces)
* [Typed variables](docs/translation_files.md#Variables) inside [translation strings](docs/definitions.md#Translation-strings)
* [Fallback languages](docs/definitions.md#Fallback-languages)
* [Printf type formatters](docs/translation_files.md#Printf-format-specifiers) are available and also contain i18n outputs
* [Plurality rules](docs/translation_files.md#Plurality-rules)
* [Embedded translations](docs/translation_files.md#Embedded-translations)

Translation data and rules are stored in optimized blobs similar to how Go’s native i18n package stores its data.

Instead of trying to extract your in-source-code translations though a tool, this lets you handle them manually. I find that keeping translation strings outside of source files and instead using constants is much cleaner.

Gol10n is available under the same style of BSD license as the Go language, which can be found in the LICENSE file.

# Installation
Gol10n is available using the standard go get command.

Install by running:

```
go get github.com/dakusan/gol10n
```

# YAML formatting by example
> [!note]
> Parts of this example will be used in other documentation examples
```yaml
Settings:
LanguageName: English
LanguageIdentifier: en-US
MissingPluralRule: A translation rule could not be found for the given plurality
FallbackLanguage: en-US #Ignored on the default language

NameSpaceExample:
TranslationID: TranslationValue
Foo天६_: 😭Bar {{*TranslationID}} {{*_animalsGroupNames.Cow}}
BorrowedNumberOfBooks:
\TestVal: Test
=0: You have no books borrowed
=1: You have one book and are encouraged to borrow more
<=10: You have {{.PluralCount}} books and are within your borrowing limit. If you were a cow, this would be called a {{.OtherVar}}
^: You have {{.PluralCount}} books and are over your limit of 10 books
OtherVar: VariableTranslation
\Translator: Dakusan
WelcomeTitle:
^: Welcome to our hotel <b>{{.Name|-10}}</b>.\nYour stay is for {{.NumDay天s|08.2}} days. Your checkout is on {{.CheckoutDay!%x %X}} and your cost will be {{.Cost}}
Name: String #Inlined printf rules. Left justify with spaces, width of 10
CheckoutDay: DateTime
Cost: Currency
NumDay天s: IntegerWithSymbols #Inlined printf rules. Right justify with '0', Total width of 8. Include 2 digits of decimal

_animalsGroupNames:
Wolf: Pack
Cow:
=1: Lonely
<=12: Herd
^: Flink
Crow:
<10: Murder
<100: Genocide
~101-164: Too many
^: Run for your life
```

[JSON](docs/translation_files.md#JSON-files) parsing is also available.

# Command line interface
```
gol10n.exe [Mode] [flags]:
Modes (Mutually exclusive):
Directory mode: [No arguments given]
Processes all files in the “InputPath” directory
Can be used in conjunction with -w
File mode: [arg1=language identifier]
Processes a single language file
The default language will need to be processed if a compiled dictionary does not exist
Can be used in conjunction with -s or -f
-s, --single-file Mode=File. The default language will not be processed
This will only work if a compiled dictionary already exists
-f, --fallbacks Mode=File. Also process the language’s fallback files
-w, --watch Mode=Directory. Continually watches the directory for relevant changes
Only processes and updates the necessary files when a change is detected
--create-settings Create the default settings-gol10n.json file
-h, --help This help prompt
File flags (Modify how non-translation-text-files are interacted with):
-d, --go-dictionary[=false] Output the go dictionary files when processing the default language (default true)
-c, --output-compiled[=false] Output the compiled translation files and dictionary (default true)
-i, --ignore-timestamps Always read from translation text files [ignore compiled files even if they are newer]
The following are for overriding settings from settings-gol10n.json. If not given, the values from the settings file will be used:
-l, --default-language string The identifier for the default language
-p, --input-path string The directory with the translation text files
-g, --go-path string The directory to output the generated Go dictionary files to
Each namespace gets its own directory and file in the format “$NamespaceName/translationIDs.go”
-o, --output-path string The directory to output the compiled binary translation files to
Each language gets its own .gtr or .gtr.gz (gzip compressed) file
-m, --compress-compiled Whether the compiled binary translation files are saved as .gtr or .gtr.gz (gzip compressed)
-b, --allow-big-strings If translation strings can be larger than 64KB
If true, and a large translation is found, then compiled binary files will become larger
-j, --allow-json-comma If JSON files can have trailing commas. If true, a sanitization process is ran over the JSON
Command line display modifiers:
See using_in_go.md#ProcessedFile Mode=Directory, -w, -f
-t, --table[=false] Output an ascii table of the processed languages and their flags (default true)
-v, --verbose Output a list of processed files and their processing flags
-x, --warnings[=false] Output a list of warnings when processing non-default language translation files (default true)
```

There are also [automatic](docs/using_in_go.md#Automatically-saving-and-loading-the-language-files) and [manual](docs/using_in_go.md#Manually-loading-the-language-files) library functions available that duplicate all command line functionality.

# Example “Get” translation function calls
[Indexed functions](docs/language_get_functions.md#Indexed-functions) examples:<br>
> ```golang
> str, err := Language.Get(_animalsGroupNames.Wolf)
> ```
> Yields
> ```golang
> str := "Pack"
> err := nil
> ```
> ```golang
> str := Language.MustGetPlural(NameSpaceExample.BorrowedNumberOfBooks, 9, _animalsGroupNames.Cow)
> ```
> Yields
> ```golang
> str := "You have 9 books and are within your borrowing limit. If you were a cow, this would be called a Herd"
> ```
[Named functions](docs/language_get_functions.md#Named-functions) example:<br>
> [!important]
> The parameter order is based upon the order in the [translation text file](docs/translation_files.md#Text-processing-rules), **NOT** the order in the translation string
> ```golang
> const numDays = 1001
> str := Language.MustGetNamed(
> "NameSpaceExample", "WelcomeTitle", //Namespace, TranslationID
> "Frodo", //Variable: Name
> time.Now().Add(time.Hour*24*numDays), //Variable: CheckoutDay
> currency.GBP.Amount(1275.98), //Variable: Cost
> numDays, //Variable: NumDay天s
> )
> ```
> Yields
> ```golang
> str := "Welcome to our hotel <b>Frodo </b>.\nYour stay is for 1,001 days. Your checkout is on 12/25/2123 09:18:24 PM and your cost will be £ 1,275.98"
> ```
# Settings file
The settings file, `gol10n-settings.yaml`, requires the following variables:
* **DefaultLanguage**: The [identifier](docs/definitions.md#Language-identifiers) for the [default language](docs/definitions.md#The-default-language).
* **InputPath**: The directory with the [translation text files](docs/translation_files.md).
* **GoOutputPath**: The directory to output the [generated Go dictionary files](docs/using_in_go.md#generated-go-dictionary-files) to. Each [namespace](docs/definitions.md#Namespaces) gets its own directory and file in the format `$NamespaceName/translationIDs.go`.
* **CompiledOutputPath**: The directory to output the [compiled binary translation files](docs/definitions.md#Compiled-binary-translation-files) to. Each language gets its own .gtr or .gtr.gz (gzip compressed) file.
* **CompressCompiled**: A boolean specifying whether the [compiled binary translation files](docs/definitions.md#Compiled-binary-translation-files) are saved as .gtr or .gtr.gz (gzip compressed).
* **AllowBigStrings**: A boolean that specifies if translation strings can be larger than 64KB. If true, and a large translation string is found, then [compiled binary translation files](docs/definitions.md#Compiled-binary-translation-files) will become larger.
* **AllowJSONTrailingComma**: A boolean that specifies if [JSON](docs/translation_files.md#JSON-files) files can have trailing commas. If true, a sanitization process is ran over the JSON that changes the regular expression `,\s*\n\s*}` to just `}`.

These settings are only used when using this library in a [command line interface](#Command-line-interface). When calling the go functions [automatically](docs/using_in_go.md#Automatically-saving-and-loading-the-language-files) or [manually](docs/using_in_go.md#Manually-loading-the-language-files), these settings are part of the function parameters.

# Additional reading:
* [Definitions](docs/definitions.md)
* [Translation Files](docs/translation_files.md)
* [Using in Go](docs/using_in_go.md)
* [Language Get() Functions](docs/language_get_functions.md)
* [Misc](docs/misc.md) (Hard and soft limits and TODO section)

0 comments on commit 1dacd65

Please sign in to comment.