Permalink
Browse files

i18n: add infrastructure for translating Git with gettext

Change the skeleton implementation of i18n in Git to one that can show
localized strings to users for our C, Shell and Perl programs using
either GNU libintl or the Solaris gettext implementation.

This new internationalization support is enabled by default. If
gettext isn't available, or if Git is compiled with
NO_GETTEXT=YesPlease, Git falls back on its current behavior of
showing interface messages in English. When using the autoconf script
we'll auto-detect if the gettext libraries are installed and act
appropriately.

This change is somewhat large because as well as adding a C, Shell and
Perl i18n interface we're adding a lot of tests for them, and for
those tests to work we need a skeleton PO file to actually test
translations. A minimal Icelandic translation is included for this
purpose. Icelandic includes multi-byte characters which makes it easy
to test various edge cases, and it's a language I happen to
understand.

The rest of the commit message goes into detail about various
sub-parts of this commit.

= Installation

Gettext .mo files will be installed and looked for in the standard
$(prefix)/share/locale path. GIT_TEXTDOMAINDIR can also be set to
override that, but that's only intended to be used to test Git itself.

= Perl

Perl code that's to be localized should use the new Git::I18n
module. It imports a __ function into the caller's package by default.

Instead of using the high level Locale::TextDomain interface I've
opted to use the low-level (equivalent to the C interface)
Locale::Messages module, which Locale::TextDomain itself uses.

Locale::TextDomain does a lot of redundant work we don't need, and
some of it would potentially introduce bugs. It tries to set the
$TEXTDOMAIN based on package of the caller, and has its own
hardcoded paths where it'll search for messages.

I found it easier just to completely avoid it rather than try to
circumvent its behavior. In any case, this is an issue wholly
internal Git::I18N. Its guts can be changed later if that's deemed
necessary.

See <AANLkTilYD_NyIZMyj9dHtVk-ylVBfvyxpCC7982LWnVd@mail.gmail.com> for
a further elaboration on this topic.

= Shell

Shell code that's to be localized should use the git-sh-i18n
library. It's basically just a wrapper for the system's gettext.sh.

If gettext.sh isn't available we'll fall back on gettext(1) if it's
available. The latter is available without the former on Solaris,
which has its own non-GNU gettext implementation. We also need to
emulate eval_gettext() there.

If neither are present we'll use a dumb printf(1) fall-through
wrapper.

= About libcharset.h and langinfo.h

We use libcharset to query the character set of the current locale if
it's available. I.e. we'll use it instead of nl_langinfo if
HAVE_LIBCHARSET_H is set.

The GNU gettext manual recommends using langinfo.h's
nl_langinfo(CODESET) to acquire the current character set, but on
systems that have libcharset.h's locale_charset() using the latter is
either saner, or the only option on those systems.

GNU and Solaris have a nl_langinfo(CODESET), FreeBSD can use either,
but MinGW and some others need to use libcharset.h's locale_charset()
instead.

=Credits

This patch is based on work by Jeff Epler <jepler@unpythonic.net> who
did the initial Makefile / C work, and a lot of comments from the Git
mailing list, including Jonathan Nieder, Jakub Narebski, Johannes
Sixt, Erik Faye-Lund, Peter Krefting, Junio C Hamano, Thomas Rast and
others.

[jc: squashed a small Makefile fix from Ramsay]

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
  • Loading branch information...
1 parent bc1bbe0 commit 5e9637c629702e3d41ad01d95956d1835d7338e0 @avar avar committed with gitster Nov 17, 2011
@@ -81,6 +81,10 @@ For shell scripts specifically (not exhaustive):
are ERE elements not BRE (note that \? and \+ are not even part
of BRE -- making them accessible from BRE is a GNU extension).
+ - Use Git's gettext wrappers in git-sh-i18n to make the user
+ interface translatable. See "Marking strings for translation" in
+ po/README.
+
For C programs:
- We use tabs to indent, and interpret tabs as taking up to
@@ -144,6 +148,9 @@ For C programs:
- When we pass <string, length> pair to functions, we should try to
pass them in that order.
+ - Use Git's gettext wrappers to make the user interface
+ translatable. See "Marking strings for translation" in po/README.
+
Writing Documentation:
Every user-visible change should be reflected in the documentation.
View
12 INSTALL
@@ -106,6 +106,18 @@ Issues of note:
history graphically, and in git-gui. If you don't want gitk or
git-gui, you can use NO_TCLTK.
+ - A gettext library is used by default for localizing Git. The
+ primary target is GNU libintl, but the Solaris gettext
+ implementation also works.
+
+ We need a gettext.h on the system for C code, gettext.sh (or
+ Solaris gettext(1)) for shell scripts, and libintl-perl for Perl
+ programs.
+
+ Set NO_GETTEXT to disable localization support and make Git only
+ use English. Under autoconf the configure script will do this
+ automatically if it can't find libintl on the system.
+
- Some platform specific issues are dealt with Makefile rules,
but depending on your specific installation, you may not
have all the libraries/tools needed, or you may have
View
@@ -43,6 +43,22 @@ all::
# Define EXPATDIR=/foo/bar if your expat header and library files are in
# /foo/bar/include and /foo/bar/lib directories.
#
+# Define NO_GETTEXT if you don't want Git output to be translated.
+# A translated Git requires GNU libintl or another gettext implementation,
+# plus libintl-perl at runtime.
+#
+# Define HAVE_LIBCHARSET_H if you haven't set NO_GETTEXT and you can't
+# trust the langinfo.h's nl_langinfo(CODESET) function to return the
+# current character set. GNU and Solaris have a nl_langinfo(CODESET),
+# FreeBSD can use either, but MinGW and some others need to use
+# libcharset.h's locale_charset() instead.
+#
+# Define LIBC_CONTAINS_LIBINTL if your gettext implementation doesn't
+# need -lintl when linking.
+#
+# Define NO_MSGFMT_EXTENDED_OPTIONS if your implementation of msgfmt
+# doesn't support GNU extensions like --check and --statistics
+#
# Define HAVE_PATHS_H if you have paths.h and want to use the default PATH
# it specifies.
#
@@ -301,6 +317,7 @@ gitexecdir = libexec/git-core
mergetoolsdir = $(gitexecdir)/mergetools
sharedir = $(prefix)/share
gitwebdir = $(sharedir)/gitweb
+localedir = $(sharedir)/locale
template_dir = share/git-core/templates
htmldir = share/doc/git-doc
ETC_GITCONFIG = $(sysconfdir)/gitconfig
@@ -309,7 +326,7 @@ lib = lib
# DESTDIR=
pathsep = :
-export prefix bindir sharedir sysconfdir gitwebdir
+export prefix bindir sharedir sysconfdir gitwebdir localedir
CC = gcc
AR = ar
@@ -322,6 +339,7 @@ RPMBUILD = rpmbuild
TCL_PATH = tclsh
TCLTK_PATH = wish
XGETTEXT = xgettext
+MSGFMT = msgfmt
PTHREAD_LIBS = -lpthread
PTHREAD_CFLAGS =
GCOV = gcov
@@ -621,6 +639,7 @@ LIB_OBJS += entry.o
LIB_OBJS += environment.o
LIB_OBJS += exec_cmd.o
LIB_OBJS += fsck.o
+LIB_OBJS += gettext.o
LIB_OBJS += graph.o
LIB_OBJS += grep.o
LIB_OBJS += hash.o
@@ -817,12 +836,14 @@ ifeq ($(uname_S),Linux)
NO_STRLCPY = YesPlease
NO_MKSTEMPS = YesPlease
HAVE_PATHS_H = YesPlease
+ LIBC_CONTAINS_LIBINTL = YesPlease
endif
ifeq ($(uname_S),GNU/kFreeBSD)
NO_STRLCPY = YesPlease
NO_MKSTEMPS = YesPlease
HAVE_PATHS_H = YesPlease
DIR_HAS_BSD_GROUP_SEMANTICS = YesPlease
+ LIBC_CONTAINS_LIBINTL = YesPlease
endif
ifeq ($(uname_S),UnixWare)
CC = cc
@@ -889,6 +910,7 @@ ifeq ($(uname_S),SunOS)
NO_MKSTEMPS = YesPlease
NO_REGEX = YesPlease
NO_FNMATCH_CASEFOLD = YesPlease
+ NO_MSGFMT_EXTENDED_OPTIONS = YesPlease
ifeq ($(uname_R),5.6)
SOCKLEN_T = int
NO_HSTRERROR = YesPlease
@@ -1012,6 +1034,7 @@ ifeq ($(uname_S),GNU)
NO_STRLCPY=YesPlease
NO_MKSTEMPS = YesPlease
HAVE_PATHS_H = YesPlease
+ LIBC_CONTAINS_LIBINTL = YesPlease
endif
ifeq ($(uname_S),IRIX)
NO_SETENV = YesPlease
@@ -1228,6 +1251,7 @@ ifneq (,$(wildcard ../THIS_IS_MSYSGIT))
EXTLIBS += /mingw/lib/libz.a
NO_R_TO_GCC_LINKER = YesPlease
INTERNAL_QSORT = YesPlease
+ HAVE_LIBCHARSET_H = YesPlease
else
NO_CURL = YesPlease
endif
@@ -1405,6 +1429,11 @@ endif
ifdef NEEDS_LIBGEN
EXTLIBS += -lgen
endif
+ifndef NO_GETTEXT
+ifndef LIBC_CONTAINS_LIBINTL
+ EXTLIBS += -lintl
+endif
+endif
ifdef NEEDS_SOCKET
EXTLIBS += -lsocket
endif
@@ -1447,9 +1476,11 @@ ifdef NO_SYMLINK_HEAD
BASIC_CFLAGS += -DNO_SYMLINK_HEAD
endif
ifdef GETTEXT_POISON
- LIB_OBJS += gettext.o
BASIC_CFLAGS += -DGETTEXT_POISON
endif
+ifdef NO_GETTEXT
+ BASIC_CFLAGS += -DNO_GETTEXT
+endif
ifdef NO_STRCASESTR
COMPAT_CFLAGS += -DNO_STRCASESTR
COMPAT_OBJS += compat/strcasestr.o
@@ -1612,6 +1643,10 @@ ifdef HAVE_PATHS_H
BASIC_CFLAGS += -DHAVE_PATHS_H
endif
+ifdef HAVE_LIBCHARSET_H
+ BASIC_CFLAGS += -DHAVE_LIBCHARSET_H
+endif
+
ifdef DIR_HAS_BSD_GROUP_SEMANTICS
COMPAT_CFLAGS += -DDIR_HAS_BSD_GROUP_SEMANTICS
endif
@@ -1632,6 +1667,10 @@ ifdef GIT_TEST_CMP_USE_COPIED_CONTEXT
export GIT_TEST_CMP_USE_COPIED_CONTEXT
endif
+ifndef NO_MSGFMT_EXTENDED_OPTIONS
+ MSGFMT += --check --statistics
+endif
+
ifeq ($(TCLTK_PATH),)
NO_TCLTK=NoThanks
endif
@@ -1662,6 +1701,7 @@ ifndef V
QUIET_GEN = @echo ' ' GEN $@;
QUIET_LNCP = @echo ' ' LN/CP $@;
QUIET_XGETTEXT = @echo ' ' XGETTEXT $@;
+ QUIET_MSGFMT = @echo ' ' MSGFMT $@;
QUIET_GCOV = @echo ' ' GCOV $@;
QUIET_SP = @echo ' ' SP $<;
QUIET_SUBDIR0 = +@subdir=
@@ -1688,6 +1728,7 @@ bindir_SQ = $(subst ','\'',$(bindir))
bindir_relative_SQ = $(subst ','\'',$(bindir_relative))
mandir_SQ = $(subst ','\'',$(mandir))
infodir_SQ = $(subst ','\'',$(infodir))
+localedir_SQ = $(subst ','\'',$(localedir))
gitexecdir_SQ = $(subst ','\'',$(gitexecdir))
template_dir_SQ = $(subst ','\'',$(template_dir))
htmldir_SQ = $(subst ','\'',$(htmldir))
@@ -1743,7 +1784,7 @@ ifndef NO_TCLTK
$(QUIET_SUBDIR0)gitk-git $(QUIET_SUBDIR1) all
endif
ifndef NO_PERL
- $(QUIET_SUBDIR0)perl $(QUIET_SUBDIR1) PERL_PATH='$(PERL_PATH_SQ)' prefix='$(prefix_SQ)' all
+ $(QUIET_SUBDIR0)perl $(QUIET_SUBDIR1) PERL_PATH='$(PERL_PATH_SQ)' prefix='$(prefix_SQ)' localedir='$(localedir_SQ)' all
endif
ifndef NO_PYTHON
$(QUIET_SUBDIR0)git_remote_helpers $(QUIET_SUBDIR1) PYTHON_PATH='$(PYTHON_PATH_SQ)' prefix='$(prefix_SQ)' all
@@ -1793,6 +1834,7 @@ sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
-e 's|@SHELL_PATH@|$(SHELL_PATH_SQ)|' \
-e 's|@@DIFF@@|$(DIFF_SQ)|' \
-e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
+ -e 's|@@LOCALEDIR@@|$(localedir_SQ)|g' \
-e 's/@@NO_CURL@@/$(NO_CURL)/g' \
-e $(BROKEN_PATH_FIX) \
$@.sh >$@+
@@ -2045,6 +2087,9 @@ config.sp config.s config.o: EXTRA_CPPFLAGS = \
attr.sp attr.s attr.o: EXTRA_CPPFLAGS = \
-DETC_GITATTRIBUTES='"$(ETC_GITATTRIBUTES_SQ)"'
+gettext.sp gettext.s gettext.o: EXTRA_CPPFLAGS = \
+ -DGIT_LOCALE_PATH='"$(localedir_SQ)"'
+
http.sp http.s http.o: EXTRA_CPPFLAGS = \
-DGIT_HTTP_USER_AGENT='"git/$(GIT_VERSION)"'
@@ -2118,17 +2163,37 @@ XGETTEXT_FLAGS = \
XGETTEXT_FLAGS_C = $(XGETTEXT_FLAGS) --language=C \
--keyword=_ --keyword=N_ --keyword="Q_:1,2"
XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell
+XGETTEXT_FLAGS_PERL = $(XGETTEXT_FLAGS) --keyword=__ --language=Perl
LOCALIZED_C := $(C_OBJ:o=c)
LOCALIZED_SH := $(SCRIPT_SH)
+LOCALIZED_PERL := $(SCRIPT_PERL)
+
+ifdef XGETTEXT_INCLUDE_TESTS
+LOCALIZED_C += t/t0200/test.c
+LOCALIZED_SH += t/t0200/test.sh
+LOCALIZED_PERL += t/t0200/test.perl
+endif
po/git.pot: $(LOCALIZED_C)
$(QUIET_XGETTEXT)$(XGETTEXT) -o$@+ $(XGETTEXT_FLAGS_C) $(LOCALIZED_C)
$(QUIET_XGETTEXT)$(XGETTEXT) -o$@+ --join-existing $(XGETTEXT_FLAGS_SH) \
$(LOCALIZED_SH)
+ $(QUIET_XGETTEXT)$(XGETTEXT) -o$@+ --join-existing $(XGETTEXT_FLAGS_PERL) \
+ $(LOCALIZED_PERL)
mv $@+ $@
pot: po/git.pot
+POFILES := $(wildcard po/*.po)
+MOFILES := $(patsubst po/%.po,po/build/locale/%/LC_MESSAGES/git.mo,$(POFILES))
+
+ifndef NO_GETTEXT
+all:: $(MOFILES)
+endif
+
+po/build/locale/%/LC_MESSAGES/git.mo: po/%.po
+ $(QUIET_MSGFMT)mkdir -p $(dir $@) && $(MSGFMT) -o $@ $<
+
FIND_SOURCE_FILES = ( git ls-files '*.[hcS]' 2>/dev/null || \
$(FIND) . \( -name .git -type d -prune \) \
-o \( -name '*.[hcS]' -type f -print \) )
@@ -2147,7 +2212,8 @@ cscope:
### Detect prefix changes
TRACK_CFLAGS = $(CC):$(subst ','\'',$(ALL_CFLAGS)):\
- $(bindir_SQ):$(gitexecdir_SQ):$(template_dir_SQ):$(prefix_SQ)
+ $(bindir_SQ):$(gitexecdir_SQ):$(template_dir_SQ):$(prefix_SQ):\
+ $(localedir_SQ)
GIT-CFLAGS: FORCE
@FLAGS='$(TRACK_CFLAGS)'; \
@@ -2184,6 +2250,7 @@ endif
ifdef GIT_TEST_CMP_USE_COPIED_CONTEXT
@echo GIT_TEST_CMP_USE_COPIED_CONTEXT=YesPlease >>$@
endif
+ @echo NO_GETTEXT=\''$(subst ','\'',$(subst ','\'',$(NO_GETTEXT)))'\' >>$@
@echo GETTEXT_POISON=\''$(subst ','\'',$(subst ','\'',$(GETTEXT_POISON)))'\' >>$@
### Detect Tck/Tk interpreter path changes
@@ -2299,6 +2366,11 @@ install: all
$(MAKE) -C templates DESTDIR='$(DESTDIR_SQ)' install
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(mergetools_instdir_SQ)'
$(INSTALL) -m 644 mergetools/* '$(DESTDIR_SQ)$(mergetools_instdir_SQ)'
+ifndef NO_GETTEXT
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(localedir_SQ)'
+ (cd po/build/locale && $(TAR) cf - .) | \
+ (cd '$(DESTDIR_SQ)$(localedir_SQ)' && umask 022 && $(TAR) xof -)
+endif
ifndef NO_PERL
$(MAKE) -C perl prefix='$(prefix_SQ)' DESTDIR='$(DESTDIR_SQ)' install
$(MAKE) -C gitweb install
@@ -2435,6 +2507,7 @@ clean:
$(RM) $(TEST_PROGRAMS)
$(RM) -r bin-wrappers
$(RM) -r $(dep_dirs)
+ $(RM) -r po/build/
$(RM) *.spec *.pyc *.pyo */*.pyc */*.pyo common-cmds.h $(ETAGS_TARGET) tags cscope*
$(RM) -r autom4te.cache
$(RM) config.log config.mak.autogen config.mak.append config.status config.cache
View
@@ -35,6 +35,9 @@ NO_CURL=@NO_CURL@
NO_EXPAT=@NO_EXPAT@
NO_LIBGEN_H=@NO_LIBGEN_H@
HAVE_PATHS_H=@HAVE_PATHS_H@
+HAVE_LIBCHARSET_H=@HAVE_LIBCHARSET_H@
+NO_GETTEXT=@NO_GETTEXT@
+LIBC_CONTAINS_LIBINTL=@LIBC_CONTAINS_LIBINTL@
NEEDS_LIBICONV=@NEEDS_LIBICONV@
NEEDS_SOCKET=@NEEDS_SOCKET@
NEEDS_RESOLV=@NEEDS_RESOLV@
View
@@ -636,6 +636,12 @@ AC_CHECK_LIB([c], [basename],
AC_SUBST(NEEDS_LIBGEN)
test -n "$NEEDS_LIBGEN" && LIBS="$LIBS -lgen"
+AC_CHECK_LIB([c], [gettext],
+[LIBC_CONTAINS_LIBINTL=YesPlease],
+[LIBC_CONTAINS_LIBINTL=])
+AC_SUBST(LIBC_CONTAINS_LIBINTL)
+test -n "$LIBC_CONTAINS_LIBINTL" || LIBS="$LIBS -lintl"
+
## Checks for header files.
AC_MSG_NOTICE([CHECKS for header files])
#
@@ -818,6 +824,19 @@ AC_CHECK_HEADER([paths.h],
[HAVE_PATHS_H=])
AC_SUBST(HAVE_PATHS_H)
#
+# Define NO_GETTEXT if you don't want Git output to be translated.
+# A translated Git requires GNU libintl or another gettext implementation
+AC_CHECK_HEADER([libintl.h],
+[NO_GETTEXT=],
+[NO_GETTEXT=YesPlease])
+AC_SUBST(NO_GETTEXT)
+#
+# Define HAVE_LIBCHARSET_H if have libcharset.h
+AC_CHECK_HEADER([libcharset.h],
+[HAVE_LIBCHARSET_H=YesPlease],
+[HAVE_LIBCHARSET_H=])
+AC_SUBST(HAVE_LIBCHARSET_H)
+#
# Define NO_STRCASESTR if you don't have strcasestr.
GIT_CHECK_FUNC(strcasestr,
[NO_STRCASESTR=],
View
@@ -1099,6 +1099,8 @@ int main(int argc, char **argv)
struct credentials *cred = NULL;
int i;
+ git_setup_gettext();
+
git_extract_argv0_path(argv[0]);
for (i = 1; i < argc; i++) {
View
@@ -3292,6 +3292,8 @@ int main(int argc, const char **argv)
git_extract_argv0_path(argv[0]);
+ git_setup_gettext();
+
if (argc == 2 && !strcmp(argv[1], "-h"))
usage(fast_import_usage);
Oops, something went wrong.

0 comments on commit 5e9637c

Please sign in to comment.