Add support for ASP.NET Core

[jonnybee]

i18n should also support ASP.NET Core on both .NET Core (netstandard1.6) and .NET 4.5.1 and higher (the same platforms supported by ASP.NET Core). These apps will always run in Kestrel and i18n should plug in as middleware, f.ex:

public class Program
{
    public static void Main(string[] args)
    {
        var host = new WebHostBuilder()
            .UseKestrel()
            .UseStartup<Startup>()
            .Usei18n()
            .Build();
        host.Run();
    }
}

I wonder if this would be best suited to do as a major overhaul as i18n should publish NuGet packages with support for (at least) 2 different target framework (net451 and netandard1.6) and webframeworks like ASP.NET and AspNetCore.

[chapeti]

100% agree... I will be very happy with this enhancement... +1

[jonnybee]

I have spent some evenings working on i18n and ASP.NET Core and have a prototype working. There is quite a lot to be done and one must decide on whether to embrace the new IoC approach or do the minimal work and keep the static and singelton classes.

Some the required changes is:

• change to use MemoryCache
• rework code as there is no HttpContext.Current
• use the New Configuration system in ASP.NET Core
• Modify i18nSettings - just Properties and registered as singelton in IoC
• EarlyUrlLocalizer must be reworked some (or a new i18nEarlyUrlLocalizerMiddleware must be created)
• i18nTranslationMiddleware must be built to support translation
• and a number of changes to only use types/packages in netstandard1.6
• create assemblies for both NET451 and netstandard1.6

Basically - the i18n middleware must add it's own MemoryStream into the pipeline only for the URI's to be translated. This will result in i18n having a buffer of the whole output.

I do wonder whether this should be added into the existing branch with asp.net support or if this should be a separate branch for asp.net core. I would like to chip in and help make the code as we use i18n in our apps.

[turquoiseowl]

Basically - the i18n middleware must add it's own MemoryStream into the pipeline only for the URI's to be translated. This will result in i18n having a buffer of the whole output.

Would you elaborate on this? Are the nuggets translated somewhere else? Are you meaning a duplication of the response buffer? Are there performance implications?

Is this a fair summation of the situation:

Path 1: Keep as much as possible from current codebase (e.g. static singletons rather than embrace new IoC/DI model) and merge into current project.

Path 2: As Path 1 but fork.

Path 3: Start new project from scratch, adopt .NET Core / ASP.NET Core best practices and borrow wherever appropriate from current project.

If path 3 is chosen, we can link to it from here wherever it ends up.

I have to say I'm not currently working on ASP.NET Core but fully appreciate the need to do this (esp. as ASP.NET Core still seems to be in love with resource files!).

[jonnybee]

In order to minimize risk in the existing codebase I would choose either 2 or 3.
A version 1 could be Path2 so the documentation would need a minimum of updates and then continue to rework so it would fully embrace the .NET Core best practices and a new documentation.

Nugget is translated with the standard NuggetLocalizerForApp but to intercept Response.Body the middleware must use it's own MemoryStream, like this:

    public async Task Invoke(HttpContext context)
    {
        // should check for wheter to localize content or not 
        // if not then just call await _next.Invoke(context);
        var existingBody = context.Response.Body;
        using (var newBody = new MemoryStream())
        {
            context.Response.Body = newBody;
            await _next.Invoke(context);   // call the next Middleware

            // must set the original body 
            context.Response.Body = existingBody;
            // read content from body and replace nuggets 
            newBody.Seek(0, SeekOrigin.Begin);
            var content = new StreamReader(newBody).ReadToEnd();
            var newContent = LocalizedApplication.NuggetLocalizerForApp
                                 .ProcessNuggets(content, context.GetRequestUserLanguages());
            // Send our modified content to the response body.
            await context.Response.WriteAsync(newContent);
        }
   }

See also this blog post: https://www.billbogaiv.com/posts/using-aspnet-cores-middleware-to-modify-response-body

So to minimize performance issues the test for whether content should be localized or not should be done before this code block.

[jonnybee]

It is not that hard to load Resources from another storage type than .resx files in ASP.NET Core but I really like the PostBuild to extract the nesources and the simplicity of translation with .po files.

In Our apps we load translations from database and I have created our own DbTranslationRepository with EF.

[turquoiseowl]

Agree with ruling out path 1. I'm worried Path 2 could end up pretty messy. What does your prototype look like in that regard?

[jonnybee]

I think it would be easiest to start with a new repo (doesn't need all history) and it would be possible to do a very similar approach to use the LocalizedApplication.Current and the other static helpers.

My prototype goes further and use the builtin IoC container and keeps a Singeton instance of

• ILocalizedApplication
• IRootServices (got circular References when LocalizedApplication implemented IRootServices)
• ITranslationRepository
• ILanguageTag
• IUrlLocalizer
• ITextLocalizer
• INuggetLocalizer
• IEarlyUrlLocalizer
• i18nSettings (with configuration read from Configuration in startup.cs)

All that is needed to use i18n in MVC Core is 2 lines with AddI18N/UseI18N in Startup.cs:

    // This method gets called by the runtime. Use this method to add services to the container.
    public void ConfigureServices(IServiceCollection services)
    {
        // Add framework services.
        services.AddMvc();
        // create i18nSettings and register classes in IoC container 
        services.AddI18N(Configuration);  
    }

    // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
    public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
    {
        // Configure i18n middleware 
        app.UseI18N();

Example configuration in appsettings.json:

"i18n": {
    "AvailableLanguages": "en;nb;sv;se;dk",
    "DefaultLanguage": "en",
    "MessageContextEnabledFromComment": "true",
    "UrlLocalizationScheme":  "Scheme1"
}

There is still work to be done on some of the static members used for configuration and the helper extension methods.

My code compiles and runs with ASP.NET Core in DOTNETCORE. I havent tried to add Projects for NET451 yet (but should be fairly easy). The idea was to have most of the code in "Shared Projects" and then add the most platform specifc implementation in targeted projects. IE: code may be in a "Shared Project" but is actually included and compiled in each Project.

There is also the event hooks that either must be kept or rewritten to an interface so that devs can create and register their own implementation to override the default.

[turquoiseowl]

Do you mean a shared/common i18n.Common project, then dependent projects off of that, say i18n.ForCore and i18n.ForNet4x (or whatever)?

If so, that means re-factoring what we have now, right?

Then if so, that means going from i18n v2 to i18n v3?

[jonnybee]

Yes, Shared Projects is feature that first came in VS2013 Update 2. Previously you could "share" code by linking files into separate Projects (where each Project was compiled to a dll). A Shared Project is never compiled, it doean't have any References. I have a i18n.Domain.Shared project that just contains code (and has no references - is never compiled to dll). Then I reference this project into i18n.Domain.netstandard to compile the code for DOTNETCORE 1.0. I can also reference this project into i18n.Domain.Net that targets .net 4.5.1 framework. So it is a useful feature for having code in just one "shared project" and create "thin" projects for each specific target or place some platform specific code in that project. You can also use conditional compilation symbols to "tailor" the shared code for each target.

in i18n.netstandard I would reference i18n.Shared but may also place the code for i18nTranlationMiddleware in the i18n.netstandard project.

F.ex: In DOTNETCORE there is no Thread.CurrentThread.CurrentCulture/CurrentUiCulture.
So the code must be like this:

        SetPrincipalAppLanguageForRequestHandlers 
               = delegate(HttpContext context, ILanguageTag langtag)
        {
            if (langtag != null)
            {
#if NETCORE  
                CultureInfo.CurrentCulture = 
                CultureInfo.CurrentUICulture = langtag.GetCultureInfo();
#else
                System.Threading.Thread.CurrentThread.CurrentCulture = 
                System.Threading.Thread.CurrentThread.CurrentUICulture = langtag.GetCultureInfo();
#endif
            }
        };

And may be placed in a Shared project.

In terms of i18n for ASP.NET Core there must be 2 folders in the nuget package:
lib
\net451
\netstandard1.6

The net451 folder will contain i18n.dll and i18n.Domain.dll compiled for .NET 4.5.1
and in netstandard1.6 folder will contain the same assemblies compiled for DOTNETCORE 1.0

Yes it means re-factoring what we have now and could possibly also be used for support of the standard ASP.NET MVC and ASP.NET WebPages.

Yes, I would put his into a v3.

[turquoiseowl]

I notice you reference HttpContext in your above code snippet.

It was with this class that my concerns about the complexity of all this began.

My understanding is that the HttpContext class is very different in ASP.NET Core to HttpContext/HttpContextBase in ASP.NET pre-Core. Is that right? If so, is all your access to it wrapped in the #if conditionals?

[jonnybee]

I focused only on ASP.NET CORE and used HttpContext directly. And yes there are some differences such as HttpContext Request.Uri does not exist. BTW, there is no HttpContextBase in ASP.NET Core.

You get access to the HttpContext in the middleware. This is the signatur for the middlevare:

public async Task Invoke(HttpContext context)
{     
}

And from the middleware I would pass it into the EarlyUrlLocalizer.

[jonnybee]

In terms of code I would have some specific code in each target project. Like for ASP.NET Core use HttpContext and for ASP.NET MVC use HttpContextBase.

For ASP.NET Core I would have EarlyUrlLocalizer implemented as its own middleware class.
Then another middleware for translation of the body that would only hook in when the request path is something i18n should translate.

[turquoiseowl]

But HttpContext will either be Microsoft.AspNetCore.Http.HttpContext or System.Web.HttpContext, no? So is your using statement for the above snippet in #if conditionals?

[stajs]

I'm keen to test/contribute; is there a branch/fork/nuget somewhere I can grab to try it out?

[rsurban]

Where does this enhancement stand? I'd like to use this feature in a project I'm developing in ASP.Net Core. Is this the version at https://github.com/jonnybee/i18n? If not, is there a fork of the project with the work you've done so far that I can get a look at?

[jonnybee]

My proof-of-concept is not in a published branch. The fork at https://github.com/jonnybee/i18n does not contain these changes.

Support for i18n in ASP.NET Core needs some careful design and guidance as to how big changes to make (minimalistic or refactor/rewrite) and whether code/docs should be put into it's own repo or to be a part of the existing repo. There will also be a need for several new published NuGet packages and documentation.

Remember: ASP.NET Core supports both .NetCore and .Net Full Framework 4.5.1 and newer runtime.

[amainiii]

We're building a new ASP.NET Core project now and would really like to use i18n. Is there anything I can do to raise the priority of this enhancement? An increasing number of virtual machines running on Azure are Linux and Microsoft's interest in migrating to ASP.NET Core is vital to the future of .NET. Since i18n is the preeminent internationalization library for .NET, I think this enhancement would be a necessary step in its evolution. jonnybee - I appreciate the significance of the architectural changes that you and Martin discussed last fall. Nevertheless, you indicated you had a prototype back in October. Is there some way we can get a copy of it?

[turquoiseowl]

@amainiii I agree with your comment. From my own situation just now all I can say is sponsorship would raise the priority.

[amainiii]

@turquoiseowl Please contact me privately to explore sponsorship options. My email is in my profile, or if you prefer, Twitter or LinkedIn

[mmunchandersen]

Hi @turquoiseowl & @jonnybee
I've used i18n in a mvc 5 project. It's really great.
Now, I'm setting up a asp.net core 2 project and looking at a localization solution.

I would like to somehow contribute to porting i18n to core 2, however, I have no idea where the project stands and what is needed.

Could you share some info, please?

Thanks in advance

Regards Morten

[dalton5]

Hi,

Are there some news on asp.net core support?

Thanks,

[HenricObjektvision]

I love TurquoiseoOwl i18n, but I don't know the inner workings of this project enough to modify it. Especially since it seem like it would require quite a rewrite of alot of things.

So I chose to do my own custom solution instead. Thought it might help someone, or help this project along, since nothing seems to have happend in several years regarding this issue.
Tested in .NET Core 3.0+

First the Middleware:

    public static class BuilderExtensions
    {
        public static IApplicationBuilder UseI18nPoMiddleware(this IApplicationBuilder app, Ii18nTranslator translator)
        {
            return app.UseMiddleware<I18nPoMiddleware>(translator);
        }
    }

    public class I18nPoMiddleware
    {
        RequestDelegate _next;

        private static Ii18nTranslator _translator;

        private static readonly IEnumerable<string> validContentTypes = new HashSet<string>() { "text/html", "application/json", "application/javascript" };

        // Blacklists files or folders that we should not translate (for instance, Javascript files might contain triple arrays that would trigger a translation by mistake).
        // This can also be used to increase performance, since the content type filtering is available at a very late stage, when we've already done much of the work, causing performance to be a potential issue
        public static IEnumerable<string> BlackList = new HashSet<string>() { "/lib/", "/styles/", "/fonts/", "/images/" };

        public I18nPoMiddleware(RequestDelegate next, Ii18nTranslator translator)
        {
            _next = next;

            _translator = translator;
        }


        public async Task Invoke(HttpContext context, ILogger<I18nPoMiddleware> logger)
        {

            var startTotalTime = DateTime.Now;

            bool modifyResponse = (_translator.IsValid() && !BlackList.Any(bl => context.Request.Path.Value.ToLower().Contains(bl)));
            Stream originBody = null;

            if (modifyResponse)
            {
                //uncomment this line only if you need to read context.Request.Body stream
                //context.Request.EnableBuffering();

                originBody = ReplaceBody(context.Response);
            }


            var startNormalTime = DateTime.Now;
            try
            {
                await _next.Invoke(context);
            }
            catch
            {

                // Before passing on the exception, we need to restore the Response body, so the real Exception handlers can write to the right object
                if (modifyResponse)
                {
                    ReturnBody(context.Response, originBody);
                }

                throw;
            }
            var endNormalTime = DateTime.Now;


            if (modifyResponse)
            {
                //as we replaced the Response.Body with a MemoryStream instance before,
                //here we can read/write Response.Body

                var contentType = context.Response.ContentType?.ToLower();
                contentType = contentType?.Split(';', StringSplitOptions.RemoveEmptyEntries).FirstOrDefault();   // Filter out text/html from "text/html; charset=utf-8"

                if (validContentTypes.Contains(contentType))
                {
                    string responseBody = null;
                    using (var streamReader = new StreamReader(context.Response.Body))
                    {
                        // Read the body
                        context.Response.Body.Seek(0, SeekOrigin.Begin);
                        responseBody = await streamReader.ReadToEndAsync();
                    }


                    responseBody = _translator.Translate(responseBody);


                    // Create a new stream with the modified body, and reset the content length to match the new stream
                    var requestContent = new StringContent(responseBody, Encoding.UTF8, contentType);
                    context.Response.Body = await requestContent.ReadAsStreamAsync();//modified stream
                    context.Response.ContentLength = context.Response.Body.Length;

                }

                //finally, write modified data to originBody and set it back as Response.Body value
                ReturnBody(context.Response, originBody);


                var endTotalTime = DateTime.Now;

                var normalTime = (endNormalTime - startNormalTime);
                var totalTime = (endTotalTime - startTotalTime);

                //logger.LogDebug($"Request path: {context.Request.Path.Value}, Total time: {totalTime.ToString("ss':'FFF")}, Normal request time: {normalTime.ToString("ss':'FFF")}, Extra time: {(totalTime - normalTime).ToString("ss':'FFF")}");
            }


        }


        private Stream ReplaceBody(HttpResponse response)
        {
            var originBody = response.Body;
            response.Body = new MemoryStream();
            return originBody;
        }

        private void ReturnBody(HttpResponse response, Stream originBody)
        {
            response.Body.Seek(0, SeekOrigin.Begin);
            response.Body.CopyToAsync(originBody);
            response.Body = originBody;
        }

    }

Then we have a translator class, and the NuggetReplacer that does the actual translations.
I'm pretty sure the code below could be exchanged for components in the Turquoiseowl i18n project, but I haven't looked in to this.

My custom NuggetReplacer uses a component called Karambolo to extract the translations from the Po file.

    public interface Ii18nTranslator
    {
        public bool IsValid();
        public string Translate(string text);
        public void ReloadTranslations();
    }

    public class I18nPoTranslator : Ii18nTranslator
    {

        private POCatalog _poCatalog = new POCatalog();
        private string _poFilePath = null;

        public bool IsValid() => 
            _poCatalog != null && _poCatalog.Count > 0;


        public I18nPoTranslator(string poFilePath)
        {
            _poFilePath = poFilePath;

            ReloadTranslations();
        }


        public string Translate(string text)
        {
            if (text == null)
                return text;

            return NuggetReplacer.ReplaceNuggets(_poCatalog, text);
        }

        public void ReloadTranslations()
        {

            if (System.IO.File.Exists(_poFilePath))
            {
                lock (_poCatalog)
                {
                    _poCatalog = PoFileParser.ExtractTranslations(_poFilePath);
                }
            }

        }

    }

    public class NuggetReplacer
    {

        public static Regex NuggetFindRegex = new Regex(@"\[\[\[(.*?)\]\]\]", RegexOptions.Compiled);

        public static string ReplaceNuggets(POCatalog catalog, string text)
        {

            return NuggetFindRegex.Replace(text, delegate (Match match) {
                string textInclNugget = match.Value;
                var textExclNugget = match.Groups[1].Value;
                string searchText = textExclNugget;

                if (textExclNugget.IndexOf("///") >= 0)
                {
                    // Remove comments
                    searchText = textExclNugget.Substring(0, textExclNugget.IndexOf("///"));
                }

                var translation = catalog.GetTranslation(new Karambolo.PO.POKey(searchText));

                if (!string.IsNullOrEmpty(translation))
                {
                    // Translation found. Replace the nuggeted text with the translation
                    return translation;
                }

                // No translation found. Remove the nugget/comments
                return searchText;
            });
        }
        
    }

I then configure the middleware in Startup.cs -> Configure at the top of the method, before any other middleware, and add it so the translator can be injected aswell in the ConfigureServices method (optional)

private Ii18nTranslator translator { get; set; }

public Startup(IConfiguration configuration, IWebHostEnvironment environment, ILoggerFactory loggerFactory)
{
    // Create a global instance of the translator, to let us use it both through the middleware and Injection
    string webRootPath = environment.WebRootPath;
    translator = new I18nPoTranslator(Path.Combine(webRootPath, "locale", "fi_FI.po"));
}


public void ConfigureServices(IServiceCollection services)
{
    ....
    // Add the translator to Injection, to enable Email view translation and such
    services.TryAddSingleton<Ii18nTranslator>(translator);
    ....
}


public void Configure(IApplicationBuilder app, IWebHostEnvironment env, IHostApplicationLifetime applicationLifetime)
{
    // This needs to be at the top, or very close, since we need to "hijack" the Middleware flow
    string contentRootPath = env.ContentRootPath;
    app.UseI18nPoMiddleware(translator);

    // The rest goes here

}

Combine this with the i18n.PostBuild.exe for pot-extraction - and you've got a complete working solution.
It's not as smart as TurquoiseOwl i18n, and I'm sure there are things that could be smarter/more flexible/faster. But this project seems to be stuck, so at least my code provides a pretty simple working solution.

[Updated 2020-03-13, because I discovered the previous code had problems with error/excption handling, not passing exceptions down to the exception handler middlewares, and not restoring the Response.Body when exceptions occurred. Also added the i18nPoTranslator middle step, to enable translation both through middleware and injection. It makes the code listings a bit longer, but also alot more versitile]

[peters]

@HenricObjektvision A .NET core version is available at: https://github.com/fintermobilityas/i18n.core

It does not support all the features this project does but it made it possible for us to migrate a large MVC enterprise application without replacing the underlying localization technology.

[ajbeaven]

@turquoiseowl where are we at with this one? I'd be willing to put something towards sponsorship for this task if it meant it got things moving.

@peters awesome! Would be nice to get an idea of what is or isn't supported compared to this version.