Question: How do I properly use ResourceDescription to embed resource information?

[JoshVarty]

This is a followup to: #7772 and a recent Stack Overflow question

I should preface this by saying I have only a cursory understanding of how resources are embedded within DLLs, so it's possible I'm misusing or misunderstanding this API.

Scenario: I'm working on a Visual Studio plugin that compiles and runs our customer's code as they write it. We've run into some issues when emitting from the Visual Studio Workspace in which resource files are not automatically embedded in emitted DLLs. @jasonmalinowski explained that the VisualStudioWorkspace wasn't built to support emitting directly, so this is understandable behavior.

Yesterday I worked on a quick workaround that used EnvDTE to discover resource files. My plan was to create any required ResoureDescription automatically and pass them to Emit().

My approach is essentially:

string fileName = "MyResources.resx";
string filePath = "C:\Users\Josh\...\MyResources.resx";
var resourceDescription = new Microsoft.CodeAnalysis.ResourceDescription(resourceFileName, () => new FileStream(resourceFilePath, FileMode.Open), isPublic: true);
var resourceDescriptions = new List<ResourceDescription>() { resourceDescription };

var compilation = await project.GetCompilationAsync();
var result = compilation.Emit("C:\Users\Josh\OutputPath\ConsoleApplication1.exe", manifestResources: resourceDescriptions);

However, when I run this DLL I'm still receiving an error telling me the resources weren't embedded properly:

Unhandled MissingManifestResourceException Could not find any resources appropriate for the specified culture (or the neutral culture) in the given assembly. Make sure "ConsoleApplication1.MyResources.resources" was correctly embedded or linked into assembly "ConsoleApplication1". 

My next step was to compare the DLL produced when building the same project (ConsoleApplication1) in VS with the output DLL I've emitted. I just opened both up in Notepad++ and noticed a few differences:

In my output DLL, there is a full plaintext XML ResX schema. The keys and values are present.

In Visual Studio's output DLL, the keys and values are present, but there isn't a plaintext resx schema. It looks like it's been serialized or something. (Once again, I'm a little over my head here)

Am I misusing the API? Could someone help point me in the right direction on this one?

[davkean]

RESXs don't actually get embedded in dlls, instead, they get transformed via ResGen.exe into a binary ".resources" file (ResourceManager format) and that gets embedded into the dll.

To do this, you can either call ResGen yourself, either directly or via the GenerateResource MSBuild task. Alternatively, but harder, would be to directly use the ResXResourceReader to read from RESX and ResourceWriter to write directly to this binary format. You can see the code that the GenerateResource task does to do this here: https://github.com/Microsoft/msbuild/blob/a4e0dd5f4b31a6c9acb1bab25ac401c438c3dfac/src/XMakeTasks/GenerateResource.cs.

[JoshVarty]

Thanks, I'll take a look at the links you've provided and try to work on a temporary solution until I can better plan how we'll handle MSBuild tasks in general.

One last question, so does the manifestResources argument in Emit() accept these binary .resources files you mentioned?

[JoshVarty]

One other question: I'm emitting the DLLs to memory, I'm guessing there's no way to co-erce MSBuild ot GenerateResource.cs to operate on a stream? If not, then my best option is to essentially rewrite this stuff myself, correct?

[jasonmalinowski]

@JoshVarty: nope, unfortunately not. Fundamentally, MSBuild isn't designed to be a "zero-impact" build system. It'd be great if all operations operated on streams and you could build without impacting the disk, but unless you've got a time machine that's not going to happen.

[JoshVarty]

That's fine, I don't mind doing some extra work here.

Just a quick sanity check before I do: If I use ResXResourceReader and ResourceWriter to create the right binary, do I just pass this to Emit() as a ResourceDescription and it'll embed it properly?

[davkean]

I'm new to Rolsyn, so I'll have to let @jasonmalinowski answer that.

[jasonmalinowski]

I have no idea actually. We should ask @dotnet/roslyn-compiler.

[JoshVarty]

I just went ahead and implemented it and it seemed to work.

[JoshVarty]

I think I'll go ahead and close this now as I've gotten the base case working. Once I'm done I'll probably write something up for others who are interested in using this API.

Thanks guys!

[RenniePet]

@JoshVarty "Once I'm done I'll probably write something up for others who are interested in using this API." Did you ever get around to doing this? Or is the code you wrote that implemented this available in one of your projects? Thanks.

[JoshVarty]

@RenniePet

If you want to support all conceivable resource files (eg. .edmx files for Entity Framework) my approach ended up not really scaling because you have to understand how to read all those different resources. That's why I didn't end up writing about it.

If you want to support .resx files I believe the mistake I was made above was how I was creating the ResourceDescription. I believe you need to read from the .resx file using a ResourceReader and use a ResourceWriter to write to a MemoryStream which you pass to Roslyn and it will insert the resource into the binary.

var resourceDescription = new Microsoft.CodeAnalysis.ResourceDescription(resourceFullName, () => ProcessFile(resourceFilePath), isPublic: true);

Where my implementation of ProcessFile was something like:

private static MemoryStream ProcessFile(string inFile)
{
    var readers = new List<ReaderInfo>();
    var resources = readResources(inFile);
    using (var outStream = writeResources(resources))
    {
        //outstream is closed, so we create a new memory stream based on its buffer.
        var openStream = new MemoryStream(outStream.GetBuffer());
        return openStream;
    }
}

private static MemoryStream writeResources(ReaderInfo resources)
{
    var memoryStream = new MemoryStream();
    using (var resourceWriter = new ResourceWriter(memoryStream))
    {
        writeResources(resources, resourceWriter);
    }
    return memoryStream;
}

private static void writeResources(ReaderInfo readerInfo, ResourceWriter resourceWriter)
{
    foreach (ResourceEntry entry in readerInfo.resources)
    {
        string key = entry.Name;
        object value = entry.Value;
        resourceWriter.AddResource(key, value);
    }
}

private static ReaderInfo readResources(string fileName)
{
    ReaderInfo readerInfo = new ReaderInfo();
    var path = Path.GetDirectoryName(fileName);
    var resXReader = new ResXResourceReader(fileName);
    resXReader.BasePath = path;

    using (resXReader)
    {
        IDictionaryEnumerator resEnum = resXReader.GetEnumerator();
        while (resEnum.MoveNext())
        {
            string name = (string)resEnum.Key;
            object value = resEnum.Value;
            addResource(readerInfo, name, value, fileName);
        }
    }

    return readerInfo;
}

private static void addResource(ReaderInfo readerInfo, string name, object value, string fileName)
{
    ResourceEntry entry = new ResourceEntry(name, value);

    if (readerInfo.resourcesHashTable.ContainsKey(name))
    {
        //Duplicate resource name. We'll ignore and continue.
        return;
    }

    readerInfo.resources.Add(entry);
    readerInfo.resourcesHashTable.Add(name, value);
}

internal sealed class ReaderInfo
{
    // We use a list to preserve the resource ordering (primarily for easier testing),
    // but also use a hash table to check for duplicate names.
    public ArrayList resources { get; }
    public Hashtable resourcesHashTable { get; }

    public ReaderInfo()
    {
        resources = new ArrayList();
        resourcesHashTable = new Hashtable(StringComparer.OrdinalIgnoreCase);
    }
}

/// <summary>
/// Name value resource pair to go in resources list
/// </summary>
private class ResourceEntry
{
    public ResourceEntry(string name, object value)
    {
        this.Name = name;
        this.Value = value;
    }

    public string Name { get; }
    public object Value { get; }
}

[RenniePet]

@JoshVarty Thank you very much for your reply. After many difficulties I've come up with something that seems to work for me, at least for .resx and for .txt embedded resources.

  /// <summary>
  /// Method to analyze the .csproj file and determine if there are embedded resource files and 
  /// to convert them into a collection of ResourceDescription objects.
  ///
  /// Embedded resource files are typically .resx files, but there are many other possibilities: 
  /// .txt, .xml, graphics files, etc.
  ///
  /// Parts of this code based on this Stack Overflow answer:
  /// https://stackoverflow.com/a/44142655/253938
  /// </summary>
  /// <returns>collection of ResourceDescription objects, 
  ///          or null if no embedded resource files or error encountered</returns>
  [SuppressMessage("Microsoft.Design", "CA1031:DoNotCatchGeneralExceptionTypes")]
  private static IEnumerable<ResourceDescription> GetManifestResources(string csprojFileName)
  {
     try
     {
        XDocument csprojAsXml = XDocument.Load(csprojFileName);
        XNamespace xmlNamespace = "http://schemas.microsoft.com/developer/msbuild/2003";

        IEnumerable<XElement> projectNamespaceElements = 
                                        csprojAsXml.Descendants(xmlNamespace + "RootNamespace");
        XElement projectNamespaceElement = projectNamespaceElements.First();
        if (projectNamespaceElement == null)
        {
           DisplayErrorOrInfo(
                       "Unable to determine default namespace for project " + csprojFileName);
           return null;
        }
        string projectNamespace = projectNamespaceElement.Value;

        List<ResourceDescription> resourceDescriptions = new List<ResourceDescription>();
        foreach (XElement embeddedResourceElement in
                                     csprojAsXml.Descendants(xmlNamespace + "EmbeddedResource"))
        {
           XAttribute includeAttribute = embeddedResourceElement.Attribute("Include");
           if (includeAttribute == null)
              continue;  // Shouldn't be possible

           string resourceFilename = includeAttribute.Value;

           // ReSharper disable once AssignNullToNotNullAttribute
           string resourceFullFilename = 
                          Path.Combine(Path.GetDirectoryName(csprojFileName), resourceFilename);

           string resourceName = 
                        resourceFilename.EndsWith(".resx", StringComparison.OrdinalIgnoreCase) ?
                           resourceFilename.Remove(resourceFilename.Length - 5) + ".resources" :
                           resourceFilename;

           resourceDescriptions.Add(
                new ResourceDescription(projectNamespace + "." + resourceName, 
                                        () => ProvideResourceData(resourceFullFilename), true));
        }

        return resourceDescriptions.Count == 0 ? null : resourceDescriptions;
     }
     catch (Exception e)
     {
        DisplayErrorOrInfo("Exception while processing embedded resource files: " + e.Message);
        return null;
     }
  }

  /// <summary>
  /// Method that gets called by ManagedResource.WriteData() in project CodeAnalysis during code 
  /// emitting to get the data for an embedded resource file. Caller guarantees that the
  /// returned Stream object gets disposed.
  /// </summary>
  /// <param name="resourceFullFilename">full path and filename for resource file to embed</param>
  /// <returns>MemoryStream containing .resource file data for a .resx file, 
  ///          or FileStream providing image of a non-.resx file</returns>
  [SuppressMessage("Microsoft.Reliability", "CA2000:Dispose objects before losing scope")]
  private static Stream ProvideResourceData(string resourceFullFilename)
  {
     // For non-.resx files just create a FileStream object to read the file as binary data
     if (!resourceFullFilename.EndsWith(".resx", StringComparison.OrdinalIgnoreCase))
        return new FileStream(resourceFullFilename, FileMode.Open);

     // Remainder of this method converts a .resx file into .resource file data and returns it 
     //  as a MemoryStream
     MemoryStream shortLivedBackingStream = new MemoryStream();
     using (ResourceWriter resourceWriter = new ResourceWriter(shortLivedBackingStream))
     {
        resourceWriter.TypeNameConverter = TypeNameConverter;
        using (ResXResourceReader resourceReader = new ResXResourceReader(resourceFullFilename))
        {
           IDictionaryEnumerator dictionaryEnumerator = resourceReader.GetEnumerator();
           while (dictionaryEnumerator.MoveNext())
           {
              string resourceKey = dictionaryEnumerator.Key as string;
              if (resourceKey != null)  // Should not be possible
                 resourceWriter.AddResource(resourceKey, dictionaryEnumerator.Value);
           }
        }
     }

     // This needed because shortLivedBackingStream is now closed
     return new MemoryStream(shortLivedBackingStream.GetBuffer());
  }

  /// <summary>
  /// This is needed to fix a "Could not load file or assembly 'System.Drawing, Version=4.0.0.0"
  /// exception, although I'm not sure why that exception was occurring.
  ///
  /// See also here: https://github.com/dotnet/corefx/issues/11083 - it says it doesn't work,
  /// but it did save the day for me.
  /// </summary>
  private static string TypeNameConverter(Type objectType)
  {
     // ReSharper disable once PossibleNullReferenceException
     return objectType.AssemblyQualifiedName.Replace("4.0.0.0", "2.0.0.0");
  }

If you have any suggestions I'd be glad to hear them. Thanks again.