Cannot find a way to build dependency dlls (libsodium and libzmq) into ZeroMQ.dll manifest #13

websteroy · 2015-02-16T04:57:03Z

I need to deploy libzmq.dll and libsodium.dll as part of ZeroMQ.dll's manifest. There's code in the Platform.Win32.cs source file that extracts these dependencies to AppData's temporary folder. However, I need to know if there is already a mechanism that is used to create a manifest for ZeroMQ.dll that would contain these dlls as part of the manifest. Right now, in code, I do not see any such thing.

In summary, my goal is to be able to deploy the ZeroMQ.dll, with manifest that contains the unmanaged dependencies which can be extracted at runtime. Do you have some ready made configuration that I could use?

@metadings Can you please help me here?

Thanks!

metadings · 2015-02-16T13:11:38Z

I don't get your question...

ZeroMQ does actually load the files from i386/amd64 folder, then it tries to load the files from system paths and finally it tries to extract a resource and load it. However there is no resource.

What do you mean by manifest?

websteroy · 2015-02-17T04:59:52Z

Hi Uli,

Yeah, my question is about the loading of the resource. Please ignore the previous message for now.

So far, what I have done is that I changed the "Build Action" property of the libzmq.dll and libsodium.dll to "Embedded Resource".

In method ExtractManifestResource, the resourceName variable contains a string like "libzmq.amd64.dll". But, shouldn't it be "ZeroMQ.amd64.libzmq.dll"?

The following is the existing code extract from file lib\Platform.Win32.cs:

string resourceName = string.Format(string.Format("{0}.{1}{2}", libraryName, arch, LibraryFileExtension));

libraryName = libzmq
arch = amd64

I changed it to:

string resourceName = string.Format(string.Format("ZeroMQ.{0}.{1}{2}", arch, libraryName, LibraryFileExtension));

And, now it works like a charm.

As far as I understand, the resource is built into the assembly based on the folder structure used. So, ZeroMQ\amd64\libzmq.dll gets converted to ZeroMQ.amd64.libzmq.dll.

Please correct me if I'm wrong.

P.S. My goal is to deploy ZeroMQ.dll and its dependencies as a single dll. I do not want to place libzmq.dll and libsodium.dll separately.

metadings · 2015-02-18T06:54:13Z

Hey littlesnake!

I like very much, that you're using the embedded resource at compilation; that's how I did it back then.

However I don't like to say twice ZeroMQ and libzmq.dll/ libzmq.so. Can't you rename that resource, to be amd64.libzmq.dll for example?

websteroy · 2015-02-18T08:09:26Z

The embedded resources which are part of the assembly are accessed based on their location in the assembly. so, if I want to access and extract libzmq.dll which is in folder amd64, which in turn is in assembly "ZeroMQ", we should form the absolute string - "ZeroMQ.amd64.libzmq.dll". That is how they are referenced in the assembly. Current implementation forms the string - "libzmq.amd64.dll".

Here's a snapshot of the resources present in the assembly. You can see what they are named as.

I'm not using the term ZeroMQ, it is automatically used by the build to identify the resource correctly within the assembly.

metadings · 2015-02-18T09:09:22Z

hehe, you are really good in VC#... ~~Would you like to make a pull request, to rename the resource files in the Posix and Win32 implementations?~~ well wait...

Now you want to do a git pull origin master, the files Platform.Posix and .Win32 are now being updated to ZeroMQ.amd64.libzmq.dll/ZeroMQ.amd64.libzmq.so...

websteroy · 2015-02-18T09:35:52Z

Yes, I will do that. I can check for Win32, however checking for Posix would be difficult.

Meanwhile, a few more questions:

Any reason why clrzmq4 does not use the LGPL license used by the libzmq project? I assume you want to allow the binding to be used by commercial tools if they need it. Am I right?
This binding doesn't appear on the CLR bindings page. http://zeromq.org/bindings:clr Any reason why?

metadings · 2015-02-18T09:41:35Z

I dislike the idea of "a license" - which license should I use? Public Domain, without warrant.

No, I didn't actually change the CLR bindings page, would you like to update the clrzmq binding?

websteroy · 2015-02-18T09:50:43Z

I think you should use the same license used by the libzmq project - LGPL
Or maybe you can even take Pieter's opinion on this.
I think the CLR bindings page should be updated to make this binding official. I see you already have Pieter's approval for the binding (http://grokbase.com/t/zeromq/zeromq-dev/1519tz7jz6/new-clrzmq-add-it-to-github). So, why not add it to the bindings page and make it official to the public?

metadings · 2015-02-18T09:55:27Z

How to change the bindings:clr page, do I need an editor account or just a git push ...? ;-)

There are some cool examples in the zguide, HWClient and HWServer, but also the Espresso...

Did you try them?

Updating... Please update your git repository, using git pull origin master... well now do it, this is the patch...

I need an update for the Platform.*.cs files... The program is written to load the defaults first (should we have default paths, like Environment variables?), then override by Platform (Win32 or Posix or Posix > MacOSX). There may also be someone who checks the installed runtime, and loads them by the boost naming scheme...

I am testing this on

Windows 7 x86 Core 2 Duo Notebook with VC2010 and .NET 4.x, and
ubuntu 14.04 amd64 Core 2 Quad Desktop with GNU C v4.8.2 compiler and mono 3.12.

websteroy · 2015-02-18T10:11:23Z

I'm new to this. So, even I don't know how the page should be updated.

metadings · 2015-02-18T10:13:52Z

Well, let's wait on Pieter's opinion on the licensing issue... I'm also glad to update the AssemblyInfo.cs and ZeroMQ.nuspec... How should I say in "Authors" and "Copyright"?

Please download clrzmq4 using
git clone https://github.com/zeromq/clrzmq4
then you just need
git pull origin master
to get the current ZeroMQ CLR namespace on your computer...

websteroy · 2015-02-20T10:17:11Z

Uli, one more question.

The currrent repository contains libzmq version 4.1.
Is it possible that after I fix the issue, we could release a build with the stable release 4.0.4?

I would prefer to work with a stable release instead of an RC.

metadings · 2015-02-20T11:18:20Z

zeromq4-1 is almost stable, but you can also use zeromq4-x. I don't want to move back to zeromq4-x.

Just replace the files in i386 and amd64.

The zmq.sizeof_zmq_msg_t is also changing with zeromq4-x to zeromq4-1, ~~so you have to set~~

public static readonly int sizeof_zmq_msg_t = sizeof_zmq_msg_t_v3;

Now there is if (minor == 0) sizeof_zmq_msg_t = sizeof_zmq_msg_t_v3; - you just need to place the libzmq.dll/.so to the i386/amd64 folder.

I do not recommend switching back, looking on libzmq :-)

hintjens · 2015-02-20T11:24:29Z

For licensing I'd suggest using MPL v2 as we're doing in newer projects.
There's a page on the community wiki that explains this.
On Feb 18, 2015 11:13 AM, "Uli Riehm" notifications@github.com wrote:

Well, let's wait on Pieter's opinion on the licensing issue... I'm also
glad to update the AssemblyInfo.cs and ZeroMQ.nuspec... How should I say
in "Author" and "License"?

—
Reply to this email directly or view it on GitHub
#13 (comment).

metadings · 2015-02-20T11:40:31Z

I would like to say "Alle Angaben ohne Gewähr.", however Americans like to say "Alle Angaben ohne irgendeine Gewähr, explizit und inklusive ... "

I am using LGPL now.

metadings · 2015-02-21T01:11:00Z

Beware the v4.0.4 is the file in miru's archives, they are originally from 2014-03-15!

Current version of zeromq4-x is v4.0.5 (v4.0.6 the next one).

We need to update the Installers for Windows section on zeromq.org... Shouldn't we prefer zeromq4-1 instead of zeromq4-x now and use libzmq then?

gillima · 2015-10-17T12:17:43Z

Using a code like this would allow to add the libraries to any assembly of the Project. So a recompile of clrzmq4 would not be necessary.

            if (resourceStream == null)
            {
                // Locate the resource in any of the current loaded assemblies
                var resourceAssembly = AppDomain.CurrentDomain
                    .GetAssemblies()
                    .FirstOrDefault(ass => ass.GetManifestResourceNames().Contains(resourceName));

                if (resourceAssembly != null)
                    resourceStream = resourceAssembly.GetManifestResourceStream(resourceName);
            }

Also the name of a embedded resource can be defined for Visual Studio. It's not possible with the UI, but by editing the .csproj with a text editor the names can be defined like this:

 <ItemGroup>
    <EmbeddedResource Include="amd64\libsodium.dll">
      <LogicalName>ZeroMQ.libsodium.amd64.dll</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="amd64\libsodium.so">
      <LogicalName>ZeroMQ.libsodium.amd64.so</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="amd64\libzmq.dll">
      <LogicalName>ZeroMQ.libzmq.amd64.dll</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="amd64\libzmq.so">
      <LogicalName>ZeroMQ.libzmq.amd64.so</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="i386\libsodium.dll">
      <LogicalName>ZeroMQ.libsodium.i386.dll</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="i386\libsodium.so">
      <LogicalName>ZeroMQ.libsodium.i386.so</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="i386\libzmq.dll">
      <LogicalName>ZeroMQ.libzmq.i386.dll</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="i386\libzmq.so">
      <LogicalName>ZeroMQ.libzmq.i386.so</LogicalName>
    </EmbeddedResource>
  </ItemGroup>

metadings · 2015-10-17T21:19:45Z

Yes, that looks correctly... I decided not to have it as resource, because it will add some weight to the binary. You need to (re)compile ZeroMQ/clrzmq4 with your EmbeddedResources.

I find it very interesting, that you ask AppDomain.CurrentDomain.GetAssemblies() for having one of these resources. I didn't think about the possibilities out there :)
But: There are serious security problems with loading just some byte[] from some assembly...

metadings · 2015-10-17T22:14:34Z

AppDomain.CurrentDomain.GetAssemblies() on a x86 Windows machine

[
    "Assembly" { "Location": "C:\Windows\Microsoft.NET\Framework\v4.0.30319\mscorlib.dll" },
    "Assembly" { "Location": "C:\Projects\YourProject\bin\Debug\YourProject.exe" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll" },
    "Assembly" { "Location": "C:\Projects\YourProject\bin\Debug\ZeroMQ.dll" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Xml\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.dll" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Security\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Security.dll" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Data.SqlXml\v4.0_4.0.0.0__b77a5c561934e089\System.Data.SqlXml.dll" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_32\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll" }
]

AppDomain.CurrentDomain.GetAssemblies() on an amd64 Linux machine

[
    "Assembly" { "Location": "/usr/lib/mono/4.5/mscorlib.dll" },
    "Assembly" { "Location": "/home/metadings/Projects/YourProject/bin/Debug/YourProject.exe" },
    "Assembly" { "Location": "/usr/lib/mono/gac/System.Core/4.0.0.0__b77a5c561934e089/System.Core.dll" },
    "Assembly" { "Location": "/home/metadings/Projects/YourProject/bin/Debug/ZeroMQ.dll" },
    "Assembly" { "Location": "/usr/lib/mono/gac/System/4.0.0.0__b77a5c561934e089/System.dll" },
    "Assembly" { "Location": "/usr/lib/mono/gac/System.Configuration/4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll" },
    "Assembly" { "Location": "/usr/lib/mono/gac/System.Xml/4.0.0.0__b77a5c561934e089/System.Xml.dll" },
    "Assembly" { "Location": "/home/metadings/Projects/YourProject/bin/Debug/System.Data.SQLite.dll" },
    "Assembly" { "Location": "/usr/lib/mono/gac/System.Data/4.0.0.0__b77a5c561934e089/System.Data.dll" },
    "Assembly" { "Location": "/usr/lib/mono/gac/Mono.Security/4.0.0.0__0738eb9f132ed756/Mono.Security.dll" }
]

gillima · 2015-10-18T07:07:16Z

Might be correct, but is it less secure than have them in subfolders of the application...

metadings · 2015-10-18T11:12:44Z

Hm... It's difficult to weight out flexibility and security... Don't you mean "more" secure? :)
You are right, probably should I trust a compiled assembly more than some loosely cupled files.

I think there should be some configuration directive where you can tell, if you first want the libzmq.dll from C:\Windows\system32, or you first want the one from ~\amd64, or you're just trusting the version in the resources; a place where I also can say from which assemblies I want libzmq.* to be extracted; a place where I also can say to which temp folder I want the extracted assembly resource to be.

gillima · 2015-10-18T12:22:52Z

Sounds cool to me. Should not be to hard to achieve as long as the configuration is done before the first call into platform...

metadings · 2015-10-18T12:28:17Z

This is so awesome...! :) I need to copy over my HTTPDealerDevice... STREAM sockets do split the message by 8192 bytes; my HTTPDealerDevice actually does read the Content-Length from the header and then REQuests/REPlies the full message...

asgerhallas · 2016-01-22T12:45:11Z

Hi, I'm having some trouble using this in an ASP.NET environment where it seems to try to locate the asseblies from my user/AppData/Temp directory. After having read this thread, I'm actually not really sure what the conclusion is...

Is it possible to get the asseblies from embedded ressources in the current library (as it is on nuget)?
Or is this a change coming?
Or am I just out of luck? :)

metadings · 2016-01-22T13:20:18Z

OH, I didn't test this in an ASP.NET environment...
(I'm running my own one, I don't know how to call it, it's just ZeroMQ, an HttpDealerDevice and a bit of CSharpCodeProvider ;))

You need to add the libraries as an embedded resource and recompile ZeroMQ/clrzmq4, instead of just adding them using "copy if newer".
This is not done because I want to be flexible in using libzmq and I also don't want the file becoming too big, however you can do so.

asgerhallas · 2016-01-25T09:11:10Z

Ok, thank you for the quick response (and sorry for my late one). That'll work :)
Would you consider maybe having to nuget packages, one as now and one with the files embedded? For easier access to this solution?

metadings · 2016-01-25T14:35:31Z

Yes, I plan to do so... I need new binaries, because I compiled zeromq4-1 a year ago.
However, I don't have a Windows machine, so this is being delayed right now...

(I need a Windows binary using VC2010 (?), a Linux binary using GCC 4.8.4 (?) and maybe also MacOSX and Android binaries.)

asgerhallas · 2016-01-25T15:12:09Z

Fantastic!

Now I don't know how this works exactly, but isn't it the same binaries as those deployed on the ZeroMQ website via the Windows installer?

Or else I might be able to do the Windows build unless it's seriously complicated :)

metadings · 2016-01-25T15:20:37Z

Well it is a bit complicated... :-)

And NO, the binary deployed in the "Windows installer" on the website is seriously outdated, it dates back to 2014.

asgerhallas · 2016-01-26T15:22:02Z

Ok. I'll stand by then. Let me know if there's anything I should do! :)

asgerhallas · 2016-02-16T06:41:11Z

Hi again, I see there's a new release, cool! Does this mean there's a chance of a nuget-package with embedded binaries? :)

miaf · 2016-07-20T13:41:11Z

Hi there. Sorry I am very interested by the discussion but I am not getting you well. I am facing the same problem when trying to let my frontend asp.net core communicate with the backend and get the following error details
InnerException:
HResult=-2147024894
Message=UnmanagedLibrary[libzmq] Unable to load binary "libzmq" from folders
C:\Works\Pro\HSMS\WebPortal\bin\Debug\net461\win7-x86\i386\libzmq.dll
C:\Works\Pro\HSMS\WebPortal\bin\Debug\net461\win7-x86\x86\libzmq.dll
C:\Windows\system32\libzmq.dll
For simplicity, I just copy the i386 folder into C:\Works\Pro\HSMS\WebPortal\bin\Debug\net461\win7-x86\ and everything is working find. I think trying the others way work as well.

metadings · 2019-03-09T12:57:02Z

Closing this issue. It's too old

metadings closed this as completed Jan 25, 2016

metadings reopened this Jan 25, 2016

metadings closed this as completed Mar 9, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot find a way to build dependency dlls (libsodium and libzmq) into ZeroMQ.dll manifest #13

Cannot find a way to build dependency dlls (libsodium and libzmq) into ZeroMQ.dll manifest #13

websteroy commented Feb 16, 2015

metadings commented Feb 16, 2015

websteroy commented Feb 17, 2015

metadings commented Feb 18, 2015

websteroy commented Feb 18, 2015

metadings commented Feb 18, 2015

websteroy commented Feb 18, 2015

metadings commented Feb 18, 2015

websteroy commented Feb 18, 2015

metadings commented Feb 18, 2015

websteroy commented Feb 18, 2015

metadings commented Feb 18, 2015

websteroy commented Feb 20, 2015

metadings commented Feb 20, 2015

hintjens commented Feb 20, 2015

metadings commented Feb 20, 2015

metadings commented Feb 21, 2015

gillima commented Oct 17, 2015

metadings commented Oct 17, 2015

metadings commented Oct 17, 2015

gillima commented Oct 18, 2015

metadings commented Oct 18, 2015

gillima commented Oct 18, 2015

metadings commented Oct 18, 2015

asgerhallas commented Jan 22, 2016

metadings commented Jan 22, 2016

asgerhallas commented Jan 25, 2016

metadings commented Jan 25, 2016

asgerhallas commented Jan 25, 2016

metadings commented Jan 25, 2016

asgerhallas commented Jan 26, 2016

asgerhallas commented Feb 16, 2016

miaf commented Jul 20, 2016

metadings commented Mar 9, 2019

Cannot find a way to build dependency dlls (libsodium and libzmq) into ZeroMQ.dll manifest #13

Cannot find a way to build dependency dlls (libsodium and libzmq) into ZeroMQ.dll manifest #13

Comments

websteroy commented Feb 16, 2015

metadings commented Feb 16, 2015

websteroy commented Feb 17, 2015

metadings commented Feb 18, 2015

websteroy commented Feb 18, 2015

metadings commented Feb 18, 2015

websteroy commented Feb 18, 2015

metadings commented Feb 18, 2015

websteroy commented Feb 18, 2015

metadings commented Feb 18, 2015

websteroy commented Feb 18, 2015

metadings commented Feb 18, 2015

websteroy commented Feb 20, 2015

metadings commented Feb 20, 2015

hintjens commented Feb 20, 2015

metadings commented Feb 20, 2015

metadings commented Feb 21, 2015

gillima commented Oct 17, 2015

metadings commented Oct 17, 2015

metadings commented Oct 17, 2015

gillima commented Oct 18, 2015

metadings commented Oct 18, 2015

gillima commented Oct 18, 2015

metadings commented Oct 18, 2015

asgerhallas commented Jan 22, 2016

metadings commented Jan 22, 2016

asgerhallas commented Jan 25, 2016

metadings commented Jan 25, 2016

asgerhallas commented Jan 25, 2016

metadings commented Jan 25, 2016

asgerhallas commented Jan 26, 2016

asgerhallas commented Feb 16, 2016

miaf commented Jul 20, 2016

metadings commented Mar 9, 2019