I'll have to check up on that, as the code belongs to my former employer, Unwire.

I have seen that, yes. It would help a lot if this would be possible!

OK. I have been given the go to issue Handsoap under LGPL 3. That should be compatible with GPL, I assume?

Yes, that would be fine (as can be seen at http://www.fsf.org/licensing/licenses/gpl-faq.html#AllCompatibility) - thanks a lot for the help and the fast response!

I've added a method to XmlDocument, so that you can turn the xml-header off:

doc.xml_header = false

This is available from 0.5.8

Thanks!

What's wrong with github, in this respect? You can install with:

gem install troelskn-handsoap -s http://gems.github.com

I'm pretty sure (although I don't recall the exact syntax), that you can specify source of dependencies, in the gemspec.

Okay, I have just tested installing a local test gem with a dependency on troelskn-handsoap and this works. However, I don't know if a gem served by Rubyforge will work. Will have to test this.

However, there is no option for specifying the source of a gem dependency in the gemspec.

However, I don't know if a gem served by Rubyforge will work.

I don't see why it shouldn't.I don't think Rubyforge has a privileged status over github, with regards to gem.

I'd rather not have to publish the gem in multiple places, just to save people from adding github as a gem source. There are lots of other gems that are distributed through github (For example Rails), so I think it's fair to assume that users add it as a source.

Rails is hosted on github but the official gems are distributed via Rubyforge (see Rails rubyforge project).

Although I kinda like having rubyforge as central gem repo, I'm not against adding github as gem source. I have pushed a test gem to rubyforge with a dependency on troelskn-handsoap and it worked. So I will specify troelskn-handsoap in the gemspec. However, I think you would probably avoid confusion if you released your handsoap library as handsoap gem and not as troelskn-handsoap.

Gem is now available through gemcutter. Please file a new issue on the tracker for unwire/handsoap, if you still think we should distribute through Rubyforge.

Gemcutter is fine, thanks!

Great effort, thanks !

This has been fixed in 0.5.5. The generator will now decide, if there are multiple portType's available in WSDL.

Those are good suggestions. Making it possible to override the endpoint on the instance level should be fairly simple and makes good sense. Likewise, making it possible to set the http-driver on the instance level shouldn't be a big deal to implement either. I wonder what the use-case for this is though?

Regarding http-auth, you should be able to use basic www-authentication by setting the request headers. Have a look in the wiki for an example. That is unless I misunderstood your request?

Silly me. You specifically said "Digest authentication". No, this would require that the http-drivers were extended to allow the functionality. Again, this is probably fairly simple to implement, since some (Probably all, but I haven't checked) of the underlying drivers have support for it, so it should merely be a matter of wrapping that up nicely.

Could you explain why you want a on_after_create_http_client callback?

Regarding instance level endpoint: A nice solution would probably be to create a #uri method that returns nil by default and change the line in #dispatch to say

response = send_http_request(uri || self.class.uri, doc.to_s, headers)

This would allow a subclass to override the uri method for custom endpoint uri.

Rational behind instance level endpoint: Some SOAP services are implemented by a number of clients, for example, Intel AMT SOAP services are implemented by all devices featuring the vPro sign. Therefore there only needs to be one service class with customizable endpoint URIs. I currently use the on_after_create_http_request hook to overwrite the request.url with the customized one.

Regarding http-driver on the instance level: As far as I know not all http drivers implement the digest authentication, httpclient surely does and I think curb does as well. However, the ruby built-in net/http library does not support digest authentication.

It would be useful on the instance level because consider two different libraries for accessing services via soap built on handsoap. One needs digest authentication and chooses to use httpclient for this since it is a pure ruby solution. The other library doesn't want to depend on any third-party gems except handsoap and therefore uses the net/http driver and REXML and sets them via class level accessors. When one uses both libraries the behaviour may be different to the case where only one of the libraries is used.

Regarding the on_after_create_http_client callback: I have seen something like this mentioned in another issue but found no implementation. The reason for such a callback is that it would allow one to customize the http access, for example, by setting the used authentication credentials.

My current solution to the digest authentication problem is that I copied the #send_http_request method to my service base class and modified the way the http driver is created and called, ie. I pass the credentials to http.send_http_request(request, username, password). For this I also needed to copy the httpclient driver nearly verbatimly and then changed the method signature to include the username and password and added one line to add the credentials to the created httpclient object. As you can see, the callback would help to avoid code duplication.

Apart from these small issues, the handsoap library rules! ;-)

I have seen something like this mentioned in another issue but found no implementation. The reason for such a callback is that it would allow one to customize the http access, for example, by setting the used authentication credentials.

on_after_create_http_request existed in older versions. I removed it, when I introduced the current http-abstraction. I don't think it should be nescesarry to re-introduce it - The idea is to put all such things on the http-request and then let the abstraction layer translate to each underlying driver. There may be some features, that are simply not supported by the driver. In this case, I think a runtime error would be in place. Although, digest is relatively simple, so it could be implemented in userland code, if net/http doesn't support it directly. I'll see if I can find time to look at it in the near future.

This would also be okay. Then it would be nice to set the authentication credentials (username, password, ev. domain) in the http request callback or even with the same system proposed for URI handling (ie. a class setting and an object setting).

I have added a new method to Http::Request in the repository. To authenticate, use:

class MyService < Handsoap::Service
  def on_after_create_http_request(http_request)
    http_request.set_auth @user, @pass
  end

The underlying drivers should then automagically determine which type of authentication the server wants, and use that. Both httpclient and curb supports at least basic and digest, and they may support other types, if the required libraries are installed etc. The bare-bones net::http only supports basic out of the box.

I haven't tested it in any way, since it's fairly complicated to do. Can you test this against your service, to verify that it works?

The 0.5.6 version works like a charm, thanks! I have tried it with httpclient and curb and both work as expected.

0.5.7 now has hooks that allow to specify the endpoint on a per-instance level.

Thanks, this does the trick!

One other small request: some SOAP services don't like the <?xml version="1.0" ?> header (Intel AMT SOAP services being one of them). It seems to be trivial to add an accessor to Handsoap::XmlMason::Document which controls whether an XML header is written or not.

Yes, that should be possible. Could you please open a new issue for this?

Will do that!

After some more investigation, I have found that the problem is not with chunked responses, but rather with the 100-Continue header and curl. I'll try to put a test case together.

Fixed in repo

0.5.0 should solve this issue. Access the individual parts through response.http_response.parts

OK. I'll add that patch, but I think you may be right about putting an interface before the http client library.

OK, if I have some time I'll fork the project an add an HTTP client interface. Besides from solving this problem, it is also possible to use a mock HTTP client for testing purposes.

I already have a http abstraction from another project that I could pull in. My biggest concern is really about adding complexity - and I'm more and more convinced that it is worth it.

My idea was to create different classes that implement the send_http_request method, and adding an API to set additional HTTP headers (e.g. Handsoap::Service.http_headers[:Authorization] = 'blah').

But if you already have something available, it may be easier to use that than writing it from scratch.

Yes, I actually have meant to document that for a while now, but I can't figure out where to put it in the documentation. Perhaps using the wiki would make sense here, as there are different ways to do authentication over soap.

In theory, I agree on your suggestion about abstracting the difference between curb and httpclient out, but it can quickly become rather complicated. For now, I have just accepted the lack of portability in return for less complex code. Not an overly consistent choice, I know ...

FWIW, I think the following variation is portable:

def on_after_create_http_client(http_client)
 # beware the missing chomp, it will make you cry....
 auth = Base64.encode64("#{username}:#{password}").chomp
 http_client.headers['Authorization'] = "Basic #{auth}"
end

Personally, I prefer using the Github project's wiki. For this issue, I would probably have added my code example to it myself :-)

I've enabled the wiki and started a page on authentication: http://wiki.github.com/troelskn/handsoap/authentication

I have now released 0.5.0, which puts the http interaction in a separate driver layer. This gives you much better control over the interaction. At present, I haven't exposed method for dealing with SSL certificates, but it's just a matter of adding them in. Note that this changes the API so code which used on_after_create_http_client pre 0.5.0 will have to be rewritten.

I made a slightly different change, since Tempfile and StringIO share a common interface.

This is odd. The xml parser is supposed to raise an exception, but this is then caught and handled higher up in the library. Can you give me a complete stack trace?

Github is hatcheting the stack trace, but I did send it to you in an attachment to your email address. I will resend in case it didn't get to you.

Doh just realized you said "Stack Trace" not wiredump...can you give me some guidance on how to do the stack trace?

I know you can put it inside a begin..rescue..end block (error.backtrace), but I don't see a good place in xml_query_front.rb to do that.

If I wrap my own code that way, it doesn't produce anything more than what we already have.

You need to catch the exception on the top level of your test-script, and print out the stack trace. Eg. wrap your code in something like:

begin
  # call code here
rescue Exception => ex
  puts ex.to_s + "\n" +  ex.backtrace.join("\n")
end

.. normally this would happen automatically, but if you're running the code inside a testcase, it may be why the stack trace is squelched. You can also try to run the testcase with --verbose option to get the same output.

Isn't testcase part of Rails? I've never used Rails, and am not using any formalized "testcase" libs or anything. This is really strange. It doesn't even show the normal level of verbosity in the trace when the exception is raised, and I can't seem to make it produce more.

I ran my code using "irb -tracer" which I can send to you. It is really verbose though, and quite the output is quite large. I don't think its of much value.

Forgot to add my code so you can see what I am doing.

Here's the snippet:

system_service = NBI::SystemService.new
resp = system_service.login($config.username, $config.password, $config.domain)

if system_service.logged_in? then

puts "Login Successful"
begin
    resp = system_service.logout()
rescue => error
    puts error.backtrace
end

else

puts "Error logging in"

end

Here's the login() and logout() methods:

    def login(username, password, domain_name="global")
        $log.info "LoginRequest =>"
        begin
            response = invoke("ns2:LoginRequest") do |message|
                message.add("ns2:domainName", domain_name)
                message.add("ns2:userName", username)
                message.add("ns2:password", password)
            end
        rescue => error
            puts error.backtrace
        end
        $log.info "LoginResponse <="
        parse_login_response(response)
    end

    def logout
        begin
            invoke("ns2:LogoutRequest") do |message|
                message.add("ns2:authToken", @auth_token)
            end
        rescue => error
            puts error.backtrace
        end
    end

And here is the output:

mcnultyd@deskcat:~/nsm-api/handsoap-ver$ ruby test.rb
Wiredump device = FILE

<File:0x7f52756be120>

<File:0x7f52756bdf18>

<File:0x7f52756bdfb8>

<File:0x7f52756be058>

Exception LoadError' at /usr/lib/ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- curb<br/> Login Successful<br/> ExceptionHandsoap::XmlQueryFront::ParseError' at /var/lib/gems/1.8/gems/troelskn-handsoap-0.4.3/lib/handsoap/xml_query_front.rb:60 - Handsoap::XmlQueryFront::ParseError
mcnultyd@deskcat:~/nsm-api/handsoap-ver$

As you can see, I've put exception handlers around everything that seems to matter. The exception is never caught. What am I doing wrong here?

Can you mail me test.rb?

Turned out that this was a problem with the client code. The global $DEBUG flag caused some confusion.

It looks like I made a mistake. It doesn't make sense to use http://www.w3.org/2003/05/soap-envelope for the soap-envelope. I think I copied this from an example, which must have gotten it wrong. The services I have used have been v. 1.1, so that's why I haven't noticed before. I'll make the change and push an update.

I've done a little more research, and it looks like http://www.w3.org/2003/05/soap-envelope is the correct current namespace for SOAP 1.2. However, it appears that during the drafting phase for the SOAP 1.2 specification, that namespace changed a few times, which is why you still see other ones (like http://www.w3.org/2001/12/soap-envelope) "in the wild" (in some of the w3schools SOAP articles, for instance). You can see older versions of the spec by following the "Previous Versions" link at the top of the current spec.

What I don't know is whether it makes sense to support any older versions of the SOAP 1.2 namespace other than the one that made it into the Recommendation.

Aha. Thanks for the research then. I would say that it does make sense, since service-providers have a way of misinterpreting the protocol all the time, and Handsoap is foremost a pragmatic tool. We could add an :envelope_namespace parameter to the endpoint class-method, instead of the current :version.

I'll have a closer look at it tomorrow.

It's now possible to specify the envelope namespace explicitly in endpoint. Use as:

class MyService < Handsoap::Service
  endpoint :uri => 'blah', :envelope_namespace => 'lorem', :request_content_type => 'application/soap+xml'
end

Or:

class MyService < Handsoap::Service
  endpoint :uri => 'blah', :envelope_namespace => 'lorem', :version => 2
end

Update to 0.4.0 for this.

Hi. I didn't see your message before now. Can you mail me a copy of the service class and a dump of the http-activity? The issue tracker here at github messed the formatting up. My address is troelskn@gmail.com

Thank you troelskn. I am pretty sure this is a problem with the service I am trying to consume, or with my code. I sent the information you requested however, perhaps you will see something. Thanks very much!

We have resolved this issue over mail. The problem was related to the service returning multipart responses. Handsoap now supports that feature.

I have the same issue. I think its because Response.fault is using soap_namespace that I set in on_create_document. But a fault response has a generic nampespace.

I'm logging faults by overriding the on_fault method but i would like to actually get the code and reason captured in the log also.

Just saw this message now, so sorry for the late reply.

If you're using a 1.2 service, I think the problem may be related to issue#7.

More generally, a better solution could be to move the document-parsing into the service class. Then it would be possible to override the default parsing code with your own. That would also allow you to parse out more detailed information that your fault might contain. I'll have a stab at that.

OK. I have now moved the fault-parsing into the service class, which means that you can hook into it, by overriding parse_soap_fault. This also simplified the code a bit, so that I could remove Response from the code-base. Update to 0.4.0 to get this new functionality.

Thank you for the effort.
I'm trying right now and I'm struggling with some xpath problem so I can't tell exactly for sure if it's solving my issues, but it seems so.

I think that Nokogiri's documentation is very poor. I don't understand how to define namespaces to parse my returned xml.

Yeah, well actually you should blame Handsoap for that. The XML parsers are wrapped in a thin abstraction layer XmlQueryFront. To use a namespace in a query, you can do one of two things. Either you register the namespace globally for the document:

doc.add_namespace(prefix, uri)

You can then use prefix for doc.

Alternatively, you can pass the namespace in the call to xpath, like this:

doc.xpath('//foo:WhatEver', 'foo' => uri)

Since you usually use the same namespace all over a single service, it's convenient to register the prefixes globally in the on_response_document hook:

class MyService < Handsoap::Service
  def on_response_document(doc)
    doc.add_namespace(prefix, uri)
  end
end

Thanks for the tip. Yesterday I've ended using the second "manual" option, because I think the first didn't work for (I must have done something wrong).

Anyway, It's definitly better to be able to customize the fault handling. A little too much overriding for my taste, but still a huge improvement. Thanks again.

What's the output of uname -a?

After some investigation, I believe this issue to be fixed. For the curious, it was due to differences between gnu readlink and bsd readlink

Fixed in: http://github.com/troelskn/handsoap/commit/00e8da7cf277350090c40697995b93bc6fac9ba1

The xml_to_str et al methods are deprecated now. You should replace code like this:

xml_to_str(node, 'foo/bar')

With:

(node/'foo/bar').to_s

Or:

node.xpath('foo/bar').first.to_s

I am adding a patch for 0.3.2 to create backwards compatibility for these helpers, but you should get rid of them.

You can now use the :raw option to add raw xml. Eg.:

element.set_value "<[[CDATA[bla]]>", :raw