New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get_roster via XML-RPC no longer works #845

Closed
encodeltd opened this Issue Nov 24, 2015 · 14 comments

Comments

Projects
None yet
4 participants
@encodeltd

encodeltd commented Nov 24, 2015

get_roster command no longer works via XML-RPC in Ejabberd 15.10

The following change introduced the issue:

cf975da#diff-0ba032512fdf1ca7756850d6e3a2ab55L365

The issue can be easily replicated by running test suite of pyejabberd (https://github.com/dirkmoors/pyejabberd) against 15.10.

Running get_roster via ejabberdctl using changed attribute (server instead of host) works fine, though.

Error thrown is:

The call provided additional unused arguments: [{user,<<"testuser_15">>},{host,<<"example.com">>}]

@mremond

This comment has been minimized.

Member

mremond commented Nov 24, 2015

Did you try using the admin parameter to act as an admin ?

Reference is: http://docs.ejabberd.im/admin/guide/oauth/

@mremond mremond added this to the ejabberd 15.12 milestone Nov 24, 2015

@encodeltd

This comment has been minimized.

encodeltd commented Nov 24, 2015

Thank you. Going to try that.

I assume the change relates to http://docs.ejabberd.im/admin/guide/managing/#restrict-execution-with-accesscommands , specifically in the change in default behaviour (ie. "Before ejabberd 15.09 the default value was to not define any restriction: []")

I also assume that 15.09 introduced backwards incompatible change when the host attribute got renamed to server which will need to be somehow handled in pyejabberd, too.

Is there a list of all backwards incompatible changes so I could refactor all affected parts of pyejabberd?

@encodeltd

This comment has been minimized.

encodeltd commented Nov 24, 2015

I did a bit more testing.

Firstly, here are relevant XML-RPC parts from ejabberd.yml of Ejabberd 15.10 server where I run the tests:

listen:
  - 
    module: ejabberd_xmlrpc
    ip: "127.0.0.1"
    port: 4560
    access_commands:
      xmlrpc_access:
        commands: all
        options: {}

access:
  xmlrpc_access:
    xmlrpc_users: allow

acl:
  xmlrpc_users:
    user:
      - "xmlrpc": "example.com"

Secondly, I use extauth for authentication and I am not using OAuth at all.

According to "Admin only" section in "List of commands available with OAuth support" (http://docs.ejabberd.im/admin/guide/oauth/) as admin I can run commands such as connected_users_number and get_roster.

pyejabberd implements (among others) connected_users_number and get_roster and my testing script looks like this:

from pyejabberd import EjabberdAPIClient


# invoke XML-RPC client
client = EjabberdAPIClient(host='127.0.0.1',
                           port=4560,
                           username='xmlrpc',
                           password='super strong password for testing xmlrpc',
                           user_domain='example.com',
                           protocol='http')
# test echo
print client.echo('ping pong')
# test connected users number
print client.connected_users_number()
# test get_roster
print client.get_roster(user='test', host='example.com')

Test result:

$ python test_xmlrpc.py 
ping pong
0
Traceback (most recent call last):
  File "test_xmlrpc.py", line 18, in <module>
    print client.get_roster(user='test', host='example.com')
  File "/usr/local/lib/python2.7/dist-packages/pyejabberd/client.py", line 396, in get_roster
    return self._call_api(definitions.GetRoster, user=user, host=host)
  File "/usr/local/lib/python2.7/dist-packages/pyejabberd/client.py", line 471, in _call_api
    response = method(self.auth, arguments)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1233, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1591, in __request
    verbose=self.__verbose
  File "/usr/lib/python2.7/xmlrpclib.py", line 1273, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1306, in single_request
    return self.parse_response(response)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1482, in parse_response
    return u.close()
  File "/usr/lib/python2.7/xmlrpclib.py", line 794, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault -120: 'Error -120\nThe call provided additional unused arguments:\n[{host,<<"example.com">>},{user,<<"test">>}]'>

As you can see, connected_users_number returned correct value (zero connected users), but get_roster failed.

I have also tried refactored version where instead of host, I pass in server attribute, resulting in the same failure.

I also tested add_rosteritem and delete_rosteritem commands and those work fine as well.

Lastly, admin parameter is not being passed in for any of the commands (connected_users_number, add_rosteritem, delete_rosteritem, get_roster) and apart from get_roster they all work fine.

Thoughts?

@badlop

This comment has been minimized.

Member

badlop commented Nov 24, 2015

I can reproduce the bug. It affects to commands that are defined with policy=user.

Until we find a proper fix, you can bypass the problematic code with this change:

--- a/src/ejabberd_xmlrpc.erl
+++ b/src/ejabberd_xmlrpc.erl
@@ -262,7 +262,7 @@ get_auth(AuthList) ->
     Admin =
         case lists:keysearch(admin, 1, AuthList) of
             {value, {admin, true}} -> true;
-            _ -> false
+            _ -> true
         end,
     try get_attrs([user, server, token], AuthList) of
         [U, S, T] -> {U, S, {oauth, T}, Admin}
@encodeltd

This comment has been minimized.

encodeltd commented Nov 25, 2015

Thank you. Good to know it wasn't my PEBKAC.

I applied the workaround to bypass the issue.

The ejabberd.yml setup and the testing script are the same per my previous comment.

Now, when I run the testing script I am getting:

<Fault -118: "Error -118\nA problem '{error,account_unprivileged}' occurred executing the command connected_users_number with arguments\n[]">

Is that expected behaviour ie. do I need to change anything in ejabberd.yml, or have I stumbled upon something else?

Additionally, I only tried to run the test script with get_roster test only and I am getting:

<Fault -120: 'Error -120\nThe call provided additional unused arguments:\n[{user,<<"test">>},{server,<<"example.com">>}]'>

But, ejabberdctl help get_roster says that user and server are valid args for get_roster:

ejabberdctl help get_roster

  Command Name: get_roster

  Arguments: user::binary
             server::binary
@badlop

This comment has been minimized.

Member

badlop commented Nov 25, 2015

Fault -118: "Error -118\nA problem '{error,account_unprivileged}' occurred executing the command connected_users_number with arguments\n[]"

Then either disable access restrictions with this:

  -
    port: 4560
    module: ejabberd_xmlrpc
    access_commands: {}

or provide auth credentials in the xmlrpc call. In your previous example, it seemed you provide it. This is my example Python script, in case you want to give another try:

import xmlrpclib

server_url = 'http://127.0.0.1:4560'
server = xmlrpclib.ServerProxy(server_url)

EJABBERD_XMLRPC_LOGIN = {'user': 'admin', 'server': 'localhost', 'password': 'mypass1234'}

def calling(command, data):
    fn = getattr(server, command)
    return fn(EJABBERD_XMLRPC_LOGIN, data)

print ""
print "Calling with auth details:"
result = calling('get_roster', {'user':'user1', 'server':'localhost'})
print result

Fault -120: 'Error -120\nThe call provided additional unused arguments:\n[{user,<<"test">>},{server,<<"example.com">>}]'

This appears to me when I remove the workaround, and gets fixed when I apply it. Are you sure you applied the workaround, compiled, copied the resulting ejabberd_xmlrpc.beam to overwrite the old one and restarted ejabberd?

badlop added a commit to processone/docs.ejabberd.im that referenced this issue Nov 26, 2015

@encodeltd

This comment has been minimized.

encodeltd commented Dec 15, 2015

My apologies for not following up earlier.

Regarding whether I applied the workaround. I did. I provision ejabberd via Ansible to assure the install is always correct ie. the installation is done from scratch.

I also did check ejabberd help which gives a different output when the workaround is in place so I can confirm the workaround was in place during my test.

You mentioned that a proper fix will be required (ie. not the workaround). I am happy to wait until that is that case and then test the final solution for you whenever it's ready.

I will try again to test the workaround with 15.11 (previously I was testing against 15.10 and I have since updated my provisioning playbooks to 15.11).

I will test with your Python script and also will do a test via pyejabberd to make sure the issue after the workaround is applied is not in pyejabberd.

I will run the tests and will report back here by Fri Dec 18 (NZ time).

@badlop

This comment has been minimized.

Member

badlop commented Dec 16, 2015

You mentioned that a proper fix will be required

Right. But later, after more looking, no patch or workaround in the code is required. The only requirement is to add the argument

'admin': True

to the auth credentials in the XML-RPC client call. See the Python example in
http://docs.ejabberd.im/admin/guide/oauth/#xml-rpc-examples

@badlop badlop closed this Dec 17, 2015

@zyfmix

This comment has been minimized.

zyfmix commented Jan 14, 2016

error still with adding 'admin': True param,my python code below:

import xmlrpclib
server_url = 'http://42.96.194.60:4560'
server = xmlrpclib.ServerProxy(server_url, verbose=True)

EJABBERD_XMLRPC_LOGIN = {'user': 'zyf', 'server': 'coopens.com',
                         'password': '123456', 'admin': True}

def ejabberdctl(command, data):
    fn = getattr(server, command)
    return fn(EJABBERD_XMLRPC_LOGIN, data)

# result = ejabberdctl('register', {'user':'ggeo224', 'host':'coopens.com', 'password':'gogo11'})
# result = ejabberdctl('check_account', {'user': 'zyfer', 'host': 'coopens.com'})
result = ejabberdctl('get_roster', {'user': 'zlh', 'server': 'coopens.com'})
print result

What's wrong, How can I solve this problem?(ejabberd15.11 with ubuntu14.04)

@badlop

This comment has been minimized.

Member

badlop commented Jan 15, 2016

Your script works for me if I configure my ejabberd as shown in this comment: #845 (comment)

@zyfmix

This comment has been minimized.

zyfmix commented Jan 16, 2016

Hi @badlop , Thank you for your reply,I have change my ejabberd.yml below:

listen:
  -
    port: 4560
    ip: "127.0.0.1"
    module: ejabberd_xmlrpc
    maxsessions: 10
    timeout: 5000
    access_commands:
      xmlrpc_access:
        commands: all
        options: {}
acl:
  xmlrpc_users:  #+++#
    user:
      - "zyf": "coopens.com"
access:
  xmlrpc_access:
    xmlrpc_users: allow

and test with my python code above,but it crashed error:

xmlrpclib.Fault: <Fault -120: 'Error -120\nThe call provided additional unused arguments:\n[{user,<<"zlh">>},{server,<<"coopens.com">>}]'>

The ejabberd show debug info:

12:03:44.899 [info] (#Port<0.15039>) Accepted connection 127.0.0.1:37894 -> 127.0.0.1:4560
12:03:44.899 [debug] S: [{[],ejabberd_xmlrpc}]
12:03:44.899 [info] started: {gen_tcp,#Port<0.15039>}
12:03:44.899 [debug] (#Port<0.15039>) http query: 'POST' <<"/RPC2">>
12:03:44.899 [debug] client data: <<"<?xml version='1.0'?>\n<methodCall>\n<methodName>get_roster</methodName>\n<params>\n<param>\n<value><struct>\n<member>\n<name>admin</name>\n<value><boolean>1</boolean></value>\n</member>\n<member>\n<name>password</name>\n<value><string>123456</string></value>\n</member>\n<member>\n<name>user</name>\n<value><string>zyf</string></value>\n</member>\n<member>\n<name>server</name>\n<value><string>coopens.com</string></value>\n</member>\n</struct></value>\n</param>\n<param>\n<value><struct>\n<member>\n<name>user</name>\n<value><string>zlh</string></value>\n</member>\n<member>\n<name>server</name>\n<value><string>coopens.com</string></value>\n</member>\n</struct></value>\n</param>\n</params>\n</methodCall>\n">>
12:03:44.899 [debug] [<<"RPC2">>] matches []
12:03:44.900 [debug] got XML-RPC request: {call,get_roster,[{struct,[{admin,true},{password,<<"123456">>},{user,<<"zyf">>},{server,<<"coopens.com">>}]},{struct,[{user,<<"zlh">>},{server,<<"coopens.com">>}]}]}
12:03:44.900 [warning] Error -120
The call provided additional unused arguments:
[{user,<<"zlh">>},{server,<<"coopens.com">>}]

and i lookup the mod_admin_extra.erl#L369 get_roaster command don't defined the param user and server.
can you help me how to figure out what's couse the "additional unused arguments*" error?

@badlop

This comment has been minimized.

Member

badlop commented Jan 18, 2016

user and server params are not explicitly defined in mod_admin_extral.erl, because policy=user, and this way the arguments are defined internally in ejabberd_commands.erl So, that's normal.

I tried your config with my call, and it works if I set the proper admin user. I only get that error if the admin account does not exist, or if the password is wrong. So, check the account zyf@coopens.com exist, and that the password is 123456.

@zyfmix

This comment has been minimized.

zyfmix commented Jan 19, 2016

After follow issues ejabberd/issues/771, I add 'commands_admin_access: xmlrpc_access' in my ejabberd.yml and it work perfect.
@badlop Thank you very much for help me to solve this question ^_^.

encodeltd pushed a commit to encodeltd/pyejabberd that referenced this issue Jan 24, 2016

Marek Kuziel
added check_account command implementation; fixed get_roster args (ch…
…anged in ejabberd 15.09); fixed auth in EjabberdAPIClient - from 15.09 onwards, {admin,true} must be present - see processone/ejabberd#845 for further information
@encodeltd

This comment has been minimized.

encodeltd commented Jan 25, 2016

Following up after doing more testing and updating pyejabberd implementation.

I wrote up some docs on how to setup Ejabberd XMLRPC with explanation of changes in 15.09.

See here: https://github.com/encodeltd/pyejabberd/blob/6078e6647ee38ff3b6f85bb3481ec828b21ff25b/README.rst#ejabberd-xmlrpc-setup

I guess this is also related to #771

In case you'd find anything useful in that write up I am happy to contribute it to the Ejabberd docs, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment