keyerror in samba config restore #1585

Closed
phillxnet opened this Issue Dec 14, 2016 · 3 comments

Comments

Projects
None yet
3 participants
@phillxnet
Member

phillxnet commented Dec 14, 2016

Under certain circumstance, when restoring a previously saved config backup, the samba service config fails to get restored: a key error is logged:

ERROR [storageadmin.util:47] exception: Samba could not be configured. Try again. Exception: 'global_config'
Traceback (most recent call last): File "/opt/rockstor/src/rockstor/smart_manager/views/samba_service.py", line 61, in post
    gc_lines = config['global_config'].split('\n')
KeyError: 'global_config'
DEBUG [storageadmin.util:48] Current Rockstor version: 3.8.16-0
ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/smb/config. payload: {'config': {u'workgroup': u'MYGROUP2'}}. 
exception: 500 Server Error: INTERNAL SERVER ERROR. Moving on.

Note in the above log that the only config element was that of the workgroup. This is suspected as the trigger to reproduce the issue.

@MFlyer

This comment has been minimized.

Show comment
Hide comment
@MFlyer

MFlyer Dec 14, 2016

Member

Hi @phillxnet, probably this issue starts when saving config: samba view had lots of mods not reproduced in config backup func. Will check it

Member

MFlyer commented Dec 14, 2016

Hi @phillxnet, probably this issue starts when saving config: samba view had lots of mods not reproduced in config backup func. Will check it

@MFlyer

This comment has been minimized.

Show comment
Hide comment
@MFlyer

MFlyer Dec 15, 2016

Member

Having a complete restore (never had it before this) I got a bounch of errors :/

[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:242] restore starting...
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:244] restore submitted...
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:55] Started restoring users and groups.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/groups. payload: {u'admin': True, u'groupname': u'rockstor', u'gid': 11201}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/groups. payload: {u'admin': False, u'groupname': u'SIVAG\\domain admins', u'gid': 10512}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/users. payload: {u'username': u'rockstor', u'public_key': None, u'shell': u'/bin/bash', u'group': 1, u'uid': 11201, u'admin': True, u'homedir': u'/home/rockstor', u'gid': 11201, u'user': 1, 'password': 'rockstor', u'email': None, u'smb_shares': []}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/users. payload: {u'username': u'SIVAG\\administrator', u'public_key': None, u'shell': u'/bin/sh', u'group': 2, u'uid': 10500, u'admin': False, u'homedir': None, u'gid': 10512, u'user': None, 'password': 'rockstor', u'email': None, u'smb_shares': []}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:71] Finished restoring users and groups.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:75] Started restoring Samba exports.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/samba. payload: {u'comment': u'', u'read_only': u'no', u'browsable': u'yes', u'snapshot_prefix': u'Data5', u'share': 3, 'shares': [u'storage'], u'shadow_copy': True, u'guest_ok': u'yes', u'path': u'/mnt2/storage'}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/samba. payload: {u'comment': u'', u'read_only': u'no', u'browsable': u'yes', u'snapshot_prefix': None, u'share': 19, 'shares': [u'Test4'], u'shadow_copy': False, u'guest_ok': u'no', u'path': u'/mnt2/Test4'}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:83] Finished restoring Samba exports.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:102] Started restoring NFS exports.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/adv-nfs-exports. payload: {'entries': []}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:122] Finished restoring NFS exports.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:87] Started restoring AFP exports.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:98] Finished restoring AFP exports.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:126] Started restoring services.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:135] services = {u'active-directory': {'config': {u'username': u'Administrator', u'domain': u'sivag.local', u'workgroup': u'SIVAG', u'rfc2307': False, u'password': u'omittedbyFlyer', u'idmap_range': u'10000 - 999999'}}, u'rockstor': {'config': {u'network_interface': u'eth0', u'listener_port': u'443'}}, u'nut': {'config': {u'shutdowntimer': u'0', u'upsname': u'ups', u'driver': u'apcsmart-old', u'nutserver': u'localhost', u'mode': u'standalone', u'upsmon': u'master', u'desc': u'Rockstor U', u'password': u'asa', u'port': u'auto', u'nutuser': u'asa'}}, u'replication': {'config': {u'network_interface': u'eth0', u'listener_port': u'10002'}}, u'smartd': {'config': {u'custom_config': u''}}, u'shellinaboxd': {'config': {u'detach': False, u'css': u'white-on-black', u'shelltype': u'LOGIN'}}, u'docker': {'config': {u'root_share': u'storage'}}, u'smb': {'config': {u'workgroup': u'SIVAG'}}, u'ntpd': {'config': {u'server': u'ntp1.inrim.it'}}}
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/active-directory/config. payload: {'config': {u'username': u'Administrator', u'domain': u'sivag.local', u'workgroup': u'SIVAG', u'rfc2307': False, u'password': u'omissis', u'idmap_range': u'10000 - 999999'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/rockstor/config. payload: {'config': {u'network_interface': u'eth0', u'listener_port': u'443'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/nut/config. payload: {'config': {u'shutdowntimer': u'0', u'upsname': u'ups', u'driver': u'apcsmart-old', u'nutserver': u'localhost', u'mode': u'standalone', u'upsmon': u'master', u'desc': u'Rockstor U', u'password': u'asa', u'port': u'auto', u'nutuser': u'asa'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/replication/config. payload: {'config': {u'network_interface': u'eth0', u'listener_port': u'10002'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/smartd/config. payload: {'config': {u'custom_config': u''}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/shellinaboxd/config. payload: {'config': {u'detach': False, u'css': u'white-on-black', u'shelltype': u'LOGIN'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/docker/config. payload: {'config': {u'root_share': u'storage'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/smb/config. payload: {'config': {u'workgroup': u'SIVAG'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/ntpd/config. payload: {'config': {u'server': u'ntp1.inrim.it'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:138] Finished restoring services.
Member

MFlyer commented Dec 15, 2016

Having a complete restore (never had it before this) I got a bounch of errors :/

[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:242] restore starting...
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:244] restore submitted...
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:55] Started restoring users and groups.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/groups. payload: {u'admin': True, u'groupname': u'rockstor', u'gid': 11201}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/groups. payload: {u'admin': False, u'groupname': u'SIVAG\\domain admins', u'gid': 10512}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/users. payload: {u'username': u'rockstor', u'public_key': None, u'shell': u'/bin/bash', u'group': 1, u'uid': 11201, u'admin': True, u'homedir': u'/home/rockstor', u'gid': 11201, u'user': 1, 'password': 'rockstor', u'email': None, u'smb_shares': []}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/users. payload: {u'username': u'SIVAG\\administrator', u'public_key': None, u'shell': u'/bin/sh', u'group': 2, u'uid': 10500, u'admin': False, u'homedir': None, u'gid': 10512, u'user': None, 'password': 'rockstor', u'email': None, u'smb_shares': []}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:71] Finished restoring users and groups.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:75] Started restoring Samba exports.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/samba. payload: {u'comment': u'', u'read_only': u'no', u'browsable': u'yes', u'snapshot_prefix': u'Data5', u'share': 3, 'shares': [u'storage'], u'shadow_copy': True, u'guest_ok': u'yes', u'path': u'/mnt2/storage'}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/samba. payload: {u'comment': u'', u'read_only': u'no', u'browsable': u'yes', u'snapshot_prefix': None, u'share': 19, 'shares': [u'Test4'], u'shadow_copy': False, u'guest_ok': u'no', u'path': u'/mnt2/Test4'}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:83] Finished restoring Samba exports.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:102] Started restoring NFS exports.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/adv-nfs-exports. payload: {'entries': []}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:122] Finished restoring NFS exports.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:87] Started restoring AFP exports.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:98] Finished restoring AFP exports.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:126] Started restoring services.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:135] services = {u'active-directory': {'config': {u'username': u'Administrator', u'domain': u'sivag.local', u'workgroup': u'SIVAG', u'rfc2307': False, u'password': u'omittedbyFlyer', u'idmap_range': u'10000 - 999999'}}, u'rockstor': {'config': {u'network_interface': u'eth0', u'listener_port': u'443'}}, u'nut': {'config': {u'shutdowntimer': u'0', u'upsname': u'ups', u'driver': u'apcsmart-old', u'nutserver': u'localhost', u'mode': u'standalone', u'upsmon': u'master', u'desc': u'Rockstor U', u'password': u'asa', u'port': u'auto', u'nutuser': u'asa'}}, u'replication': {'config': {u'network_interface': u'eth0', u'listener_port': u'10002'}}, u'smartd': {'config': {u'custom_config': u''}}, u'shellinaboxd': {'config': {u'detach': False, u'css': u'white-on-black', u'shelltype': u'LOGIN'}}, u'docker': {'config': {u'root_share': u'storage'}}, u'smb': {'config': {u'workgroup': u'SIVAG'}}, u'ntpd': {'config': {u'server': u'ntp1.inrim.it'}}}
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/active-directory/config. payload: {'config': {u'username': u'Administrator', u'domain': u'sivag.local', u'workgroup': u'SIVAG', u'rfc2307': False, u'password': u'omissis', u'idmap_range': u'10000 - 999999'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/rockstor/config. payload: {'config': {u'network_interface': u'eth0', u'listener_port': u'443'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/nut/config. payload: {'config': {u'shutdowntimer': u'0', u'upsname': u'ups', u'driver': u'apcsmart-old', u'nutserver': u'localhost', u'mode': u'standalone', u'upsmon': u'master', u'desc': u'Rockstor U', u'password': u'asa', u'port': u'auto', u'nutuser': u'asa'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/replication/config. payload: {'config': {u'network_interface': u'eth0', u'listener_port': u'10002'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/smartd/config. payload: {'config': {u'custom_config': u''}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/shellinaboxd/config. payload: {'config': {u'detach': False, u'css': u'white-on-black', u'shelltype': u'LOGIN'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/docker/config. payload: {'config': {u'root_share': u'storage'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/smb/config. payload: {'config': {u'workgroup': u'SIVAG'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] ERROR [storageadmin.views.config_backup:51] Exception occured while creating resource: https://localhost/api/sm/services/ntpd/config. payload: {'config': {u'server': u'ntp1.inrim.it'}}. exception: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /o/token/ (Caused by <class 'socket.error'>: [Errno 111] Connection refused). Moving on.
[15/Dec/2016 11:24:05] DEBUG [storageadmin.views.config_backup:138] Finished restoring services.

@schakrava schakrava added the bug label Dec 17, 2016

@schakrava schakrava added this to the Pinnacles milestone Dec 17, 2016

@MFlyer

This comment has been minimized.

Show comment
Hide comment
@MFlyer

MFlyer Feb 1, 2017

Member

Got it, both samba and other exceptions.
To @phillxnet , samba error: samba_service.py looks for a global_config item (our textarea with samba params = samba values), but on configuration restore from backup global_config doesn't exists and we have only config field from db with all samba params like param : val

Probably having PR directly over #1576 fork

Member

MFlyer commented Feb 1, 2017

Got it, both samba and other exceptions.
To @phillxnet , samba error: samba_service.py looks for a global_config item (our textarea with samba params = samba values), but on configuration restore from backup global_config doesn't exists and we have only config field from db with all samba params like param : val

Probably having PR directly over #1576 fork

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment