Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'Settings' object has no attribute 'CUMULUS' #161

Open
dasginganinja opened this issue Nov 25, 2013 · 18 comments
Open

'Settings' object has no attribute 'CUMULUS' #161

dasginganinja opened this issue Nov 25, 2013 · 18 comments

Comments

@dasginganinja
Copy link

Hello all. I've done some searching on this issue and haven't been able to turn up too much so I figured I would finally ask in here. I've been trying to get File Conveyor working with Rackspace's Cloud Files but have been having horrible luck doing so. I installed originally with the instructions from INSTALL.txt and when starting up an instance with a fully configured config.xml file I got these errors while using "transport = 'cloudfiles'".

/usr/local/src/fileconveyor/fileconveyor# python arbitrator.py
/usr/local/src/fileconveyor/fileconveyor/filter.py:10: DeprecationWarning: the sets module is deprecated
  from sets import Set, ImmutableSet
2013-11-25 17:03:39,878 - Arbitrator                - WARNING  - File Conveyor is initializing.
2013-11-25 17:03:39,888 - Arbitrator                - INFO     - Loading config file.
2013-11-25 17:03:39,890 - Arbitrator.Config         - INFO     - Parsing sources.
2013-11-25 17:03:39,890 - Arbitrator.Config         - INFO     - Parsing servers.
2013-11-25 17:03:39,894 - Arbitrator.Config         - INFO     - Parsing rules.
2013-11-25 17:03:39,894 - Arbitrator                - WARNING  - Loaded config file.
Traceback (most recent call last):
  File "arbitrator.py", line 1185, in <module>
    run_file_conveyor()
  File "arbitrator.py", line 1168, in run_file_conveyor
    arbitrator = Arbitrator(os.path.join(FILE_CONVEYOR_PATH, "config.xml"), restart)
  File "arbitrator.py", line 142, in __init__
    transporter_class = self._import_transporter(transporter_name)
  File "arbitrator.py", line 1148, in _import_transporter
    module = __import__(module_name, globals(), locals(), ["TRANSPORTER_CLASS"], -1)
  File "/usr/local/src/fileconveyor/fileconveyor/transporters/transporter_cloudfiles.py", line 2, in <module>
    from cumulus.storage import CloudFilesStorage
  File "/usr/local/lib/python2.6/dist-packages/cumulus/storage.py", line 11, in <module>
    from cumulus.settings import CUMULUS
  File "/usr/local/lib/python2.6/dist-packages/cumulus/settings.py", line 33, in <module>
    if "FILTER_LIST" in settings.CUMULUS.keys():
  File "/usr/local/lib/python2.6/dist-packages/django/conf/__init__.py", line 55, in __getattr__
    return getattr(self._wrapped, name)
AttributeError: 'Settings' object has no attribute 'CUMULUS'

My Python 2.6 dist packages looks like so:

/usr/local/src/fileconveyor/fileconveyor# ls /usr/local/lib/python2.6/dist-packages/ -1
Babel-1.3-py2.6.egg-info
Django-1.6-py2.6.egg-info
argparse-1.2.1-py2.6.egg-info
argparse.py
argparse.pyc
babel
boto
boto-1.6b-py2.6.egg-info
cloudfiles
cssutils
cssutils-0.9.10-py2.6.egg-info
cumulus
distribute-0.6.49-py2.6.egg
django
django_cumulus-1.0.11-py2.6.egg-info
easy-install.pth
encutils
fileconveyor-0.3_dev-py2.6.egg
fileconveyor.egg-link
httplib2
httplib2-0.8-py2.6.egg-info
iso8601
iso8601-0.1.8-py2.6.egg-info
keyring
keyring-3.2.1-py2.6.egg-info
mock-1.0.1-py2.6.egg-info
mock.py
mock.pyc
novaclient
os_diskconfig_python_novaclient_ext
os_diskconfig_python_novaclient_ext-0.1.1-py2.6.egg-info
os_networksv2_python_novaclient_ext-0.21-py2.6.egg-info
os_networksv2_python_novaclient_ext.py
os_networksv2_python_novaclient_ext.pyc
os_virtual_interfacesv2_python_novaclient_ext-0.14-py2.6.egg-info
os_virtual_interfacesv2_python_novaclient_ext.py
os_virtual_interfacesv2_python_novaclient_ext.pyc
pbr
pbr-0.5.23-py2.6.egg-info
pip
pip-1.4.1-py2.6.egg-info
prettytable-0.7.2-py2.6.egg-info
prettytable.py
prettytable.pyc
pyinotify-0.9.4-py2.6.egg-info
pyinotify.py
pyinotify.pyc
pyrax
pyrax-1.4.10-py2.6.egg-info
python_cloudfiles-1.7.10-py2.6.egg-info
python_novaclient-2.15.0-py2.6.egg-info
python_swiftclient-1.8.0-py2.6.egg-info
pytz
pytz-2013.8-py2.6.egg-info
rackspace_auth_openstack
rackspace_auth_openstack-1.1-py2.6.egg-info
rackspace_novaclient-1.4-py2.6.egg-info
rax_default_network_flags_python_novaclient_ext
rax_default_network_flags_python_novaclient_ext-0.1.3-py2.6.egg-info
rax_scheduled_images_python_novaclient_ext
rax_scheduled_images_python_novaclient_ext-0.2.1-py2.6.egg-info
requests
requests-2.0.1-py2.6.egg-info
setuptools-0.6c11-py2.6.egg-info
setuptools.pth
simplejson
simplejson-3.3.1-py2.6.egg-info
six-1.4.1-py2.6.egg-info
six.py
six.pyc
swiftclient

Can somebody point me in the right direction?

Thanks!

@dasginganinja
Copy link
Author

Here is what my config.xml looks like:

/usr/local/src/fileconveyor/fileconveyor# cat config.xml
<?xml version="1.0" encoding="UTF-8"?>
<config>
  <!-- Sources -->
  <sources ignoredDirs="">
    <source name="test" scanPath="/var/www" documentRoot="/var/www" basePath="/" />
  </sources>

  <!-- Servers -->
  <servers>
    <server name="rackspace" transporter="cloudfiles">
      <username>###</username>
      <api_key>###</api_key>
      <container>###</container>
    </server>
  </servers>
 <!-- Rules -->
  <rules>
    <rule for="test" label="Media or images only">
              <filter>
                <paths>sites/all:sites/default/files</paths>
                <extensions>js:ico:gif:png:jpg:jpeg:svg:swf:css</extensions>
              </filter>
              <processorChain>
              </processorChain>
              <destinations>
                <destination server="rackspace" path="" />
              </destinations>
            </rule>
  </rules>
</config>

@dasginganinja
Copy link
Author

I've got a little bit more of an update for you.

I brought up a brand new Vagrant Debian Squeeze box, installed git, pip, and File Conveyor per INSTALL.txt. After configuring config.xml for cloudfiles I am still getting the following error after a

root@vagrant-debian-squeeze:~# python src/fileconveyor/fileconveyor/arbitrator.py
/root/src/fileconveyor/fileconveyor/filter.py:10: DeprecationWarning: the sets module is deprecated
  from sets import Set, ImmutableSet
2013-11-26 08:31:08,075 - Arbitrator                - WARNING  - File Conveyor is initializing.
2013-11-26 08:31:08,081 - Arbitrator                - WARNING  - Loaded config file.
Traceback (most recent call last):
  File "src/fileconveyor/fileconveyor/arbitrator.py", line 1185, in <module>
    run_file_conveyor()
  File "src/fileconveyor/fileconveyor/arbitrator.py", line 1168, in run_file_conveyor
    arbitrator = Arbitrator(os.path.join(FILE_CONVEYOR_PATH, "config.xml"), restart)
  File "src/fileconveyor/fileconveyor/arbitrator.py", line 142, in __init__
    transporter_class = self._import_transporter(transporter_name)
  File "src/fileconveyor/fileconveyor/arbitrator.py", line 1148, in _import_transporter
    module = __import__(module_name, globals(), locals(), ["TRANSPORTER_CLASS"], -1)
  File "/root/src/fileconveyor/fileconveyor/transporters/transporter_cloudfiles.py", line 2, in <module>
    from cumulus.storage import CloudFilesStorage
  File "/usr/local/lib/python2.6/dist-packages/cumulus/storage.py", line 11, in <module>
    from cumulus.settings import CUMULUS
  File "/usr/local/lib/python2.6/dist-packages/cumulus/settings.py", line 33, in <module>
    if "FILTER_LIST" in settings.CUMULUS.keys():
  File "/usr/local/lib/python2.6/dist-packages/django/conf/__init__.py", line 55, in __getattr__
    return getattr(self._wrapped, name)
AttributeError: 'Settings' object has no attribute 'CUMULUS'

Here's the output from pip:

root@vagrant-debian-squeeze:~# pip freeze
Babel==1.3
Django==1.6
argparse==1.2.1
boto==1.6b
cssutils==0.9.10
distribute==0.6.49
django-cumulus==1.0.11
-e git+https://github.com/wimleers/fileconveyor@ca3585edbdcc75973c35bd8ee4d52601153bf6a3#egg=fileconveyor-master
httplib2==0.8
iso8601==0.1.8
keyring==3.2.1
mock==1.0.1
os-diskconfig-python-novaclient-ext==0.1.1
os-networksv2-python-novaclient-ext==0.21
os-virtual-interfacesv2-python-novaclient-ext==0.14
pbr==0.5.23
prettytable==0.7.2
pyinotify==0.9.4
pyrax==1.4.10
python-apt==0.7.100.1-squeeze1
python-cloudfiles==1.7.10
python-novaclient==2.15.0
python-swiftclient==1.8.0
pytz==2013.8
rackspace-auth-openstack==1.1
rackspace-novaclient==1.4
rax-default-network-flags-python-novaclient-ext==0.1.3
rax-scheduled-images-python-novaclient-ext==0.2.1
reportbug==4.12.6
requests==2.0.1
simplejson==3.3.1
six==1.4.1
wsgiref==0.1.2

@omnia-ibrahim
Copy link

I have the same errors, any help?

@dasginganinja
Copy link
Author

None here. I'm still waiting on Wim Leers to respond. I'm assuming that this is because one of the packages is at a non-working version.

Can somebody who has a working copy of fileconveyor please put up a pip listing of the currently installed modules? We might be able to determine which module is the cause of the current problem that we are all experiencing with getting this module working.

Again, this problem is only for the select few of us that will be utilizing cloudfiles. Signs point to the cumulus module. I think I'm going to start from there as I need to have this fixed by the end of the day.

@omnia-ibrahim
Copy link

What other alternatives if this is not fixed? i want to use cloud files

@chrisivens
Copy link
Contributor

When I was last using fileconveyor, it was at the time when the cloudfiles stuff was just a pull request so it's a while ago. Having said that, I did get it running and unfortunately I haven't got access to that machine any more. On looking at the docs for django-cumulus though, it does mention the CUMULUS object.

CUMULUS = {
    'USERNAME': 'YourUsername',
    'API_KEY': 'YourAPIKey',
    'CONTAINER': 'ContainerName'
}

Now I suspect that it'll need adding to the django-settings.py file but this is just a hunch and I can no longer test with any Rackspace stuff without setting up a new account.

@omnia-ibrahim
Copy link

I tried adding the CUMULUS object in both the django-settings in fileconveyor and also in the django settings file itself and both didn't work :(

@chrisivens
Copy link
Contributor

My old fork of fileconveyor has this in the django_settings.py file. I suspect I've missed out the api key etc:

SECRET_KEY='change me for something better'
# Dummy settings for `django-storages`.
MEDIA_URL=''
MEDIA_ROOT=''
# `backends/ftp.py`
FTP_STORAGE_LOCATION=''
# `backends/sftp.py`
SFTP_STORAGE_HOST=''
# django-cumulus
#CUMULUS['USERNAME'] = '';
#CUMULUS['CUMULUS_API_KEY'] = '';
#CUMULUS['CONTAINER'] = '';
CUMULUS_API_KEY = '';
CUMULUS_USERNAME = '';
CUMULUS_CONTAINER = '';
CUMULUS = {
  'AUTH_URL' : 'uk_authurl'
}

It doesn't look like my settings.py file differs much. You can look over my fork if you wish but I warn you, it's deliberately broken to prevent local deletions propagating to Cloudfiles. Oh, and it's quite a bit behind.

@omnia-ibrahim
Copy link

its almost the same as mine, i have a feeling its not related to the settings in fileconveyor, but something with django and its settings, but i have no idea why its not working

@dasginganinja
Copy link
Author

Good morning all! I don't have any updates on my end other than the fact that I've tried multiple versions of cumulus, django, etc. and still can't get it working. I created a gist (https://gist.github.com/dasginganinja/7769677) with some outputs that might be helpful in diagnosing. In the gist I give the setup done to the box to get it up and running, the strace output for starting the arbitrator, and a pip freeze to show what python modules are actually being used.

Hopefully this helps somebody diagnose further! Here's a quick shortcut to the strace output: https://gist.github.com/dasginganinja/7769677/raw/696b112bf25a967ecab63cf08330ac4f6569425b/Strace+Output.txt

@dasginganinja
Copy link
Author

Okay here's another little update. I went ahead and put the CUMULUS object into /usr/lib/local/python2.6/dist-packages/django/conf/global_settings.py with my correct settings. Here is the output when I run now:

tom@dev2-lamp:/tmp$ sudo vim /usr/local/lib/python2.6/dist-packages/django/conf/global_settings.py
tom@dev2-lamp:/tmp$ python /usr/local/lib/python2.6/dist-packages/fileconveyor-0.3_dev-py2.6.egg/fileconveyor/arbitrator.py
/usr/local/lib/python2.6/dist-packages/fileconveyor-0.3_dev-py2.6.egg/fileconveyor/filter.py:10: DeprecationWarning: the sets module is deprecated
  from sets import Set, ImmutableSet
2013-12-03 10:22:22,742 - Arbitrator                - WARNING  - File Conveyor is initializing.
2013-12-03 10:22:22,748 - Arbitrator                - WARNING  - Loaded config file.
Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/fileconveyor-0.3_dev-py2.6.egg/fileconveyor/arbitrator.py", line 1185, in <module>
    run_file_conveyor()
  File "/usr/local/lib/python2.6/dist-packages/fileconveyor-0.3_dev-py2.6.egg/fileconveyor/arbitrator.py", line 1168, in run_file_conveyor
    arbitrator = Arbitrator(os.path.join(FILE_CONVEYOR_PATH, "config.xml"), restart)
  File "/usr/local/lib/python2.6/dist-packages/fileconveyor-0.3_dev-py2.6.egg/fileconveyor/arbitrator.py", line 142, in __init__
    transporter_class = self._import_transporter(transporter_name)
  File "/usr/local/lib/python2.6/dist-packages/fileconveyor-0.3_dev-py2.6.egg/fileconveyor/arbitrator.py", line 1162, in _import_transporter
    self.logger.error("The Transporter module '%s' was found, but its Transporter class '%s' could not be found."  % (module_name, classname))
UnboundLocalError: local variable 'classname' referenced before assignment

As you can see, at this point this error is now what is being shown in #162.

@omnia-ibrahim
Copy link

i think he is asking for the cloud files storage, i added it to settings DEFAULT_FILE_STORAGE = 'cumulus.storage.CloudFilesStorage', but also didn't work

@dasginganinja
Copy link
Author

At this point all I think we need to do is find out why 'classname' isn't getting set. I'm sure that will open a whole other can of worms but that's the next step.

@omnia-ibrahim
Copy link

you reached anything about this class issue? i couldn't solve it :s

@dasginganinja
Copy link
Author

I went back through and took a look at the PIP Install log for this program and noticed the following lines:


  Running setup.py install for python-cloudfiles
    cloudfiles/__init__.py:90: DeprecationWarning: python-cloudfiles has been deprecated as of August 1, 2013. Please see https://github.com/openstack/python-swiftclient.
      DeprecationWarning)
    /tmp/pip_build_root/python-cloudfiles/setup.py:11: DeprecationWarning: python-cloudfiles has been deprecated as of August 1, 2013. Please see https://github.com/openstack/python-swiftclient.
      DeprecationWarning)
    warning: no previously-included files found matching 'python_cloudfiles.egg-info'

I wonder if this has something to do with it....

@ihuro
Copy link

ihuro commented Dec 14, 2013

Update django-cumulus to 1.0.10: easy_install "django-cumuls==1.0.10" or pip install django-cumulus==1.0.10

In setup.py the dependencies must be >=1.0.5 and <=1.0.10 for now

@omnia-ibrahim
Copy link

The error is now gone thanksss :)

But i have another issue, when saving images i got this error: Filter queue: dropped 'files/blue-search-icon_2.png' because it doesn't match any rules

@stifboy
Copy link

stifboy commented Mar 28, 2014

I have put the cumulus object into django/conf/global_settings.py with my settings but now I am getting this error

Last login: Fri Mar 28 22:28:25 2014 from 196.201.228.134 [root@af-server ~]# sudo python src/fileconveyor/fileconveyor/arbitrator.py /root/src/fileconveyor/fileconveyor/filter.py:10: DeprecationWarning: the sets m odule is deprecated from sets import Set, ImmutableSet 2014-03-28 22:34:38,206 - Arbitrator - WARNING - File Conveyor i s initializing. 2014-03-28 22:34:38,210 - Arbitrator - WARNING - Loaded config f ile. Traceback (most recent call last): File "src/fileconveyor/fileconveyor/arbitrator.py", line 1185, in run_file_conveyor() File "src/fileconveyor/fileconveyor/arbitrator.py", line 1168, in run_file_con veyor arbitrator = Arbitrator(os.path.join(FILE_CONVEYOR_PATH, "config.xml"), rest art) File "src/fileconveyor/fileconveyor/arbitrator.py", line 142, in **init** transporter_class = self._import_transporter(transporter_name) File "src/fileconveyor/fileconveyor/arbitrator.py", line 1162, in _import_tran sporter self.logger.error("The Transporter module '%s' was found, but its Transporte r class '%s' could not be found." % (module_name, classname)) UnboundLocalError: local variable 'classname' referenced before assignment [root@af-server ~]#

I have already updated django-cumulus to 1.0.10 and still get the error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants