Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade path and tags database #32

Closed
nickbabcock opened this Issue Apr 3, 2018 · 6 comments

Comments

Projects
None yet
2 participants
@nickbabcock
Copy link

nickbabcock commented Apr 3, 2018

Coming from 1.0.2-2 and upgrading to 1.1.2-1, I see several errors related to tags:

curl -s "http://graphite/tags?pretty=1"

{
  "error": "no such table: tags_tag"
}
curl -X POST "http://graphite/tags/tagSeries" \
   --data-urlencode 'path=disk.used;rack=a1;datacenter=dc1;server=web01'
{"error": "no such table: tags_series"}

These are the tables I see in graphite.db. Since I did no configuring on my end I'm assuming this includes the local tags db.

account_mygraph             dashboard_dashboard_owners
account_profile             dashboard_template
account_variable            dashboard_template_owners
account_view                django_admin_log
account_window              django_content_type
auth_group                  django_migrations
auth_group_permissions      django_session
auth_permission             events_event
auth_user                   tagging_tag
auth_user_groups            tagging_taggeditem
auth_user_user_permissions  url_shortener_link
dashboard_dashboard

Is there a migration guide / am I missing something?

@deniszh

This comment has been minimized.

Copy link
Member

deniszh commented Apr 4, 2018

@nickbabcock
I think you need to use separate database (MySQL, PostgreSQL) for any serious use of tags, I doubt that SQLite will able to handle an even minimal load.
But yes, database schema migration is needed, and there's no migration guide, despite official for Graphite 1.1.x - https://graphite.readthedocs.io/en/latest/releases/1_1_1.html#upgrading
You need to run something like

cp /opt/graphite/storage/graphite.db \
      /opt/graphite/storage/graphite.db.backup-`date +%Y%m%d_%H%M%S`
PYTHONPATH=/opt/graphite/webapp django-admin.py migrate \
      --noinput --settings=graphite.settings --run-syncdb

in the container for that.

@deniszh deniszh added the question label Apr 4, 2018

@nickbabcock

This comment has been minimized.

Copy link
Author

nickbabcock commented Apr 5, 2018

Interestingly that command returns:

Operations to perform:
  Synchronize unmigrated apps: browser, composer, functions, metrics, render, staticfiles, whitelist
  Apply all migrations: account, admin, auth, contenttypes, dashboard, events, sessions, tagging, tags, url_shortener
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
Running migrations:
  Applying account.0001_initial...Traceback (most recent call last):
  File "/usr/local/bin/django-admin.py", line 5, in <module>
    management.execute_from_command_line()
// [Snip]
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py", line 326, in execute
    return Database.Cursor.execute(self, query)
django.db.utils.OperationalError: table "account_mygraph" already exists

Since I only use graphite-web as a mean as interfacing with carbon with functions for grafana, a quick scan of graphite.db didn't reveal anything I'd care too much about if I dropped sqlite and made a fresh start with postgres.

@deniszh

This comment has been minimized.

Copy link
Member

deniszh commented Apr 5, 2018

That's a good move, indeed - postgresql should run much better.

@nickbabcock

This comment has been minimized.

Copy link
Author

nickbabcock commented Apr 5, 2018

I did have a couple questions. I noticed that in order to get the initial migrations to run for postgres (and I'm assuming any non-sqlite db) I had to force /usr/local/bin/manage.sh to run on every startup, which I believe should be ok as the migrations appear idempotent. Plus on future graphite db updates, I should be able to automatically migrate to them without exec-ing into the container. For whatever reason, 01_conf_init.sh didn't kick off the migration process.

FROM graphiteapp/graphite-statsd:1.1.3

COPY carbon.conf storage-schemas.conf /opt/graphite/conf/

# Copy over settings for SECRET_KEY and DATABASES
COPY local_settings.py /opt/graphite/webapp/graphite/

# Graphite does not come bundled with a python postgres driver so we manually
# pip install psycopg2. We pin to 2.7.4 for reproducibility.
RUN pip install psycopg2-binary==2.7.4

CMD /usr/local/bin/manage.sh && /sbin/my_init

And my compose file if necessary:

version: '3'
services:
  graphite:
    container_name: 'graphite'
    build: .
    restart: 'unless-stopped'
    ports:
      - '8181:80'
      - '2003:2003'
    links:
      - db
    # Snipped volumes
  db:
    container_name: 'graphite-db'
    image: 'postgres:10.3-alpine'
    restart: 'unless-stopped'
    # Snipped volumes
    environment:
      - POSTGRES_PASSWORD=graphitedbpassword

Do you see anything amiss?

@deniszh

This comment has been minimized.

Copy link
Member

deniszh commented Apr 5, 2018

LGTM

@nickbabcock

This comment has been minimized.

Copy link
Author

nickbabcock commented Apr 5, 2018

Excellent, works like a charm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.