New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Counting guest views #40

Closed
dyzajash opened this Issue Sep 29, 2015 · 21 comments

Comments

Projects
None yet
2 participants
@dyzajash

dyzajash commented Sep 29, 2015

Hello,

I have one small problem with your app. It counts only views from authenticated users.

Link for my app: https://github.com/dyzajash/TB_cms/tree/develop
Python version: 2.7.5
Django: 1.8.4

show.html layout:

{% extends "shared/layout.html" %}
{% load staticfiles %}
{% load markdown_filter %}
{% load download_formatter %}
{% load hitcount_tags %}

{% block title %}{{ post.title }}{% endblock title %}

{% block content %}
    <div class="news"> <!-- news -->
        <h1>{{ post.title }}</h1>

        <div class="news-views">{% get_hit_count for post %}</div>
        <div class="news-author">{{ post.author }}</div>

        <div class="news-content">
            {% if post.manga %}
                <div class="news-img"><img src="{{ post.manga.news_image.url }}" alt="{{ post.title }}-image"></div>
            {% else %}
                {{ post.content|markdownify|safe }}
            {% endif %}
            <div class="chapters"><!-- chapters -->
                {{ post.download|blob_to_html|safe }}
            </div>
            <!-- chapters -->

            {% if user.is_authenticated %}
                <a href="{% url 'edit-news' post.slug %}" class="red-button">Edytuj</a>
            {% endif %}
            <a href="{% url 'home' %}" class="gold-button">Wróć</a>
        </div>
    </div> <!-- news -->
{% endblock %}
{% block extra_scripts %}
    <script src="{% static 'hitcount/hitcount-jquery.js' %}"></script>
    {% insert_hit_count_js_variables for post %}
{% endblock %}

settings (as base.py):

# Application definition

INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.sitemaps',
    'django.contrib.sites',
    'django_forms_bootstrap',
    'hitcount',
)

PROJECT_APPS = (
    'accounts',
    'api',
    'blog',
    'dashboard',
    'projects',
    'utils'
)

INSTALLED_APPS += PROJECT_APPS
SESSION_SAVE_EVERY_REQUEST = True

and nginx conf:

    location / {
        proxy_pass http://127.0.0.1:8001;
        proxy_set_header X-Forwarded-Host $server_name;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_set_header X-Real-IP $remote_addr;
        add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
    }
@thornomad

This comment has been minimized.

Owner

thornomad commented Sep 29, 2015

Hmm. What is the output from your JavaScript console? If you are using the example JavaScript it should output the logic behind counting or ignoring a hit.

@dyzajash

This comment has been minimized.

dyzajash commented Sep 29, 2015

Try out yoursef: http://dev.team-black.pl/
Click on 'Czytaj więcej' (Read more)

JS output when i'm logged in: "Hit counted: user authentication"

JS output when I'm view site as guest:
nothing.

Btw. I added $ajax.setup with csrf_middleware_token because ajax POST got 403 errors.

@thornomad

This comment has been minimized.

Owner

thornomad commented Sep 29, 2015

What does your urls.py look like? Looks like authentication is required to
retrieve the ajax URL ... That is: the JavaScript request is never getting
to the view ... Can look on a computer later but you need to resolve that
403 ...
On Tue, Sep 29, 2015 at 12:10 PM Mateusz notifications@github.com wrote:

Try out yoursef: http://dev.team-black.pl/
Click on 'Czytaj więcej' (Read more)

JS output:
when i'm logged in: "Hit counted: user authentication"

JS output when I'm view site as guest:
POST http://dev.team-black.pl/hitcount/hit/ajax/ 403 (FORBIDDEN)


Reply to this email directly or view it on GitHub
#40 (comment)
.

@dyzajash

This comment has been minimized.

dyzajash commented Sep 29, 2015

Urls.py:

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    # REST api
    url(r'^api/v1/', include('tb_cms.apps.api.urls', namespace='v1')),
    # login / logout helper
    url(r'^accounts/', include('tb_cms.apps.accounts.urls', namespace='accounts')),
    # sitemap
    url(r'^sitemap\.xml$', sitemap, {'sitemaps': sitemaps}, name='django.contrib.sitemaps.views.sitemap'),
    # project apps
    url(r'^$', 'tb_cms.apps.blog.views.home', name='home'),
    url(r'^crew/$', 'tb_cms.apps.blog.views.crew', name='crew'),
    url(r'^faq/$', 'tb_cms.apps.blog.views.faq', name='faq'),
    url(r'^contact/$', 'tb_cms.apps.blog.views.contact', name='contact'),
    url(r'^recruitment/$', 'tb_cms.apps.blog.views.recruitment', name='recruitment'),
    url(r'^blog/', include('tb_cms.apps.blog.urls')),
    url(r'^projects/', include('tb_cms.apps.projects.urls')),
    url(r'^dashboard/', include('tb_cms.apps.dashboard.urls')),
    # hitcount
    url(r'hitcount/', include('hitcount.urls', namespace='hitcount')),
] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)  # dev only

I resolved 403 by adding on layout, but still no console.log as guest:

<script>
  $.ajaxSetup({data: {
    csrfmiddlewaretoken: '{{ csrf_token }}'
  }});
</script>
@thornomad

This comment has been minimized.

Owner

thornomad commented Sep 29, 2015

Interesting. I wonder why the method in the script file is unable to get
the token from the cookie in your case. On my test systems it works fine.

https://github.com/thornomad/django-hitcount/blob/master/hitcount/static/hitcount/hitcount-jquery.js

I'm not sure I can replicate that error.
On Tue, Sep 29, 2015 at 12:58 PM Mateusz notifications@github.com wrote:

Urls.py:

urlpatterns = [
url(r'^admin/', include(admin.site.urls)),
# REST api
url(r'^api/v1/', include('tb_cms.apps.api.urls', namespace='v1')),
# login / logout helper
url(r'^accounts/', include('tb_cms.apps.accounts.urls', namespace='accounts')),
# sitemap
url(r'^sitemap.xml$', sitemap, {'sitemaps': sitemaps}, name='django.contrib.sitemaps.views.sitemap'),
# project apps
url(r'^$', 'tb_cms.apps.blog.views.home', name='home'),
url(r'^crew/$', 'tb_cms.apps.blog.views.crew', name='crew'),
url(r'^faq/$', 'tb_cms.apps.blog.views.faq', name='faq'),
url(r'^contact/$', 'tb_cms.apps.blog.views.contact', name='contact'),
url(r'^recruitment/$', 'tb_cms.apps.blog.views.recruitment', name='recruitment'),
url(r'^blog/', include('tb_cms.apps.blog.urls')),
url(r'^projects/', include('tb_cms.apps.projects.urls')),
url(r'^dashboard/', include('tb_cms.apps.dashboard.urls')),
# hitcount
url(r'hitcount/', include('hitcount.urls', namespace='hitcount')),
] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT) # dev only

I resolved 403 by adding:

<script> $.ajaxSetup({data: { csrfmiddlewaretoken: '{{ csrf_token }}' }}); </script>


Reply to this email directly or view it on GitHub
#40 (comment)
.

@dyzajash

This comment has been minimized.

dyzajash commented Sep 29, 2015

I found that Gunicorn+Nginx needs this 2 extra headers:

proxy_set_header Host $http_host;
proxy_pass_header X-CSRF-TOKEN;

And everything in your script work fine without defining ajax.setup.

But still I got no output as guest in console.log.

Btw. when i run django in dev mode and come as guest I get error 500:

2015-09-30 01:14:02,347  [django.request:256]  ERROR - Internal Server Error: /hitcount/hit/ajax/
Traceback (most recent call last):
  File "/opt/tb_cms/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/tb_cms/lib/python2.7/site-packages/django/utils/decorators.py", line 145, in inner
    return func(*args, **kwargs)
  File "/opt/tb_cms/lib/python2.7/site-packages/hitcount/views.py", line 124, in update_hit_count_ajax
    response = _update_hit_count(request, hitcount)
  File "/opt/tb_cms/lib/python2.7/site-packages/hitcount/views.py", line 85, in _update_hit_count
    hit.save()
  File "/opt/tb_cms/lib/python2.7/site-packages/hitcount/models.py", line 147, in save
    super(Hit, self).save(*args, **kwargs)
  File "/opt/tb_cms/lib/python2.7/site-packages/django/db/models/base.py", line 734, in save
    force_update=force_update, update_fields=update_fields)
  File "/opt/tb_cms/lib/python2.7/site-packages/django/db/models/base.py", line 762, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/opt/tb_cms/lib/python2.7/site-packages/django/db/models/base.py", line 846, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/opt/tb_cms/lib/python2.7/site-packages/django/db/models/base.py", line 885, in _do_insert
    using=using, raw=raw)
  File "/opt/tb_cms/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/tb_cms/lib/python2.7/site-packages/django/db/models/query.py", line 920, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/opt/tb_cms/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 974, in execute_sql
    cursor.execute(sql, params)
  File "/opt/tb_cms/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/opt/tb_cms/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/opt/tb_cms/lib/python2.7/site-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/opt/tb_cms/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/opt/tb_cms/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 318, in execute
    return Database.Cursor.execute(self, query, params)
IntegrityError: hitcount_hit.session may not be NULL
@thornomad

This comment has been minimized.

Owner

thornomad commented Sep 30, 2015

Interesting - it would seem that on your dev mode it is not finding a session_key which is needed to save a Hit(session=session_key):

session_key = request.session.session_key

Have you tried cloning the project and just running the example app? I'm not sure exactly what's different about your setup that is causing the issue but would love to find out!

@dyzajash

This comment has been minimized.

dyzajash commented Sep 30, 2015

Finally I found solution!

Your example app just works, so I have run pip freeze and found 2 extra deps: requests and pytz.
After installing this two deps, everything starts working.

But I found another issue:
This two middlewares:

    'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
    'django.middleware.security.SecurityMiddleware',

blocks CSRF-XToken in WebBrowser (Chromium / Firefox) incognito-mode and script got POST 403 error.

Same problem is described here: http://stackoverflow.com/questions/18557196/tastypie-csrftoken-not-set-in-incognito-mode

@thornomad

This comment has been minimized.

Owner

thornomad commented Oct 1, 2015

You know I never tested it "Incognito" -- but I will! Did you try manually setting the token ? Does that circumvent the incognito issue? It may be easier, rather than relying on cookies, to pass the token at the template level. Perhaps safer as well.

@thornomad

This comment has been minimized.

Owner

thornomad commented Oct 1, 2015

Hi again - so I tried adding both of these middleware to my example project (since they are not there by default).

MIDDLEWARE_CLASSES = (
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'django.middleware.security.SecurityMiddleware',
)

But I do not receive any 403 errors and the Hits are being counted as expected. I haven't been able to reproduce this ...

@dyzajash

This comment has been minimized.

dyzajash commented Oct 1, 2015

Try upgrade Django to 1.8.4 ;). It looks like bug inside Django's 1.8.4 django.contrib.sessions.middleware.SessionMiddleware. After downgrading to 1.8.2 everything just works fine.

@thornomad

This comment has been minimized.

Owner

thornomad commented Oct 1, 2015

Yea, interesting -- it works in 1.8.2 and 1.8.3 but not 1.8.4 ... I wonder if they know this is a bug or is that as intended?

It looks like you don't even need the non-standard middleware to cause this problem either:

MIDDLEWARE_CLASSES = (
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    # 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    # 'django.middleware.security.SecurityMiddleware'
)
@dyzajash

This comment has been minimized.

dyzajash commented Oct 1, 2015

Someone added ticket for broken sessions: https://code.djangoproject.com/ticket/25458

@thornomad

This comment has been minimized.

Owner

thornomad commented Oct 1, 2015

Yea, the issue is that suddenly request.session.session_key is returning None in Icognito mode ... that's in 1.8.4 .... in <1.8.4 it returns a key.

@thornomad

This comment has been minimized.

Owner

thornomad commented Oct 1, 2015

I don't think it just icognito ... try clearing your cookies for that site with django 1.8.4 and then see if you get that error in a regular browser. I think the SESSION_SAVE_EVERY_REQUEST = True is not working ...

@thornomad

This comment has been minimized.

Owner

thornomad commented Oct 1, 2015

@thornomad

This comment has been minimized.

Owner

thornomad commented Oct 2, 2015

I pushed a change to the develop branch -- can you test it and see if it fixes your problem? I am just forcing the save of a session to get the session_key.

pip install -e git://github.com/thornomad/django-hitcount@develop#egg=django-hitcount

Let me know if this fixes it for you and I will upgrade the pip repository.

@dyzajash

This comment has been minimized.

dyzajash commented Oct 2, 2015

So in 1.8.4 it's not a bug, it's a feature 😸

1.1.1 works with guests on normal mode. In incognito still 403.

@thornomad

This comment has been minimized.

Owner

thornomad commented Oct 2, 2015

Yea - smile. A feature!

v1.1.1 should fix the 500 error -- I haven't seen a 403 error yet. I would guess a 403 error is a problem with the CSRF token. Are you testing this on the example project in development mode? Or on your own server/environment?

Does the 403 error only appear in 1.8.4 as well? The session_key issue should not throw a 403 ... it would be 500 (which is what I saw).

Can we start with the example project in dev mode to try and narrow down what is causing it?

$ git clone git@github.com:thornomad/django-hitcount.git@develop
$ cd django-hitcount/example_project
$ pip install -r requirements.txt   # sqlite requires pytz
$ python manage.py migrate          # will load some data fixtures for you
$ python manage.py runserver        # should be all set!
@dyzajash

This comment has been minimized.

dyzajash commented Oct 2, 2015

@thornomad

This comment has been minimized.

Owner

thornomad commented Oct 2, 2015

Okay. I am not able to reproduce this yet on my end. I am using ./manage.py runserver in the example_project folder. Then opening Chrome/Icognito and browsing to one of the "Post" pages. This results in a valid POST request and the hist is counted. I don't see any 403 errors on my end. I've tried this in Firefox/Private as well. Django 1.8.4.

October 02, 2015 - 14:46:22
Django version 1.8.4, using settings 'example_project.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
[02/Oct/2015 14:46:55] "GET /1/ HTTP/1.1" 200 3670
[02/Oct/2015 14:46:55] "POST /hitcount/hit/ajax/ HTTP/1.1" 200 50

I'm not sure what is different about our setups ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment