Centralized tokenization #454

Closed
wants to merge 55 commits into
from

Conversation

Projects
None yet
6 participants

crodjer commented Oct 20, 2012

Adding centralized tokenization to django.
Merge work done under my GSoC 2012 regarding centralized tokenization system in django.

Merged from yarko/django

crodjer and others added some commits May 18, 2012

Add myself to authors
Signed-off-by: Rohan Jain <crodjer@gmail.com>
Add Token class.
Conflicts:
	django/utils/crypto.py

Fix repeated imports due to conflicts

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Actually use length parameter
Signed-off-by: Rohan Jain <crodjer@gmail.com>
By default, return whole digest.
Signed-off-by: Rohan Jain <crodjer@gmail.com>
added update method to token, and dog strings for base digests
Conflicts:
	django/utils/crypto.py

Indentation fixes.
Restored lost default length=None as function parameters

Signed-off-by: Rohan Jain <crodjer@gmail.com>
modify init
Signed-off-by: Rohan Jain <crodjer@gmail.com>
Make random not the default
Signed-off-by: Rohan Jain <crodjer@gmail.com>
updates for new crypto
Conflicts:
	django/middleware/csrf.py

Neglecting the changes for csrf, because they are irrelevant in the
new context.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
added test subset to use prior to our security commits; might not be …
…complete, but should hit what we touch

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Fix bugs introduced with the merge
Missing import in file cache backend.
Buggy import of random functionality.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Added usage of django.utils.crypto's Token class
Conflicts:
	django/utils/cache.py

Three cache tests fail. None of my merges should have affected them.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Default value in token should be string
md5 raises an exception in case it is provided None as the argument.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Remove token system usage from part of cache util
Otherwise some tests fail. Will add token system and handle the
failures later, when tokenization is stable.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
changed Token naming to be consistent w/ rest of code base;
Conflicts:
	django/middleware/csrf.py
	django/utils/crypto.py

Failures were present in cache too, fixed them.
CSRF still doesn't use the token class.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
re-structuring crypto.py - not complete, but we need to share what's …
…here now...

Conflicts:
	django/utils/crypto.py

Broken build. Fixed class naming.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Adding random token
Conflicts:
	django/utils/crypto.py

Broken build, s/Token/BaseToken/g

Signed-off-by: Rohan Jain <crodjer@gmail.com>
re-structuring crypto.py - updated HashToken; ready to merge w/ Ryans
Conflicts:
	django/utils/crypto.py

Broken build. _build_token missing.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Updated random token
Conflicts:
	django/utils/crypto.py

Broken. Missing _build_token

Signed-off-by: Rohan Jain <crodjer@gmail.com>
removed length options from HashToken, per suggestion from Paul
Signed-off-by: Rohan Jain <crodjer@gmail.com>
removed base62 function (we are using baseconv.py utilities instead)
Conflicts:
	django/utils/crypto.py

Broken. Missing _build_token

Signed-off-by: Rohan Jain <crodjer@gmail.com>
added README.sec for this working group;
Signed-off-by: Rohan Jain <crodjer@gmail.com>
Split tokens into django.utils.tokens and reverted crypto.py
Conflicts:
	django/utils/cache.py
	django/utils/crypto.py

Cache doesn't use token system at most places.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
now uses django.utils.tokens instead of django.utils.crypto
Signed-off-by: Rohan Jain <crodjer@gmail.com>
beginning to migrate files to new django/utils/token (WIP)
Conflicts:
	django/contrib/sessions/backends/base.py
	django/contrib/staticfiles/storage.py
	django/core/cache/backends/filebased.py
	django/middleware/csrf.py
	django/templatetags/cache.py
	django/utils/cache.py
	tests/regressiontests/cache/tests.py

Failing tests, can't import django.test.client

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Fix wrong module name in utils.tokens import
Tests pass now

Signed-off-by: Rohan Jain <crodjer@gmail.com>
continuing to migrate files to new django/utils/token (WIP)
Conflicts:
	django/contrib/formtools/utils.py
	django/db/backends/util.py
	tests/regressiontests/file_uploads/tests.py
	tests/regressiontests/file_uploads/views.py

Fixes: Token import module name

Signed-off-by: Rohan Jain <crodjer@gmail.com>
fixed conversion typo; updated test script; all hashlib changes, sele…
…ct md5, but not all sh1; tests now failing (next step) (WIP)

Conflicts:
	django/middleware/csrf.py

Still not using tokens in CSRF

Signed-off-by: Rohan Jain <crodjer@gmail.com>
fixed: cache tests now pass (removed hardcoded constants for hash com…
…pare values)

Conflicts:
	tests/regressiontests/cache/tests.py

Still tests fail. Fixing in next.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Cache uses HashToken. Tests pass.
Changes to `cache.CacheUtils.test_learn_cache_key` might need
attention.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
all tests pass, w/ cache on.
Conflicts:
	tests/regressiontests/staticfiles_tests/tests.py
	tests/test_sqlite.py

One test failing,
staticfiles_test.TestCollectionCachedStorage.test_path_with_querystring

Signed-off-by: Rohan Jain <crodjer@gmail.com>
moved contrib.auth.tokens functionality into utils.tokens; all tests …
…pass

Conflicts:
	django/contrib/auth/forms.py
	django/utils/tokens.py

Signed-off-by: Rohan Jain <crodjer@gmail.com>
adding postgres tests; now using memcached
Signed-off-by: Rohan Jain <crodjer@gmail.com>
now using memcached
Signed-off-by: Rohan Jain <crodjer@gmail.com>
added pip requirements file for tests
Signed-off-by: Rohan Jain <crodjer@gmail.com>
bug: missing import in tokens.py
Signed-off-by: Rohan Jain <crodjer@gmail.com>
catch OverflowError, since it's also valid to get from sqlite
Signed-off-by: Rohan Jain <crodjer@gmail.com>
session keys need to fit w/in 40 chars, so adapt sha256 output
Signed-off-by: Rohan Jain <crodjer@gmail.com>
fixed typo in new short_alphanumeric.
Signed-off-by: Rohan Jain <crodjer@gmail.com>
corrected to use RandomToken where appropriate (still needs testing o…
…n jython)

Conflicts:
	django/middleware/csrf.py

CSRF really uses tokens system now

Signed-off-by: Rohan Jain <crodjer@gmail.com>

yarko and others added some commits Sep 18, 2011

removed unnecessary imports
Signed-off-by: Rohan Jain <crodjer@gmail.com>
Don't install dev django every time
Using django.pth to automatically use dev django all the time in the
environment

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Cache Test: Custom key generator encoding
Otherwise, this fails with a unicode decode error for memcache key

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Remove trailing whitespaces
Signed-off-by: Rohan Jain <crodjer@gmail.com>
Introduced RandomToken in contrib.auth
API of HashToken doesn't seem like usable for password hasher yet.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
RandomToken: Length is an optional init param
Signed-off-by: Rohan Jain <crodjer@gmail.com>
Move contrib.auth.tokens to utils.tokens
Signed-off-by: Rohan Jain <crodjer@gmail.com>
Configurable algorithm in HashTokens
 - Allow configuring the algorithm being used while initializing
   HashToken.
 - Move remaining code base to HashToken instead of hashlib

Crypto can't import from HashToken.

One staticfiles test is failing:
staticfiles_tests.TestCollectionCachedStorage.test_path_with_querystring

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Remove tokenization from staticfiles
Tests passed

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Migrate static files back to HashToken
Tests pass

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Custom character set support in random token
Signed-off-by: Rohan Jain <crodjer@gmail.com>
Prevent coed repetition, a base token class.
`RandomToken` and `HashToken` now implement over `Token` class, which
provides the basic token fetching methods like hex, alphanumeric,
digits, lower_alphanumeric, readable_alphabet and custom characters.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Support all algorithms that hashlib supports
Signed-off-by: Rohan Jain <crodjer@gmail.com>
Introduce random token in more modules
Signed-off-by: Rohan Jain <crodjer@gmail.com>
Add documentation of centralized tokenization
Documents the API and basic examples for usage.

Signed-off-by: Rohan Jain <crodjer@gmail.com>
Owner

apollo13 commented Oct 24, 2012

@PaulMcMillan any input on this ticket?

Owner

timgraham commented Aug 13, 2013

Hi Rohan, do you think there's any point in leaving this pull request open at this point? It seems like it still needs quite a bit of work to get it into a mergeable state.

crodjer commented Aug 14, 2013

@timgraham True. I guess if it is ever needed to be done, these changes can be used as a reference. Shall I close it?

Owner

timgraham commented Aug 14, 2013

Sounds good, thanks for the reply.

@timgraham timgraham closed this Aug 14, 2013

nanuxbe pushed a commit to nanuxbe/django that referenced this pull request Jul 2, 2016

Merge pull request #454 from MarkusH/test-fixup
Removed stale import and sorted imports
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment