Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/testing' into stable
Browse files Browse the repository at this point in the history 
* origin/testing: (35 commits)
  Bump version.
  Make waitress listen on all interfaces in development.ini.  Resolves #371.
  Use unicode instead of strings in test (prevents SQLAlchemy warnings).
  Make scaffold use the name of project as the name of application.  Closes #457.
  Make kotti_tinymce a requirement in the scaffold.  Fixes #447.
  Add myself.
  Reorder children on append.
  Re-enable failing assert.  Fixed with ``reorder_on_append``.
  Revert "Remove assert that fails for no reason on Postgres."
  Upgrade Babel to 2.1.1.
  Upgrade Babel to 2.1.1.
  Add ``kotti.modification_date_excludes`` configuration option.
  Remove assert that fails for no reason on Postgres.
  Add changenotes.
  Add ``kotti.modification_date_excludes`` configuration option.
  Prevent some SQLAlchemy warnings that have been present since SQLAlchemy 1.0.
  Order conf_defaults and conf_dotted alphabetically.
  Upgrade requirements to their latest respective version.
  Added '--statistics' option to invocation of `msgfmt` to bring attention to possible fuzzy translations; see wichert/lingua#59 Additionally now we print file being currently compiled so that we know which statistic is for which language. Added '--no-wrap' option to invocation of `msgmerge` to avoid wrapping original msgids. Wrapping needlessly "mangles" original msgids. Please note that both options were lost during migration to lingua 3.6.1 in commit 285bad5
  Prevent some SQLAlchemy warnings that have been present since SQLAlchemy 1.0.
  ...
  • Loading branch information
@disko
disko committed Sep 27, 2015
2 parents 2fc61ee + 65b21fd commit f413c83
Show file tree
Hide file tree
Showing 26 changed files with 1,311 additions and 153 deletions.
25 changes: 25 additions & 0 deletions CHANGES.txt
View file
@@ -1,6 +1,31 @@
Change History
==============

1.2.0 - 2015-09-27
------------------

- **Greatly** reduce the number of queries that are sent to the DB:
- Add caching for the root node.
- Use eager / joined loading for local_groups.
- Don't query principals for roles
- Add "missing" foreign key indices (with corresponding migration step).
- Add a ``kotti.modification_date_excludes`` configuration option.
It takes a list of attributes in dotted name notation that should not trigger
an update of ``modification_date`` on change. Defaults to
``kotti.resources.Node.position``.
- Don't try to set a caching header from the NewRequest handler when Pyramid's
tweens didn't follow the usual chain of calls. This fixes compatibility with
``bowerstatic``.
- Don't assume ``renderer_name`` exists in a rendering event (ex.
BeforeRender). The official docstring of ``pyramid.interfaces.IRenderer`` is
a bit ambigous in regards to what the ``system`` parameter should include
when a renderer gets called. This fixes compatibility with
``pyramid_layout``.
- Add a ``kotti.modification_date_excludes`` configuration option.
It takes a list of attributes in dotted name notation that should not trigger
an update of ``modification_date`` on change. Defaults to
``kotti.resources.Node.position``.

1.1.5 - 2015-09-04
------------------

Expand Down
1 change: 1 addition & 0 deletions CONTRIBUTORS.txt
View file
Expand Up @@ -103,3 +103,4 @@ Contributors
- Ichim Tiberiu, 2015/01/13
- Martin Peeters, 2015/02/21
- Davide Moro, 2015/04/02
- Matt Russell, 2015/09/27
1 change: 0 additions & 1 deletion development.ini
View file
Expand Up @@ -28,7 +28,6 @@ pipeline =

[server:main]
use = egg:waitress#main
host = 127.0.0.1
port = 5000

[alembic]
Expand Down
102 changes: 52 additions & 50 deletions docs/developing/basic/configuration.rst
View file
Expand Up @@ -37,52 +37,54 @@ Overview of settings
This table provides an overview of available settings.
All these settings must go into the ``[app:kotti]`` section of your Paste Deploy configuration file.

============================ ==================================================
Setting Description
============================ ==================================================
**kotti.site_title** The title of your site
**kotti.secret** Secret token used for the initial admin password
kotti.secret2 Secret token used for email password reset token

**sqlalchemy.url** `SQLAlchemy database URL`_
**mail.default_sender** Sender address for outgoing email
mail.host Email host to send from

pyramid.includes List of Python configuration hooks
kotti.available_types List of active content types
kotti.base_includes List of base Python configuration hooks
kotti.zcml_includes List of packages to include the ZCML from
kotti.configurators List of advanced functions for config
kotti.request_factory Override Kotti's default request factory
kotti.root_factory Override Kotti's default Pyramid *root factory*
kotti.populators List of functions to fill initial database
kotti.search_content Override Kotti's default search function

kotti.asset_overrides Override Kotti's templates
kotti.templates.api Override ``api`` object available in templates
kotti.fanstatic.view_needed List of static resources used for public interface
kotti.fanstatic.edit_needed List of static resources used for edit interface

kotti.authn_policy_factory Component used for authentication
kotti.authz_policy_factory Component used for authorization
kotti.session_factory Component used for sessions

kotti.caching_policy_chooser Component for choosing the cache header policy
kotti.url_normalizer Component used for url normalization

kotti.date_format Date format to use, default: ``medium``
kotti.datetime_format Datetime format to use, default: ``medium``
kotti.time_format Time format to use, default: ``medium``
kotti.max_file_size Max size for file uploads, default: ```10`` (MB)

kotti.depot.*.* Configure the blob storage. More details below

kotti.sanitizers Configure available :ref:`sanitizers`.
kotti.sanitize_on_write Configure :ref:`sanitizers` to be used on write
access to resource objects.

pyramid.default_locale_name Set the user interface language, default ``en``
============================ ==================================================
================================ ==============================================
Setting Description
================================ ==============================================
**kotti.site_title** The title of your site
**kotti.secret** Secret token used for the initial admin
password
kotti.secret2 Secret token used for email password reset
token
**sqlalchemy.url** `SQLAlchemy database URL`_
**mail.default_sender** Sender address for outgoing email
mail.host Email host to send from
pyramid.includes List of Python configuration hooks
kotti.available_types List of active content types
kotti.base_includes List of base Python configuration hooks
kotti.zcml_includes List of packages to include the ZCML from
kotti.configurators List of advanced functions for config
kotti.request_factory Override Kotti's default request factory
kotti.root_factory Override Kotti's default Pyramid
*root factory*
kotti.populators List of functions to fill initial database
kotti.search_content Override Kotti's default search function
kotti.asset_overrides Override Kotti's templates
kotti.templates.api Override ``api`` object available in
templates
kotti.fanstatic.view_needed List of static resources used for public
interface
kotti.fanstatic.edit_needed List of static resources used for edit
interface
kotti.authn_policy_factory Component used for authentication
kotti.authz_policy_factory Component used for authorization
kotti.session_factory Component used for sessions
kotti.caching_policy_chooser Component for choosing the cache header policy
kotti.url_normalizer Component used for url normalization
kotti.date_format Date format to use, default: ``medium``
kotti.datetime_format Datetime format to use, default: ``medium``
kotti.time_format Time format to use, default: ``medium``
kotti.max_file_size Max size for file uploads,
default: ```10`` (MB)
kotti.modification_date_excludes List of attributes in dotted name notation
that should not trigger an update of
``modification_date`` on change.
kotti.depot.*.* Configure the blob storage. More details below
kotti.sanitizers Configure available :ref:`sanitizers`.
kotti.sanitize_on_write Configure :ref:`sanitizers` to be used on
write access to resource objects.
pyramid.default_locale_name Set the user interface language,
default ``en``
================================ ==============================================

Only the settings in bold letters required.
The rest has defaults.
Expand Down Expand Up @@ -317,26 +319,26 @@ Blob storage configuration
--------------------------

By default, Kotti will store blob data (files uploaded in File and Image instances) in the database.
Internally, Kotti integrates with :app:`filedepot`, so it is possible to use any :app:`filedepot` compatible storage, including those provided by :app:`filedepot` itself:
Internally, Kotti integrates with ``filedepot``, so it is possible to use any ``filedepot`` compatible storage, including those provided by ``filedepot`` itself:

- :class:`depot.io.local.LocalFileStorage`
- :class:`depot.io.awss3.S3Storage`
- :class:`depot.io.gridfs.GridFSStorage`

The default storage for :app:`Kotti` is :class:`~kotti.filedepot.DBFileStorage`.
The default storage for Kotti is :class:`~kotti.filedepot.DBFileStorage`.
The benefit of storing files in ``DBFileStorage`` is having *all* content in a single place (the DB) which makes backups, exporting and importing of your site's data easy, as long as you don't have too many or too large files.
The downsides of this approach appear when your database server resides on a different host (network performance becomes a greater issue) or your DB dumps become too large to be handled efficiently.

To configure a depot, several ``kotti.depot.*.*`` lines need to be added.
The number in the first position is used to group backend configuration and to order the file storages in the configuration of :app:`filedepot`.
The number in the first position is used to group backend configuration and to order the file storages in the configuration of ``filedepot``.
The depot configured with number 0 will be the default depot, where all new blob data will be saved.
There are 2 options that are required for every storage configuration: ``name`` and ``backend``.
The ``name`` is a unique string that will be used to identify the path of saved files (it is recorded with each blob info), so once configured for a particular storage, it should never change.
The ``backend`` should point to a dotted path for the storage class.
Then, any number of keyword arguments can be added, and they will be passed to the backend class on initialization.

Example of a possible configurationi that stores blob data on the disk, in
``/var/local/files`` using the :app:`filedepot` :class:`depot.io.local.LocalFileStorage` provided backend.
``/var/local/files`` using the ``filedepot`` :class:`depot.io.local.LocalFileStorage` provided backend.
Kotti's default backend, ``DBFileStorage`` has been moved to position **1** and all data stored there will continue to be available.
See :ref:`blobs` to see how to migrate blob data between storages.

Expand Down
4 changes: 3 additions & 1 deletion docs/developing/basic/security.rst
View file
Expand Up @@ -108,9 +108,10 @@ The default workflow is quite useful for websites, but sometimes you need someth
Just point the ``kotti.use_workflow`` setting to your zcml file:

.. code-block:: ini
kotti.use_workflow = kotti_yourplugin:workflow.zcml
The simplest way to deal with workflow definitions is::
The simplest way to deal with workflow definitions is:

1. create a copy of the default workflow definition and
2. customize it (change permissions, add new states, permissions, transitions, initial state and so on).
Expand Down Expand Up @@ -252,6 +253,7 @@ Here it is, our "custom" workflow definition assigned to our ``ICustomContent``
Last you have to tell Kotti to register your new custom workflow including our ``zcml`` file:
.. code-block:: ini
kotti.zcml_includes = kotti_wf:workflow.zcml
Special cases:
Expand Down
5 changes: 3 additions & 2 deletions i18n.sh
View file
Expand Up @@ -30,12 +30,13 @@ if [ $# -eq 0 ]; then

echo "Update translations"
for po in "$LOCALES_PATH"/*/LC_MESSAGES/$DOMAIN.po; do
msgmerge -o "$po" "$po" "$LOCALES_PATH"/$DOMAIN.pot
msgmerge --no-wrap -o "$po" "$po" "$LOCALES_PATH"/$DOMAIN.pot
done

echo "Compile message catalogs"
for po in "$LOCALES_PATH"/*/LC_MESSAGES/*.po; do
msgfmt -o "${po%.*}.mo" "$po"
echo "Compiling file $po..."
msgfmt --statistics -o "${po%.*}.mo" "$po"
done

# first argument represents language identifier, create catalog
Expand Down
84 changes: 44 additions & 40 deletions kotti/__init__.py
View file
Expand Up @@ -54,9 +54,15 @@ def none_factory(**kwargs): # pragma: no cover

# All of these can be set by passing them in the Paste Deploy settings:
conf_defaults = {
'kotti.templates.api': 'kotti.views.util.TemplateAPI',
'kotti.configurators': '',
'pyramid.includes': '',
'kotti.alembic_dirs': 'kotti:alembic',
'kotti.asset_overrides': '',
'kotti.authn_policy_factory': 'kotti.authtkt_factory',
'kotti.authz_policy_factory': 'kotti.acl_factory',
'kotti.available_types': ' '.join([
'kotti.resources.Document',
'kotti.resources.File',
'kotti.resources.Image',
]),
'kotti.base_includes': ' '.join([
'kotti',
'kotti.filedepot',
Expand All @@ -76,43 +82,26 @@ def none_factory(**kwargs): # pragma: no cover
'kotti.views.navigation',
'kotti.views.users',
]),
'kotti.zcml_includes': ' '.join([
]),
'kotti.asset_overrides': '',
'kotti.use_tables': '',
'kotti.request_factory': 'kotti.request.Request',
'kotti.root_factory': 'kotti.resources.default_get_root',
'kotti.populators': 'kotti.populate.populate',
'kotti.available_types': ' '.join([
'kotti.resources.Document',
'kotti.resources.File',
'kotti.resources.Image',
]),
'kotti.search_content': 'kotti.views.util.default_search_content',
'kotti.authn_policy_factory': 'kotti.authtkt_factory',
'kotti.authz_policy_factory': 'kotti.acl_factory',
'kotti.session_factory': 'kotti.beaker_session_factory',
'kotti.principals_factory': 'kotti.security.principals_factory',
'kotti.caching_policy_chooser': (
'kotti.views.cache.default_caching_policy_chooser'),
'kotti.url_normalizer': 'kotti.url_normalizer.url_normalizer',
'kotti.url_normalizer.map_non_ascii_characters': True,
'kotti.use_workflow': 'kotti:workflow.zcml',
'kotti.configurators': '',
'kotti.date_format': 'medium',
'kotti.datetime_format': 'medium',
'kotti.time_format': 'medium',
'kotti.max_file_size': '10',
'kotti.depot.0.name': 'dbfiles',
'kotti.depot.0.backend': 'kotti.filedepot.DBFileStorage',
'kotti.depot.0.name': 'dbfiles',
'kotti.fanstatic.edit_needed': 'kotti.fanstatic.edit_needed',
'kotti.fanstatic.view_needed': 'kotti.fanstatic.view_needed',
'kotti.static.edit_needed': '', # BBB
'kotti.static.view_needed': '', # BBB
'kotti.alembic_dirs': 'kotti:alembic',
'kotti.max_file_size': '10',
'kotti.modification_date_excludes': ' '.join([
'kotti.resources.Node.position',
]),
'kotti.populators': 'kotti.populate.populate',
'kotti.principals_factory': 'kotti.security.principals_factory',
'kotti.register': 'False',
'kotti.register.group': '',
'kotti.register.role': '',
'pyramid_deform.template_search_path': 'kotti:templates/deform',
'kotti.request_factory': 'kotti.request.Request',
'kotti.root_factory': 'kotti.resources.default_get_root',
'kotti.sanitizers': ' '.join([
'xss_protection:kotti.sanitizers.xss_protection',
'minimal_html:kotti.sanitizers.minimal_html',
Expand All @@ -123,24 +112,39 @@ def none_factory(**kwargs): # pragma: no cover
'kotti.resources.Content.title:no_html',
'kotti.resources.Content.description:no_html',
]),
'kotti.search_content': 'kotti.views.util.default_search_content',
'kotti.session_factory': 'kotti.beaker_session_factory',
'kotti.static.edit_needed': '', # BBB
'kotti.static.view_needed': '', # BBB
'kotti.templates.api': 'kotti.views.util.TemplateAPI',
'kotti.time_format': 'medium',
'kotti.url_normalizer': 'kotti.url_normalizer.url_normalizer',
'kotti.url_normalizer.map_non_ascii_characters': True,
'kotti.use_tables': '',
'kotti.use_workflow': 'kotti:workflow.zcml',
'kotti.zcml_includes': ' '.join([
]),
'pyramid.includes': '',
'pyramid_deform.template_search_path': 'kotti:templates/deform',
}

conf_dotted = set([
'kotti.templates.api',
'kotti.configurators',
'kotti.base_includes',
'kotti.request_factory',
'kotti.root_factory',
'kotti.populators',
'kotti.available_types',
'kotti.search_content',
'kotti.authn_policy_factory',
'kotti.authz_policy_factory',
'kotti.session_factory',
'kotti.principals_factory',
'kotti.available_types',
'kotti.base_includes',
'kotti.caching_policy_chooser',
'kotti.configurators',
'kotti.fanstatic.edit_needed',
'kotti.fanstatic.view_needed',
'kotti.modification_date_excludes',
'kotti.populators',
'kotti.principals_factory',
'kotti.request_factory',
'kotti.root_factory',
'kotti.search_content',
'kotti.session_factory',
'kotti.templates.api',
'kotti.url_normalizer',
])

Expand Down
32 changes: 32 additions & 0 deletions kotti/alembic/versions/4a3de0d0804a_add_foreignkey_indices.py
View file
@@ -0,0 +1,32 @@
"""Add ForeignKey indices
Revision ID: 4a3de0d0804a
Revises: 37a05f6246af
Create Date: 2015-08-31 12:37:26.493958
"""

from alembic import op

# revision identifiers, used by Alembic.
revision = '4a3de0d0804a'
down_revision = '37a05f6246af'


def upgrade():
op.create_index(
'ix_nodes_parent_id', 'nodes', ['parent_id', ])
op.create_index(
'ix_local_groups_node_id', 'local_groups', ['node_id', ])
op.create_index(
'ix_local_groups_principal_name', 'local_groups', ['principal_name', ])
op.create_index(
'ix_tags_to_contents_tag_id', 'tags_to_contents', ['tag_id', ])
op.create_index(
'ix_tags_to_contents_content_id', 'tags_to_contents', ['content_id', ])


def downgrade():
op.drop_index('ix_nodes_parent_id', 'nodes')
op.drop_index('ix_local_groups_node_id', 'local_groups')
op.drop_index('ix_local_groups_principal_name', 'local_groups')
11 changes: 10 additions & 1 deletion kotti/events.py
View file
Expand Up @@ -23,11 +23,13 @@
from sqlalchemy.orm import load_only
import venusian
from sqlalchemy.orm import mapper
from sqlalchemy_utils.functions import has_changes
from pyramid.location import lineage
from pyramid.threadlocal import get_current_request
from zope.deprecation.deprecation import deprecated

from kotti import DBSession
from kotti import get_settings
from kotti.resources import Content
from kotti.resources import LocalGroup
from kotti.resources import Node
Expand Down Expand Up @@ -255,7 +257,14 @@ def set_modification_date(event):
:type event: :class:`ObjectUpdate`
"""

event.object.modification_date = datetime.now()
exclude = []

for e in get_settings()['kotti.modification_date_excludes']:
if isinstance(event.object, e.class_):
exclude.append(e.key)

if has_changes(event.object, exclude=exclude):
event.object.modification_date = datetime.now()


def delete_orphaned_tags(event):
Expand Down
Binary file added kotti/locale/pl/LC_MESSAGES/Kotti.mo
View file
Binary file not shown.

0 comments on commit f413c83

Please sign in to comment.