Skip to content

Commit

Permalink
Merge pull request #2106 from openedx/feanil/redirect_release_notes
Browse files Browse the repository at this point in the history 
feanil/redirect release notes
  • Loading branch information
@feanil
feanil committed Feb 13, 2023
2 parents 885f2dd + cf90b90 commit 40a9b2e
Show file tree
Hide file tree
Showing 37 changed files with 92 additions and 6,583 deletions.
2 changes: 1 addition & 1 deletion en_us/install_operations/source/platform_releases/eucalyptus.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ What's Included in Eucalyptus

The Open edX Eucalyptus release contains several new features for learners,
course teams, and developers. For more information, see
:ref:`openreleasenotes:Open edX Eucalyptus Release`.
`Open edX Eucalyptus Release <https://docs.openedx.org/en/latest/community/release_notes/eucalyptus.html>`.

*******************************
What Is the Eucalyptus Git Tag?
Expand Down
76 changes: 2 additions & 74 deletions en_us/open_edx_release_notes/source/CSMHE/CSMHE_overview.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,77 +4,5 @@
Overview of the ``courseware_studentmodulehistory`` Changes
##########################################################################

This topic provides background information about the
``courseware_studentmodulehistory`` table and why a new database configuration
is needed.

.. contents::
:local:
:depth: 1

For procedures about how to upgrade all Open edX instances that follow master,
see :ref:`CSMHE Procedures`.

.. note:: The changes described in this section are a part of the upgrade to
the Open edX Eucalyptus release.

****************************************************************
What Is the ``courseware_studentmodulehistory`` Table?
****************************************************************

The ``courseware_studentmodulehistory`` database table contains a record for
each attempt that learners make to answer problem types that are implemented in
the edX platform by the ``capa_module`` XBlock correctly. This database table
is a standard Django model. It has an ``id`` column with a type of 32-bit
signed integer, and therefore has a maximum capacity of 2,147,483,647 records.

.. _What Is the Issue:

************************
What Is the Issue?
************************

Typically, ``courseware_studentmodulehistory`` is the largest table in the
database. It can be twice as large as the next largest table,
``courseware_studentmodule``.

On the edx.org site, two records are written to this table every second. Before
this table for edx.org reached even half of its maximum capacity, edX began to
design a replacement with a higher capacity form.

*******************************
What Is the Replacement Table?
*******************************

The new database table,
``coursewarehistoryextended_studentmodulehistoryextended``, uses a custom
Django field type to give the ``id`` column a type of 64-bit unsigned integer,
which offers an exponentially larger storage capacity than the
``courseware_studentmodulehistory`` table.

.. _Why Is A New Database Needed:

********************************
Why Is A New Database Needed?
********************************

By design, the ``coursewarehistoryextended_studentmodulehistoryextended`` table
must be created in a new database, ``edxapp_csmh``. The new database will
coexist alongside the existing ``edxapp`` database.

Depending on your operational needs, you can either create this database in
your existing database infrastructure or stand a new database server up.

EdX chose to set up a new database to address several requirements.

* Distribute write load across different backends.
* Reclaim storage on our main database instance, shifting that storage to a
less powerful and less expensive system.
* Enable smaller, faster, and less disruptive backups.
* Ensure faster disaster recovery by having smaller backups.

The edx-platform repository master branch writes records to this database and
table as of 5 May 2016.


.. include:: ../../../links/links.rst
.. This is a placeholder for redirects to the new release notes site.
See https://docs.openedx.org/en/latest/community/release_notes/index.html
23 changes: 2 additions & 21 deletions en_us/open_edx_release_notes/source/CSMHE/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,31 +4,12 @@
Replacing the ``courseware_studentmodulehistory`` Table
########################################################

This section describes a change to the ``courseware_studentmodulehistory``
database table.

.. note:: The changes described in this section are a part of the upgrade to
the Open edX Eucalyptus release.

The change to the ``courseware_studentmodulehistory`` database table requires a
new database configuration, and offers an optional data migration. This change
will be released to the edx-platform repository on 5 May 2016, and will affect
all Open edX installations that follow master on that date.

.. important:: If you are responsible for maintaining an Open edX instance,
including a devstack, fullstack, or production installation, you must prepare
for this change before you upgrade to the 5 May 2016 version of master.

This section presents an overview of the change followed by options for
completing the required and optional procedures. EdX recommends that you review
all of the topics in this section before you choose an option for your Open edX
installation.
.. This is a placeholder for redirects to the new release notes site.
See https://docs.openedx.org/en/latest/community/release_notes/index.html
.. toctree::
:maxdepth: 2

CSMHE_overview
migration_options
migration_procedures

.. include:: ../../../links/links.rst
153 changes: 2 additions & 151 deletions en_us/open_edx_release_notes/source/CSMHE/migration_options.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,154 +4,5 @@
Options for Updating ``courseware_studentmodulehistory``
##########################################################

This topic outlines the options for updating to the new database and
table configuration required by the ``courseware_studentmodulehistory`` change.

.. contents::
:local:
:depth: 1

Because each Open edX installation has a unique set of constraints
and requirements, edX recommends that you review all of these options before
selecting one for your instance or instances.

*******************************
Keep, and Query, Both Tables
*******************************

This option is suitable for many instances with small databases, such as a
fullstack or small production instance, that do not have the performance
considerations or other operational needs described in the :ref:`overview<CSMH
Overview>`.

For instances with relatively small databases, you set up the new database and
table and then configure the system to read from both tables, without migrating
data. An outline of the steps you need to complete follows.

#. Create the ``edxapp_csmh`` database.

#. Update ``lms.yml`` with a new entry in the DATABASES section.

#. Update ``lms.yml`` to set ``"ENABLE_CSMH_EXTENDED": true``. Leave
``"ENABLE_READING_FROM_MULTIPLE_HISTORY_TABLES": true``.

#. Run django migrations to generate the new table.

A system with this configuration writes only to the new
``coursewarehistoryextended_studentmodulehistoryextended`` table, but it reads
from both that table and the ``courseware_studentmodulehistory`` table in the
``edxapp`` database.

For more information, see :ref:`CSMHE Procedures`.

**************************************
Reprovision or Reinstall Your Devstack
**************************************

.. note:: This option is suitable only for devstacks that were provisioned
recently. If your devstack dates from before March 2016, follow the destroy
and reprovision instructions.

To set up devstack with the new database and SQL table, you can reprovision. If
you choose this option for updating devstack, no further configuration or
migration procedures are required. Before you reprovision, make sure that any
active development is committed in your devstack's edx-platform GitHub
repository. If that repository has changes, Ansible™ will fail to provision
because it cannot update. After vagrant provision completes, your devstack will
be running the release branch of edx-platform.

.. code-block:: bash
cd devstack
vagrant provision
Reprovisioning adds the ``edxapp_csmh`` database and its
``coursewarehistoryextended_studentmodulehistoryextended`` table. The
``lms.yml`` feature flags that control use of the
``coursewarehistoryextended_studentmodulehistoryextended`` table have the
following settings.

.. code-block:: bash
"ENABLE_CSMH_EXTENDED": true
"ENABLE_READING_FROM_MULTIPLE_HISTORY_TABLES": true
A system with this configuration writes to the new
``coursewarehistoryextended_studentmodulehistoryextended`` table only, but
queries both tables.

.. note:: Only set ``ENABLE_READING_FROM_MULTIPLE_HISTORY_TABLES`` to
``true`` for an existing site that does not choose the data migration
option. For new installations, this setting should be ``false``.

You can also completely destroy and rebuild your devstack if you prefer by
using ``vagrant destroy`` before running ``vagrant provision``. If you use this
approach, update your Vagrantfile to ensure that you retrieve the newest base
box. For more information, see the "Download the Vagrantfile" section of the
devstack `wiki`_.

.. code-block:: bash
cd devstack
vagrant destroy
vagrant provision
Please note, this procedure does not remove your git source control directories
under the devstack directory.

.. _Migrate All Data to One Table:

******************************
Migrate All Data to One Table
******************************

This option is suitable for installations that have a large number of records
in the ``coursewarehistoryextended_studentmodulehistoryextended`` table, such
as large production instances.

If you select this option, you set up the new database and table and then
migrate all existing data to the new table. When the process is complete, the
system uses only the new table. This is the procedure that edX followed for
edx.org and edX Edge.

For more information, see :ref:`Why Is A New Database Needed`.

An outline of the steps you complete follows.

#. Create the ``edxapp_csmh`` database.

#. Update ``lms.yml`` with a new entry in the DATABASES section.

If you use the edxapp Ansible role to update ``lms.yml``, the system
automatically merges an update to the ``edxapp_databases`` dictionary in
`edxapp/defaults/main.yml`_.

#. Update ``lms.yml`` to set ``"ENABLE_CSMH_EXTENDED": true``.

#. Run migrations to create the new database table.

#. Deploy so that all new data is being written to the new
``coursewarehistoryextended_studentmodulehistoryextended`` table.

#. Migrate all data from ``courseware_studentmodulehistory`` to
``coursewarehistoryextended_studentmodulehistoryextended``.

#. Update ``lms.yml`` to set
``"ENABLE_READING_FROM_MULTIPLE_HISTORY_TABLES": false``.

#. Truncate ``courseware_studentmodulehistory``.

As soon as you deploy a system with ``ENABLE_CSMH_EXTENDED`` enabled, the
system writes only to the
``coursewarehistoryextended_studentmodulehistoryextended`` table, but it reads
from both that table and the ``courseware_studentmodulehistory`` table. To
reduce the overhead of querying two tables in two databases, you migrate data
and then set ``"ENABLE_READING_FROM_MULTIPLE_HISTORY_TABLES": false``.

For more information, see :ref:`CSMHE Procedures`.

.. include:: ../../../links/links.rst

.. _edxapp/defaults/main.yml: https://github.com/openedx/configuration/blob/master/playbooks/roles/edxapp/defaults/main.yml#L635

.. _wiki: https://openedx.atlassian.net/wiki/display/OpenOPS/Running+Devstack#RunningDevstack-InstallingtheOpenedXDeveloperStack
.. This is a placeholder for redirects to the new release notes site.
See https://docs.openedx.org/en/latest/community/release_notes/index.html

0 comments on commit 40a9b2e

Please sign in to comment.