Skip to content

Commit

Permalink
Remove cdb and tdb CNID backends, Issue #535
Browse files Browse the repository at this point in the history 
- Removed in-memory tdb backend fallback
- Abort when dbd connection has a fatal error
- Rewrote docs to clarify that dbd is the only option
  • Loading branch information
@rdmark
rdmark committed Dec 25, 2023
1 parent e7fbb4b commit 013b5c6
Show file tree
Hide file tree
Showing 70 changed files with 97 additions and 5,345 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/build.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ on:
env:
APT_PACKAGES: |
libtool libtool-bin automake autoconf libssl-dev libgcrypt-dev libkrb5-dev libpam0g-dev \
libdb-dev libtdb-dev libmysqlclient-dev libavahi-client-dev libacl1-dev libcrack2-dev \
libdb-dev libmysqlclient-dev libavahi-client-dev libacl1-dev libcrack2-dev \
libcups2-dev tcpd libkrb5-dev
jobs:
Expand Down
41 changes: 0 additions & 41 deletions configure.ac
View file
Original file line number Diff line number Diff line change
Expand Up @@ -199,44 +199,6 @@ AC_ARG_ENABLE(ddp,
]
)

AC_MSG_CHECKING([whether to enable debug code])
AC_ARG_ENABLE(debug1,
[ --enable-debug1 enable debug code],[
if test "$enableval" != "no"; then
if test "$enableval" = "yes"; then
AC_DEFINE(DEBUG1, 1, [Define if debugging information should be included])
else
AC_DEFINE_UNQUOTED(DEBUG1, $enableval, [Define if debugging information should be included])
fi
AC_MSG_RESULT([yes])
else
AC_MSG_RESULT([no])
fi
],[
AC_MSG_RESULT([no])
]
)

AC_MSG_CHECKING([whether to enable verbose debug code])
AC_ARG_ENABLE(debug,
[ --enable-debug enable verbose debug code],[
if test "$enableval" != "no"; then
if test "$enableval" = "yes"; then
AC_DEFINE(DEBUG, 1, [Define if verbose debugging information should be included])
else
AC_DEFINE_UNQUOTED(DEBUG, $enableval, [Define if verbose debugging information should be included])
fi
AC_MSG_RESULT([yes])
else
AC_MSG_RESULT([no])
AC_DEFINE(NDEBUG, 1, [Disable assertions])
fi
],[
AC_MSG_RESULT([no])
AC_DEFINE(NDEBUG, 1, [Disable assertions])
]
)

AC_MSG_CHECKING([whether to enable debugging with debuggers])
AC_ARG_ENABLE(debugging,
[ --enable-debugging disable SIGALRM timers and DSI tickles (eg for debugging with gdb/dbx/...)],[
Expand Down Expand Up @@ -1212,16 +1174,13 @@ AC_CONFIG_FILES([Makefile
libatalk/atp/Makefile
libatalk/bstring/Makefile
libatalk/cnid/Makefile
libatalk/cnid/cdb/Makefile
libatalk/cnid/last/Makefile
libatalk/cnid/dbd/Makefile
libatalk/cnid/tdb/Makefile
libatalk/compat/Makefile
libatalk/dsi/Makefile
libatalk/nbp/Makefile
libatalk/netddp/Makefile
libatalk/util/Makefile
libatalk/tdb/Makefile
libatalk/unicode/Makefile
libatalk/unicode/charsets/Makefile
libatalk/vfs/Makefile
Expand Down
42 changes: 11 additions & 31 deletions doc/manual/man/man5/AppleVolumes.default.5.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

<manvolnum>5</manvolnum>

<refmiscinfo class="date">03 May 2023</refmiscinfo>
<refmiscinfo class="date">24 Dec 2023</refmiscinfo>

<refmiscinfo class="source">:NETATALK_VERSION:</refmiscinfo>
</refmeta>
Expand Down Expand Up @@ -622,56 +622,36 @@
folder in the volume root.</para>

<variablelist>
<varlistentry>
<term>cdb</term>

<listitem>
<para>"Concurrent database", backend is based on Sleepycat's Berkeley
DB. With this backend several <command>afpd</command> daemons access
the CNID database directly. Berkeley DB locking is used to
synchronize access, if more than one <command>afpd</command> process
is active for a volume. The drawback is, that the crash of a single
<command>afpd</command> process might corrupt the database.</para>
</listitem>
</varlistentry>

<varlistentry>
<term>dbd</term>

<listitem>
<para>Access to the CNID database is restricted to the
<para>The "Database Daemon" backend is built on Berkeley DB.
Access to the CNID database is restricted to the
<command>cnid_metad</command> daemon process.
<command>afpd</command> processes communicate with the daemon for
database reads and updates. If built with Berkeley DB transactions
the probability for database corruption is practically zero, but
performance can be slower than with <option>cdb</option></para>
database reads and updates.</para>
</listitem>
</varlistentry>

<varlistentry>
<term>last</term>

<listitem>
<para>This backend is an exception, in terms of ID persistency. ID's
are only valid for the current session. This is basically what
<command>afpd</command> did in the 1.5 (and 1.6) versions. This
backend is still available, as it is useful for e.g. sharing
cdroms.</para>
<para>A simple backend without ID persistency.
IDs are only valid for the current session. This is basically what
<command>afpd</command> did in the 1.5 and 1.6 versions. This
backend can be useful for sharing read-only
filesystems, such as CR-ROMs.</para>

<para><emphasis role="bold">Warning</emphasis>: It is
<emphasis>NOT</emphasis> recommended to use this backend for volumes
anymore, as <command>afpd</command> now relies heavily on a
<emphasis>NOT</emphasis> recommended to use this backend for writable
voumes, as <command>afpd</command> relies heavily on a
persistent ID database. Aliases will likely not work and filename
mangling is not supported.</para>
</listitem>
</varlistentry>
</variablelist>

<para>Even though <command>./configure --help</command> might show that
there are other CNID backends available, be warned those are likely broken
or mainly used for testing. Don't use them unless you know what you're
doing, they may be removed without further notice from future
versions.</para>
</refsect1>

<refsect1>
Expand Down
8 changes: 0 additions & 8 deletions doc/manual/man/man5/afpd.conf.5.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -735,14 +735,6 @@
</listitem>
</varlistentry>

<varlistentry>
<term>-nodebug</term>

<listitem>
<para>Disables debugging.</para>
</listitem>
</varlistentry>

<varlistentry>
<term>-sleep <replaceable>[number]</replaceable></term>

Expand Down
6 changes: 2 additions & 4 deletions doc/manual/man/man8/afpd.8.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

<manvolnum>8</manvolnum>

<refmiscinfo class="date">01 Mar 2023</refmiscinfo>
<refmiscinfo class="date">24 Dec 2023</refmiscinfo>

<refmiscinfo class="source">:NETATALK_VERSION:</refmiscinfo>
</refmeta>
Expand Down Expand Up @@ -89,9 +89,7 @@
<term>-d</term>

<listitem>
<para>Specifies that the daemon should not fork. If netatalk has
been configured with <emphasis>--enable-debug1</emphasis>, a trace
of all AFP commands will be written to stdout.</para>
<para>Prevents the daemon from forking, for debugging purposes.</para>
</listitem>
</varlistentry>

Expand Down
5 changes: 2 additions & 3 deletions doc/manual/man/man8/papd.8.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

<manvolnum>8</manvolnum>

<refmiscinfo class="date">01 Mar 2023</refmiscinfo>
<refmiscinfo class="date">24 Dec 2023</refmiscinfo>

<refmiscinfo class="source">:NETATALK_VERSION:</refmiscinfo>
</refmeta>
Expand Down Expand Up @@ -210,8 +210,7 @@
<term>-d</term>

<listitem>
<para>Do not fork or disassociate from the terminal. Write some
debugging information to stderr.</para>
<para>Do not fork or disassociate from the terminal.</para>
</listitem>
</varlistentry>

Expand Down
50 changes: 6 additions & 44 deletions doc/manual/netatalk/configuration.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -220,60 +220,22 @@
</itemizedlist>
</note>

<sect3>
<title>cdb<indexterm>
<primary>CDB</primary>

<secondary>"cdb" CNID backend</secondary>
</indexterm></title>

<para>The "concurrent database" backend is based on
Berkeley DB. With this backend, several afpd daemons access the CNID
database directly. Berkeley DB locking is used to synchronize access,
if more than one afpd process is active for a volume. The drawback is,
that the crash of a single afpd process might corrupt the database.
cdb should only be used when sharing home directories for a larger
number of users <emphasis>and</emphasis> it has been determined that a
large number of <command>cnid_dbd</command> processes is
problematic.</para>
</sect3>

<sect3>
<title>dbd<indexterm>
<primary>DBD</primary>

<secondary>"dbd" CNID backend</secondary>
</indexterm></title>

<para>Access to the CNID database is restricted to the cnid_dbd daemon
<para>The "Database Daemon" backend is built on Berkeley DB.
Access to the CNID database is restricted to the cnid_dbd daemon
process. afpd processes communicate with the daemon for database reads
and updates. The probability for database corruption is practically
zero. As a database process gets spawned for each volume, you're
probably better off using cdb for sharing home directories for a
larger number of users.</para>
zero.</para>

<para>This is the default backend since Netatalk 2.1.</para>
</sect3>

<sect3>
<title>tdb<indexterm>
<primary>tdb</primary>

<secondary>"tdb" CNID backend</secondary>
</indexterm></title>

<para><abbrev>tdb</abbrev> is another persistent CNID database, it's
Samba's <emphasis>Trivial Database</emphasis>. It could be used
instead of <abbrev>cdb</abbrev> for user volumes.<important>
<para>Only ever use it for volumes that are
<emphasis>not</emphasis> shared and accessed by multiple clients
at once !</para>
</important>This backend is also used internally (as in-memory CNID
database) as a fallback in case opening the primary database can't be
opened, because <abbrev>tdb</abbrev> can work as in-memory database.
This of course means upon restart the CNIDs are gone.</para>
</sect3>

<sect3>
<title>last<indexterm>
<primary>Last</primary>
Expand All @@ -282,9 +244,9 @@
</indexterm></title>

<para>The last backend is a semi-persistent backend. IDs will be
reused and, what is much worse, you can get duplicate IDs. You should
use it for sharing cdroms only, <emphasis>don't</emphasis> use it for
sharing normal volumes.</para>
reused and, what is much worse, you can get duplicate IDs.
You should use it for sharing read-only volumes only.
<emphasis>Don't</emphasis> use it for sharing normal volumes.</para>
</sect3>
</sect2>

Expand Down
54 changes: 5 additions & 49 deletions doc/manual/netatalk/upgrade.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -291,14 +291,14 @@
properly.</para>

<para>That leaves the CNID schemes that use persistent storage for CNIDs.
Netatalk 2.0 supports two: <emphasis remap="B">cdb</emphasis> and
<emphasis remap="B">dbd</emphasis>. Both are based on the Berkeley DB
Netatalk 2.3 supports one:
<emphasis remap="B">dbd</emphasis>. It is based on the Berkeley DB
database library as before. One difference is, though, that you are not
restricted to using a single scheme for all of your volumes that has to be
determined at compile time. The CNID scheme (also called a "CNID backend")
is now a runtime option for a volume. That means that you can make the
choice per volume based on your requirements. Here are the properties as
well as the advantages and disadvantages of the three supported schemes:
well as the advantages and disadvantages of the two supported schemes:
<orderedlist>
<listitem>
<para><emphasis role="bold">last<indexterm>
Expand All @@ -309,23 +309,6 @@
possible.</para>
</listitem>

<listitem>
<para><emphasis role="bold">cdb<indexterm>
<primary>CDB</primary>

<secondary>"cdb" CNID backend</secondary>
</indexterm>: </emphasis>Roughly analogous to the Netatalk 1.6.x
versions with what was called then the "DID scheme" option set to
"cnid" and the "CNID with Concurrent Data Store" option set to
"yes". Access to the CNID database for a volume happens directly
from the Netatalk afpd daemons. A Berkeley DB locking scheme (the
"Concurrent Data Store" bit) is used to avoid database
inconsistencies. Robustness is much improved compared to previous
releases. The CNID database can only become corrupted if an afpd
daemon crashes unexpectedly, is killed by the administrator or the
whole machine crashes.</para>
</listitem>

<listitem>
<para><emphasis role="bold">dbd<indexterm>
<primary>DBD</primary>
Expand All @@ -339,34 +322,12 @@
design combined with the transactional updates makes the CNID
database crashproof: Any of the participating afpd daemons, the
database daemon itself or the whole machine can crash and the CNID
database should still be in a consistent state. The downside to this
is that the speed of updates and retrieval is slower than with the
<emphasis remap="B">cdb</emphasis> scheme. If this is a problem, you
might want to disable transactions at Netatalk compile time
(currently, the default is to compile without transactions anyway).
That will give you safety against afpd crashing, but not if the
machine goes down unexpectedly. Also, have a look at the nosync
database should still be in a consistent state.
Also, have a look at the nosync
option documented in the cnid_dbd manual page.</para>
</listitem>
</orderedlist></para>

<para>It is also possible to switch between <emphasis
remap="B">cdb</emphasis> and <emphasis remap="B">dbd</emphasis> for a
given volume, since they use the same database format. You just have to
shut down all processes accessing the database cleanly, make the necessary
configuration changes and restart. Please note, that you can easily
specify a default CNID backend for all shares by applying the
<option>cnidscheme</option><indexterm>
<primary>cnidscheme</primary>

<secondary>specifying a CNID backend</secondary>
</indexterm> option to the "<option>:DEFAULT:</option>" share (compare
with the <citerefentry>
<refentrytitle>AppleVolumes.default</refentrytitle>

<manvolnum>5</manvolnum>
</citerefentry> manual page for details).</para>

<para>Note that the <emphasis remap="B">dbd</emphasis> backend needs an
auxiliary daemon, called <command>cnid_metad</command><indexterm>
<primary>cnid_metad</primary>
Expand All @@ -378,11 +339,6 @@
process list even though the <emphasis remap="B">dbd</emphasis> backend is
used please check for errors in the startup scripts.</para>

<para>If you compile Netatalk 2.0 yourself and invoke configure --help,
you'll notice that there are in fact more CNID backends to chose from.
Don't use any of them. They are either broken or incomplete. Some of them
might turn into something useful in the future.</para>

<sect2>
<title>How to upgrade if no persistent CNID storage was used</title>

Expand Down

0 comments on commit 013b5c6

Please sign in to comment.