Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[2.3] Remove cdb and tdb CNID backends #611

Merged
merged 1 commit into from
Dec 27, 2023
Merged

[2.3] Remove cdb and tdb CNID backends #611

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ on:
env:
APT_PACKAGES: |
libtool libtool-bin automake autoconf libssl-dev libgcrypt-dev libkrb5-dev libpam0g-dev \
libdb-dev libtdb-dev libmysqlclient-dev libavahi-client-dev libacl1-dev libcrack2-dev \
libdb-dev libmysqlclient-dev libavahi-client-dev libacl1-dev libcrack2-dev \
libcups2-dev tcpd libkrb5-dev

jobs:
Expand Down
3 changes: 0 additions & 3 deletions configure.ac
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1174,16 +1174,13 @@ AC_CONFIG_FILES([Makefile
libatalk/atp/Makefile
libatalk/bstring/Makefile
libatalk/cnid/Makefile
libatalk/cnid/cdb/Makefile
libatalk/cnid/last/Makefile
libatalk/cnid/dbd/Makefile
libatalk/cnid/tdb/Makefile
libatalk/compat/Makefile
libatalk/dsi/Makefile
libatalk/nbp/Makefile
libatalk/netddp/Makefile
libatalk/util/Makefile
libatalk/tdb/Makefile
libatalk/unicode/Makefile
libatalk/unicode/charsets/Makefile
libatalk/vfs/Makefile
Expand Down
42 changes: 11 additions & 31 deletions doc/manual/man/man5/AppleVolumes.default.5.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

<manvolnum>5</manvolnum>

<refmiscinfo class="date">03 May 2023</refmiscinfo>
<refmiscinfo class="date">24 Dec 2023</refmiscinfo>

<refmiscinfo class="source">:NETATALK_VERSION:</refmiscinfo>
</refmeta>
Expand Down Expand Up @@ -622,56 +622,36 @@
folder in the volume root.</para>

<variablelist>
<varlistentry>
<term>cdb</term>

<listitem>
<para>"Concurrent database", backend is based on Sleepycat's Berkeley
DB. With this backend several <command>afpd</command> daemons access
the CNID database directly. Berkeley DB locking is used to
synchronize access, if more than one <command>afpd</command> process
is active for a volume. The drawback is, that the crash of a single
<command>afpd</command> process might corrupt the database.</para>
</listitem>
</varlistentry>

<varlistentry>
<term>dbd</term>

<listitem>
<para>Access to the CNID database is restricted to the
<para>The "Database Daemon" backend is built on Berkeley DB.
Access to the CNID database is restricted to the
<command>cnid_metad</command> daemon process.
<command>afpd</command> processes communicate with the daemon for
database reads and updates. If built with Berkeley DB transactions
the probability for database corruption is practically zero, but
performance can be slower than with <option>cdb</option></para>
database reads and updates.</para>
</listitem>
</varlistentry>

<varlistentry>
<term>last</term>

<listitem>
<para>This backend is an exception, in terms of ID persistency. ID's
are only valid for the current session. This is basically what
<command>afpd</command> did in the 1.5 (and 1.6) versions. This
backend is still available, as it is useful for e.g. sharing
cdroms.</para>
<para>A simple backend without ID persistency.
IDs are only valid for the current session. This is basically what
<command>afpd</command> did in the 1.5 and 1.6 versions. This
backend can be useful for sharing read-only
filesystems, such as CR-ROMs.</para>

<para><emphasis role="bold">Warning</emphasis>: It is
<emphasis>NOT</emphasis> recommended to use this backend for volumes
anymore, as <command>afpd</command> now relies heavily on a
<emphasis>NOT</emphasis> recommended to use this backend for writable
voumes, as <command>afpd</command> relies heavily on a
persistent ID database. Aliases will likely not work and filename
mangling is not supported.</para>
</listitem>
</varlistentry>
</variablelist>

<para>Even though <command>./configure --help</command> might show that
there are other CNID backends available, be warned those are likely broken
or mainly used for testing. Don't use them unless you know what you're
doing, they may be removed without further notice from future
versions.</para>
</refsect1>

<refsect1>
Expand Down
8 changes: 0 additions & 8 deletions doc/manual/man/man5/afpd.conf.5.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -735,14 +735,6 @@
</listitem>
</varlistentry>

<varlistentry>
<term>-debug</term>

<listitem>
<para>Prevents afpd from forking, for debugging purposes.</para>
</listitem>
</varlistentry>

<varlistentry>
<term>-sleep <replaceable>[number]</replaceable></term>

Expand Down
50 changes: 6 additions & 44 deletions doc/manual/netatalk/configuration.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -220,60 +220,22 @@
</itemizedlist>
</note>

<sect3>
<title>cdb<indexterm>
<primary>CDB</primary>

<secondary>"cdb" CNID backend</secondary>
</indexterm></title>

<para>The "concurrent database" backend is based on
Berkeley DB. With this backend, several afpd daemons access the CNID
database directly. Berkeley DB locking is used to synchronize access,
if more than one afpd process is active for a volume. The drawback is,
that the crash of a single afpd process might corrupt the database.
cdb should only be used when sharing home directories for a larger
number of users <emphasis>and</emphasis> it has been determined that a
large number of <command>cnid_dbd</command> processes is
problematic.</para>
</sect3>

<sect3>
<title>dbd<indexterm>
<primary>DBD</primary>

<secondary>"dbd" CNID backend</secondary>
</indexterm></title>

<para>Access to the CNID database is restricted to the cnid_dbd daemon
<para>The "Database Daemon" backend is built on Berkeley DB.
Access to the CNID database is restricted to the cnid_dbd daemon
process. afpd processes communicate with the daemon for database reads
and updates. The probability for database corruption is practically
zero. As a database process gets spawned for each volume, you're
probably better off using cdb for sharing home directories for a
larger number of users.</para>
zero.</para>

<para>This is the default backend since Netatalk 2.1.</para>
</sect3>

<sect3>
<title>tdb<indexterm>
<primary>tdb</primary>

<secondary>"tdb" CNID backend</secondary>
</indexterm></title>

<para><abbrev>tdb</abbrev> is another persistent CNID database, it's
Samba's <emphasis>Trivial Database</emphasis>. It could be used
instead of <abbrev>cdb</abbrev> for user volumes.<important>
<para>Only ever use it for volumes that are
<emphasis>not</emphasis> shared and accessed by multiple clients
at once !</para>
</important>This backend is also used internally (as in-memory CNID
database) as a fallback in case opening the primary database can't be
opened, because <abbrev>tdb</abbrev> can work as in-memory database.
This of course means upon restart the CNIDs are gone.</para>
</sect3>

<sect3>
<title>last<indexterm>
<primary>Last</primary>
Expand All @@ -282,9 +244,9 @@
</indexterm></title>

<para>The last backend is a semi-persistent backend. IDs will be
reused and, what is much worse, you can get duplicate IDs. You should
use it for sharing cdroms only, <emphasis>don't</emphasis> use it for
sharing normal volumes.</para>
reused and, what is much worse, you can get duplicate IDs.
You should use it for sharing read-only volumes only.
<emphasis>Don't</emphasis> use it for sharing normal volumes.</para>
</sect3>
</sect2>

Expand Down
54 changes: 5 additions & 49 deletions doc/manual/netatalk/upgrade.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -291,14 +291,14 @@
properly.</para>

<para>That leaves the CNID schemes that use persistent storage for CNIDs.
Netatalk 2.0 supports two: <emphasis remap="B">cdb</emphasis> and
<emphasis remap="B">dbd</emphasis>. Both are based on the Berkeley DB
Netatalk 2.3 supports one:
<emphasis remap="B">dbd</emphasis>. It is based on the Berkeley DB
database library as before. One difference is, though, that you are not
restricted to using a single scheme for all of your volumes that has to be
determined at compile time. The CNID scheme (also called a "CNID backend")
is now a runtime option for a volume. That means that you can make the
choice per volume based on your requirements. Here are the properties as
well as the advantages and disadvantages of the three supported schemes:
well as the advantages and disadvantages of the two supported schemes:
<orderedlist>
<listitem>
<para><emphasis role="bold">last<indexterm>
Expand All @@ -309,23 +309,6 @@
possible.</para>
</listitem>

<listitem>
<para><emphasis role="bold">cdb<indexterm>
<primary>CDB</primary>

<secondary>"cdb" CNID backend</secondary>
</indexterm>: </emphasis>Roughly analogous to the Netatalk 1.6.x
versions with what was called then the "DID scheme" option set to
"cnid" and the "CNID with Concurrent Data Store" option set to
"yes". Access to the CNID database for a volume happens directly
from the Netatalk afpd daemons. A Berkeley DB locking scheme (the
"Concurrent Data Store" bit) is used to avoid database
inconsistencies. Robustness is much improved compared to previous
releases. The CNID database can only become corrupted if an afpd
daemon crashes unexpectedly, is killed by the administrator or the
whole machine crashes.</para>
</listitem>

<listitem>
<para><emphasis role="bold">dbd<indexterm>
<primary>DBD</primary>
Expand All @@ -339,34 +322,12 @@
design combined with the transactional updates makes the CNID
database crashproof: Any of the participating afpd daemons, the
database daemon itself or the whole machine can crash and the CNID
database should still be in a consistent state. The downside to this
is that the speed of updates and retrieval is slower than with the
<emphasis remap="B">cdb</emphasis> scheme. If this is a problem, you
might want to disable transactions at Netatalk compile time
(currently, the default is to compile without transactions anyway).
That will give you safety against afpd crashing, but not if the
machine goes down unexpectedly. Also, have a look at the nosync
database should still be in a consistent state.
Also, have a look at the nosync
option documented in the cnid_dbd manual page.</para>
</listitem>
</orderedlist></para>

<para>It is also possible to switch between <emphasis
remap="B">cdb</emphasis> and <emphasis remap="B">dbd</emphasis> for a
given volume, since they use the same database format. You just have to
shut down all processes accessing the database cleanly, make the necessary
configuration changes and restart. Please note, that you can easily
specify a default CNID backend for all shares by applying the
<option>cnidscheme</option><indexterm>
<primary>cnidscheme</primary>

<secondary>specifying a CNID backend</secondary>
</indexterm> option to the "<option>:DEFAULT:</option>" share (compare
with the <citerefentry>
<refentrytitle>AppleVolumes.default</refentrytitle>

<manvolnum>5</manvolnum>
</citerefentry> manual page for details).</para>

<para>Note that the <emphasis remap="B">dbd</emphasis> backend needs an
auxiliary daemon, called <command>cnid_metad</command><indexterm>
<primary>cnid_metad</primary>
Expand All @@ -378,11 +339,6 @@
process list even though the <emphasis remap="B">dbd</emphasis> backend is
used please check for errors in the startup scripts.</para>

<para>If you compile Netatalk 2.0 yourself and invoke configure --help,
you'll notice that there are in fact more CNID backends to chose from.
Don't use any of them. They are either broken or incomplete. Some of them
might turn into something useful in the future.</para>

<sect2>
<title>How to upgrade if no persistent CNID storage was used</title>

Expand Down
2 changes: 0 additions & 2 deletions etc/afpd/afp_options.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,8 +216,6 @@ int afp_options_parseline(char *buf, struct afp_options *options)
options->server = opt;

/* parse toggles */
if (strstr(buf, " -debug"))
options->flags |= OPTION_DEBUG;
#ifdef USE_SRVLOC
if (strstr(buf, " -slp"))
options->flags &= ~OPTION_NOSLP;
Expand Down
2 changes: 1 addition & 1 deletion etc/afpd/directory.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -825,7 +825,7 @@ struct dir *dir_add(struct vol *vol, const struct dir *dir, struct path *path, i
adp = &ad;

/* Get CNID */
if ((id = get_id(vol, adp, &path->st, dir->d_did, path->u_name, len)) == 0) { /* 2 */
if ((id = get_id(vol, adp, &path->st, dir->d_did, path->u_name, len)) == CNID_INVALID) { /* 2 */
err = 1;
goto exit;
}
Expand Down
46 changes: 10 additions & 36 deletions etc/afpd/file.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,8 +191,7 @@ char *set_name(const struct vol *vol, char *data, cnid_t pid, char *name, cnid_t
*
* 1. Get the objects CNID as stored in its adouble file
* 2. Get the objects CNID from the database
* 3. If there's a problem with a "dbd" database, fallback to "tdb" in memory
* 4. In case 2 and 3 differ, store 3 in the adouble file
* 3. Store in the adouble file
*
* @param vol (rw) volume
* @param adp (rw) adouble struct of object upath, might be NULL
Expand All @@ -208,11 +207,9 @@ uint32_t get_id(struct vol *vol,
const char *upath,
const int len)
{
static int first = 1; /* mark if this func is called the first time */
u_int32_t adcnid;
u_int32_t dbcnid = CNID_INVALID;

restart:
if (vol->v_cdb != NULL) {
/* prime aint with what we think is the cnid, set did to zero for
catching moved files */
Expand All @@ -224,6 +221,14 @@ uint32_t get_id(struct vol *vol,
switch (errno) {
case CNID_ERR_CLOSE: /* the db is closed */
break;
case CNID_ERR_DB:
LOG(log_error, logtype_afpd,
"get_id: Connection to the CNID backend DB failed. "
"This is now treated as a fatal error. "
"Is the cnid_metad process running? "
"Please escalate to the upstream project "
"if you suspect this is a bug");
exit(EXITERR_SYS);
case CNID_ERR_PARAM:
LOG(log_error, logtype_afpd, "get_id: Incorrect parameters passed to cnid_add");
afp_errno = AFPERR_PARAM;
Expand All @@ -232,41 +237,11 @@ uint32_t get_id(struct vol *vol,
afp_errno = AFPERR_PARAM;
goto exit;
default:
/* Close CNID backend if "dbd" and switch to temp in-memory "tdb" */
/* we have to do it here for "dbd" because it uses "lazy opening" */
/* In order to not end in a loop somehow with goto restart below */
/* */
if (first && (strcmp(vol->v_cnidscheme, "dbd") == 0)) { /* (3) */
cnid_close(vol->v_cdb);
free(vol->v_cnidscheme);
vol->v_cnidscheme = strdup("tdb");

int flags = CNID_FLAG_MEMORY;
if ((vol->v_flags & AFPVOL_NODEV)) {
flags |= CNID_FLAG_NODEV;
}
LOG(log_error, logtype_afpd, "Reopen volume %s using in memory temporary CNID DB.",
vol->v_path);
vol->v_cdb = cnid_open(vol->v_path, vol->v_umask, "tdb", flags, NULL, NULL);
if (vol->v_cdb) {
vol->v_flags &= ~AFPVOL_CACHE;
if (!(vol->v_flags & AFPVOL_TM)) {
vol->v_flags |= AFPVOL_RO;
setmessage("Something wrong with the volume's CNID DB, using temporary CNID DB instead."
"Check server messages for details. Switching to read-only mode.");
kill(getpid(), SIGUSR2);
}
goto restart; /* now try again with the temp CNID db */
} else {
setmessage("Something wrong with the volume's CNID DB, using temporary CNID DB failed too!"
"Check server messages for details, can't recover from this state!");
}
}
afp_errno = AFPERR_MISC;
goto exit;
}
}
else if (adp && (adcnid != dbcnid)) { /* 4 */
else if (adp && (adcnid != dbcnid)) { /* 3 */
/* Update the ressource fork. For a folder adp is always null */
LOG(log_debug, logtype_afpd, "get_id(%s/%s): calling ad_setid(old: %u, new: %u)",
getcwdpath(), upath, htonl(adcnid), htonl(dbcnid));
Expand All @@ -277,7 +252,6 @@ uint32_t get_id(struct vol *vol,
}

exit:
first = 0;
return dbcnid;
}

Expand Down