Skip to content
Permalink
Browse files

pgBackRest is now pure C.

Remove embedded Perl from the distributed binary.  This includes code, configure, Makefile, and packages.  The distributed binary is now pure C.

Remove storagePathEnforceSet() from the C Storage object which allowed Perl to write outside of the storage base directory.  Update mock/all and real/all integration tests to use storageLocal() where they were violating this rule.

Remove "c" option that allowed the remote to tell if it was being called from C or Perl.

Code to convert options to JSON for passing to Perl (perl/config.c) has been moved to LibC since it is still required for Perl integration tests.

Update build and installation instructions in the user guide.

Remove all Perl unit tests.

Remove obsolete Perl code.  In particular this included all the Perl protocol code which required modifications to the Perl storage, manifest, and db objects that are still required for integration testing but only run locally.  Any remaining Perl code is required for testing, documentation, or code generation.

Rename perlReq to binReq in define.yaml to indicate that the binary is required for a test.  This had been the actual meaning for quite some time but the key was never renamed.
  • Loading branch information
dwsteele committed Dec 13, 2019
1 parent 1f2ce45 commit f0ef73db7009cd6e08740d270a6ee7565efc9f8c
Showing with 229 additions and 22,405 deletions.
  1. +1 −1 build/lib/pgBackRestBuild/Config/BuildDefine.pm
  2. +0 −13 build/lib/pgBackRestBuild/Config/Data.pm
  3. +0 −128 build/lib/pgBackRestBuild/Embed/Build.pm
  4. +6 −0 doc/xml/release.xml
  5. +24 −55 doc/xml/user-guide.xml
  6. +0 −1 lib/pgBackRest/Backup/Info.pm
  7. +0 −48 lib/pgBackRest/Common/Cipher.pm
  8. +0 −187 lib/pgBackRest/Common/Io/Process.pm
  9. +69 −109 lib/pgBackRest/Db.pm
  10. +0 −1 lib/pgBackRest/LibCAuto.pm
  11. +0 −196 lib/pgBackRest/Main.pm
  12. +1 −4 lib/pgBackRest/Manifest.pm
  13. +0 −293 lib/pgBackRest/Protocol/Base/Master.pm
  14. +0 −247 lib/pgBackRest/Protocol/Base/Minion.pm
  15. +0 −148 lib/pgBackRest/Protocol/Command/Master.pm
  16. +0 −62 lib/pgBackRest/Protocol/Command/Minion.pm
  17. +0 −489 lib/pgBackRest/Protocol/Helper.pm
  18. +0 −51 lib/pgBackRest/Protocol/Local/Master.pm
  19. +0 −667 lib/pgBackRest/Protocol/Local/Process.pm
  20. +0 −76 lib/pgBackRest/Protocol/Remote/Master.pm
  21. +0 −122 lib/pgBackRest/Protocol/Remote/Minion.pm
  22. +0 −180 lib/pgBackRest/Protocol/Storage/File.pm
  23. +6 −28 lib/pgBackRest/Protocol/Storage/Helper.pm
  24. +0 −342 lib/pgBackRest/Protocol/Storage/Remote.pm
  25. +1 −1 libc/LibC.xs
  26. +0 −1 libc/Makefile.PL
  27. +2 −7 src/perl/config.c → libc/xs/config/configTest.xsh
  28. +0 −2 libc/xs/storage/storage.xs
  29. +5 −13 src/Makefile.in
  30. +0 −3 src/build.auto.h.in
  31. +0 −1 src/command/archive/get/get.c
  32. +2 −2 src/command/backup/pageChecksum.c
  33. +0 −1 src/command/info/info.c
  34. +13 −34 src/common/exit.c
  35. +0 −9 src/config/config.auto.c
  36. +1 −4 src/config/config.auto.h
  37. +1 −1 src/config/config.h
  38. +0 −23 src/config/define.auto.c
  39. +0 −1 src/config/define.auto.h
  40. +1 −2 src/config/load.c
  41. +0 −8 src/config/parse.auto.c
  42. +1 −54 src/configure
  43. +0 −6 src/configure.ac
  44. +2 −2 src/info/infoPg.c
  45. +1 −8 src/main.c
  46. +0 −14 src/perl/config.h
  47. +0 −11,537 src/perl/embed.auto.c
  48. +0 −276 src/perl/exec.c
  49. +0 −18 src/perl/exec.h
  50. +0 −2,337 src/perl/libc.auto.c
  51. +0 −3 src/protocol/helper.c
  52. +1 −19 src/storage/storage.c
  53. +0 −4 src/storage/storage.intern.h
  54. +8 −89 test/define.yaml
  55. +3 −3 test/lib/pgBackRestTest/Common/DefineTest.pm
  56. +5 −9 test/lib/pgBackRestTest/Common/JobTest.pm
  57. +3 −3 test/lib/pgBackRestTest/Common/ListTest.pm
  58. +0 −62 test/lib/pgBackRestTest/Module/Common/CommonEncodePerlTest.pm
  59. +0 −533 test/lib/pgBackRestTest/Module/Common/CommonIniPerlTest.pm
  60. +0 −259 test/lib/pgBackRestTest/Module/Common/CommonIoBufferedPerlTest.pm
  61. +0 −209 test/lib/pgBackRestTest/Module/Common/CommonIoHandlePerlTest.pm
  62. +0 −72 test/lib/pgBackRestTest/Module/Common/CommonIoProcessPerlTest.pm
  63. +0 −46 test/lib/pgBackRestTest/Module/Common/CommonLogPerlTest.pm
  64. +0 −249 test/lib/pgBackRestTest/Module/Info/InfoInfoArchivePerlTest.pm
  65. +0 −262 test/lib/pgBackRestTest/Module/Info/InfoInfoBackupPerlTest.pm
  66. +0 −1,733 test/lib/pgBackRestTest/Module/Manifest/ManifestAllPerlTest.pm
  67. +2 −1 test/lib/pgBackRestTest/Module/Mock/MockAllTest.pm
  68. +0 −135 test/lib/pgBackRestTest/Module/Protocol/ProtocolCommonMinionPerlTest.pm
  69. +0 −155 test/lib/pgBackRestTest/Module/Protocol/ProtocolHelperPerlTest.pm
  70. +7 −6 test/lib/pgBackRestTest/Module/Real/RealAllTest.pm
  71. +0 −116 test/lib/pgBackRestTest/Module/Storage/StorageHelperPerlTest.pm
  72. +0 −351 test/lib/pgBackRestTest/Module/Storage/StoragePerlTest.pm
  73. +36 −0 test/patch/debian-package.patch
  74. +20 −0 test/patch/rhel-package.patch
  75. +0 −13 test/src/module/common/exitTest.c
  76. +0 −112 test/src/module/perl/configTest.c
  77. +0 −82 test/src/module/perl/execTest.c
  78. +6 −6 test/src/module/protocol/protocolTest.c
  79. +0 −5 test/src/module/storage/posixTest.c
  80. +1 −55 test/test.pl
@@ -134,7 +134,7 @@ sub helpFormatText
$strText .= "\n";
}

# Escape perl special characters
# Escape special characters
$strLine =~ s/\"/\\"/g;

my $strPart;
@@ -151,8 +151,6 @@ use constant CFGOPT_OUTPUT => 'output';

# Command-line only local/remote options
#-----------------------------------------------------------------------------------------------------------------------------------
use constant CFGOPT_C => 'c';
push @EXPORT, qw(CFGOPT_C);
use constant CFGOPT_COMMAND => 'command';
push @EXPORT, qw(CFGOPT_COMMAND);
use constant CFGOPT_PROCESS => 'process';
@@ -1038,17 +1036,6 @@ my %hConfigDefine =

# Command-line only local/remote options
#-------------------------------------------------------------------------------------------------------------------------------
&CFGOPT_C =>
{
&CFGDEF_TYPE => CFGDEF_TYPE_BOOLEAN,
&CFGDEF_INTERNAL => true,
&CFGDEF_DEFAULT => false,
&CFGDEF_COMMAND =>
{
&CFGCMD_REMOTE => {},
}
},

&CFGOPT_COMMAND =>
{
&CFGDEF_TYPE => CFGDEF_TYPE_STRING,

This file was deleted.

@@ -14,6 +14,12 @@
<release-list>
<release date="XXXX-XX-XX" version="2.21dev" title="UNDER DEVELOPMENT">
<release-core-list>
<release-feature-list>
<release-item>
<p><backrest/> is now pure <proper>C</proper>.</p>
</release-item>
</release-feature-list>

<release-improvement-list>
<release-item>
<release-item-contributor-list>
@@ -83,13 +83,10 @@

<variable key="pg-group">postgres</variable>

<variable key="perl-lib-path">/usr/share/perl5</variable>
<variable key="perl-bin-path">/usr/bin</variable>

<variable key="backrest-repo-path">/var/lib/pgbackrest</variable>
<variable key="backrest-repo-cipher-type">aes-256-cbc</variable>
<variable key="backrest-repo-cipher-pass">zWaf6XtpjIVZC5444yXB+cgFDFl7MxGlgkZSaoPvTGirhPygu4jOKOXf9LO4vjfO</variable>
<variable key="br-bin">{[perl-bin-path]}/pgbackrest</variable>
<variable key="br-bin">/usr/bin/pgbackrest</variable>
<variable key="br-user">pgbackrest</variable>
<variable key="br-group">{[br-user]}</variable>
<variable key="br-home-path">/home/{[br-user]}</variable>
@@ -448,43 +445,37 @@
<block-define if="'{[package]}' eq 'none'" id="br-install">
<p><backrest/> needs to be installed from a package or installed manually as shown here.</p>

<execute-list host="{[br-install-host]}">
<title>Copy <backrest/> binary from build host</title>
<execute-list host="{[host-build]}">
<title>Install dependencies</title>

<execute user="root">
<exe-cmd>scp {[host-build]}:/build/pgbackrest-release-{[version]}/src/pgbackrest /usr/bin</exe-cmd>
<exe-cmd-extra>2>&amp;1</exe-cmd-extra>
<execute if="{[os-type-is-debian]}" user="root" pre="y">
<exe-cmd>
apt-get install postgresql-client libxml2</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
</execute>

<execute user="root">
<exe-cmd>chmod 755 /usr/bin/pgbackrest</exe-cmd>
<execute if="{[os-type-is-centos]}" user="root" pre="y">
<exe-cmd>
yum install postgresql-libs
</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
</execute>
</execute-list>

<!-- Perl installation -->
<p><backrest/> contains embedded Perl which requires some additional packages.</p>

<execute-list host="{[br-install-host]}">
<title>Install required Perl packages</title>

<execute if="{[os-type-is-debian]}" user="root" pre="y">
<exe-cmd>apt-get install perl</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
</execute>
<title>Copy <backrest/> binary from build host</title>

<execute if="{[os-type-is-centos6]}" user="root" pre="y">
<exe-cmd>yum install perl perl-Time-HiRes perl-parent perl-JSON
perl-Digest-SHA</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
<execute user="root">
<exe-cmd>scp {[host-build]}:/build/pgbackrest-release-{[version]}/src/pgbackrest /usr/bin</exe-cmd>
<exe-cmd-extra>2>&amp;1</exe-cmd-extra>
</execute>

<execute if="{[os-type-is-centos7]}" user="root" pre="y">
<exe-cmd>yum install perl perl-Time-HiRes perl-Digest-SHA perl-JSON-PP</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
<execute user="root">
<exe-cmd>chmod 755 /usr/bin/pgbackrest</exe-cmd>
</execute>
</execute-list>

<p>Finally, <backrest/> requires log and configuration directories and a configuration file.</p>
<p><backrest/> requires log and configuration directories and a configuration file.</p>

<execute-list host="{[br-install-host]}">
<title>Create <backrest/> configuration file and directories</title>
@@ -638,7 +629,7 @@

<p>This user guide is intended to be followed sequentially from beginning to end &amp;mdash; each section depends on the last. For example, the <link section="/backup">Backup</link> section relies on setup that is performed in the <link section="/quickstart">Quick Start</link> section. Once <backrest/> is up and running then skipping around is possible but following the user guide in order is recommended the first time through.</p>

<p>Although the examples are targeted at {[user-guide-os]} and <postgres/> {[pg-version]}, it should be fairly easy to apply this guide to any Unix distribution and <postgres/> version. The only OS-specific commands are those to create, start, stop, and drop <postgres/> clusters. The <backrest/> commands will be the same on any Unix system though the locations to install Perl libraries and executables may vary.
<p>Although the examples are targeted at {[user-guide-os]} and <postgres/> {[pg-version]}, it should be fairly easy to apply this guide to any Unix distribution and <postgres/> version. The only OS-specific commands are those to create, start, stop, and drop <postgres/> clusters. The <backrest/> commands will be the same on any Unix system though the location to install the executable may vary.

Configuration information and documentation for PostgreSQL can be found in the <postgres/> <link url='http://www.postgresql.org/docs/{[pg-version]}/static/index.html'>Manual</link>.</p>

@@ -766,42 +757,20 @@

<execute if="{[os-type-is-debian]}" user="root" pre="y">
<exe-cmd>
apt-get install build-essential libssl-dev libxml2-dev libperl-dev zlib1g-dev
libpq-dev</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
</execute>

<execute if="{[os-type-is-centos6]}" user="root" pre="y">
<exe-cmd>
yum install build-essential gcc openssl-devel libxml2-devel
postgresql-devel perl-ExtUtils-Embed
</exe-cmd>
apt-get install make gcc libpq-dev libssl-dev libxml2-dev</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
</execute>

<execute if="{[os-type-is-centos7]}" user="root" pre="y">
<execute if="{[os-type-is-centos]}" user="root" pre="y">
<exe-cmd>
yum install build-essential gcc make openssl-devel libxml2-devel
postgresql-devel perl-ExtUtils-Embed
yum install make gcc postgresql-devel openssl-devel libxml2-devel
</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
</execute>
</execute-list>

<p><backrest/> supports 32-bit distributions that build Perl with 64-bit integer support.</p>

<execute-list host="{[host-build]}">
<title>Check for 64-bit integers</title>

<execute output="y">
<exe-cmd>perl -V | grep USE_64_BIT_INT</exe-cmd>
</execute>
</execute-list>

<p>The <backrest/> executable is written in C. This allows certain time-critical commands (like async <cmd>archive-push</cmd>/<cmd>archive-get</cmd>) to run more quickly.</p>

<execute-list host="{[host-build]}">
<title>Build <backrest/> package</title>
<title>Compile <backrest/></title>

<execute>
<exe-cmd>cd /build/pgbackrest-release-{[version]}/src &amp;&amp; ./configure</exe-cmd>
@@ -22,7 +22,6 @@ use pgBackRest::Common::Log;
use pgBackRest::Config::Config;
use pgBackRest::InfoCommon;
use pgBackRest::Manifest;
use pgBackRest::Protocol::Helper;
use pgBackRest::Protocol::Storage::Helper;
use pgBackRest::Storage::Helper;

This file was deleted.

0 comments on commit f0ef73d

Please sign in to comment.
You can’t perform that action at this time.