Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

* Add new cron-style facility for determining HouseKeeping jobs.

* Default is no change, i.e. no cron.

* The recommended method to add the file is:

	HouseKeepingCron  <crontab

  That will use the file etc/lib/crontab by default in the tarball,
  or /etc/interchangec/crontab in an LSB configuration.

* Requires the Set::Crontab module, which has been added to
  Bundle::Interchange.

* Structure of the crontab file is just like crontab(5) in UNIX
  except that a seconds column is added.

  The targets are GlobalSub or anything which you can make run
  with Vend::Dispatch::run_macro. Bear in mind there is no
  catalog context.

  Two special targets exist, :reconfig and :jobs. They allow calling
  of the catalog reconfig routines and jobs routines, respectively.
  The etc/reconfig and etc/jobsqueue files will be ignored if these
  targets are not present -- a warning will be issued at startup
  (and crontab change) if they are not there.

  A target prepended with > runs *after* the reconfig/restart/jobs/pid
  mgmt cycle. Normal specifications run before.

  The basic entry to implement "HouseKeeping 5" would be:

     HouseKeeping 1
	 HouseKeepingCron <<EOC
	 */5 * * * * * :restart
	 */5 * * * * * :jobs
	 EOC

  (Note that would normally be in etc/lib/crontab or /etc/interchange/crontab.)

  To only check the jobs queue every five minutes (on the minute), you
  do:

	 */5 * * * * * :restart
	 0 */5 * * * * :jobs

  If you want to run the GlobalSub "checkit" once a day at 4am, you would
  do:

	 0 0 4 * * * checkit

* If you set HouseKeeping to a granularity besides 1 (or if for some
  reason Interchange skips a second), it does the cron check for
  every intervening second. This ensures a job will not be skipped.
  The :restart and :jobs entries will only run once, but if you have
  a frequent GlobalSub job that pushes the granularity of HouseKeeping
  it can be run twice in succession.

* WARNING: You should not put long-running jobs in a GlobalSub! You have
  been warned. Use the Jobs facility for that.

* Probably should implement the ability to call out jobs, but not quite
  sure how to specify and do. Can we just call run_jobs() directly?
  If so, then maybe an = sign introduces a job:

  	0 0 * * * * =standard_cat hourly
  	0 0 4 * * * =standard_cat daily
  	0 0 2 * * 7 =standard_cat weekly

* Include bin/crontab script to edit the crontab and submit to the
  running IC daemon. BUG: Cannot run as root.
  • Loading branch information...
commit ab7a69c1aa4ac4d17c0b33826cece1e996a657a9 1 parent 23ce74e
@perusionmike perusionmike authored
View
4 MANIFEST
@@ -298,6 +298,7 @@ code/Widget/realvalue.widget
code/Widget/select.widget
code/Widget/show.widget
code/Widget/text.widget
+code/Widget/time.widget
code/Widget/uploadblob.widget
code/Widget/uploadhelper.widget
code/Widget/value.widget
@@ -1034,6 +1035,7 @@ lib/Vend/Cart.pm
lib/Vend/Config.pm
lib/Vend/Control.pm
lib/Vend/CounterFile.pm
+lib/Vend/Cron.pm
lib/Vend/Data.pm
lib/Vend/DbSearch.pm
lib/Vend/Dispatch.pm
@@ -1117,6 +1119,7 @@ relocate.pl
scripts/compile_link.PL
scripts/config_prog.PL
scripts/configdump.PL
+scripts/crontab.PL
scripts/dump.PL
scripts/expire.PL
scripts/expireall.PL
@@ -1211,3 +1214,4 @@ WHATSNEW-4.7
WHATSNEW-4.9
WHATSNEW-5.1
WHATSNEW-5.3
+META.yml Module meta-data (added by MakeMaker)
View
1  Makefile.PL
@@ -305,6 +305,7 @@ sub initialize {
compile_link
config_prog
configdump
+ crontab
dump
expire
expireall
View
0  dist/etc/lib/crontab
No changes.
View
13 lib/Vend/Config.pm
@@ -1,6 +1,6 @@
# Vend::Config - Configure Interchange
#
-# $Id: Config.pm,v 2.174 2005-05-12 17:54:37 mheins Exp $
+# $Id: Config.pm,v 2.175 2005-05-16 21:22:28 mheins Exp $
#
# Copyright (C) 2002-2003 Interchange Development Group
# Copyright (C) 1996-2002 Red Hat, Inc.
@@ -50,8 +50,9 @@ use Vend::Parse;
use Vend::Util;
use Vend::File;
use Vend::Data;
+use Vend::Cron;
-$VERSION = substr(q$Revision: 2.174 $, 10);
+$VERSION = substr(q$Revision: 2.175 $, 10);
my %CDname;
my %CPname;
@@ -443,6 +444,7 @@ sub global_directives {
['Jobs', 'hash', 'MaxLifetime 600 MaxServers 1'],
['IPCsocket', undef, "$Global::VendRoot/etc/socket.ipc"],
['HouseKeeping', 'time', 60],
+ ['HouseKeepingCron', 'cron', ''],
['Mall', 'yesno', 'No'],
['TagGroup', 'tag_group', $StdTags],
['TagInclude', 'tag_include', 'ALL'],
@@ -3541,6 +3543,13 @@ sub parse_time {
$n;
}
+sub parse_cron {
+ my($var, $value) = @_;
+
+ return '' unless $value =~ /\s/ and $value =~ /[a-zA-Z]/;
+ return Vend::Cron::read_cron($value);
+}
+
# Determine catalog structure from Catalog config line(s)
sub parse_catalog {
my ($var, $setting) = @_;
View
163 lib/Vend/Cron.pm
@@ -0,0 +1,163 @@
+# Vend::Cron - Determine tasks to run based on time
+#
+# $Id: Cron.pm,v 2.1 2005-05-16 21:22:28 mheins Exp $
+#
+# Copyright (C) 2002-2005 Interchange Development Group
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public
+# License along with this program; if not, write to the Free
+# Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
+# MA 02111-1307 USA.
+
+package Vend::Cron;
+
+use vars qw($VERSION);
+$VERSION = substr(q$Revision: 2.1 $, 10);
+
+use POSIX qw(strftime);
+use Vend::Util;
+use Text::ParseWords;
+use strict;
+
+no warnings qw(uninitialized);
+
+use Set::Crontab;
+
+my @periods = (
+ [0 .. 59],
+ [0 .. 59],
+ [0 .. 23],
+ [1 .. 31],
+ [1 .. 12],
+ [0 .. 7],
+);
+
+sub read_cron {
+ my $lines = shift;
+#::logDebug("read_cron reading $lines") unless $Vend::Quiet;
+ my @lines = grep /\w/, split /\n/, $lines;
+ @lines = grep $_ !~ /^\s*#/, @lines;
+
+ my @cronobj;
+ for(@lines) {
+ s/\s+$//;
+ my @ary = split /\s+/, $_, 7;
+
+ if(scalar(@ary) < 7) {
+ die "Bad cron entry '$_', not right number of time specifications.";
+ }
+
+ my $thing = pop @ary;
+ if($thing !~ /[a-zA-Z]/) {
+ die "Bad cron entry '$_', no job specification.";
+ }
+ my @times;
+ for(my $i = 0; $i < @ary; $i++) {
+ $times[$i] = Set::Crontab->new($ary[$i], $periods[$i]);
+ }
+ push @cronobj, {
+ times => \@times,
+ things => [ Text::ParseWords::shellwords($thing) ],
+ original => $_,
+ };
+ }
+
+ my %wanted = qw/ :reconfig 1 :jobs 1 /;
+ for(@cronobj) {
+ my $things = $_->{things};
+ for(@$things) {
+ next unless $wanted{$_};
+ delete $wanted{$_};
+ }
+ }
+
+ for(keys %wanted) {
+ ::logGlobal("WARNING: suggested cron entry '%s' not present.", $_)
+ unless $Vend::Quiet;
+ }
+ my $obj = \@cronobj;
+#::logDebug("read_cron returning $obj") unless $Vend::Quiet;
+ return $obj;
+}
+
+sub cron {
+ my $jobspec = shift;
+ my $time = shift || time;
+
+ my @todo;
+
+ my $from;
+
+ ## We initialize this baby to make sure run for every second
+ if( ref($jobspec->[-1]) ) {
+ push @$jobspec, $time;
+ }
+
+ $from = $jobspec->[-1];
+ $jobspec->[-1] = $time + 1;
+
+#::logDebug("doing run for $from .. $time");
+ for my $runtime ($from .. $time) {
+ my @made_cut = @$jobspec;
+ pop @made_cut;
+ my @t = localtime($runtime);
+ $t[4]++;
+ splice @t, 5, 1;
+
+ for my $n (0 .. 5) {
+ my @try = splice @made_cut;
+ for(@try) {
+ $_->{times}->[$n]->contains($t[$n])
+ and push @made_cut, $_;
+ }
+ last unless @made_cut;
+ }
+ push @todo, @made_cut;
+ }
+
+ my %do;
+ my @do_before;
+ my @do_after;
+
+ my $date = POSIX::strftime("time=%H:%M:%S", localtime($time));
+ for my $obj (@todo) {
+ for(@{$obj->{things}}) {
+#::logDebug("$date spawns $_ from $obj->{original}");
+ my $j = $_;
+ if($j =~ s/^://) {
+ $do{$j} = 1;
+ }
+ elsif($j =~ s/^>//) {
+ push @do_after, $j;
+ }
+ else {
+ $j =~ s/^<//;
+ push @do_before, $j;
+ }
+ }
+ }
+
+ my @out = \%do;
+ push @out, (scalar(@do_before) ? \@do_before : undef);
+ push @out, (scalar(@do_after) ? \@do_after : undef);
+ return @out;
+
+}
+
+sub housekeeping {
+ return cron($Global::HouseKeepingCron, shift(@_));
+}
+
+1;
+__END__
+
View
18 lib/Vend/Dispatch.pm
@@ -1,6 +1,6 @@
# Vend::Dispatch - Handle Interchange page requests
#
-# $Id: Dispatch.pm,v 1.53 2005-05-03 06:03:26 mheins Exp $
+# $Id: Dispatch.pm,v 1.54 2005-05-16 21:22:28 mheins Exp $
#
# Copyright (C) 2002-2003 Interchange Development Group
# Copyright (C) 2002 Mike Heins <mike@perusion.net>
@@ -26,7 +26,7 @@
package Vend::Dispatch;
use vars qw($VERSION);
-$VERSION = substr(q$Revision: 1.53 $, 10);
+$VERSION = substr(q$Revision: 1.54 $, 10);
use POSIX qw(strftime);
use Vend::Util;
@@ -1017,10 +1017,12 @@ EOF
)
{
-#::logDebug("deliver image: method=$CGI::request_method type=$mt");
my $imgdir = $Vend::Cfg->{ImageDir};
my $fn = $CGI::path_info;
+#::logDebug("deliver image: method=$CGI::request_method type=$mt fn=$fn");
$fn =~ s:^/+::;
+ ## Won't resend any images beginning with admin/
+ $fn =~ s{^admin/}{};
if($CGI::secure) {
$imgdir = $Vend::Cfg->{ImageDirSecure}
if $Vend::Cfg->{ImageDirSecure};
@@ -1110,7 +1112,15 @@ sub run_macro {
if ($m =~ /^\w+$/) {
my $sub = $Vend::Cfg->{Sub}{$m} || $Global::GlobalSub->{$m}
or do {
- logError("Unknown Autoload macro '%s'.", $m);
+ my $call = join(',', caller());
+
+ my $msg = errmsg("Unknown macro '%s' from %s.", $m, $call);
+ if($Vend::Cfg->{CatalogName}) {
+ logError($msg);
+ }
+ else {
+ logGlobal($msg);
+ }
next;
};
$sub->($content_ref);
View
55 lib/Vend/Server.pm
@@ -1,6 +1,6 @@
# Vend::Server - Listen for Interchange CGI requests as a background server
#
-# $Id: Server.pm,v 2.63 2005-05-12 17:52:51 mheins Exp $
+# $Id: Server.pm,v 2.64 2005-05-16 21:22:28 mheins Exp $
#
# Copyright (C) 2002-2003 Interchange Development Group
# Copyright (C) 1996-2002 Red Hat, Inc.
@@ -26,7 +26,7 @@
package Vend::Server;
use vars qw($VERSION);
-$VERSION = substr(q$Revision: 2.63 $, 10);
+$VERSION = substr(q$Revision: 2.64 $, 10);
use POSIX qw(setsid strftime);
use Vend::Util;
@@ -1017,6 +1017,23 @@ sub housekeeping {
#::logDebug("called housekeeping");
return if defined $interval and ($now - $Last_housekeeping < $interval);
+ my $do;
+ my $do_before;
+ my $do_after;
+
+#my $date = POSIX::strftime("time=%H:%M:%S", localtime($now));
+ if($Global::HouseKeepingCron) {
+ ($do, $do_before, $do_after) = Vend::Cron::housekeeping($now);
+#::logDebug("got housekeeping at $date do=" . ::uneval($do));
+ }
+ else {
+ $do = {
+ restart => 1,
+ reconfig => 1,
+ jobs => 1,
+ };
+ }
+
#::logDebug("actually doing housekeeping interval=$interval now=$now last=$Last_housekeeping");
rand();
$Last_housekeeping = $now;
@@ -1069,10 +1086,25 @@ sub housekeeping {
@files = readdir Vend::Server::CHECKRUN;
closedir(Vend::Server::CHECKRUN)
or die "closedir $Global::RunDir: $!\n";
- ($reconfig) = grep $_ eq 'reconfig', @files;
+ ($reconfig) = grep $_ eq 'reconfig', @files
+ if $do->{reconfig};
($restart) = grep $_ eq 'restart', @files
if $Signal_Restart || $Global::Windows;
- ($jobs) = grep $_ eq 'jobsqueue', @files;
+ ($jobs) = grep $_ eq 'jobsqueue', @files
+ if $do->{jobs};
+
+ if($do_before) {
+ for(@$do_before) {
+#::logDebug("run before macro $_");
+ eval {
+ Vend::Dispatch::run_macro($_);
+ };
+ if($@) {
+ ::logGlobal("cron before macro '%s' failed: %s", $_, $@);
+ }
+ }
+ }
+
if($Global::PIDcheck) {
$Num_servers = 0;
@pids = grep /^pid\.\d+$/, @files;
@@ -1101,6 +1133,7 @@ EOF
last;
}
chomp $value;
+#::logDebug("restart file reads value '$value'");
}
eval {
if($directive =~ /^\s*(sub)?catalog$/i) {
@@ -1121,7 +1154,7 @@ EOF
Vend::Config::code_from_file($directive, $value, 'nohup');
}
else {
- ::change_global_directive($directive, $value);
+ ::change_global_directive("$directive $value");
}
};
if($@) {
@@ -1248,6 +1281,18 @@ EOF
}
}
+ if($do_after) {
+ for(@$do_after) {
+#::logDebug("would run after macro $_");
+ eval {
+ Vend::Dispatch::run_macro($_);
+ };
+ if($@) {
+ ::logGlobal("cron after macro '%s' failed: %s", $_, $@);
+ }
+ }
+ }
+
if($respawn) {
if($Global::PreFork) {
# We need to respawn all the servers to pick up the new config
View
205 scripts/crontab.PL
@@ -0,0 +1,205 @@
+#!/usr/local/bin/perl
+##!~_~perlpath~_~
+#
+# Interchange cron editor
+#
+# $Id: crontab.PL,v 2.1 2005-05-16 21:22:28 mheins Exp $
+#
+# Copyright (C) 2005 Interchange Development Group
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public
+# License along with this program; if not, write to the Free
+# Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
+# MA 02111-1307 USA.
+
+use lib '/usr/local/interchange/lib';
+#use lib '~_~INSTALLPRIVLIB~_~';
+use lib '/usr/local/interchange';
+#use lib '~_~INSTALLARCHLIB~_~';
+
+BEGIN {
+
+ ($Global::VendRoot = $ENV{MINIVEND_ROOT})
+ if defined $ENV{MINIVEND_ROOT};
+ $Global::VendRoot = $Global::VendRoot || '/usr/local/interchange';
+# $Global::VendRoot = $Global::VendRoot || '~_~INSTALLARCHLIB~_~';
+
+ if(-f "$Global::VendRoot/interchange.cfg") {
+ $Global::ExeName = 'interchange';
+ $Global::ConfigFile = 'interchange.cfg';
+ }
+ else {
+ $Global::ExeName = 'interchange';
+ $Global::ConfigFile = 'interchange.cfg';
+ }
+
+}
+
+### END CONFIGURATION VARIABLES
+
+use POSIX qw/tmpnam/;
+use Vend::Cron;
+
+my $prospect = tmpnam();
+
+use File::Copy;
+use Term::ReadLine;
+use FindBin;
+
+my $term = new Term::ReadLine 'Simple';
+
+use Safe;
+my $safe = new Safe;
+
+my $configstring = `$FindBin::Bin/interchange -globalconfig`;
+
+chdir $Global::VendRoot
+ or die "Unable to chdir to $Global::VendRoot: $!\n";
+
+my $Global = $safe->reval($configstring)
+ or die "Unable to read configuration via $FindBin::Bin/interchange.\n";
+
+
+my $hupit = 1;
+
+if(! $Global->{HouseKeepingCron}) {
+ warn "No HouseKeepingCron is defined, this will do nothing.\n";
+ warn "Add:\n\n\tHouseKeepingCron <crontab\n\nto interchange.cfg to activate.\n";
+ undef $hupit;
+}
+
+$Vend::Quiet = 1;
+
+my $file = "$Global->{ConfigDir}/crontab";
+
+#warn "Cron file is $file\n";
+
+if(-f $file) {
+ File::Copy::copy($file, $prospect);
+}
+
+my $ed = $ENV{VISUAL} || $ENV{EDITOR} || 'vi';
+
+my @needed = qw/
+ :restart
+ :reconfig
+ :jobs
+/;
+
+EDCRON: {
+ system "$ed $prospect";
+
+ my $status = `diff $prospect $file`;
+
+ unless($?) {
+ warn "Crontab unchanged.\n";
+ exit;
+ }
+
+ open NEWCRON, "< $prospect"
+ or die "Cannot open prospective cron file: $!\n";
+
+ local($/);
+
+ my $lines = <NEWCRON>;
+ close NEWCRON;
+
+#warn "Read lines (" . length($lines) . " bytes)\n";
+
+ my $obj;
+ eval {
+ $obj = Vend::Cron::read_cron($lines);
+ };
+
+ if(! $obj or $@) {
+ print "Cron file problem: $@\n";
+ my $prompt = "Retry? [y]";
+ my $ans = $term->readline($prompt);
+ if($ans =~ /^\s*n/) {
+ last EDCRON;
+ }
+ else {
+ redo EDCRON;
+ }
+ }
+
+ my %wanted = qw/ :reconfig 1 :jobs 1 /;
+ for(@$obj) {
+ my $things = $_->{things};
+ for(@$things) {
+ next unless $wanted{$_};
+ delete $wanted{$_};
+ }
+ }
+
+ my @errmsg;
+ for(keys %wanted) {
+ push @errmsg, sprintf("WARNING: suggested cron entry '%s' not present.", $_);
+ }
+
+ if(@errmsg) {
+ print join "\n", @errmsg, '';
+ my $prompt = "Retry? [y]";
+ my $ans = $term->readline($prompt);
+ unless ($ans =~ /^\s*n/) {
+ redo EDCRON;
+ }
+ }
+
+
+#warn "read_cron returned $obj.\n";
+
+ File::Copy::copy($prospect, $file);
+ print "Wrote crontab file $file.\n";
+#warn "copied $prospect to $file\n";
+
+ unlink $prospect;
+ if($hupit) {
+
+ open CRON, "< $file"
+ or die "Can't read cron file $file: $!\n";
+ my $lines = <CRON>;
+ close CRON;
+
+ my $rsfile = "$Global->{RunDir}/restart";
+ open RESTART, ">> $rsfile"
+ or die "Cannot write restart file: $!\n";
+ print RESTART "HouseKeepingCron <<EndOfCrontab\n";
+ print RESTART $lines;
+ print RESTART "\nEndOfCrontab\n";
+ close RESTART;
+#warn "created restart file\n";
+
+ my $pidfile = $Global->{PIDfile};
+ unless (-f $pidfile) {
+ warn "Interchange not running, cannot tell to reread ($pidfile).\n";
+ }
+ open PID, "< $pidfile"
+ or die "Cannot read PID file $pidfile: $!\n";
+
+ my $pid = <PID>;
+ close PID;
+ $pid =~ s/\s+.*//s;
+ $pid =~ s/\D+//g;
+#warn "Found pid=$pid\n";
+ chomp $pid;
+ if ($pid) {
+ kill 'HUP', $pid;
+ print "Sent HUP signal to Interchange server at $pid.\n";
+ }
+ else {
+ die "Unable to find pid at $pidfile.\n";
+ }
+ }
+}
+
View
9 scripts/interchange.PL
@@ -3,7 +3,7 @@
#
# Interchange version 5.3.1
#
-# $Id: interchange.PL,v 2.84 2005-04-30 15:17:10 mheins Exp $
+# $Id: interchange.PL,v 2.85 2005-05-16 21:22:28 mheins Exp $
#
# Copyright (C) 2002-2005 Interchange Development Group
# Copyright (C) 1996-2002 Red Hat, Inc.
@@ -639,6 +639,7 @@ sub parse_options {
serve => $modesub,
test => $modesub,
+ globalconfig => sub { $Vend::Quiet = 1; $modesub->('globalconfig') },
unixmode => \$Global::Unix_Mode,
version => sub { version(); exit 0 },
stop => \&control_interchange,
@@ -709,6 +710,7 @@ sub parse_options {
email=s
exclude|e=s
help|h
+ globalconfig
inetmode|inet|i
jobgroup=s
kill:s
@@ -813,6 +815,11 @@ print errmsg("\n##### DEBUG MODE, running in foreground #####\n") if $Global::DE
# Read interchange.cfg (or whatever its name is set to be)
global_config();
+
+ if($Vend::mode eq 'globalconfig') {
+ print Vend::Util::uneval($Global::Structure);
+ exit;
+ }
# Select locking mode
set_lock_type();
Please sign in to comment.
Something went wrong with that request. Please try again.