Version 0.43 1/4/94

I. Here is the code for and description of "whatsnewd" and "findwhatsnew":

(Connect your Gopher to "gopher.eff.org", port "5070" to play with it.)

What it does:

When a user gives a date (e.g. "1 day ago") or enters a dated bookmark
(generated by a previous query), he or she gets a gopher menu of the
gopher items that are new or changed since that date.

The generated bookmarks allow a user to see just those items that
have been created or changed since his or her last query.

I won't have time to work any more on this for a while. But here is
the code, "as is". It consists of two programs:
   findwhatsnew - which updates a database of gopher items
   whatsnewd    - a gopher protocol server
Both are written in Perl.

You're are encouraged to add to or modify this code.
(And to improve this documentation.)

[Joaquim Baptista, px@fct.unl.pt, has done this creating "traveller" a
hybrid of "whatsnew" and "veronica" with other new features. To try
it, do "gopher -ptravel gopher.fct.unl.pt 4320". As of today,
the software is in boombox.micro.umn.edu:pub/gopher/incoming.]

-- Carl Kadie (kadie@eff.org) 09/21/93

II. ============== About findwhatsnew ==============

USAGE findwhatsnew NAME PATH SERVER PORT DEPTH
OR    findwhatsnew NAME

For example:
      findwhatsnew caf "1/academic" gopher.eff.org 70 3
      findwhatsnew caf

NAME can include a directory path. For example:

      findwhatsnew /net/kragar/serv/gopher/whatsnewd/data/caf "1/academic" gopher.eff.org 70 3

Notes:

1. You will eventually need to create a whatsnewd directory.
Ours is a subdirectory of our main "gopher" directory.
It should have three subdirectories:
      bin data logs
"data" is where "whatsnewd" expects to see the database
that "findwhatsnew" creates.

2. Run "findwhatsnew" first with a name and all the parameters

   findwhatsnew NAME PATH SERVER PORT DEPTH

After that, you need not give the parameters (unless you want to
change them). e.g.

    findwhatsnew NAME

will usually be enough.

3. The parameters

   NAME -- will become the short name used to make files.
   PATH -- a gopher path
   SERVER -- aka HOST
   PORT -- usually 70
   DEPTH -- start with 1 or 2 and slowly increase until
          you are sure you have your kill and keep files the way you want

4. Files (only NAME.kill is user created)

	NAME.times - text file. Each line is a time stamp and
                      bookmark. Here is an example line:

             19921208075043<tab>0cafv02n56<tab>0/academic/news/cafv02n56<tab>gopher.eff.org<tab>70

              Dates are of the form YYYYMMDDHHMMSS and are GMT.

	NAME.bak - the old version of NAME.times
        NAME.tmp - temporary version of NAME.times
            (NAME.times is replaced by NAME.tmp at the very end of processing.)
	NAME.kill - kills and keeps certain paths. Examples are enclosed.
	NAME.save - the parameters for the program.
        NAME.tree - the explored gopher tree (nice and indented)
			This is very handy for tuning the NAME.kill file.

5. Kill file:

A kill file is used to stop whatsnewd from searching places you don't
what it too look.

Comment lines in a kill file start with a "#". Blanks lines are OK,
too.

The four commands are:

kill PATTERN
keep PATTERN
kill-subs-of PATTERN
keep PATTERN

Where PATTERN is a Perl regular expression. A kill file
may contain many instances of each type of command.

Every time an item is found, these patterns are consulted. If the item
matches at least one "keep" pattern and doesn't match any "kill"
patterns, it is included in the database.  If an item matches at least
one "keeps-subs-of" pattern and doesn't machine any
"kill-subs-of" patterns, its subitems (if any) will be explored.

For the purpose of this pattern matching an item is described as
a string of the form:

"
Name0=NAME0VALUE
Port0=PORT0VALUE
Path0=PATH0VALUE
Host0=HOST0VALUE
Name=NAMEVALUE
Type=TYPEVALUE
Port=PORTVALUE
Path=PATHVALUE
Host=HOSTVALUE
"

Where all the fields ending in 0 have values for
the item's parent. And the non-0-ending fields have values
for the item.

6. Example kill files:

============== caf.kill ==============
# Stops it from exploring "whatsnew" stuff
kill-subs-of \nPort0=5070\n
# Stops it from exploring articles of the Campus Newspapers
kill-subs-of \nPath=1/academic/newspapers
# Don't look at subdirectories of items whose path starts "m/"
kill-subs-of \nPath=m/
# Don't follow gopher link for library material back to top of CAF
kill \nPath=1/academic/library/academic\n
# Keep only those items reached from gopher.eff.org 
keep \nHost0=gopher.eff.org\n
keep \nHost0=kragar.eff.org\n

============== cso.kill ===================
# Don't go more than one step about from a uiuc.edu machine.
keep \nHost0=.*\.uiuc\.edu\n

# Don't go from a department machine back to the main campus gopher machine
kill \nType=1\nPort=70\nPath=(1/)?\nHost=harpoon\.cso\.uiuc\.edu\n
kill \nType=1\nPort=70\nPath=(1/)?\nHost=gopher\.(cso\.)?uiuc\.edu\n
kill \nHost0=s\.psych\.uiuc\.edu\n(.|\n)*\nHost=harpoon\.cso.\uiuc\.edu\n
kill \nHost0=cyberdyne\.ece\.uiuc\.edu\n(.|\n)*\nHost=gopher.\uiuc\.edu\n
kill \nHost0=s\.psych\.uiuc\.edu\n(.|\n)*\nHost=gopher\.uiuc\.edu\n

# Kill an experimental branch
kill-subs-of \nPath=1/Information about Gopher/exp\n

# Stops it from exploring "whatsnew" bookmarks
kill \nPath=.* gophergmt\n

# Don't look at subdirectories of items whose path starts "m/"
kill-subs-of \nPath=m/

kill-subs-of \nPath=1/FTP\n
kill-subs-of \nPath=1/Phone Books\n
kill-subs-of \nPath=1/UI/Timetables\n

# Don't explore the Placement Office
kill \nPort=15000\n(.|\n)*\nHost=harpoon\.cso\.uiuc\.edu\n
# Don't explore the newspaper
kill \nPath=1/UI/DI\n
kill-subs-of \nPath=1/News/Daily Illini Newspaper\n
# Don't explore Student Employment
kill-subs-of \nPath=1/UI/FinAid/Student Employment\n

kill-subs-of \nPath=1/Manuals/Unix\n
kill \nPath=1/Manuals/SM/
kill-subs-of \nPath=1/Forecasts\n
kill-subs-of \nPath=1/Current Conditions\n
kill \nPath=ftp:
kill-subs-of \nPort=70\nPath=1/Forecast\nHost=wx\.atmos\.uiuc\.edu\n
kill-subs-of \nPort=70\nPath=1/Illinois\nHost=wx\.atmos\.uiuc\.edu\n
kill-subs-of \nPort=70\nPath=1/Images\nHost=wx\.atmos\.uiuc\.edu\n
kill-subs-of \nPort=70\nPath=1/Surface\nHost=wx\.atmos\.uiuc\.edu\n
kill-subs-of \nPort=70\nPath=1/Upper\nHost=wx\.atmos\.uiuc\.edu\n
kill-subs-of \nPort=70\nPath=1/Severe\nHost=wx\.atmos\.uiuc\.edu\n
kill-subs-of \nPort=70\nPath=1/Servers\nHost=wx\.atmos\.uiuc\.edu\n

# Don't look at individual articles in _Inside Illinois_
#  Kill both ways to get to the articles
kill-subs-of \nPath0=1/UI/II\n
kill-subs-of \nPath0=1/News/Inside Illinois, the Faculty-Staff Newspaper\n

# Don't look at individual books
kill-subs-of \nPath0=1/library\n

#Kill specific manuals and scripts
kill-subs-of \nPath0=1/Documents\nHost0=wx\.atmos\.uiuc\.edu\n


=======================================================

7. What exactly the program does

It reads the old database of time stamps and bookmarks. If it sees
bookmarks, identical except from the time stamp, it uses the older
time stamp.

It explores the gopher tree within the bounderies set by the depth
bound and the kill file. It remembers the Name/Port/Path/Host of the
items that it sees. Duplicates are not explored.

If it finds an item that is mentioned in the old database,
it puts the item in the new database with the old time stamp.

If it finds an item that is not mentioned in the old database,
it puts the item in the new database, stamped with the current time.

Items from the old database that are not found, are put in the new
database anyway. They are marked "Couldn't connect: ".  From time to
time, you may want to delete these items manually from the database
(*.times file).

III. ============== About whatsnewd ==============

USAGE: whatsnewd -lLOGFILE Datadir

"whatsnewd" is a gopher-protocol server. 

If a user gives a date (e.g. "1 day ago") or enters a dated bookmark
(generated by a previous query), he or she gets a gopher menu of the
gopher items that are new or changed since that date.

Its input is:
   DATADIR  - a directory containing *.times and *.save files
                  as created by "findwhatsnew"
   LOGFILE  - a log file
   stdin - a command (i.e. gopher path). For example:
      1\      : create a menu for every database you know about
      1\NAME  : create a menu for the NAME.times database
      0\NAME : create information about the NAME.times database
      7\NAME<tab>STRING : Answer a query about NAME
      1\NAME\STRING : same as 7\NAME<tab>STRING

Its output is:
    stdout - gopher menus or text items

2. You can play with the program before installing it as a daemon.
Just run it (don't forget parameters) and type into standard input.

3. Ultimately it should be installed as a network daemon. 
(My sysadmin, Chris Davis, ckd@eff.org, did this for me).

Port 5070 is suggested, but any port should work. Here is the line
from gopher.eff.org's "etc/inetd":

gwn     stream  tcp     nowait  gopher  /serv/gopher/whatsnewd/bin/whatsnewd whatsnewd -l/serv/gopher/whatsnewd/logs/whatsnewd.log /serv/gopher/whatsnewd/data

And here is the line from gopher.eff.org's "services":

services:gwn             5070/tcp        whatsnewd       # gopher what's new

Here is where we've but it's files:
  The program is:
       /serv/gopher/whatsnewd/bin/whatsnewd
   The log file is:
      /serv/gopher/whatsnewd/logs/whatsnewd.log
   And the *.times and *.save files live in:
     /serv/gopher/whatsnewd/data

IV.  The Code: ========== findwhatsnew =================
#!/usr/bin/perl
# !/local/all/perl


# History
#  July 24, 1993   - mark old saved items to make them easier to dump
#                     if two identical items are in the *.times file,
#                                  use the older date
#  July 7, 1993    - changed a bit to make work with new minor release of Perl
#  Jan 1, 1992      - save old items, even if not found
#  Dec 9, 1992	    - output *.tree file
#  Dec 3, 1992      - change to findwhatsnew
#  Nov 23, 1992 cmk - initial development of gopherdiff

# Bugs: if gopher can supply time, that should be used instead.

$sort = "/bin/sort";
$rm = "/usr/bin/rm";

$| = 1;

($cursec,$curmin,$curhour,$curmday,$curmon,$curyear,
 $curwday,$curyday,$curisdst)
 = gmtime(time);
$curmon++;
if ($curyear > 90) {$curyear = $curyear + 1900;}
elsif ($curyear < 20) {$curyear = $curyear + 2000;}
else {die "Sorry, limited to years between 1990 and 2020";}

$curdate = sprintf("%.4d%.2d%.2d%.2d%.2d%.2d",$curyear,$curmon,$curmday,$curhour,$curmin,$cursec);

$cannotconnect = "Couldn't connect: ";
($gopherdir,$name) = @ARGV[0] =~ /(.*)([^\/]*)/;
#print STDERR "< $gopherdir $name >\n";

$bakfile = "$gopherdir$name.bak";
$oldfile = "$gopherdir$name.times";
if (open(OLD,"<$oldfile")) {
	while (<OLD>) {
		($date2,$tname2,$rest2) = /^([^\t]*)\t([^\t]*)\t(.*).$/;
		#print STDERR
                 #  "date2=$date2\n\ttname2=$tname2\n\trest2=$rest2\n";
		$rest2 =~ s/^($cannotconnect)+//;
		$tname2 =~ s/^($cannotconnect)+//;
		#print STDERR
                 #  "date2=$date2\n\ttname2=$tname2\n\trest2=$rest2\n";
	        $rem2 = @REMEM{$rest2};
                #print "before:$rest2->@REMEM{$rest2}\n";
                 # if new "tag" then just remember
	        if ($rem2 eq "") {
   	          @REMEM{$rest2} = "$date2\t$tname2";
                } else { # if have seen "tag" before keep older tag
		($date22,$tname22) = split("\t",$rem2);
                     #print "$date2<$date22\n";
                   if ($date2<$date22) {
        	          @REMEM{$rest2} = "$date2\t$tname2";
                     }
                }
                #print "after:$rest2->@REMEM{$rest2}\n";
	}
	close(OLD);
	}
#print STDERR "OLD is loaded\n";

$newfile = "$gopherdir$name.tmp";
open(NEW,">$newfile")||die("Cannot open $newfile");
$oldhandle = select(NEW); $| = 1; select ($oldhandle);

$treefile = "$gopherdir$name.tree";
open(TREE,">$treefile")||die("Cannot open $treefile");
$oldhandle = select(TREE); $| = 1; select ($oldhandle);

@kills = &LOAD_KILL_FILE($gopherdir,$name);

$save = "$gopherdir$name.save";
if ($#ARGV == 4) {
	open (SAVE,">$save");
	print SAVE join("\n",@ARGV);
	print SAVE "\n";
	close(SAVE);
  } elsif ($#ARGV == 0) {
	@ARGV=();
	open (SAVE,"<$save");
	while(<SAVE>){chop; push(@ARGV,$_);}
	close(SAVE);
   } else {
     print "USAGE findwhatsnew NAME PATH SERVER PORT DEPTH\n";
     print "OR    findwhatsnew NAME\n";
     exit;
       }
  $oridepth = @ARGV[4];
  shift;
  $start_gopher = "gopher -p \"@ARGV[0]\" @ARGV[1] @ARGV[2]";
  push(@ARGV,"");
  &RECUR(@ARGV);
  # output old items, not seen this time
  foreach $key9 (keys(%REMEM)) {
     $rem9 = @REMEM{$key9};
    if ($rem9 ne "1") {
     $rem9 =~ s/\t/\t$cannotconnect/;
     print NEW "$rem9\t$key9\r\n";
      }
     }
  unlink($bakfile) if -e $bakfile;
  rename($oldfile,$bakfile);
  close;
  exec("$sort < $newfile > $oldfile;$rm $newfile");

sub RECUR {
local($path,$server,$port,$depth) = @_;
local($newtype,$newline,$name1,$path1,$server1,$port1);
local($dir,$rem,$c);

#print STDERR "<< $server,$port,$path,$depth >>\n";

$dir = &CLIENT($server,$port,"$path\r\n");

#print STDERR "<2> $dir\n";
foreach $_ (split("\n",$dir)) {
	#print STDERR $_;
	$item = $_;
	($type1,$name1,$rest1,$path1,$server1,$port1) 
           = /^(.)([^\t]*)\t(([^\t]*)\t([^\t]*)\t([^\t]*).*)$/;
         $tname1= "$type1$name1";
	#print STDERR ">3>$type1,$name1,$rest1,$path1,$server1,$port1\n";
	#print $rest1;
	$rem = @REMEM{$rest1};
	($remdate,$remname) = split("\t",$rem);
	#print "<rem< $rem, $remdate, $remname >>\n";
	$killlong ="\nName0=$name\nPort0=$port1\nPath0=$path\nHost0=$server\nName=$name1\nType=$type1\nPort=$port1\nPath=$path1\nHost=$server1\n";
	study($killong);
	$*=1;
	$k1 = !(@kills{'kill'} && ($killlong =~ /@kills{'kill'}/));
	$k2 = (!@kills{'keep'} || ($killlong =~ /@kills{'keep'}/));
	#print STDERR ">4>Killlong=$killlong\n";
 	#print STDERR ">5>don't kill=$k1. keep=$k2 rem=$rem\n";
	if ($k1 && $k2 && $rem != 1) {
            $*=0;
		#print STDERR ">6>\n";
 	    if ($remname ne $tname1){
		print NEW "$curdate\t$item\r\n";
		} else {
 		print NEW "$remdate\t$item\r\n";
		}
		#print STDERR ">7>\n";
	    $indent = ($oridepth - $depth) * 3 + 1;
	    printf(TREE "%${indent}d:%s\n",$depth,$item);
	    @REMEM{$rest1} = 1;
		#print STDERR ">8>\n";
	   if ($type1 ==1) {
		$k1s = !(@kills{'kill-subs-of'} &&
                             ($killlong =~ /@kills{'kill-subs-of'}/));
		$k2s = (!@kills{'keep'-subs-of} || 
                             ($killlong =~ /@kills{'keep-sub-of'}/));
    	        #print "don't kill sub=$k1s. keep sub=$k2s\n";
		#print STDERR ">9>\n";
		if ($k1s && $k2s) {
			if ($depth >0) {
	 		do &RECUR($path1,$server1,$port1,$depth-1);
			} else {
			   #print STDERR "Depth bound reached. Will not explore:\n";
			   #print STDERR "$name1\n	$path1	$server1	$port1\n\n";
			}}
		#print STDERR ">10>\n";
		}}
		#print STDERR ">11>\n";
       $*=0;
}
1;
}


# Based on "client" on page 344 of _Programming Perl_ by Wall and Schwartz

sub CLIENT {

local($them,$port,$input) = @_;
local($client);

if ($them eq "error.host") { return("");}

$port = 2345 unless $port;
$them = 'localhost' unless $them;

$AF_INET =2;
$SOCK_STREAM = 1;

$SIG{'INT'} = 'dokill';

$sockaddr = 'S n a4 x8';

chop($hostname = `hostname`);

($name,$aliases,$proto) = getprotobyname('tcp');
($name,$aliases,$port) = getservbyname($port,'tcp')
     unless $port =~ /^\d+$/;;
($name,$aliases,$type,$len,$thisaddr) = gethostbyname($hostname);
($name,$aliases,$type,$len,$thataddr) = gethostbyname($them);

$this = pack($sockaddr,$AF_INET, 0, $thisaddr);
$that = pack($sockaddr,$AF_INET, $port, $thataddr);

# Make the socket filehandle
socket(S, $AF_INET, $SOCK_STREAM, $proto)|| die $!;


# Give the socket an address.
bind(S, $this) || die $!;

# Call up the server.
if (connect(S,$that)) {
	#print "connect ok\n";
}
else {
    #print STDERR "CANNOT CONNECT TO $them $port $input $!\n";
	return("");
}

# Set socket to be command buffered.

$oldhandle = select(S); $| = 1; select ($oldhandle);

#print STDERR $input;
print S $input;

READ: while (<S>) {
        #print STDERR;
	chop;chop;
	last READ if ($_ eq ".");
	$client .= "$_\n";
	}
#print STDERR "done with client\n";
return($client);
}

sub LOAD_KILL_FILE {
	local($gopherdir,$name) = @_;
	local($killfile) = "$gopherdir$name.kill";
	local(@kills) = ();
	@kills{'kill'} = '0'; @kills{'kill-subs-of'} = '0';
	@kills{'keep'} = '0'; @kills{'keep-subs-of'} = '0';
	if (open(KILL,"<$killfile")) {
	while (<KILL>) {
		if (!/^\s*(#.*)?$/) {
		/(\S+) (.*)/;
		chop;
		local($so_far) = @kills{$1};
		if ($so_far eq '') {die "Kill file command $1 not recognized";}
		if ($so_far eq '0') 
			{@kills{$1} = '';} else {@kills{$1} .= '|';}
		@kills{$1} .= $2;
		#print "kills{$1} = @kills{$1}\n";
		}}
	}
	return(@kills);
	}

V. More code: ======= whatsnewd ==========
#!/usr/bin/perl

# History
#  Jan 4, 1993 -- allow "+" to be used in place of " " (for www compatiblity)
#  July 29, 1993 - adding code to allow it to work with sequence numbers
#                    (but sequence numbers aren't used)
#  Dec 8, 1992 - fixed bug cause by limit and date interaction
#  Dec 3, 1992  - Carl M. Kadie - initial development

# Bugs and Limitations:
#       Does a linear, rather than binary search
#       Only works with GMT and relative times, no local times
#       The termcap gopher client will highlight words from search string

# Assumptions:
# 	Assumes item list is sorted.
#            (this is used to find date for new bookmark)

# Possibilities:
#      Could findwhatsnew could remember menu selection paths
#      Could use sequence numbers rather than dates.

require "getopts.pl";
&Getopts("l:");

if ($opt_l) {
	$logfile = $opt_l;
} else {
	$logfile = "/dev/null";
}


if ($#ARGV == 0) {
	($whatsnewdir) = @ARGV;
   } else {
     print "USAGE: whatsnewd -lLOGFILE Datadir\n";
     exit;
       }

$host = `hostname`;
$default_limit = 100;
$oldext = "times";
$saveext = "save";
chop($host);

@seconds{'s'} = 1;
@seconds{'m'} = @seconds{'s'} * 60;
@seconds{'h'} = @seconds{'m'} * 60;
@seconds{'d'} = @seconds{'h'} * 24;
@seconds{'w'} = @seconds{'d'} * 7;

&SERVER($port);
#&TESTSERVER($port);

sub TESTSERVER {
	$input = <STDIN>;
	chop($input);
	open(NS,">del");
	&WHATSNEW($input);
}

sub SERVER {

	$sockaddr = 'S n a4 x8';
	$mysockaddr = getsockname(STDIN);
	$remsockaddr = getpeername(STDIN);
	if ($mysockaddr) {
		($family,$myport,$myaddr) 
					= unpack($sockaddr,$mysockaddr);
		($remfamily,$remport,$remaddr) 
					= unpack($sockaddr,$remsockaddr);
		($theirname) = gethostbyaddr($remaddr,2);
		} else {
		$myport = 5070;
		$theirname = "justtesting.debug";
		}
	$input = <STDIN>;
	chop($input);
	$input =~ s/\r$//; # remove trailing \r if any
        $input =~ s/\+/ /g;  # change +'s to blanks
        $input =~ s/\?/\t/g;  # change ?'s to tabs
	#print $input;
	&WHATSNEW($input);
}

sub WHATSNEW {

($path,$search) = split("\t",@_[0]);
($type,$short_name,$style) = split("/",$path);
if ($type eq '') {$type = 1;}
if (($type ==1) && ($short_name eq '' || $short_name eq '.')) {
	opendir(DIR,$whatsnewdir) || die ($whatsnewdir,": ",$!);
	@timefiles = grep(/\.$oldext$/,readdir(DIR));
	foreach $timefile (@timefiles) {
		$timefile =~ /(.*)\.$oldext$/;
		local($short_name) = $1;
		local($path1,$host1,$port1) = &GET_GOPHER_REF($1);
		local($date) = &LASTDATE($short_name);
		&PRINT_EXTRA($short_name,$path1,$host1,$port1,$date,0,"about,whatsnew,top");
		}
	closedir(DIR);
    } else {
   local($path1,$host1,$port1) = &GET_GOPHER_REF($short_name);

  if ($type == 0 && $style eq '') {
	$curdate = &TIME2GOPHER(time,1);
	open(OLD,"<$whatsnewdir/$short_name.$oldext") || die "$whatsnewdir/$short_name.$oldext: ",$!;
	$_ = <OLD>;
	# what if empty?
	close(OLD);
	($oldestdate) = /^([^\t]*)\t/;
	$recentdate = &LASTDATE($short_name);
	&LOG_MESS("generated info on $short_name");
print "About \"What's New in $short_name\":\r
\r
To search for items that have been created or changed\r
since some date, give that date as the search string.\r
\r
Dates can be specified in two formats. For example:\r
     1 day 35 min ago\r
and\r
     19921104040013 gophergmt\r
\r
In general the formats are:\r
    {W w{weeks}} {D d{ays}} {H h{hours}} {M m{inutes}} {S s{econds}} ago\r
and\r
    YYYYMMDDHHMMSS gophergmt\r
\r
The last three items listed will be:\r
	**About \"What's New in $short_name\"\r
	**The top of the \"$short_name\"'\r
	**Bookmark for future new \"$short_name\" items (...)\r
\r
If you save the bookmark now and \"enter\" it in the the future, it will\r
display the items that have change between now until then.\r
\r
==================Some Status Info ==========================\r
   The current date is $curdate gophergmt. The date of the oldest\r
$short_name item is $oldestdate gophergmt. The date of the most recent\r
$short_name item is $recentdate gophergmt.\r
.\r\n";
    } elsif ($type == 0) {
print "\r
The default limit on the number of items returned is $default_limit.\r
To change this, add, for example, \"30 limit\" or \"no limit\" to your\r
search string.\r
.\r\n";
    } elsif ($type == 1 && $style eq "") {
	$date = &LASTDATE($short_name);
	&PRINT_EXTRA($short_name,$path1,$host1,$port1,$date,0,"about,whatsnew,top");
    } elsif ($type == 1 || $type == 7){
 	#print "style=$style, search=$search\n";
	if ($type == 1) {$search = $style;}
	#print "style=$style, search=$search\n";
	($since,$limit) = &PROCESS_DATE($search);
	#print "since=$since, limit=$limit\n";
	$date = $since;
	#$last = &LASTDATE($short_name);
	open(OLD,"<$whatsnewdir/$short_name.$oldext") || die "$whatsnewdir/$short_name.$oldext: ",$!;
	#print "limit=$limit\n";
	local($ll) = $limit;
	&LOG_MESS("searched $short_name with @_[0]");
	LOOP5: while(<OLD>) {
            if (/^[0-9]{14,14}\t/) { # old format
		($date,$rest) = /^([^\t]*)\t(.*)/;
                } else {
		($seq,$date,$rest) = /^([^\t]*)\t([^\t]*)\t(.*)/;
                 }
		# print "$date > $search ? \n";
		if ($date >$since) {
			$ll --;
			#print "ll=$ll\n";
			last LOOP5 if $ll == -1;
 			print "$rest\n";
                    }
		}
	if ($ll == -1) {
	print "0**Limit of $limit Reached. Select for more information	0/$short_name/limit	$host	$myport\r\n";
	$date = &LASTDATE($short_name);
	}
	#print "date=$date\n";
	if ($type==1) {
	&PRINT_EXTRA($short_name,$path1,$host1,$port1,$date,1,"about,whatsnew,top,bookmark");
	} else {
	&PRINT_EXTRA($short_name,$path1,$host1,$port1,$date,1,"about,top,bookmark");
	print ".\r\n";}
        }}

#    Binary search -- coding not finished
#	$filelen = (stat(OLD))[7];
#	$lo = 0;
#	$hi = $filelen - 3;
#	NARROW: while (1) {
#		last NARROW if $lo == $hi;
#		$mid = int(($lo + $hi) /2);
#		($date1,$date2) = &RETURN_TIME($mid);
#		if ($date1 )}

sub RETURN_TIME {
	local($pos) = @_;
	local($c,$date);
	BACK: for (; $pos >= 0; $pos--){
		seek(OLD,$pos,0);
		read(OLD,$c,1);
                if ($c eq "\n") {
			$pos++;
			last BACK;
			}
		}
 	seek(OLD,$pos,0);
	$_ = <OLD>;
	($date1) = /([0-9]{14,14})\t/;
	return($date1);
	}
}

sub PROCESS_DATE {
	@command = split(" ",@_[0]);
	$key = pop(@command);
	#print "key=$key\n";

	$limit = $default_limit;
	if ($key =~ /^limit$/i) {
		$limit = pop(@command);
		$limit = int($limit);
		if ($limit < 0) {$limit = 0;}
		if ($limit == "no") {$limit = -1;}
		$key = pop(@command);
		}
	if ($key =~ /^gophergmt$/i) {
		return(@command[0],$limit);
	} elsif ($key =~ /^ago$/i){
	local($time) = time();
	LOOP1: while(@command) {
		$num = shift(@command);
		$unit = shift(@command);
		#print ">2>",substr($unit,0,1),"<2<\n";
		$factor = @seconds{substr($unit,0,1)};
		if (! $factor) {
			&LOG_MESS("Don't know unit \"$unit\"");
			die("Don't know unit \"$unit\"");
			}
		$time = $time - $num * $factor;
		#print ">3>$time<3<\n";
		}
	if ($time < 0) {$time = 0;}
	local($gopht) = &TIME2GOPHER($time,1);
	#print ">4>$gopht<4<\n";
	return ($gopht,$limit);
	}
	else {
		&LOG_MESS("I don't know time format \"$key\"");
		die("I don't know time format \"$key\"");
		}
 }
	
sub TIME2GOPHER {
	#print "time=$time\n";
	local($time,$gmt_p) = @_;
        local($cursec,$curmin,$curhour,$curmday,$curmon,$curyear,
	      $curwday,$curyday,$curisdst);
	if ($gmt_p) {
	     ($cursec,$curmin,$curhour,$curmday,$curmon,$curyear,
	      $curwday,$curyday,$curisdst)
	        = gmtime($time);
	   } else {
	     ($cursec,$curmin,$curhour,$curmday,$curmon,$curyear,
	      $curwday,$curyday,$curisdst)
	        = localtime($time);
		}
	#print ">1>",gmtime(time),"<1<\n";
	$curmon++;
     if ($curyear >= 70) {$curyear = $curyear + 1900;}
          elsif ($curyear < 20) {$curyear = $curyear + 2000;}
          else {die "Sorry, limited to years between 1970 and 2020";}
     return(sprintf("%.4d%.2d%.2d%.2d%.2d%.2d",$curyear,$curmon,
                      $curmday,$curhour,$curmin,$cursec));
        }

sub LASTDATE {
	local($short_name) = @_;
	open(OLD,"<$whatsnewdir/$short_name.$oldext") || die "$whatsnewdir/$short_name.$oldext: ",$!;
	local($filelen) = (stat(OLD))[7];
	local($date) = &RETURN_TIME($filelen-3);
	return($date);
}

sub LOG_MESS {
	local($date) = `date`;
	chop($date);
	$date =~ s/[^ ]* ([0-9]+)$/$1/;# remove timezone info
	open(LOG,">>$logfile")||die("$logfile: $!");
	flock(LOG,2);
	seek(LOG,0,2); # in case someone appended while we waited
	print LOG "$date $$ $theirname : @_[0]\n";
	flock(LOG,8);
	close(LOG);
	}

sub PRINT_EXTRA {
	local($short_name,$path1,$host1,$port1,$date,$star_p,$include) =@_;
	#print "datebpe=$date\n";

	if (index($include,"about") >= $[){
	if ($star_p) {print "0**";} else {print "0";}
	print "About \"What's New in $short_name?\"	0/$short_name	$host	$myport\r\n";}

	if (index($include,"whatsnew") >= $[){
	if ($star_p) {print "7**";} else {print "7";}
	print "What's New in $short_name? (e.g.: 1 day 2 hours ago)	7/$short_name	$host	$myport\r\n";}

	if (index($include,"top") >= $[){
	if ($star_p) {print "1**";} else {print "1";}
	print "Top of \"$short_name\"	$path1	$host1	$port1\r\n";}

	if (index($include,"bookmark") >= $[){ 
	if ($star_p) {print "1**";} else {print "1";}
	print "Bookmark for future new \"$short_name\" items ($date gophergmt)	1/$short_name/$date gophergmt	$host	$myport\r\n";}
	}

sub GET_GOPHER_REF {
	local($short_name) = @_;
	#print $short_name,"\n";
	#print "$whatsnewdir/$short_name.$saveext\n";
	if (! open(SAVE,"<$whatsnewdir/$short_name.$saveext")) {
	       &LOG_MESS("Can't find file $whatsnewdir/$short_name.$saveext");
               die $!;
	 }
	<SAVE>;
	$path1 = <SAVE>; chop($path1);
	$host1 = <SAVE>; chop($host1);
	$port1 = <SAVE>; chop($port1);
	close(SAVE);
	#print "$path1,$host1,$port1\n";
	return($path1,$host1,$port1);
	}