Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better handling of symlink creation in Configure #13890

Closed
p5pRT opened this issue May 31, 2014 · 16 comments

Comments

@p5pRT
Copy link
Collaborator

@p5pRT p5pRT commented May 31, 2014

Migrated from rt.perl.org#122002 (status was 'resolved')

Searchable as RT122002$

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented May 31, 2014

From afresh1@openbsd.org

Created by afresh1@openbsd.org

This brings in improved symlinking of the build tree as as been used
by OpenBSD for the last couple of years.

Perl Info

Flags:
    category=install
    severity=low

Site configuration information for perl 5.18.2:

Configured by root at Thu Jan  1  0:00:00 UTC 1970.

Summary of my perl5 (revision 5 version 18 subversion 2) configuration:
   
  Platform:
    osname=openbsd, osvers=5.5, archname=amd64-openbsd
    uname='openbsd'
    config_args='-dsE -Dopenbsd_distribution=defined -Dccflags=-DNO_LOCALE_NUMERIC -DNO_LOCALE_COLLATE -Dmksymlinks'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-DNO_LOCALE_NUMERIC -DNO_LOCALE_COLLATE -fno-strict-aliasing -fno-delete-null-pointer-checks -pipe -fstack-protector -I/usr/local/include',
    optimize='-O2',
    cppflags='-DNO_LOCALE_NUMERIC -DNO_LOCALE_COLLATE -fno-strict-aliasing -fno-delete-null-pointer-checks -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.2.1 20070719 ', gccosandvers='openbsd5.5'
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags ='-Wl,-E  -fstack-protector'
    libpth=/usr/lib
    libs=-lm -lutil -lc
    perllibs=-lm -lutil -lc
    libc=/usr/lib/libc.so.74.2, so=so, useshrplib=true, libperl=libperl.so.15.0
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-R/usr/libdata/perl5/amd64-openbsd/5.18.2/CORE'
    cccdlflags='-DPIC -fPIC ', lddlflags='-shared -fPIC  -fstack-protector'

Locally applied patches:
    


@INC for perl 5.18.2:
    /usr/local/libdata/perl5/site_perl/amd64-openbsd
    /usr/libdata/perl5/site_perl/amd64-openbsd
    /usr/local/libdata/perl5/site_perl
    /usr/libdata/perl5/site_perl
    /usr/libdata/perl5/amd64-openbsd/5.18.2
    /usr/local/libdata/perl5/amd64-openbsd/5.18.2
    /usr/libdata/perl5
    /usr/local/libdata/perl5
    .


Environment for perl 5.18.2:
    HOME=/home/afresh1
    LANG (unset)
    LANGUAGE (unset)
    LC_CTYPE=en_US.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/afresh1/.plenv/bin:/home/afresh1/bin:/home/afresh1/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/usr/games:/home/afresh1/.plenv/bin:.
    PERL_BADLANG (unset)
    SHELL=/bin/ksh

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented May 31, 2014

From afresh1@openbsd.org

0001-Better-handling-of-symlink-creation.patch
From 10e4b04f4d8ef6c60f55544a64b7e6e13fc38bbf Mon Sep 17 00:00:00 2001
From: Andrew Fresh <afresh1@openbsd.org>
Date: Sat, 31 May 2014 15:37:52 -0700
Subject: [PATCH] Better handling of symlink creation

OpenBSD heavily uses the mksymlinks define to build outside of the src
tree.  This combines two commits from OpenBSD that improve the
robustness and speed of creating the symlinks.

From r1.24 by deraadt@:
> After the MANIFEST-based symbolic link creation loop, there is a check
> for a specific link to see if things worked out.  Add a check for the very
> last file in the MANIFEST, as well, since we are trying to spot a very
> odd bug where symbolic links are not being created.  Hopefully this will
> help us diagnose it.

http://www.openbsd.org/cgi-bin/cvsweb/src/gnu/usr.bin/perl/Configure#rev1.24

And from r1.25 by millert@:
> More efficient method of building the symlink tree that makes better
> use of awk.  Slightly faster and works around an apparent namei or
> buffer cache related bug on arm.

http://www.openbsd.org/cgi-bin/cvsweb/src/gnu/usr.bin/perl/Configure#rev1.25
---
 Configure | 35 ++++++++++++++---------------------
 1 file changed, 14 insertions(+), 21 deletions(-)

diff --git a/Configure b/Configure
index 5b3169b..93672cd 100755
--- a/Configure
+++ b/Configure
@@ -2795,38 +2795,31 @@ $define|true|[yY]*)
 	*)	case "$lns:$issymlink" in
 		*"ln"*" -s:"*"test -"?)
 			echo "Creating the symbolic links..." >&4
-			echo "(First creating the subdirectories...)" >&4
 			cd ..
-			awk '{print $1}' $src/MANIFEST | grep / | sed 's:/[^/]*$::' | sort -u | while true; do
-				read directory
-				test -z "$directory" && break
-				mkdir -p $directory
-			done
+			awk -v src="$src" '{
+				dir=$1;
+				if (!sub(/\/[^\/]*$/, "", dir)) { dir = "." }
+				mf[dir] = mf[dir]" "src"/"$1;
+			} END {
+				for (d in mf) {
+					if (d != ".") { system("mkdir -p "d) }
+					system("ln -sf "mf[d]" "d);
+				}
+			}' $src/MANIFEST
 			# Sanity check 1.
 			if test ! -d t/base; then
 				echo "Failed to create the subdirectories.  Aborting." >&4
 				exit 1
 			fi
-			echo "(Then creating the symlinks...)" >&4
-			awk '{print $1}' $src/MANIFEST | while true; do
-				read filename
-				test -z "$filename" && break
-				if test -f $filename; then
-					if $issymlink $filename; then
-						rm -f $filename
-					fi
-				fi
-				if test -f $filename; then
-					echo "$filename already exists, not symlinking."
-				else
-					ln -s $src/$filename $filename
-				fi
-			done
 			# Sanity check 2.
 			if test ! -f t/base/lex.t; then
 				echo "Failed to create the symlinks (t/base/lex.t missing).  Aborting." >&4
 				exit 1
 			fi
+			if test ! -f x2p/walk.c; then
+				echo "Failed to create the symlinks (x2p/walk.c missing).  Aborting." >&4
+				exit 1
+			fi
 			cd UU
 			;;
 		*)	echo "(I cannot figure out how to do symbolic links, ignoring mksymlinks)." >&4
-- 
1.9.2

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 1, 2014

From @jkeenan

On Sat May 31 15​:56​:30 2014, afresh1@​openbsd.org wrote​:

This is a bug report for perl from afresh1@​openbsd.org,
generated with the help of perlbug 1.39 running under perl 5.18.2.

-----------------------------------------------------------------
[Please describe your issue here]

This brings in improved symlinking of the build tree as as been used
by OpenBSD for the last couple of years.

This ticket would benefit from evaluation by our Configure experts and by anyone familiar with the various shell programs used in the patch.

I have pushed it for smoking to​: smoke-me/jkeenan/122002-configure-symlink

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 1, 2014

The RT System itself - Status changed from 'new' to 'open'

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 4, 2014

From @tonycoz

On Sat May 31 15​:56​:30 2014, afresh1@​openbsd.org wrote​:

This brings in improved symlinking of the build tree as as been used
by OpenBSD for the last couple of years.

This looks sane to me, except that the file you're testing, x2p/walk.c is no longer part of the perl tree.

Tony

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 4, 2014

From @doughera88

On Tue, Jun 03, 2014 at 05​:19​:37PM -0700, "Tony Cook via RT" wrote​:

On Sat May 31 15​:56​:30 2014, afresh1@​openbsd.org wrote​:

This brings in improved symlinking of the build tree as as been used
by OpenBSD for the last couple of years.

This looks sane to me, except that the file you're testing, x2p/walk.c is no longer part of the perl tree.

Unfortunately, it's "new" awk syntax (nawk) and not portable to 'old' awk.
On Solaris, at least, /usr/bin/awk is still the old awk. I haven't
looked very carefully, but the first thing I saw was the 'sub' function,
which is definitely new awk. The -v var=value stuff is also the new awk
syntax, but old awk can take var=value assignments after the program,
and I think at least some 'new' awks can as well, so that's probably
not a blocker. The 'sub' stuff is, however.

--
  Andy Dougherty doughera@​lafayette.edu

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 4, 2014

From @tonycoz

On Tue, Jun 03, 2014 at 09​:02​:08PM -0400, Andy Dougherty wrote​:

On Tue, Jun 03, 2014 at 05​:19​:37PM -0700, "Tony Cook via RT" wrote​:

On Sat May 31 15​:56​:30 2014, afresh1@​openbsd.org wrote​:

This brings in improved symlinking of the build tree as as been used
by OpenBSD for the last couple of years.

This looks sane to me, except that the file you're testing, x2p/walk.c is no longer part of the perl tree.

Unfortunately, it's "new" awk syntax (nawk) and not portable to 'old' awk.
On Solaris, at least, /usr/bin/awk is still the old awk. I haven't
looked very carefully, but the first thing I saw was the 'sub' function,
which is definitely new awk. The -v var=value stuff is also the new awk
syntax, but old awk can take var=value assignments after the program,
and I think at least some 'new' awks can as well, so that's probably
not a blocker. The 'sub' stuff is, however.

Thanks, I'll have to remember to test on Solaris, since it's so far
behind the times ;)

Tony

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 11, 2014

From @tonycoz

On Sat May 31 15​:56​:30 2014, afresh1@​openbsd.org wrote​:

This brings in improved symlinking of the build tree as as been used
by OpenBSD for the last couple of years.

As Andy pointed out, this isn't portable.

Would you like to submit an updated patch, or should I reject it?

Tony

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 11, 2014

From afresh1@openbsd.org

On Tue, Jun 10, 2014 at 10​:32​:25PM -0700, Tony Cook via RT wrote​:

On Sat May 31 15​:56​:30 2014, afresh1@​openbsd.org wrote​:

This brings in improved symlinking of the build tree as as been used
by OpenBSD for the last couple of years.

As Andy pointed out, this isn't portable.

Would you like to submit an updated patch, or should I reject it?

I'm having trouble finding a copy of old awk. I have an in-progress
patch that I want to test, but not sure it works on old awk, or if it is
better than what is there. If someone has a pointer to where I can get
a copy of olde awk, I'll submit a new patch.

Do we need to support spaces in the path to the perl source tree?

If someone else wants to give it a shot, this is what I have so far.

+ awk "{
+ d=''
+ n=split(\$1, array, '/')
+ if (n == 1) { d = '.' }
+ for (i=0; i<n; i++) {
+ if (d) { d=d'/' }
+ d = d array[i]
+ }
+ mf[d] = mf[d]' $src/'\$1
+ } END {
+ for (d in mf) {
+ if (d != '.') { system('mkdir -p 'd) }
+ system('ln -sf 'mf[d]' 'd);
+ }
+ }" $src/MANIFEST

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 12, 2014

From @tonycoz

On Tue Jun 10 22​:40​:21 2014, afresh1@​openbsd.org wrote​:

On Tue, Jun 10, 2014 at 10​:32​:25PM -0700, Tony Cook via RT wrote​:

On Sat May 31 15​:56​:30 2014, afresh1@​openbsd.org wrote​:

This brings in improved symlinking of the build tree as as been used
by OpenBSD for the last couple of years.

As Andy pointed out, this isn't portable.

Would you like to submit an updated patch, or should I reject it?

I'm having trouble finding a copy of old awk. I have an in-progress
patch that I want to test, but not sure it works on old awk, or if it is
better than what is there. If someone has a pointer to where I can get
a copy of olde awk, I'll submit a new patch.

Do we need to support spaces in the path to the perl source tree?

If someone else wants to give it a shot, this is what I have so far.

+ awk "{
+ d=''
+ n=split(\$1, array, '/')
+ if (n == 1) { d = '.' }
+ for (i=0; i<n; i++) {
+ if (d) { d=d'/' }
+ d = d array[i]
+ }
+ mf[d] = mf[d]' $src/'\$1
+ } END {
+ for (d in mf) {
+ if (d != '.') { system('mkdir -p 'd) }
+ system('ln -sf 'mf[d]' 'd);
+ }
+ }" $src/MANIFEST

There's two problems, after testing on Solaris 11​:

a) it doesn't support ' for string quoting, and produced syntax errors until I converted each ' to '"

b) system() isn't documented as a built-in - calling it doesn't produce an error, but it doesn't appear to execute its argument

As an aside Configure defines a mkdir_p macro - does that mean mkdir -p isn't portable?

Tony

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 12, 2014

From @doughera88

On Thu, Jun 12, 2014 at 12​:07​:42AM -0700, "Tony Cook via RT" wrote​:

On Tue Jun 10 22​:40​:21 2014, afresh1@​openbsd.org wrote​:

On Tue, Jun 10, 2014 at 10​:32​:25PM -0700, Tony Cook via RT wrote​:

On Sat May 31 15​:56​:30 2014, afresh1@​openbsd.org wrote​:

This brings in improved symlinking of the build tree as as been used
by OpenBSD for the last couple of years.

As Andy pointed out, this isn't portable.

Would you like to submit an updated patch, or should I reject it?

I'm having trouble finding a copy of old awk. I have an in-progress
patch that I want to test, but not sure it works on old awk, or if it is
better than what is there. If someone has a pointer to where I can get
a copy of olde awk, I'll submit a new patch.

I don't know any easy source for it. (Debian distributes
a package called "original-awk", and Kernighan's web site,
http​://www.cs.princeton.edu/~bwk/btl.mirror/ , also has source,
but I think both are for 'nawk', not the very old 'awk'.

Anyway, I think the simplest thing to do is perhaps to use the awk associative arrays to
reduce the number of system calls needed, but still use sed to break pathnames up into
directory and filename, and still use /bin/sh to do the actual calls to mkdir and ln.

Here's a version that should be about as fast as your original, but work
with either new or old awk. It does have two disadvantages​:
  1. It doesn't do the detailed item-by-item error checking of the original
  (but the OpenBSD patch included some spot checking, so that's probably
  ok).
  2. It does end up with some very long lines, depending on the length of
  $src, which might cause a /bin/sh line length problem, but didn't in
  any of my quick tests.

  #!/bin/sh
  src="$HOME/src/perl/perl-blead/" # Set here just for testing
  test -d UU || mkdir UU
  # Split all MANIFEST directory/file files into a pair of
  # entries separated by a space.
  # Then for each directory, bundle up all files to be linked into
  # that directory into a single ln command for efficiency.
  awk '{print $1}' $src/MANIFEST | sed -e 's​:/\([^/]*\)$​: \1​:' |
  awk 'NF == 1 {
  dir=".";
  file=$1 "";
  }
  NF == 2 {
  dir=$1 "";
  file=$2 "";
  }
  {
  print "# dir = ", dir, "file = ", file
  mf[dir] = mf[dir]" "src"/"dir"/"file;
  } END {
  for (d in mf) {
  if (d != ".") { print("mkdir -p "d) }
  print("ln -sf "mf[d]" "d);
  }
  }' src="$src" > UU/mksymlinks.$$
  sh UU/mksymlinks.$$
  rm UU/mksymlinks.$$

Do we need to support spaces in the path to the perl source tree?

Yes, ideally, but my version above would need an extra set of quotes in the last
print line to deal with it.

There's two problems, after testing on Solaris 11​:

As an aside Configure defines a mkdir_p macro - does that mean mkdir -p isn't portable?

Back in 1994, yes. My recollection is that the systems that didn't
support mkdir -p also didn't support symbolic links, so this section
wasn't relevant anyway.

In any case, since -Dmksymlinks is an opt-in item, I wouldn't worry about
it.

--
  Andy Dougherty doughera@​lafayette.edu

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 25, 2014

From @tonycoz

On Thu Jun 12 08​:29​:26 2014, doughera wrote​:

On Thu, Jun 12, 2014 at 12​:07​:42AM -0700, "Tony Cook via RT" wrote​:

On Tue Jun 10 22​:40​:21 2014, afresh1@​openbsd.org wrote​:

On Tue, Jun 10, 2014 at 10​:32​:25PM -0700, Tony Cook via RT wrote​:

On Sat May 31 15​:56​:30 2014, afresh1@​openbsd.org wrote​:

This brings in improved symlinking of the build tree as as been
used
by OpenBSD for the last couple of years.

As Andy pointed out, this isn't portable.

Would you like to submit an updated patch, or should I reject it?

I'm having trouble finding a copy of old awk. I have an in-
progress
patch that I want to test, but not sure it works on old awk, or if
it is
better than what is there. If someone has a pointer to where I can
get
a copy of olde awk, I'll submit a new patch.

I don't know any easy source for it. (Debian distributes
a package called "original-awk", and Kernighan's web site,
http​://www.cs.princeton.edu/~bwk/btl.mirror/ , also has source,
but I think both are for 'nawk', not the very old 'awk'.

Anyway, I think the simplest thing to do is perhaps to use the awk
associative arrays to
reduce the number of system calls needed, but still use sed to break
pathnames up into
directory and filename, and still use /bin/sh to do the actual calls
to mkdir and ln.

Here's a version that should be about as fast as your original, but
work
with either new or old awk. It does have two disadvantages​:
1. It doesn't do the detailed item-by-item error checking of the
original
(but the OpenBSD patch included some spot checking, so that's
probably
ok).
2. It does end up with some very long lines, depending on the
length of
$src, which might cause a /bin/sh line length problem, but
didn't in
any of my quick tests.

I've attached it as a patch against Configure. Tested on Solaris 11.

Please test it elsewhere.

Do we need to support spaces in the path to the perl source tree?

Yes, ideally, but my version above would need an extra set of quotes
in the last
print line to deal with it.

Unfortunately Configure fails well before the mksymlinks logic when the source
path includes spaces, so I didn't try to follow-up on spaces.

Tony

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 25, 2014

From @tonycoz

0001-Better-handling-of-symlink-creation.patch
From 2520356405baac1c040f3ca75098dc9da9a4961c Mon Sep 17 00:00:00 2001
From: Andrew Fresh <afresh1@openbsd.org>
Date: Sat, 31 May 2014 15:37:52 -0700
Subject: [PATCH] Better handling of symlink creation

OpenBSD heavily uses the mksymlinks define to build outside of the src
tree.  This combines two commits from OpenBSD that improve the
robustness and speed of creating the symlinks.

From r1.24 by deraadt@:
> After the MANIFEST-based symbolic link creation loop, there is a check
> for a specific link to see if things worked out.  Add a check for the very
> last file in the MANIFEST, as well, since we are trying to spot a very
> odd bug where symbolic links are not being created.  Hopefully this will
> help us diagnose it.

http://www.openbsd.org/cgi-bin/cvsweb/src/gnu/usr.bin/perl/Configure#rev1.24

And from r1.25 by millert@:
> More efficient method of building the symlink tree that makes better
> use of awk.  Slightly faster and works around an apparent namei or
> buffer cache related bug on arm.

http://www.openbsd.org/cgi-bin/cvsweb/src/gnu/usr.bin/perl/Configure#rev1.25

Re-worked by Andy Armstrong to support the old awk used by systems
such as Solaris, see [perl #122002].
---
 Configure |   45 ++++++++++++++++++++++++---------------------
 1 file changed, 24 insertions(+), 21 deletions(-)

diff --git a/Configure b/Configure
index 4e6315d..cd69bcc 100755
--- a/Configure
+++ b/Configure
@@ -2798,38 +2798,41 @@ $define|true|[yY]*)
 	*)	case "$lns:$issymlink" in
 		*"ln"*" -s:"*"test -"?)
 			echo "Creating the symbolic links..." >&4
-			echo "(First creating the subdirectories...)" >&4
 			cd ..
-			awk '{print $1}' $src/MANIFEST | grep / | sed 's:/[^/]*$::' | sort -u | while true; do
-				read directory
-				test -z "$directory" && break
-				mkdir -p $directory
-			done
+			awk '{print $1}' $src/MANIFEST | sed -e 's:/\([^/]*\)$: \1:' |
+			awk 'NF == 1 {
+				dir=".";
+				file=$1 "";
+			     }
+			     NF == 2 {
+				dir=$1 "";
+				file=$2 "";
+			     }
+			     {
+				 print "# dir = ", dir, "file = ", file
+				 mf[dir] = mf[dir]" "src"/"dir"/"file;
+			     } END {
+				 for (d in mf) {
+				     if (d != ".") { print("mkdir -p "d) }
+				     print("ln -sf "mf[d]" "d);
+				 }
+			     }' src="$src" > UU/mksymlinks.$$
+			sh UU/mksymlinks.$$
+			rm UU/mksymlinks.$$
 			# Sanity check 1.
 			if test ! -d t/base; then
 				echo "Failed to create the subdirectories.  Aborting." >&4
 				exit 1
 			fi
-			echo "(Then creating the symlinks...)" >&4
-			awk '{print $1}' $src/MANIFEST | while true; do
-				read filename
-				test -z "$filename" && break
-				if test -f $filename; then
-					if $issymlink $filename; then
-						rm -f $filename
-					fi
-				fi
-				if test -f $filename; then
-					echo "$filename already exists, not symlinking."
-				else
-					ln -s $src/$filename $filename
-				fi
-			done
 			# Sanity check 2.
 			if test ! -f t/base/lex.t; then
 				echo "Failed to create the symlinks (t/base/lex.t missing).  Aborting." >&4
 				exit 1
 			fi
+			if test ! -f win32/win32.c; then
+				echo "Failed to create the symlinks (win32/win32.c missing).  Aborting." >&4
+				exit 1
+			fi
 			cd UU
 			;;
 		*)	echo "(I cannot figure out how to do symbolic links, ignoring mksymlinks)." >&4
-- 
1.7.10.4

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 26, 2014

From andrew@afresh1.com

On Tue, Jun 24, 2014 at 08​:14​:25PM -0700, Tony Cook via RT wrote​:

Re-worked by Andy Armstrong to support the old awk used by systems
such as Solaris, see [perl #122002].

This seems to work for me, I think that generating the script is a great
solution.

I would like to see proper whitespace but that is not up to me.

I assume there is a way to do this in a single awk processes since there
are tests that guarantee MANIFEST is sorted and we don't actually care
that they are. I haven't had time to actually get an old awk interpreter
compiled under OpenBSD so can't actually try to rewrite that for that
though.

---
Configure | 45 ++++++++++++++++++++++++---------------------
1 file changed, 24 insertions(+), 21 deletions(-)

diff --git a/Configure b/Configure
index 4e6315d..cd69bcc 100755
--- a/Configure
+++ b/Configure
@​@​ -2798,38 +2798,41 @​@​ $define|true|[yY]*)
*) case "$lns​:$issymlink" in
*"ln"*" -s​:"*"test -"?)
echo "Creating the symbolic links..." >&4
- echo "(First creating the subdirectories...)" >&4
cd ..
- awk '{print $1}' $src/MANIFEST | grep / | sed 's​:/[^/]*$​::' | sort -u | while true; do
- read directory
- test -z "$directory" && break
- mkdir -p $directory
- done
+ awk '{print $1}' $src/MANIFEST | sed -e 's​:/\([^/]*\)$​: \1​:' |
+ awk 'NF == 1 {
+ dir=".";
+ file=$1 "";
+ }
+ NF == 2 {
+ dir=$1 "";
+ file=$2 "";
+ }
+ {
+ print "# dir = ", dir, "file = ", file
+ mf[dir] = mf[dir]" "src"/"dir"/"file;
+ } END {
+ for (d in mf) {
+ if (d != ".") { print("mkdir -p "d) }
+ print("ln -sf "mf[d]" "d);
+ }
+ }' src="$src" > UU/mksymlinks.$$
+ sh UU/mksymlinks.$$
+ rm UU/mksymlinks.$$
# Sanity check 1.
if test ! -d t/base; then
echo "Failed to create the subdirectories. Aborting." >&4
exit 1
fi
- echo "(Then creating the symlinks...)" >&4
- awk '{print $1}' $src/MANIFEST | while true; do
- read filename
- test -z "$filename" && break
- if test -f $filename; then
- if $issymlink $filename; then
- rm -f $filename
- fi
- fi
- if test -f $filename; then
- echo "$filename already exists, not symlinking."
- else
- ln -s $src/$filename $filename
- fi
- done
# Sanity check 2.
if test ! -f t/base/lex.t; then
echo "Failed to create the symlinks (t/base/lex.t missing). Aborting." >&4
exit 1
fi
+ if test ! -f win32/win32.c; then
+ echo "Failed to create the symlinks (win32/win32.c missing). Aborting." >&4
+ exit 1
+ fi
cd UU
;;
*) echo "(I cannot figure out how to do symbolic links, ignoring mksymlinks)." >&4
--
1.7.10.4

--
andrew - http​://afresh1.com

Software doesn't do what you want it to do, it does what you tell it do.
  -- Stefan G. Weichinger.

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Nov 4, 2014

From @tonycoz

On Tue Jun 24 20​:14​:25 2014, tonyc wrote​:

On Thu Jun 12 08​:29​:26 2014, doughera wrote​:

On Thu, Jun 12, 2014 at 12​:07​:42AM -0700, "Tony Cook via RT" wrote​:

On Tue Jun 10 22​:40​:21 2014, afresh1@​openbsd.org wrote​:

On Tue, Jun 10, 2014 at 10​:32​:25PM -0700, Tony Cook via RT wrote​:

On Sat May 31 15​:56​:30 2014, afresh1@​openbsd.org wrote​:

This brings in improved symlinking of the build tree as as
been
used
by OpenBSD for the last couple of years.

As Andy pointed out, this isn't portable.

Would you like to submit an updated patch, or should I reject
it?

I'm having trouble finding a copy of old awk. I have an in-
progress
patch that I want to test, but not sure it works on old awk, or
if
it is
better than what is there. If someone has a pointer to where I
can
get
a copy of olde awk, I'll submit a new patch.

I don't know any easy source for it. (Debian distributes
a package called "original-awk", and Kernighan's web site,
http​://www.cs.princeton.edu/~bwk/btl.mirror/ , also has source,
but I think both are for 'nawk', not the very old 'awk'.

Anyway, I think the simplest thing to do is perhaps to use the awk
associative arrays to
reduce the number of system calls needed, but still use sed to break
pathnames up into
directory and filename, and still use /bin/sh to do the actual calls
to mkdir and ln.

Here's a version that should be about as fast as your original, but
work
with either new or old awk. It does have two disadvantages​:
1. It doesn't do the detailed item-by-item error checking of the
original
(but the OpenBSD patch included some spot checking, so that's
probably
ok).
2. It does end up with some very long lines, depending on the
length of
$src, which might cause a /bin/sh line length problem, but
didn't in
any of my quick tests.

I've attached it as a patch against Configure. Tested on Solaris 11.

Please test it elsewhere.

Applied as 7191ba8.

Tony

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Nov 4, 2014

@tonycoz - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.