Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Porting/release_managers_guide.pod: need advice re new directories created by CPAN synch #15775

Closed
p5pRT opened this issue Dec 19, 2016 · 8 comments
Closed

Comments

@p5pRT
Copy link

@p5pRT p5pRT commented Dec 19, 2016

Migrated from rt.perl.org#130375 (status was 'resolved')

Searchable as RT130375$

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 19, 2016

From @jkeenan

Created by @jkeenan

Over the weekend both BinGOs and I were, in the course of helping to
prepare for this week's monthly development version "blead-point"
release, bit by the same problem.

We were attempting to synchronize the Test-Simple distribution found
underneath the 'cpan/' directory in the core distribution with the
latest version of that distro on CPAN. Specifically, we wanted to go
from 1.302067 to 1.302071. We (or, at least, I -- I'm assuming we each
went through the same steps) got to the same point in the instructions
in Porting/release_managers_guide.pod concerning CPAN-to-blead
synchronization​:

#####
=item *

Run a full configure/build/test cycle.

=item *

If everything is ok, commit the changes.
#####

Note the absence of guidance concerning what to do if everything is
*not* ok. In the course of the final 'make test_harness', I got a
failure in 't/porting/regen.t' which, when I ran that file by itself,
appeared as​:

#####
not ok # Makefile.SH is up to date
ok # win32/Makefile is up to date
ok # win32/makefile.mk is up to date
#####

This was puzzling, as the 'diff' of the old Test-Simple against the new
showed no changes in Makefile.SH. I sprinkled 't/porting/regen.t' with
'print' statements, ran it through the debugger, etc., with no good
results. Finally I noticed this line in that test file​:

#####
require './regen/regen_lib.pl';
#####

... and wondered what would happen when I ran that program by itself
with the newly build './perl'. When I did so, I got a whole lot of
changes in Makefile.SH; see attachment. I re-ran 'make test_porting'
and got PASS. I then re-ran the full configure-build-test cycle and got
PASS. And today on advice from arc on #p5p, I pushed to blead. In a
response to an inquiry from Test-Simple maintainer Exodist, arc said,

#####
(03​:10​:59 PM) arc​: Exodist​: not afaict — it's a p5-internal thing about
dealing with the fact that the lib/Test2/Event/TAP/ directory is new in
the latest version (containing a file Version.pm)
(03​:11​:36 PM) arc​: we have to list all cpan dists' directories in the
makefile, so that "make clean" can clean up the versions that are
created under lib/ by the build system
#####

I ran a couple of 'ack's over the core distribution to try to find any
discussion of this requirement and couldn't find anything quickly.
Hence, I infer that the release manager's guide needs updating. We
need​:

* explicit documentation of this requirement for Makefile.SH, including
why ('make clean') we need it at all;

* to decide where in the discussion of CPAN-to-blead synchronization
guidance should go -- perhaps earlier than the last step;

* to decide whether 'regen/regen_lib.pl' should be discussed in the RLM
-- it currently is not.

* discussion as to whether there are any other ways in which a
CPAN-to-blead synch could go wrong at the very last step and how such
failures should be handled.

Thank you very much.
Jim Keenan

Perl Info

Flags:
     category=docs
     severity=low

Site configuration information for perl 5.24.0:

Summary of my perl5 (revision 5 version 24 subversion 0) configuration:

   Platform:
     osname=linux, osvers=3.13.0-85-generic, archname=x86_64-linux
     uname='linux zareason 3.13.0-85-generic #129-ubuntu smp thu mar 17 
20:50:15 utc 2016 x86_64 x86_64 x86_64 gnulinux '
     config_args='-de 
-Dprefix=/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0 
-Aeval:scriptdir=/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/bin'
     hint=recommended, useposix=true, d_sigaction=define
     useithreads=undef, usemultiplicity=undef
     use64bitint=define, use64bitall=define, uselongdouble=undef
     usemymalloc=n, bincompat5005=undef
   Compiler:
     cc='cc', ccflags ='-fwrapv -fno-strict-aliasing -pipe 
-fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE 
-D_FILE_OFFSET_BITS=64',
     optimize='-O2',
     cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector 
-I/usr/local/include'
     ccversion='', gccversion='4.8.4', gccosandvers=''
     intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, 
doublekind=3
     d_longlong=define, longlongsize=8, d_longdbl=define, 
longdblsize=16, longdblkind=3
     ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', 
lseeksize=8
     alignbytes=8, prototype=define
   Linker and Libraries:
     ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
     libpth=/usr/local/lib 
/usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed 
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib 
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /lib64 /usr/lib64
     libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc
     perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
     libc=libc-2.19.so, so=so, useshrplib=false, libperl=libperl.a
     gnulibc_version='2.19'
   Dynamic Linking:
     dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
     cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib 
-fstack-protector'


Characteristics of this binary (from libperl):
   Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE
                         PERL_DONT_CREATE_GVSV
                         PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
                         PERL_PRESERVE_IVUV USE_64_BIT_ALL USE_64_BIT_INT
                         USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE
                         USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME
                         USE_PERLIO USE_PERL_ATOF
   Built under linux
   Compiled at May  9 2016 18:38:59
   %ENV:
     PERLBREW_BASHRC_VERSION="0.67"
     PERLBREW_HOME="/home/jkeenan/.perlbrew"
     PERLBREW_MANPATH="/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/man"
 
PERLBREW_PATH="/home/jkeenan/perl5/perlbrew/bin:/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/bin"
     PERLBREW_PERL="perl-5.24.0"
     PERLBREW_ROOT="/home/jkeenan/perl5/perlbrew"
     PERLBREW_VERSION="0.67"
     PERL_WORKDIR="/home/jkeenan/gitwork/perl"
   @INC:
 
/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/lib/site_perl/5.24.0/x86_64-linux
     /home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/lib/site_perl/5.24.0
     /home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/lib/5.24.0/x86_64-linux
     /home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/lib/5.24.0
     .

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 19, 2016

From @jkeenan

Makefile.SH.10003fd.963d205.diff
diff --git a/10003fd..963d205 b/963d205
index 10003fd..963d205 100644
--- a/10003fd..963d205
+++ b/963d205
@@ -1351,23 +1351,23 @@ _cleaner2:
 	-rmdir lib/Unicode/Collate/CJK lib/Unicode/Collate lib/Tie/Hash
 	-rmdir lib/Thread lib/Text lib/Test2/Util lib/Test2/IPC/Driver
 	-rmdir lib/Test2/IPC lib/Test2/Hub/Interceptor lib/Test2/Hub
-	-rmdir lib/Test2/Formatter lib/Test2/Event lib/Test2/API lib/Test2
-	-rmdir lib/Test/use lib/Test/Tester lib/Test/Builder/Tester
-	-rmdir lib/Test/Builder/IO lib/Test/Builder lib/Test lib/Term
-	-rmdir lib/TAP/Parser/YAMLish lib/TAP/Parser/SourceHandler
-	-rmdir lib/TAP/Parser/Scheduler lib/TAP/Parser/Result
-	-rmdir lib/TAP/Parser/Iterator lib/TAP/Parser lib/TAP/Harness
-	-rmdir lib/TAP/Formatter/File lib/TAP/Formatter/Console
-	-rmdir lib/TAP/Formatter lib/TAP lib/Sys/Syslog lib/Sys lib/Sub
-	-rmdir lib/Search lib/Scalar lib/Pod/Text lib/Pod/Simple
-	-rmdir lib/Pod/Perldoc lib/PerlIO/via lib/PerlIO lib/Perl
-	-rmdir lib/Parse/CPAN lib/Parse lib/Params lib/Net/FTP lib/Module/Load
-	-rmdir lib/Module/CoreList lib/Module lib/Memoize lib/Math/BigInt
-	-rmdir lib/Math/BigFloat lib/Math lib/MIME lib/Locale/Maketext
-	-rmdir lib/Locale/Codes lib/Locale lib/List/Util lib/List lib/JSON/PP
-	-rmdir lib/JSON lib/IPC lib/IO/Uncompress/Adapter lib/IO/Uncompress
-	-rmdir lib/IO/Socket lib/IO/Compress/Zlib lib/IO/Compress/Zip
-	-rmdir lib/IO/Compress/Gzip lib/IO/Compress/Base
+	-rmdir lib/Test2/Formatter lib/Test2/Event/TAP lib/Test2/Event
+	-rmdir lib/Test2/API lib/Test2 lib/Test/use lib/Test/Tester
+	-rmdir lib/Test/Builder/Tester lib/Test/Builder/IO lib/Test/Builder
+	-rmdir lib/Test lib/Term lib/TAP/Parser/YAMLish
+	-rmdir lib/TAP/Parser/SourceHandler lib/TAP/Parser/Scheduler
+	-rmdir lib/TAP/Parser/Result lib/TAP/Parser/Iterator lib/TAP/Parser
+	-rmdir lib/TAP/Harness lib/TAP/Formatter/File
+	-rmdir lib/TAP/Formatter/Console lib/TAP/Formatter lib/TAP
+	-rmdir lib/Sys/Syslog lib/Sys lib/Sub lib/Search lib/Scalar
+	-rmdir lib/Pod/Text lib/Pod/Simple lib/Pod/Perldoc lib/PerlIO/via
+	-rmdir lib/PerlIO lib/Perl lib/Parse/CPAN lib/Parse lib/Params
+	-rmdir lib/Net/FTP lib/Module/Load lib/Module/CoreList lib/Module
+	-rmdir lib/Memoize lib/Math/BigInt lib/Math/BigFloat lib/Math lib/MIME
+	-rmdir lib/Locale/Maketext lib/Locale/Codes lib/Locale lib/List/Util
+	-rmdir lib/List lib/JSON/PP lib/JSON lib/IPC lib/IO/Uncompress/Adapter
+	-rmdir lib/IO/Uncompress lib/IO/Socket lib/IO/Compress/Zlib
+	-rmdir lib/IO/Compress/Zip lib/IO/Compress/Gzip lib/IO/Compress/Base
 	-rmdir lib/IO/Compress/Adapter lib/IO/Compress lib/IO
 	-rmdir lib/I18N/LangTags lib/I18N lib/Hash/Util lib/Hash lib/HTTP
 	-rmdir lib/Filter/Util lib/Filter lib/File/Spec lib/ExtUtils/Typemaps

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 30, 2016

From @jkeenan

On Mon, 19 Dec 2016 00​:21​:28 GMT, jkeen@​verizon.net wrote​:

Porting/release_managers_guide.pod needs advise re new directories
created by CPAN synch

This is a bug report for perl from jkeenan@​cpan.org,
generated with the help of perlbug 1.40 running under perl 5.24.0.

-----------------------------------------------------------------
[Please describe your issue here]

Over the weekend both BinGOs and I were, in the course of helping to
prepare for this week's monthly development version "blead-point"
release, bit by the same problem.

We were attempting to synchronize the Test-Simple distribution found
underneath the 'cpan/' directory in the core distribution with the
latest version of that distro on CPAN. Specifically, we wanted to go
from 1.302067 to 1.302071. We (or, at least, I -- I'm assuming we
each
went through the same steps) got to the same point in the instructions
in Porting/release_managers_guide.pod concerning CPAN-to-blead
synchronization​:

#####
=item *

Run a full configure/build/test cycle.

=item *

If everything is ok, commit the changes.
#####

Note the absence of guidance concerning what to do if everything is
*not* ok. In the course of the final 'make test_harness', I got a
failure in 't/porting/regen.t' which, when I ran that file by itself,
appeared as​:

#####
not ok # Makefile.SH is up to date
ok # win32/Makefile is up to date
ok # win32/makefile.mk is up to date
#####

This was puzzling, as the 'diff' of the old Test-Simple against the
new
showed no changes in Makefile.SH. I sprinkled 't/porting/regen.t'
with
'print' statements, ran it through the debugger, etc., with no good
results. Finally I noticed this line in that test file​:

#####
require './regen/regen_lib.pl';
#####

... and wondered what would happen when I ran that program by itself
with the newly build './perl'. When I did so, I got a whole lot of
changes in Makefile.SH; see attachment. I re-ran 'make test_porting'
and got PASS. I then re-ran the full configure-build-test cycle and
got
PASS. And today on advice from arc on #p5p, I pushed to blead. In a
response to an inquiry from Test-Simple maintainer Exodist, arc said,

#####
(03​:10​:59 PM) arc​: Exodist​: not afaict — it's a p5-internal thing
about
dealing with the fact that the lib/Test2/Event/TAP/ directory is new
in
the latest version (containing a file Version.pm)
(03​:11​:36 PM) arc​: we have to list all cpan dists' directories in the
makefile, so that "make clean" can clean up the versions that are
created under lib/ by the build system
#####

I ran a couple of 'ack's over the core distribution to try to find any
discussion of this requirement and couldn't find anything quickly.
Hence, I infer that the release manager's guide needs updating. We
need​:

* explicit documentation of this requirement for Makefile.SH,
including
why ('make clean') we need it at all;

* to decide where in the discussion of CPAN-to-blead synchronization
guidance should go -- perhaps earlier than the last step;

* to decide whether 'regen/regen_lib.pl' should be discussed in the
RLM
-- it currently is not.

* discussion as to whether there are any other ways in which a
CPAN-to-blead synch could go wrong at the very last step and how such
failures should be handled.

Thank you very much.
Jim Keenan

From irc.perl.ort #p5p​:

#####
(09​:21​:11 AM) kid51​: arc​: Perhaps in the RMG we should (a) mention synch-with-cpan *first*; (b) discuss what we're trying to achieve; then (c) discuss specific distros where synch-with-cpan is likely to be insufficient
(09​:25​:20 AM) arc​: that sounds like a good plan
#####

Aaron, if we decide to go with the approach above, the language attached might be a starting point.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 30, 2016

From @jkeenan

0001-Discuss-Porting-sync-with-cpan-first.patch
From f7072e6e720b680e5f51a34821dd16baa80b0fe4 Mon Sep 17 00:00:00 2001
From: James E Keenan <jkeenan@cpan.org>
Date: Fri, 30 Dec 2016 09:58:17 -0500
Subject: [PATCH] Discuss Porting/sync-with-cpan first.

---
 Porting/release_managers_guide.pod | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/Porting/release_managers_guide.pod b/Porting/release_managers_guide.pod
index d148c21..7c48acd 100644
--- a/Porting/release_managers_guide.pod
+++ b/Porting/release_managers_guide.pod
@@ -245,6 +245,16 @@ have some extra changes.
 
 =head3 How to sync a CPAN module with a cpanE<sol> distro
 
+In most cases, once a new version of a distribution shipped with core has been
+uploaded to CPAN, the core version thereof can be synchronized automatically
+with the program F<Porting/sync-with-cpan>.  (But see the comments at the
+beginning of that program.  In particular, it has not yet been exercised on
+Windows.  On that platform it would require a set of Unix tools such as Cygwin
+and use of C<nmake> rather than C<make>).
+
+If, however, F<Porting/sync-with-cpan> does not provide good results, follow
+the steps below.
+
 =over 4
 
 =item *
@@ -326,12 +336,6 @@ If everything is ok, commit the changes.
 For entries with a non-simple C<FILES> section, or with a C<MAP>, you
 may have to take more steps than listed above.
 
-F<Porting/sync-with-cpan> is a script that automates most of the steps
-above; but see the comments at the beginning of the file.  In particular,
-it has not yet been exercised on Windows, but will certainly require a set
-of Unix tools such as Cygwin, and steps that run C<make> will need to run
-C<nmake> instead.
-
 =head3 dual-life CPAN module stability
 
 Ensure dual-life CPAN modules are stable, which comes down to:
-- 
2.7.4

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 30, 2016

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 30, 2016

From @arc

James E Keenan via RT <perlbug-followup@​perl.org> wrote​:

#####
(09​:21​:11 AM) kid51​: arc​: Perhaps in the RMG we should (a) mention synch-with-cpan *first*; (b) discuss what we're trying to achieve; then (c) discuss specific distros where synch-with-cpan is likely to be insufficient
(09​:25​:20 AM) arc​: that sounds like a good plan
#####

Aaron, if we decide to go with the approach above, the language attached might be a starting point.

Thanks, James, this looks great!

At the risk of bikeshedding, I suggest one small tweak​:

+In most cases, once a new version of a distribution shipped with core has been
+uploaded to CPAN, the core version thereof can be synchronized automatically
+with the program F<Porting/sync-with-cpan>. (But see the comments at the
+beginning of that program. In particular, it has not yet been exercised on
+Windows. On that platform it would require a set of Unix tools such as Cygwin
+and use of C<nmake> rather than C<make>).

There's definitely been substantial work (mainly by Corion++ afaict)
to make sync-with-cpan work on Windows (without any Unix-like shell
tools, and using whatever "make" utility is reported by Config.pm),
and I've tried to follow his lead in my recent changes. Perhaps it
would be better to replace the last two sentences with this​:

In particular, it has not yet been exercised on Windows as much as it
has on Unix-like platforms.

--
Aaron Crane ** http​://aaroncrane.co.uk/

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 30, 2016

From @jkeenan

On Fri, 30 Dec 2016 19​:01​:35 GMT, arc wrote​:

James E Keenan via RT <perlbug-followup@​perl.org> wrote​:

#####
(09​:21​:11 AM) kid51​: arc​: Perhaps in the RMG we should (a) mention
synch-with-cpan *first*; (b) discuss what we're trying to achieve;
then (c) discuss specific distros where synch-with-cpan is likely to
be insufficient
(09​:25​:20 AM) arc​: that sounds like a good plan
#####

Aaron, if we decide to go with the approach above, the language
attached might be a starting point.

Thanks, James, this looks great!

At the risk of bikeshedding, I suggest one small tweak​:

+In most cases, once a new version of a distribution shipped with
core has been
+uploaded to CPAN, the core version thereof can be synchronized
automatically
+with the program F<Porting/sync-with-cpan>. (But see the comments
at the
+beginning of that program. In particular, it has not yet been
exercised on
+Windows. On that platform it would require a set of Unix tools such
as Cygwin
+and use of C<nmake> rather than C<make>).

There's definitely been substantial work (mainly by Corion++ afaict)
to make sync-with-cpan work on Windows (without any Unix-like shell
tools, and using whatever "make" utility is reported by Config.pm),
and I've tried to follow his lead in my recent changes. Perhaps it
would be better to replace the last two sentences with this​:

In particular, it has not yet been exercised on Windows as much as it
has on Unix-like platforms.

I incorporated your feedback into commit b58fbdf and pushed it to blead.

Given the work that you've been doing today and recently on Porting/sync-with-cpan, I think I'll defer discussion of the other bullet points in my original post until we/I get some experience with that program and synchronization in general. So I'm marking this ticket Resolved.

Thanks to people who earlier worked on Porting/sync-with-cpan​:

Aaron Crane
Abigail
David Steinbrunner
H.Merijn Brand
Jesse Luehrs
Karl Williamson
Max Maischein
Ricardo Signes

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Dec 30, 2016

@jkeenan - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant