Skip to content

Commit

Permalink
GH-38332: [CI][Release] Resolve symlinks in RAT lint (#38337)
Browse files Browse the repository at this point in the history
### Rationale for this change

Our release script (`dev/release/02-source.sh`) resolves symlinks in source archive but our lint script (`dev/archery/archery/utils/source.py`) doesn't resolve symlinks. So we may detect RAT problem by our CI.

### What changes are included in this PR?

Resolve symlinks in our lint script too.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* Closes: #38332

Lead-authored-by: Sutou Kouhei <kou@clear-code.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
  • Loading branch information
3 people committed Oct 19, 2023
1 parent bd61239 commit 9f90995
Show file tree
Hide file tree
Showing 4 changed files with 37 additions and 9 deletions.
19 changes: 19 additions & 0 deletions cpp/cmake_modules/snappy.diff
Original file line number Diff line number Diff line change
@@ -1,3 +1,22 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
# https://github.com/google/snappy/pull/172

diff --git a/snappy.cc b/snappy.cc
index d414718..5b0d0d6 100644
--- a/snappy.cc
Expand Down
18 changes: 15 additions & 3 deletions dev/archery/archery/utils/source.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@
import os
from pathlib import Path
import subprocess
import tempfile

from .command import Command
from .git import git


Expand Down Expand Up @@ -117,10 +119,20 @@ def archive(self, path, dereference=False, compressor=None, revision=None):
raise ValueError("{} is not backed by git".format(self))

rev = revision if revision else "HEAD"
archive = git.archive("--prefix=apache-arrow/", rev,
archive = git.archive("--prefix=apache-arrow.tmp/", rev,
git_dir=self.path)

# TODO(fsaintjacques): fix dereference for
with tempfile.TemporaryDirectory() as tmp:
tmp = Path(tmp)
tar_path = tmp / "apache-arrow.tar"
with open(tar_path, "wb") as tar:
tar.write(archive)
Command("tar").run("xf", tar_path, "-C", tmp)
# Must use the same logic in dev/release/02-source.sh
Command("cp").run("-R", "-L", tmp /
"apache-arrow.tmp", tmp / "apache-arrow")
Command("tar").run("cf", tar_path, "-C", tmp, "apache-arrow")
with open(tar_path, "rb") as tar:
archive = tar.read()

if compressor:
archive = compressor(archive)
Expand Down
4 changes: 3 additions & 1 deletion dev/release/02-source.sh
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,9 @@ rm -rf ${tag}
git archive ${release_hash} --prefix ${tag}/) | \
tar xf -

# Resolve all hard and symbolic links
# Resolve all hard and symbolic links.
# If we change this, we must change ArrowSources.archive in
# dev/archery/archery/utils/source.py too.
rm -rf ${tag}.tmp
mv ${tag} ${tag}.tmp
cp -R -L ${tag}.tmp ${tag}
Expand Down
5 changes: 0 additions & 5 deletions dev/release/rat_exclude_files.txt
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,6 @@ cpp/build-support/cpplint.py
cpp/build-support/lint_exclusions.txt
cpp/build-support/iwyu/*
cpp/cmake_modules/FindPythonLibsNew.cmake
cpp/cmake_modules/SnappyCMakeLists.txt
cpp/cmake_modules/SnappyConfig.h
cpp/cmake_modules/snappy.diff
cpp/examples/parquet/parquet-arrow/cmake_modules/FindArrow.cmake
cpp/src/parquet/.parquetcppversion
cpp/src/generated/parquet_constants.cpp
Expand Down Expand Up @@ -89,8 +86,6 @@ js/yarn.lock
js/.eslintignore
python/cmake_modules
python/cmake_modules/FindPythonLibsNew.cmake
python/cmake_modules/SnappyCMakeLists.txt
python/cmake_modules/SnappyConfig.h
python/MANIFEST.in
python/manylinux1/.dockerignore
python/pyarrow/includes/__init__.pxd
Expand Down

0 comments on commit 9f90995

Please sign in to comment.