Add flags when open file #7445

scw00 · 2019-07-02T05:37:12Z

Add flags for open method. So user can use it with read only mode or write only mode.

Risk Level: Low
Testing: unit_tests (under linux platform)
Docs Changes: no docs change
Release Notes:
[Optional Fixes #Issue] no issue
[Optional Deprecated:]

scw00 · 2019-07-02T09:22:57Z

Hi, it seems the clang_tidy check run win32 implementation under linux platform. And I am not sure how to fix that.

/source/source/common/filesystem/win32/filesystem_impl.cc:2:10: error: 'io.h' file not found [clang-diagnostic-error]
#include <io.h>
         ^
Suppressed 2 warnings (1 in non-user code, 1 due to line filter).

mattklein123 · 2019-07-03T23:22:17Z

@scw00 thanks for the contribution. Can you describe the purpose of this change? AFAICT the new functionality is not used? Are you planning on a follow up change?

scw00 · 2019-07-04T00:08:01Z

Hi @mattklein123 . I am trying to implement the static file filter which return static file from local fs. But I didn't found any appropriate structure for reading a file. It is the first step to implement the opening method with reading only flag.

scw00 · 2019-07-07T01:26:04Z

Updated. For using std::bitset and set default value to open method.

mattklein123

Thanks, a few high level comments to get started. Please also check clang-tidy and format.

/wait

include/envoy/filesystem/filesystem.h

source/common/filesystem/file_shared_impl.cc

ci/run_clang_tidy.sh

source/common/filesystem/posix/filesystem_impl.cc

source/common/filesystem/win32/filesystem_impl.cc

mattklein123

Cool, thanks for working on this. A few small comments otherwise LGTM.

/wait

source/common/filesystem/file_shared_impl.cc

source/common/filesystem/posix/filesystem_impl.cc

test/common/filesystem/filesystem_impl_test.cc

mattklein123

LGTM with some nits, thanks.

/wait

source/common/access_log/access_log_manager_impl.cc

source/common/filesystem/posix/filesystem_impl.cc

source/common/filesystem/win32/filesystem_impl.cc

mattklein123 · 2019-07-30T23:13:11Z

test/common/filesystem/filesystem_impl_test.cc

@@ -12,6 +12,10 @@
 namespace Envoy {
 namespace Filesystem {

+static constexpr FlagSet DEFAULT_FLAGS{


Please use the default flags function that is already defined, same elsewhere (I already commented on this I think).

I'm not sure I can use the default function which is defined in access_log_manager_impl.h. file_system_test should not dependent on other implementations .

mattklein123

Thanks, LGTM. @jmarantz any further comments?

jmarantz

sorry I should've chimed in earlier; I do have some questions.

jmarantz · 2019-07-31T20:11:47Z

source/common/access_log/access_log_manager_impl.h

@@ -83,6 +83,9 @@ class AccessLogFileImpl : public AccessLogFile {
  void open();
  void createFlushStructures();

+  // return default flags set which used by open
+  Filesystem::FlagSet defaultFlags();


Declare this method static; it does not appear to require this.

Thanks and will do.

jmarantz · 2019-07-31T20:12:49Z

source/common/filesystem/posix/filesystem_impl.cc

-
-  fd_ = ::open(path_.c_str(), flags, mode);
+void FileImplPosix::openFile(FlagSet in) {
+  const auto flags_and_perm = translateFlag(in);


You reference 'perm' in this PR a few times; can we spell that out? I think it might mean "permanent" but I'm not sure.

You mean move this into shared interface . It seems contradictory to previous changes, see
#7445 (comment)

This is also why I make the PermAndFlags into platform implementation.

Oh it's hard to see exactly what Matt had an issue with because you squashed commits. In general please try to avoid that because we lose the comment history.

I still don't understand the meaning of 'perm'. But looking at how it's used here I retract my "make part of the interface" suggestion, and instead suggest using the correct type and name in posix for the 3rd arg to open: mode_t mode

jmarantz · 2019-07-31T20:13:54Z

source/common/filesystem/posix/filesystem_impl.h

@@ -16,8 +16,14 @@ class FileImplPosix : public FileSharedImpl {
  ~FileImplPosix() override;

 protected:
+  struct FlagsAndPerm {


Does it make sense to put FlagsAndPerm in a shared interface, rather than repeating it from win32 and posix?

Because the translateFlags method is platform specified function ( the translateFlags is in the platform class not in shared interface). So the implementation should have their own flags and perm or other something more than this two flags.

But currently we only care about the perm and flags, so I'm ok with that . And will change.

s/perm/mode/ and s/int/mode_t/ for the mode arg?

source/common/access_log/access_log_manager_impl.cc

source/common/filesystem/posix/filesystem_impl.cc

test/common/filesystem/filesystem_impl_test.cc

source/common/access_log/access_log_manager_impl.cc

source/common/filesystem/posix/filesystem_impl.cc

jmarantz · 2019-08-01T00:57:33Z

source/common/filesystem/win32/filesystem_impl.cc

-  fd_ = ::_open(path_.c_str(), flags, mode);
+void FileImplWin32::openFile(FlagSet in) {
+  const auto flags_and_perm = translateFlag(in);
+  fd_ = ::open(path_.c_str(), flags_and_perm.flags, flags_and_perm.perm);


ah I finally figured out what 'perm' meant by reading the MS docs: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/open-wopen?view=vs-2019 -- it means "permissions".

but there it's called pmode, not perm.

Thanks and will change

jmarantz · 2019-08-01T00:58:20Z

source/common/filesystem/win32/filesystem_impl.h

@@ -14,8 +14,14 @@ class FileImplWin32 : public FileSharedImpl {
  ~FileImplWin32();

 protected:
+  struct FlagsAndPerm {
+    int flags = 0;
+    int perm = 0;


jmarantz · 2019-08-01T02:03:59Z

source/common/filesystem/posix/filesystem_impl.h

@@ -16,8 +16,14 @@ class FileImplPosix : public FileSharedImpl {
  ~FileImplPosix() override;

 protected:
+  struct FlagsAndMode {
+    int flags = 0;


nit: end member variables with _

https://github.com/envoyproxy/envoy/blob/master/STYLE.md

Signed-off-by: cwsong <cwsong@iflytek.com>

scw00 · 2019-08-01T04:50:12Z

/retest

repokitteh-read-only · 2019-08-01T04:50:14Z

🤷‍♀️ nothing to rebuild.

🐱

Caused by: a #7445 (comment) was created by @scw00.

see: more, trace.

dio · 2019-08-01T05:03:25Z

/azp run envoy-macos

azure-pipelines · 2019-08-01T05:03:33Z

Azure Pipelines successfully started running 1 pipeline(s).

scw00 · 2019-08-01T05:16:04Z

Thanks @dio

scw00 · 2019-08-02T00:23:14Z

Hi @mattklein123 @jmarantz thanks for your comments. And Can I get another approve ?

scw00 · 2019-08-02T00:46:01Z

Thanks

scw00 force-pushed the open_reconstruct branch 4 times, most recently from 94d066a to 72e0a14 Compare July 2, 2019 06:51

scw00 requested review from alyssawilk and mattklein123 as code owners July 2, 2019 06:51

scw00 force-pushed the open_reconstruct branch from 72e0a14 to 2b90a64 Compare July 2, 2019 07:55

mattklein123 self-assigned this Jul 3, 2019

mattklein123 added the waiting:any label Jul 3, 2019

repokitteh-read-only bot removed the waiting:any label Jul 4, 2019

scw00 force-pushed the open_reconstruct branch 3 times, most recently from 6c4d30f to f86d9b4 Compare July 7, 2019 01:24

mattklein123 requested changes Jul 10, 2019

View reviewed changes

include/envoy/filesystem/filesystem.h Outdated Show resolved Hide resolved

source/common/filesystem/file_shared_impl.cc Outdated Show resolved Hide resolved

repokitteh-read-only bot added the waiting label Jul 10, 2019

scw00 force-pushed the open_reconstruct branch from f86d9b4 to 050e9fd Compare July 12, 2019 00:30

repokitteh-read-only bot removed the waiting label Jul 12, 2019

scw00 force-pushed the open_reconstruct branch 6 times, most recently from 45d2bc7 to 80bdf02 Compare July 12, 2019 04:51

scw00 commented Jul 12, 2019

View reviewed changes

ci/run_clang_tidy.sh Outdated Show resolved Hide resolved

scw00 commented Jul 12, 2019

View reviewed changes

source/common/filesystem/posix/filesystem_impl.cc Outdated Show resolved Hide resolved

scw00 commented Jul 12, 2019

View reviewed changes

source/common/filesystem/win32/filesystem_impl.cc Outdated Show resolved Hide resolved

mattklein123 requested changes Jul 26, 2019

View reviewed changes

source/common/filesystem/file_shared_impl.cc Outdated Show resolved Hide resolved

source/common/filesystem/posix/filesystem_impl.cc Outdated Show resolved Hide resolved

test/common/filesystem/filesystem_impl_test.cc Outdated Show resolved Hide resolved

repokitteh-read-only bot added the waiting label Jul 26, 2019

scw00 force-pushed the open_reconstruct branch from d916098 to fb3b2ba Compare July 30, 2019 09:06

repokitteh-read-only bot removed the waiting label Jul 30, 2019

scw00 force-pushed the open_reconstruct branch from fb3b2ba to 6cdc33c Compare July 30, 2019 09:08

mattklein123 requested changes Jul 30, 2019

View reviewed changes

repokitteh-read-only bot added the waiting label Jul 30, 2019

scw00 force-pushed the open_reconstruct branch from 6cdc33c to 5c9f063 Compare July 31, 2019 07:00

repokitteh-read-only bot removed the waiting label Jul 31, 2019

scw00 force-pushed the open_reconstruct branch from 5c9f063 to 0b8fb18 Compare July 31, 2019 07:13

mattklein123 previously approved these changes Jul 31, 2019

View reviewed changes

jmarantz reviewed Jul 31, 2019

View reviewed changes

mattklein123 added the waiting label Jul 31, 2019

jmarantz reviewed Aug 1, 2019

View reviewed changes

scw00 dismissed mattklein123’s stale review via ef09e7c August 1, 2019 01:55

scw00 force-pushed the open_reconstruct branch from 0b8fb18 to ef09e7c Compare August 1, 2019 01:55

repokitteh-read-only bot removed the waiting label Aug 1, 2019

jmarantz previously approved these changes Aug 1, 2019

View reviewed changes

Add flag when open file

85da87e

Signed-off-by: cwsong <cwsong@iflytek.com>

scw00 dismissed jmarantz’s stale review via 85da87e August 1, 2019 02:32

scw00 force-pushed the open_reconstruct branch from ef09e7c to 85da87e Compare August 1, 2019 02:32

jmarantz approved these changes Aug 2, 2019

View reviewed changes

jmarantz merged commit dd50110 into envoyproxy:master Aug 2, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add flags when open file #7445

Add flags when open file #7445

scw00 commented Jul 2, 2019 •

edited

Loading

scw00 commented Jul 2, 2019

mattklein123 commented Jul 3, 2019

scw00 commented Jul 4, 2019

scw00 commented Jul 7, 2019

mattklein123 left a comment

mattklein123 left a comment

mattklein123 left a comment

mattklein123 Jul 30, 2019

scw00 Jul 31, 2019

mattklein123 left a comment

jmarantz left a comment

jmarantz Jul 31, 2019

scw00 Aug 1, 2019

jmarantz Jul 31, 2019

scw00 Aug 1, 2019 •

edited

Loading

jmarantz Aug 1, 2019

jmarantz Jul 31, 2019

scw00 Aug 1, 2019

jmarantz Aug 1, 2019

jmarantz Aug 1, 2019

scw00 Aug 1, 2019

jmarantz Aug 1, 2019

jmarantz Aug 1, 2019

scw00 commented Aug 1, 2019

repokitteh-read-only bot commented Aug 1, 2019

dio commented Aug 1, 2019

azure-pipelines bot commented Aug 1, 2019

scw00 commented Aug 1, 2019

scw00 commented Aug 2, 2019

scw00 commented Aug 2, 2019

Add flags when open file #7445

Add flags when open file #7445

Conversation

scw00 commented Jul 2, 2019 • edited Loading

scw00 commented Jul 2, 2019

mattklein123 commented Jul 3, 2019

scw00 commented Jul 4, 2019

scw00 commented Jul 7, 2019

mattklein123 left a comment

Choose a reason for hiding this comment

mattklein123 left a comment

Choose a reason for hiding this comment

mattklein123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattklein123 left a comment

Choose a reason for hiding this comment

jmarantz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

scw00 Aug 1, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

scw00 commented Aug 1, 2019

repokitteh-read-only bot commented Aug 1, 2019

dio commented Aug 1, 2019

azure-pipelines bot commented Aug 1, 2019

scw00 commented Aug 1, 2019

scw00 commented Aug 2, 2019

scw00 commented Aug 2, 2019

scw00 commented Jul 2, 2019 •

edited

Loading

scw00 Aug 1, 2019 •

edited

Loading