Tidy 5.6.0 on Mac says Not a file when file is not writeable

[thvv]

I check files before install with
tidy --markup no --quiet yes --show-warnings no index.html
I get the error
Document: "index.html" is not a file!
well, it certainly is. But it is mode 444. Which is fine because I am not updating it, just checking it.
Changing the file mode to 666 makes it work.

I installed this version from macports tidy-5.6.0_0.darwin_16.x86_64.tbz2

[geoffmcl]

@thvv wow what an interesting discovery...

It seems it is not only on the MAC, but is the same in unix/linux, and Windows, using current tidy 5.7.3...

I just set a file to 444 - $ chmod 0444 temp.html, and then $ tidy temp.html and I get Document: "temp.html" is not a file! - Zute, or as Snagglepuss would say Heavens to Murgatroyd!!!

It certainly seems at least one write bit must be set, like $ chmod 0644, and tidy will be ok...

On the face of it this does not seem right. Provided you are not using the -modify option, which will try to overwite the file, it seems only read permission should be required...

AH HA! The answer is simple. The tidyDocParseFile service has the following code -

    int status = -ENOENT;
    FILE* fin = fopen( filnam, "r+" );

    if ( !fin )
    {
        TY_(ReportFileError)( doc, filnam, FILE_NOT_FILE );
        return status;
    }

    fclose( fin );

Now reading some fopen references that access string "r+" means **read/update**: Open a file for update (both for input and output). The file must exist.

And note that if I set the file read only in Windows, > attrib +R temp.html, tidy will fail at the same place...

Now after that code block, WIN32 and UNIX separate...

UNIX continues with fin = fopen( filnam, "rb" );... you will note the + has been removed...

Windows goes another code path, using in essence -

    HANDLE fp = CreateFileA( filnam, GENERIC_READ, FILE_SHARE_READ, NULL,
                              OPEN_EXISTING, 0, NULL );
    ...
    fin->map = CreateFileMapping( fp, NULL, PAGE_READONLY, 0, 0, NULL );
    ...
    data->view = MapViewOfFile( data->map, FILE_MAP_READ,
                                (DWORD)( data->pos >> 32 ),
                                (DWORD)data->pos, numb );

So we note also in Windows, that the concept of + is not required, with GENERIC_READ, PAGE_READONLY, and FILE_MAP_READ... this of course needs to be verified...

Reading this far, maybe, just maybe! there is a good case for removing that initial + also... but as I review the history, maybe NOT...

It should be noted that the above fopen( filnam, "r+" ); block was added relatively recently.. I trace it back to commit ce105dc, May 7, 2017... It was to address #391 - the fact that a directory would be treated as a file in unix/MAC...

And IIRC a fopen( dirname, "r" ); will succeed in unix - need to check this...

Maybe the whole block could be replaced with a crossported stat check, which would also yield the size and time, which would make some of the subsequent code both in windows and unix redundant...

I have been using the following, well tested, crossported service, in many projects, for a long time -

#ifdef WIN32
#define M_IS_DIR _S_IFDIR
#else // !WIN32
#define M_IS_DIR S_IFDIR
#endif
static struct stat buf;
enum DiskType {
    MDT_NONE,
    MDT_FILE,
    MDT_DIR
};

static DiskType is_file_or_directory(const char * path)
{
    if (!path)
        return MDT_NONE;
    if (stat(path, &buf) == 0)
    {
        if (buf.st_mode & M_IS_DIR)
            return MDT_DIR;
        else
            return MDT_FILE;
    }
    return MDT_NONE;
}
static size_t get_last_file_size() { return buf.st_size; }
static time_t get_last_file_time() { return buf.st_mtime; }

It seems this is a bug. Tidy should NOT have a problem with a read only files.

If the -modify option is used, then this problem should be handled at the output stage...

In any case tidy should not report is not a file! when it is a file...

What do others think... thanks...

[geoffmcl]

@thvv, just to confirm one small test done of fopen( filnam, "r" );, that is without the +, succeeds in unix, and probably in OSX, even if the input is a directory, so can not just remove it...

Am tending toward using a 'stat' test and also fail if a directory, but would need another message like say FILE_IS_DIRECTORY, "%s is a directory!" so tidy can correctly report the problem...

As stated look forward to further feedback... thanks...

[i-give-up]

Yes, encountered the same "no file" error in Windows 8.1 when the html file was opened in gVim (which I suppose locks the file from being written by another program).

I have no idea about the code but yes it shouldn't fail if the file is read-only.

[geoffmcl]

@hosiet repeated this bug in #789 ... any help appreciated... thanks...

[jidanni]

[Seen on 5.6. I wrote this offline before connecting to search for
already filed bugs.]

Tidy can't deal with readonly files even though we are not requesting overwrite

$ tidy -eq  /usr/share/doc/aptitude/html/en/ch02s04s05.html
Document: "/usr/share/doc/aptitude/html/en/ch02s04s05.html" is not a file!

First bug: It is. So don't say that. Proof:

$ tidy -eq < /usr/share/doc/aptitude/html/en/ch02s04s05.html
line 676 column 39 - Warning: trimming empty <p>

So what's the problem?

$ strace tidy -q /usr/share/doc/aptitude/html/en/ch02s04s05.html 2>&1
openat(AT_FDCWD, "/usr/share/doc/aptitude/html/en/ch02s04s05.html", O_RDWR) = -1 EACCES (Permission denied)

This is crazy.

$ cp /usr/share/doc/aptitude/html/en/ch02s04s05.html /tmp/g.html
$ strace tidy -q /tmp/g.html 2>&1
openat(AT_FDCWD, "/tmp/g.html", O_RDWR) = 3
close(3)                                = 0
openat(AT_FDCWD, "/tmp/g.html", O_RDONLY) = 3

Second bug: I am not asking tidy to overwrite the file. So don't use
that test in such cases!

Else one cannot pretty print any readonly file, no matter to STDOUT, a
pipe, or another file.

[geoffmcl]

@i-give-up, do not know about gVim in Windows, but have encountered other editors, that use some form of file locking, sometimes by just keeping the file open, but have not tested how tidy would react in that circumstance, especially that libTidy uses file mapping... that would need to be explored...

And @jidanni thank you for repeating this known bug...

In discussion, the code problem has been identified, and code solutions offered, at least for the read-only files...

Just waiting for someone to step up with a patch, or PR, to fix this crazy bug... thanks...

[geoffmcl]

20201007:

@thvv, @hosiet, all, any, still waiting for someone to step up with a patch, PR, to fix this crazy bug...

The discussion above offers solutions... need help... hopefully more that a +1... thanks...

[geoffmcl]

@thvv, @hosiet - #789, @i-give-up, @jidanni, @jmadiot, @d0ugm - #918, etc, have got around to testing the following patch...

diff --git a/src/tidylib.c b/src/tidylib.c
index 75cb1d9..4d57510 100644
--- a/src/tidylib.c
+++ b/src/tidylib.c
@@ -1125,19 +1125,26 @@ int TIDY_CALL  tidyParseSource( TidyDoc tdoc, TidyInputSource* source )
     return tidyDocParseSource( doc, source );
 }
 
-
+#ifdef WIN32
+#define M_IS_DIR _S_IFDIR
+#else // !WIN32
+#define M_IS_DIR S_IFDIR
+#endif
 int   tidyDocParseFile( TidyDocImpl* doc, ctmbstr filnam )
 {
     int status = -ENOENT;
-    FILE* fin = fopen( filnam, "r+" );
-
-    if ( !fin )
+    FILE* fin = 0;
+    struct stat sbuf = { 0 }; /* Is. #681 - read-only files */
+    if ( stat(filnam,&sbuf) != 0 )
     {
         TY_(ReportFileError)( doc, filnam, FILE_NOT_FILE );
         return status;
     }
-
-    fclose( fin );
+    if (sbuf.st_mode & M_IS_DIR) /* and /NOT/ if a DIRECTORY */
+    {
+        TY_(ReportFileError)(doc, filnam, FILE_NOT_FILE);
+        return status;
+    }
 
 #ifdef _WIN32
     return TY_(DocParseFileWithMappedFile)( doc, filnam );
@@ -1147,7 +1154,6 @@ int   tidyDocParseFile( TidyDocImpl* doc, ctmbstr filnam )
 
 #if PRESERVE_FILE_TIMES
     {
-        struct stat sbuf = { 0 };
         /* get last modified time */
         TidyClearMemory(&doc->filetimes, sizeof(doc->filetimes));
         if (fin && cfgBool(doc, TidyKeepFileTimes) &&

At present only tested it in Windows 10, and it seems to work ok...

As time permits, will get around the creating an issue-681 branch, or the like, so it can be easily pulled, checked out, and test in all OSes, distros, etc... especially the MAC...

Meantime, would really appreciate others applying the patch, and testing it in your local system, and reporting, so we can get rid of this silly BUG... thanks...