More detailed StatStruct show #39463

IanButterworth · 2021-01-31T04:48:05Z

Perhaps the show for StatStruct could display more than it currently does?

Currently

julia> f, io = mktemp();

julia> stat(f)
StatStruct(mode=0o100600, size=0)

This PR

julia> f, io = mktemp();

julia> stat(f)
StatStruct
   size: 0
 device: 16777220
  inode: 108982358
   mode: 0o100600
  nlink: 1
    uid: 501
    gid: 20
   rdev: 0
blksize: 4096
 blocks: 0
  mtime: 1.612067396520361e9
  ctime: 1.612067396520361e9

Edit: latest version

julia> stat(f)
/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_pOnU8k size: 0 bytes device: 16777225 inode: 114151349 mode: 0o100600 (-rw-------) nlink: 1 uid: 501 (ian) gid: 20 (staff) rdev: 0 blksize: 4096 blocks: 0 mtime: 2021-02-12T00:21:05-0500 (2 minutes ago) ctime: 2021-02-12T00:21:05-0500 (2 minutes ago)

julia> Base.active_repl.options.iocontext[:compact] = false;

julia> stat(f)
StatStruct for /var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_pOnU8k
   size: 0 bytes
 device: 16777225
  inode: 114151349
   mode: 0o100600 (-rw-------)
  nlink: 1
    uid: 501 (ian)
    gid: 20 (staff)
   rdev: 0
blksize: 4096
 blocks: 0
  mtime: 2021-02-12T00:21:05-0500 (2 minutes ago)
  ctime: 2021-02-12T00:21:05-0500 (2 minutes ago)

vtjnash · 2021-01-31T05:06:33Z

We typically like to keep show to one-liners, as they nest better when merged with other output. This might be good for mime-show?

For the short form, we could mimic BSD:

$ stat /dev/null 
990745296 308 crw-rw-rw- 1 root wheel 50331650 0 "Jan 30 11:31:21 2021" "Jan 30 23:58:08 2021" "Jan 30 23:58:08 2021" "Dec 31 19:00:00 1969" 131072 0 0 /dev/null
$ stat julia/VERSION 
16777220 50088609 -rw-r--r-- 1 jameson staff 0 10 "Jan 29 22:12:55 2021" "Dec  9 17:51:53 2020" "Dec  9 17:51:53 2020" "Dec  9 17:51:53 2020" 4096 8 0 julia/VERSION

For this full form, we could mimic Linux stat (BSD -x):

$ stat -x /dev/null 
  File: "/dev/null"
  Size: 0            FileType: Character Device
  Mode: (0666/crw-rw-rw-)         Uid: (    0/    root)  Gid: (    0/   wheel)
Device: 59,889552   Inode: 308    Links: 1
Access: Sat Jan 30 11:31:21 2021
Modify: Sun Jan 31 00:05:36 2021
Change: Sun Jan 31 00:05:36 2021
$ stat -x julia/VERSION 
  File: "julia/VERSION"
  Size: 10           FileType: Regular File
  Mode: (0644/-rw-r--r--)         Uid: (  501/ jameson)  Gid: (   20/   staff)
Device: 1,4   Inode: 50088609    Links: 1
Access: Fri Jan 29 22:12:55 2021
Modify: Wed Dec  9 17:51:53 2020
Change: Wed Dec  9 17:51:53 2020

IanButterworth · 2021-01-31T05:30:41Z

Yeah, nice.

Do we have functions to print the mode and any others in their more readable forms?

It'd be nice to also print the file path, but that's not part of StatStruct. Could we expand StatStruct to include it?

IanButterworth · 2021-02-01T01:32:32Z

For the moment, I'm just working on the way the fields are printed. The times now are shown in local time, with a human-friendly time difference in most significant units only, up to units of days

julia> f, io = mktemp();

julia> stat(f)
StatStruct
   size: 0
 device: 16777220
  inode: 109011905
   mode: 0o100600
  nlink: 1
    uid: 501
    gid: 20
   rdev: 0
blksize: 4096
 blocks: 0
  mtime: 2021-01-31T20:24:22-0500 (1 second ago)
  ctime: 2021-01-31T20:24:22-0500 (1 second ago)

...

julia> stat(f)
StatStruct
   size: 0
 device: 16777220
  inode: 109011905
   mode: 0o100600
  nlink: 1
    uid: 501
    gid: 20
   rdev: 0
blksize: 4096
 blocks: 0
  mtime: 2021-01-31T20:24:22-0500 (53 minutes ago)
  ctime: 2021-01-31T20:24:22-0500 (53 minutes ago)
  
...

  mtime: 2021-01-31T20:24:22-0500 (3 hours ago)
  ctime: 2021-01-31T20:24:22-0500 (3 hours ago)

Two questions:

Does anyone know of a better way to convert the mode to a readable string than mimicking this kind of thing? https://stackoverflow.com/questions/17809386/how-to-convert-a-stat-output-to-a-unix-permissions-string/17809850#17809850
Would we want to add a path field to StatStruct so that like BSD we can print the path, and help figure out the mode string wrt. being a directory or not

IanButterworth · 2021-02-03T03:58:22Z

Now with:

A desc field added to StatStruct to support a path or fd, and allow it to be printed during the show method (like BSD etc.)
Unix style mode strings. Order of operations based on python's os.stat.filemode(mode) _filemode_table

julia> d = mktempdir()
"/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_Pvk1UW"

julia> stat(d)
StatStruct for /var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_Pvk1UW
   size: 64
 device: 16777220
  inode: 109279042
   mode: 0o040700 (drwx------)
  nlink: 2
    uid: 501
    gid: 20
   rdev: 0
blksize: 4096
 blocks: 0
  mtime: 2021-02-02T22:47:34-0500 (just now)
  ctime: 2021-02-02T22:47:34-0500 (just now)


julia> f, io = mktemp()
("/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_YW2cX3", IOStream(<fd 23>))

julia> stat(f)
StatStruct for /var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_YW2cX3
   size: 0
 device: 16777220
  inode: 109279102
   mode: 0o100600 (-rw-------)
  nlink: 1
    uid: 501
    gid: 20
   rdev: 0
blksize: 4096
 blocks: 0
  mtime: 2021-02-02T22:47:42-0500 (1 second ago)
  ctime: 2021-02-02T22:47:42-0500 (1 second ago)

IanButterworth · 2021-02-03T04:03:25Z

I can work to fix the tests if we're ok adding a field, @vtjnash ?

Also, I'm still planning to make a compact show the default, just working on getting the fields right still.

Can any other fields be more richly printed?

IanButterworth · 2021-02-07T04:02:29Z

Now with username and group name lookup on Linux and MacOS

julia> stat(d)
StatStruct for /var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_jhT8ud
   size: 64
 device: 16777220
  inode: 109597893
   mode: 0o040700 (drwx------)
  nlink: 2
    uid: 501 (ian)
    gid: 20 (staff)
   rdev: 0
blksize: 4096
 blocks: 0
  mtime: 2021-02-06T22:19:05-0500 (41 minutes ago)
  ctime: 2021-02-06T22:19:05-0500 (41 minutes ago)

IanButterworth · 2021-02-12T05:25:59Z

Updated with a compact single-line form as default

julia> stat(f)
/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_pOnU8k size: 0 bytes device: 16777225 inode: 114151349 mode: 0o100600 (-rw-------) nlink: 1 uid: 501 (ian) gid: 20 (staff) rdev: 0 blksize: 4096 blocks: 0 mtime: 2021-02-12T00:21:05-0500 (2 minutes ago) ctime: 2021-02-12T00:21:05-0500 (2 minutes ago)

julia> Base.active_repl.options.iocontext[:compact] = false;

julia> stat(f)
StatStruct for /var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_pOnU8k
   size: 0 bytes
 device: 16777225
  inode: 114151349
   mode: 0o100600 (-rw-------)
  nlink: 1
    uid: 501 (ian)
    gid: 20 (staff)
   rdev: 0
blksize: 4096
 blocks: 0
  mtime: 2021-02-12T00:21:05-0500 (2 minutes ago)
  ctime: 2021-02-12T00:21:05-0500 (2 minutes ago)

IanButterworth · 2021-02-15T03:06:24Z

@vtjnash no rush on this, but do you think it's getting close?

vtjnash

Yes, this seems getting close

base/stat.jl

vtjnash · 2021-02-16T02:28:44Z

base/stat.jl

@@ -26,6 +26,7 @@ export
    uperm

 struct StatStruct
+    desc    :: Union{AbstractString, OS_HANDLE}


this will change == and hash, but I don't think people would have been relying on that?

@staticfloat just a thought.. is the StatStruct hash ever used for artifact hashing etc.?

GIven this broke existing symlink tests, I've defined custom hash and equality methods that skip desc and added a note to the struct

vtjnash · 2021-02-16T02:30:14Z

base/stat.jl

+            println(iob, " ctime: $(iso_datetime_with_relative(st.ctime, tnow))")
+        end
+    else
+        sprint() do iob


This should be the code for a different function (instead of compact):

Suggested change

sprint() do iob

function show(io::IO, ::MIME"text/plain", st::StatStruct)

You mean that you wouldn't have the user select compact or not via the compact flag?

Tests are passing now, and I believe this is the only outstanding issue, but I need help understanding what you mean. Are you suggesting the user selects the compact or full versions via some other mechanism than setting julia> Base.active_repl.options.iocontext[:compact] = false ?
Setting compact via that method is rather laborious, so I'm definitely not attached to it.

base/stat.jl

base/util.jl

base/filesystem.jl

vtjnash · 2021-04-16T20:38:46Z

bump?

IanButterworth · 2021-04-19T01:14:44Z

I won't have more time for this for a while, so feel free to push to it if you want to push it over the line @vtjnash

Edit: Just getting tests passing

staticfloat · 2021-05-12T15:29:31Z

I think we should just merge this. If noone else responds in the next day or two, I will.

vtjnash · 2021-05-12T17:28:03Z

I think there were a number of minor issues I've been meaning to address, but just haven't gotten time

IanButterworth · 2021-05-15T12:10:33Z

Happy to give a stab at finishing this, but I need some clarification on the thread above at least

vtjnash · 2021-05-16T02:16:48Z

Mostly finished. Will push when I'm done :)

vtjnash · 2021-05-20T22:43:54Z

okay, I pushed a cleanup commit (squash when merging)

IanButterworth · 2021-05-21T23:52:23Z

@vtjnash is the cause of the error here obvious?

      From worker 2:	julia: /buildworker/worker/package_linux64/build/src/sys.c:437: jl_os_get_group: Assertion `gr_mem == buf + name_size + mem_size' failed.
      From worker 2:	
      From worker 2:	signal (6): Aborted
      From worker 2:	in expression starting at /buildworker/worker/tester_linux64/build/share/julia/test/file.jl:685

IanButterworth · 2021-05-30T07:13:37Z

Bump @vtjnash

vtjnash · 2021-05-31T15:21:09Z

oops, I miscounted the bytes, and didn't realize I had asserts disabled

KristofferC · 2021-06-02T16:00:52Z

This fails on Windows: https://build.julialang.org/#/builders/66/builds/3795/steps/5/logs/stdio

Need to check for the correct substring.

JeffBezanson · 2021-06-02T21:17:18Z

src/sys.c

+#ifdef _OS_WINDOWS_
+  return UV_ENOTSUP;
+#else
+  // taken directly from libuv


Can we call libuv instead of copying the code?

I added an extra argument, so we need to upstream this first.

Need to check for the correct substring.

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

Need to check for the correct substring.

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

Need to check for the correct substring.

DilumAluthge requested a review from JeffBezanson January 31, 2021 04:50

IanButterworth force-pushed the ib/statstruct_show branch from 7634bb4 to 28dc1a7 Compare January 31, 2021 04:54

IanButterworth mentioned this pull request Feb 12, 2021

Add shorthand for controlling the repl iocontext compact setting #39633

Closed

vtjnash reviewed Feb 16, 2021

View reviewed changes

IanButterworth force-pushed the ib/statstruct_show branch 2 times, most recently from 065d46e to 511b655 Compare April 19, 2021 01:02

mbauman added the domain:display and printing Aesthetics and correctness of printed representations of objects. label Apr 19, 2021

IanButterworth force-pushed the ib/statstruct_show branch from a6acc40 to 568a69a Compare May 2, 2021 03:34

vtjnash force-pushed the ib/statstruct_show branch from 568a69a to 1e97b0d Compare May 20, 2021 22:43

IanButterworth and others added 3 commits May 31, 2021 10:36

full detail show for StatStruct

9085a7c

move platform-specific code to C, use libuv, refactor

dc77739

fixup! move platform-specific code to C, use libuv, refactor

9fc7a41

vtjnash force-pushed the ib/statstruct_show branch from 1e97b0d to 9fc7a41 Compare May 31, 2021 15:20

vtjnash merged commit 61caf7f into JuliaLang:master May 31, 2021

IanButterworth deleted the ib/statstruct_show branch May 31, 2021 21:47

vtjnash added a commit that referenced this pull request Jun 2, 2021

fix for StatStruct show test from #39463

1a01667

Need to check for the correct substring.

JeffBezanson reviewed Jun 2, 2021

View reviewed changes

vtjnash added a commit that referenced this pull request Jun 2, 2021

fix for StatStruct show test from #39463 (#41060)

0432ea3

Need to check for the correct substring.

shirodkara pushed a commit to shirodkara/julia that referenced this pull request Jun 9, 2021

More detailed StatStruct show (JuliaLang#39463)

b33d492

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

shirodkara pushed a commit to shirodkara/julia that referenced this pull request Jun 9, 2021

fix for StatStruct show test from JuliaLang#39463 (JuliaLang#41060)

1c34627

Need to check for the correct substring.

johanmon pushed a commit to johanmon/julia that referenced this pull request Jul 5, 2021

More detailed StatStruct show (JuliaLang#39463)

2ee76ba

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

johanmon pushed a commit to johanmon/julia that referenced this pull request Jul 5, 2021

fix for StatStruct show test from JuliaLang#39463 (JuliaLang#41060)

9012a3c

Need to check for the correct substring.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

More detailed StatStruct show #39463

More detailed StatStruct show #39463

IanButterworth commented Jan 31, 2021 •

edited

vtjnash commented Jan 31, 2021

IanButterworth commented Jan 31, 2021

IanButterworth commented Feb 1, 2021 •

edited

IanButterworth commented Feb 3, 2021

IanButterworth commented Feb 3, 2021

IanButterworth commented Feb 7, 2021

IanButterworth commented Feb 12, 2021

IanButterworth commented Feb 15, 2021

vtjnash left a comment

vtjnash Feb 16, 2021

IanButterworth Apr 19, 2021

IanButterworth Apr 20, 2021

vtjnash Feb 16, 2021

IanButterworth Apr 19, 2021

IanButterworth May 2, 2021

vtjnash commented Apr 16, 2021

IanButterworth commented Apr 19, 2021 •

edited

staticfloat commented May 12, 2021

vtjnash commented May 12, 2021

IanButterworth commented May 15, 2021

vtjnash commented May 16, 2021

vtjnash commented May 20, 2021

IanButterworth commented May 21, 2021

IanButterworth commented May 30, 2021

vtjnash commented May 31, 2021

KristofferC commented Jun 2, 2021

JeffBezanson Jun 2, 2021

vtjnash Jun 2, 2021

	sprint() do iob
	function show(io::IO, ::MIME"text/plain", st::StatStruct)

More detailed StatStruct show #39463

More detailed StatStruct show #39463

Conversation

IanButterworth commented Jan 31, 2021 • edited

vtjnash commented Jan 31, 2021

IanButterworth commented Jan 31, 2021

IanButterworth commented Feb 1, 2021 • edited

IanButterworth commented Feb 3, 2021

IanButterworth commented Feb 3, 2021

IanButterworth commented Feb 7, 2021

IanButterworth commented Feb 12, 2021

IanButterworth commented Feb 15, 2021

vtjnash left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vtjnash commented Apr 16, 2021

IanButterworth commented Apr 19, 2021 • edited

staticfloat commented May 12, 2021

vtjnash commented May 12, 2021

IanButterworth commented May 15, 2021

vtjnash commented May 16, 2021

vtjnash commented May 20, 2021

IanButterworth commented May 21, 2021

IanButterworth commented May 30, 2021

vtjnash commented May 31, 2021

KristofferC commented Jun 2, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

IanButterworth commented Jan 31, 2021 •

edited

IanButterworth commented Feb 1, 2021 •

edited

IanButterworth commented Apr 19, 2021 •

edited