Audit & other reports for City Hall as static text in PUBLISH folder

[tevpg]

This would make current audit info available on the ipad. This is especially useful when a person is walking around the valet reconciling system data to what's on the floor.

Effort: surprisingly little! (Except for the complications arising from use of iprint() -- see comment below)

Proposal:

• when TT saves, it also saves an audit report in text format on the shared google drive.
• ipad browser (or a person's phone for that matter) reads the file.

Comments

• opening a link to a google drive text file puts it inside a google drive context, which may not be desirable (because it uses screen real estate)
• opening as a file:// is superior, but this requires that the remote viewer has the shared google drive content presented as part of its filesystem. Don't know how fussy this is for ipad, it would certainly not be part of valets' phones.
  • if it were accessed in the filesystem, then could write as html, which means ablity to include colours etc
• embedding the google drive text file in a google sites page makes a good presentation, and the google site seems to (re-)read the content from the embedded text file with each page refresh. So this is a good solution, though it does require that the Valet google account set up a google sites page with the info on it.

Of course, the great way to do this is to have a server available, and a script that processes the TT datafiles on the fly. And while we're doing that, then of course allow the ability to update through a web interface (contention dealt with through opportunistic concurrency control). Mmmmm, nice.

Ties to (Issue #53)

[tevpg]

Current reports use iprint() with style= flag (which then calls text_style() for formatting codes which themselves derive from colorama attributes which are managed in tagtracker_config.py

Thoughts on approaches for printing the various reports to plain text in a file:

My current thinking is some combination of 4,5,6,8

1. create an alternative to iprint() which ignores the text styling, and writes to a file instead of screen (e.g. fiprint()). The reports get passed a function to do its printing - either iprint() or fiprint(). This is semi-elegant but that's a lot of layers of function to pass it through. Alternatively, could manage it as a global. [which is a minor variation on 5]
2. monkey-patch iprint() to (eg) fiprint() when starting the reports, then switch back. This is quick but fragile
3. copy the reports into another module which does file-based reports, duplicating code. (yuck)
4. create a 'printer' module (maybe import tt_print as p) that encapsulates the colour styles, and methods for printing to file or to screen (etc). Reports and whatever could be passed a trackerday object and a styler object and not much else. Because the text styling implicit in iprint() is what stops so many things from leaving tagtracker.py for other modules, this would be a bit of a breakthrough.
   • would include methods to set/restore USE_COLOUR
   • iprint(), text_style(), Block, Visit, [etc] could be moved to the utilities module
   • reports could be moved to a reporting module
   • colorama etc would move from config to utilties
   • utilities would need to import that module
5. Have iprint() read a global (sigh) to know whether printing to a file or to screen
   • this could sort of work with a tt_printer() module that would have the colour stuff in it
   • possibly push/pop current USE_COLOUR state
   • persistent concept (somewhere) of current file destination
6. redirect STDOUT to file during when printing these for-text-file report. (Requires temporarily disabling USE_COLOUR,, or otherwise dealing with control codes in the text output)
7. reports always print to both screen and file
   • so... managing a file handle could be the trick there
   • also an interesting idea would be to spool all screen i/o to a log, to help support. (Too much google drive traffic?)
8. reports return a list of lines rather than doing printing. A higher-level routing can then make determination on where to print
   • trick bit here is applying styling (or not). So perhaps returns 2 lists - one plain, one styled

[tevpg]

Discussion copied from discord;
Vladimeme — Yesterday at 9:01 AM
Quick question: what do you mean by "monkey-patch" in #2?
Toddzil — Yesterday at 9:09 AM
Would change the definition of iprint() at runtime to a different function while printing to file, then change it back.
https://stackoverflow.com/questions/5626193/what-is-monkey-patching
Stack Overflow
What is monkey patching?
I am trying to understand, what is monkey patching or a monkey patch?

Is that something like methods/operators overloading or delegating?

Does it have anything common with these things?
Image
(I suspect that doing an import of readline monkey-patches the input() function)
Vladimeme — Yesterday at 9:11 AM
Ah, I see
Now I know what it is I'm even more leary of going with option 2* there
Toddzil — Yesterday at 9:13 AM
Yeah. It violates the principle of what you see is what you get
I would hold it back for things like testing, debugging or desperation
Vladimeme — Yesterday at 9:24 AM
4: I like the organization this enables, as well as the ability to potentially use current reporting code in other modules

5: Having iprint() also do file writing - this seems potentially confusing to future code editors including us

6: This is interesting. As an alternative to temporarily flipping on and off a global, would it maybe be better for processes that read USE_COLOUR to take a colour usage argument that just defaults to reading the value of USE_COLOUR?

7: For support I quite like the idea of having I/O logged in a way we can view it remotely, but I have no idea what amount of Google drive traffic is too much. It could be that collaboratively editing a Google Slides presentation is more traffic than this and it would be totally fine
Toddzil — Yesterday at 9:24 AM
(Back to monkey-patching. Thinking a reasonable production use of it would be where you are swapping in a new funtion that has a complete superset of the functionality of the original)
Vladimeme — Yesterday at 9:25 AM
But wouldn't it be just as good to simply replace the original at its first definition?
If you're maintaining backwards compatibility with the older version
Toddzil — Yesterday at 9:28 AM
Well, a couple made-up examples.
you are redefining a built-in (like input())
you have a special case, like normally something runs in one language (LOCALE), but sometimes you want it to do side-by-side in two languages (say, english and spanish). So at the beginning of your script you would then import multi_en_sp and it would then redefine whatever routines were doing I/O.
Notice that in those examples, it's getting changed but not changed, changed back, changed to soimething else etc while running
Vladimeme — Yesterday at 9:29 AM
Ah, I was thinking of it on a smaller scale. That does make more sense
Toddzil — Yesterday at 9:29 AM
Mind you, I'm just making this stuff up. I don't have a lot of experience with this
Vladimeme — Yesterday at 9:30 AM
Right, that's fair
Toddzil — Yesterday at 9:33 AM
Your comment on (6) above. Passing a flag in is just the same as (1), isn't it? Which means every calling function has to know this low-level detail. Which is right and proper, but a lot of fuss. I'm more inclined to put it as persistence in a module, so that one might do something like:
pr.set_output("cityfile.txt")
pr.set_colour(False)
do_audit_report(pack_trackerday())
pr.reset_colour()
pr.reset_output()

(where pr is a tagtracker printing/styling module)
(You'll have noticied btw that I have also sneakily introduced the idea that a report would take as its argument a TrackerDay object -- which of course encapsulates all the data a report would need)
Vladimeme — Yesterday at 9:36 AM
Hmm, it would be pretty much the same as (1). This is tricky cause now I see the value in doing it either of these ways lol
Vladimeme — Yesterday at 9:37 AM
I do like that. Would allow us to more easily replace the unpacked globals in future

[tevpg]

Plan:

• set_output(destination:str="") in tt_print ("" resets to screen)
• (could do same for set_colour(on|off) -- add ON and OFF to globals
• for usage logging, might also make am set_echo(on|off)
• tt_print needs to keep track of its file. Open/close if destination changes
• iprint() looks at destination - uses no style if not screen
• iprint() echoes if echo location
• input() [name !!!??? tt_input seems so awkward] that will echo if called to do so
• readline import in tt_printer module
• a special city hall report that combines bits they care about
• maybe even their fullness histogram and busyness histogram
• run on publish cycle

Extra fiddly bits

• move input() to tt_printer() so can choose to echo input if desired (move readline import also)
• check that tt_globals import is actually needed in modules where it is imported
• [ ]

[tevpg]

Overall idea for published reports.

• if looking at old log, do not publish automatically (otherwise, do)
  • command 'publish' will force publishing even if old. (Prompt for confirmation if old file)
• report titles include long_date() in title
• report types:
  • summary table from audit with activity graph
  • summary table from busyness command, with fullness graph
  • right-now overview:
    • summary table from audit command
    • summary table from Busy command
  • graphs
    • busyness graph
    • fullness graph
  • summary stats
    • same as always
  • audit
    • standard audit report, for staff
• today v yesterday
  • maintain current-.txt and previous-.txt
  • publish will save to current-{subject}.txt
  • overwriting logic:
    • if not doing today's log, don't publish
    • inspect timestamp of a report
    • if timestamp is not from today
      • rename current-* to prev-*
    • write to current-*.txt
• presenting:
  • Google Site that includes the text files.
• (bug in fullness graph logic: if no events during a timeblock, it shows as empty)

[tevpg]

graphs.

Ideas below are interesting but realize now that must only choose single-width because different platforms/fonts render fancy characters in different widths.

Fullness.  Maybe O & o or O/r
Could show 
- oversize as 
  - '⚡' 0x26a1 *double width*
  - 'o'
  - 'B'
  - '■' 0x25a0

- regular as 
  - '⛮' 0x26ee *1.5x width*
  - '💪' 0x1f4aa *double width*
  - '⚙' 0x2699 *1.5x width*
  - 'r'
  - 'b'
  - '▪' 0x25aa
  
Transactions. O Ø O X
Could show
- check-ins as
  - '<'
  - 'i'
  - '↓' 0x2193
  - open circle
  - box
- check-outs as
  - '>'
  - 'o'
  - '↑' 0x2191
  - closed circle
  - box with checkmark

[tevpg]

So many kinds of reports, charts & graphs! It becomes kind of too many different commands.

operational reports:

• audit
• dataform
• recent

summary reports:

• stats
• busy
• full-graph
• busy-graph
• chart
• csv

might we combine at least the summary reports into something like:

• report [type..|all], where 'type' is
  • stat|summary
  • busy
  • activity
  • busy-graph
  • etc