feat: parse the log for a list of shreds

[gacallea]

To replicate webchuck ui and to avoid running a --status command every time, implement a shred list.

• log chuck to file, besides having VM output
• parse logs to populate the shreds table
• create a list of shreds from the table
• display shreds in top right pane.

[gacallea]

something like this: https://gist.github.com/gacallea/254cb01872a034222c6034826e4bdc61

[gacallea]

notes from discord:

Andrea C From The App — Today at 6:15 PM I guess that the first thing I'd ask
help about is: passing a log file to the script as argument "to an exposed
function" and parsing the lines instead of hard coded variables. That would be
progress 🙂 ps: right now I'm mocking everything in a script. I hope that it
will be nvim-compatible :S

Cloud — Today at 6:22 PM If you want to pass a file, you have a couple options
Pass a /path/to/the/file.ext as an argument, and use io (nvim might have extra
filesystem interaction libraries that i don't know of) to open and use it

Open the file directly, and pass the handle around (might get messy if multiple
things try to touch it?)

Or open it, read it, and use an internal buffer passed around to "write" before
actually committing it to the disk 

Andrea C From The App — Today at 6:24 PM Hi @Cloud  🙂 the latter is not
feasible for me. At some point the script will have to be in its own split
parsing the log live to view the list of shreds. adding, replacing, removing
should be live and the table and UI updated accordingly. regarding multiple
things touching it: I'm not sure how to do this but: the log should be parsed
on new lines only to avoid dupes. each new line should trigger the action to
add/remove/replace based on matching. I don't even know this is possibe. Life
would be much easier if ChucK VM would just dump do console somehere besides
its own VM.

Cloud — Today at 6:28 PM So this should be pretty easy, the only major issue is
checking when a file has been written to, otherwise you'll just be on a
time-based loop for reading from the file realistically You can read a file
continuously in lua, and when it gets appended to, your next file:read("*a")
will read from your last point in the file (position 0 if no reads/seeks have
been performed) to the now last position in the file

Andrea C From The App — Today at 6:30 PM yay 🙂

Cloud — Today at 6:31 PM Then at that point you could just use a split function (custom logic, nvim might
have one builtin) to split on newlines. I don't know how file:lines() performs
when used from the middle of a file, for example, but it might work for exactly
what you need (and someone else might be able to clarify) for splitting on \n
(or carriage returns as well, depending) 

Andrea C From The App — Today at 6:32 PM thanks @Cloud I appreciate your input.
I will need to process it once my brain is back 😛 

[gacallea]

Drawer Foxie >w< — Today at 12:23 PM
wait your 

local file = assert(io.open(logfile, "r"))
    local line = file:read("*line")

    while line do
        shreds_manage(line)
        line = file:read("*line")
    end

    file:close()
 already works on live
the "file:read()" will wait
until new input

Andrea C From The App — Today at 12:24 PM
it closes here 😦 is that because chuck log doesn't set any line terminator?

Drawer Foxie >w< — Today at 12:28 PM
oki

Andrea C From The App — Today at 1:59 PM
at this point, I'm at a loss of ideas. it works for Foxie but not here. that is the last bit I miss before I can proceed with features using that table. @Drawer Foxie >w<  which file have you tried it on? how did you launch it? It may help me troubleshoot please

Drawer Foxie >w< — Today at 2:00 PM
it waits for all input before printing the output
the code still process line by line as it received

Andrea C From The App — Today at 2:04 PM
it would help if you could tell me which file you parsed and how (e.g: lua parse_chuck.lua "log" ?)
did you do it while the file was open and a process appending to it?
please and thank you

Drawer Foxie >w< — Today at 2:07 PM
a file and have another background process which copies your sample log one by one line every second to the new log which going to be read 

Andrea C From The App — Today at 2:08 PM
thanks. while that happened, did you issue lua parse_chuck.lua on a separate terminal? because that's what I do and it exits immediately

Drawer Foxie >w< — Today at 2:08 PM
yes i do

Andrea C From The App — Today at 2:08 PM
thanks, I appreciate 🙂

also: wtf. why does this closes immediately here!?!?!

Drawer Foxie >w< — Today at 2:09 PM
did you use io.open() or io.popen?

Andrea C From The App — Today at 2:09 PM
io.open as the gist code

Drawer Foxie >w< — Today at 2:10 PM
what operating system?

Andrea C From The App — Today at 2:10 PM
local function shred_lines(logfile)
    local file = assert(io.open(logfile, "r"))
    local line = file:read("*line")

    while line do
        set_table(line)
        line = file:read("*line")
    end

    file:close()
end

macOS Ventura
lua -v
Lua 5.4.6  Copyright (C) 1994-2023 Lua.org, PUC-Rio

Drawer Foxie >w< — Today at 2:11 PM
a question, the log file is ordinary file not a fifo pipe??

Andrea C From The App — Today at 2:11 PM
chuck --loop --srate44100 --bufsize512 --dac0 --adc0 --channels2 --in2 --out2 --remote127.0.0.1 --port8888 3>&1 2>&1 | tee dada.log 

Drawer Foxie >w< — Today at 2:11 PM
ah i see

Andrea C From The App — Today at 2:12 PM
that is because chuck ony outputs to its vm. that way I can log and parse. else I'm out of luck

Drawer Foxie >w< — Today at 2:12 PM
oki
then try "tail -f -n 0 dada.log"

Andrea C From The App — Today at 2:14 PM
I guess that at this point I should share the code in its entirety to make sense of it: https://github.com/gacallea/chuck-nvim/blob/main/lua/chuck-nvim/core/utils.lua
the above command is passed from chuck.lua into utils.lua and run upon ChuckLoop autocmd. so far they are separated commands. not sure I can pipe to tail as is.

Drawer Foxie >w< — Today at 2:16 PM
the same code but use "tail -f -n 0" inplace of io.open because i thought the log was fifo pipe not regular file

Andrea C From The App — Today at 2:17 PM
ok 🙂 I tried io.popen("tail -f -n 0 " .. logfile) but it freezes everything.

Drawer Foxie >w< — Today at 2:18 PM
it reads inputs and process it and wait
what else its going to do other than looking freeze while waiting input

Andrea C From The App — Today at 2:19 PM
I see, trying the following:

local function shred_lines(logfile)
  local cmd = "tail -F --lines=1"
  local file = assert(io.popen(cmd .. " " .. logfile, "r"))
  local line = file:read "*line"

  while line do
    shreds.set_table(line)
    line = file:read "*line"
  end

  file:close()
end

Drawer Foxie >w< — Today at 2:21 PM
i see, your program either wait on first read or second read

Andrea C From The App — Today at 2:22 PM
so, that freezes nvim entirely. also it doesn't start chuck. I should probably defer that function 
for commands, see https://github.com/gacallea/chuck-nvim/blob/main/lua/chuck-nvim/core/chuck.lua

Drawer Foxie >w< — Today at 2:23 PM
wait your program is running in nvim?

Andrea C From The App — Today at 2:24 PM
it is a nvim plugin 🙂

Drawer Foxie >w< — Today at 2:24 PM
i thought its seperate program

Andrea C From The App — Today at 2:24 PM
that was for writing code that I could test
new to lua and nvim, I had to take variable out of the equation to make something of my code

Drawer Foxie >w< — Today at 2:25 PM
does the chuck.lua run in its own process?

Andrea C From The App — Today at 2:27 PM
chuck.lua defines commands to manage chuck (the program). chuck --loop runs in a terminal in its own process. its output in vm, that I log and parse with the above command and code.
basically what I'm trynig to achieve is this https://github.com/gacallea/chuck-nvim/issues/5  -- I got everything but the parsing and table.

Drawer Foxie >w< — Today at 2:29 PM
then i dont know much about lua in nvim. my rough idea would be ask the "runtime" to run a function when a file now can be read without waiting

Andrea C From The App — Today at 2:30 PM
thank you so much for your help @Drawer Foxie >w< , I really appreciate you ❤️

Drawer Foxie >w< — Today at 2:30 PM
no problem >w<

Andrea C From The App — Today at 2:30 PM
I will try to investigate autoread next and see. I also tried using jobstart and jobsend but ehm

Drawer Foxie >w< — Today at 2:31 PM
oki

after a long discord session we realized the code parsing works in a lua script and:

then i dont know much about lua in nvim. my rough idea would be ask the "runtime" to run a function when a file now can be read without waiting

[gacallea]

maybe something like this: https://unix.stackexchange.com/questions/149209/refresh-changed-content-of-file-opened-in-vim/383044#383044

[gacallea]

from nvim on matrix, this should be it:

local b = vim.api.nvim_create_buf(true, true)
local w = vim.api.nvim_open_win(b, true, { relative = "editor", row = 10, col = 10, width = 30, height = 10 })
vim.bo[b].bufhidden = "wipe"
local l = ""
local j = vim.fn.jobstart({ "tail", "-f", "foo.log" }, {
	on_stdout = function(_, data, _)
		if not vim.api.nvim_buf_is_valid(b) then
			return
		end
		data[1] = l .. data[1]
		l = data[#data]
		data[#data] = nil
		vim.api.nvim_buf_set_lines(b, -1, -1, true, data)
		vim.api.nvim_win_set_cursor(w, { vim.api.nvim_buf_line_count(b), 0 })
	end,
})
vim.api.nvim_create_autocmd("BufWipeout", {
	buffer = b,
	once = true,
	callback = function()
		vim.fn.jobstop(j)
	end,
})

will implement it tomorrow.

[gacallea]

parsing teh log now works!

local function shred_set(line)
  local action = set_action(line)
  if line and action then
    shreds.set_table(line, action) -- this works and the table is populated
    layout.shreds_tree:render() -- the UI isn't updating tho. look into set_nodes?
  end
end

local function shred_lines(logfile)
  local line = ""
  vim.fn.jobstart({ "tail", "-f", tostring(logfile) }, {
    on_stdout = function(_, data, _)
      data[1] = line .. data[1]
      line = data[#data]
      data[#data] = nil
      if #data > 0 then
        shred_set(data[#data])
      end
    end,
  })
end

[gacallea]

fixed!