stringx.expandtabs doesn't align to tabstops

[mjwhite]

stringx.expandtabs might be more useful if it matched the behaviour of terminals and text editors by aligning tabs to tabstops - that is, for each tab character, insert just the number of spaces needed to reach the next integer multiple of the tab size. Currently it replaces each tab with the specified number of spaces, regardless of where in the string it occurs.

Here's a quick go at implementing this:

function stringx.expandtabs(self,n)
    n = n or 8
    local s, nrep = self, 1
    while nrep > 0 do
        s, nrep = string.gsub(s,
            "()\t",
            function(pos)
                return string.rep(" ", n - ((pos-1) % n))
            end,
            1)
    end
    return s
end

And some tests that it behaves the same way as a terminal/editor:

asserteq(stringx.expandtabs("\t1"),
-- 12345678123456781
  "        1")

asserteq(stringx.expandtabs("\t\t1"),
-- 12345678123456781
  "                1")

asserteq(stringx.expandtabs("1234\t1"),
-- 12345678123456781
  "1234    1")

asserteq(stringx.expandtabs("12345678\t1"),
-- 12345678123456781
  "12345678        1")

asserteq(stringx.expandtabs("1\t1234567\t1"),
-- 12345678123456781
  "1       1234567 1")

I believe this matches the behaviour of str.expandtabs in Python.

[stevedonovan]

Yes, I was wondering about the exact behavior of the Python function, since stringx is so obviously a clone ;) The reference I found merely said that it turns tabs into spaces. The meaning you propose is certainly useful; perhaps an extra boolean argument to allow this to happen, and keeping the dumb functionality as the default?

[mjwhite]

Thanks for replying - yup, the Python docstring I checked is a bit unclear (the current manual is better). But anyway, this is how it behaves when you call it...

Adding a flag does avoid breaking old code; OTOH it also adds complexity (in particular, expandtabs(self,n,mode) means anytime you specify mode you also have to specify n, even though the n=8 default is probably the most common case). And all other things being equal, I suspect expand-up-to-tabstop is the more useful default in principle.

BTW, thank you for your work on penlight: it's an extremely convenient (and well-organised and well-documented) set of tools. :-)

[stevedonovan]

Thanks for your kind words! I am busy reorganizing things so that there are less inter-module dependencies. For instance, pl.data now only depends on pl.utils, and that dependency is only because the original design was that the rows would have the List metatable. (Actual loading of pl.List is done on a lazy on-demand basis). Generally things become clearer once I stopped trying to write Python ;)

Any other things which you would like to see in the library? Currently I'm contemplating adding pl.date for general data and time manipulation and parsing.

[mjwhite]

A date-time library sounds good.

What else? Hmm, JSON has made it into the Python standard library recently, which I think is a good thing - it's a widespread portable serial data format. Several languages include a basic general-purpose logging facility in their standard libraries, too, which might be useful here (especially if it fitted comfortably with lapp). Both of those would be natural enough in pure lua.

[mjwhite]

Gah, I apparently clicked the "Comment and close" button by mistake - not intentional, but I don't think I can re-open the issue myself... :-/

[stevedonovan]

Well, I shall mentally consider it still open (I don't see any way to re-open it here either)

As for JSON and logging, there's already a few Lua libraries that do that - JSON feels a little specialized for a general-purpose library.

As for date-time, I'm looking for a good model; I already know that the Java Date class is not that model ;)

[mjwhite]

Apparently there's a drop-down menu for project owners etc...

I guess it'd be nice for any new date-time stuff to interact reasonably cleanly with Lua's small set of existing built-in functions for this (os.date etc).