Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lua table support #1700

Closed
Zash opened this issue Jun 17, 2023 · 2 comments
Closed

Lua table support #1700

Zash opened this issue Jun 17, 2023 · 2 comments

Comments

@Zash
Copy link
Contributor

Zash commented Jun 17, 2023

Please describe your feature request.
A clear and concise description of what the request is and what it would solve.
Eg. I wish I could use yq to support Lua tables as input and output formats. This would allow querying and manipulating Lua based file formats like LuaRocks rockspec format or the Prosody internal data storage format.

Describe the solution you'd like
If we have data1.yml like:
(please keep to around 10 lines )

country: Australia # this place
cities:
- Sydney
- Melbourne
- Brisbane
- Perth

And we run a command:

yq --output-format lua data1.yml

it could output

country = "Australia" -- this place
cities = {
  "Sydney",
  "Melbourne",
  "Brisbane",
  "Perth",
}

or alternatively (perhaps a flag)

return {
  country = "Australia"; -- this place
  cities = {
    "Sydney",
    "Melbourne",
    "Brisbane",
    "Perth",
  }
}

Lua tables support arbitrary types as both keys and values, but in stored serialized formats this is rarely used, so keeping to a subset such as only allowing scalar keys may be sensible.

Describe alternatives you've considered
Writing simple converters to/from Lua and yaml/json etc is relatively easy, I have a few of these already, but they lack the query and manipulation capabilities and usually don't preserve comments or order of keys like yq does.

Zash added a commit to Zash/yq that referenced this issue Jul 30, 2023
Ref mikefarah#1700

Basic but working serialization to Lua tables.
Zash added a commit to Zash/yq that referenced this issue Jul 30, 2023
Ref mikefarah#1700

Basic but working serialization to Lua tables.
Zash added a commit to Zash/yq that referenced this issue Aug 3, 2023
Ref mikefarah#1700

Basic but working serialization to Lua tables.
Zash added a commit to Zash/yq that referenced this issue Aug 5, 2023
Ref mikefarah#1700

Basic but working serialization to Lua tables.
mikefarah pushed a commit that referenced this issue Aug 11, 2023
* Implement basic Lua output support

Ref #1700

Basic but working serialization to Lua tables.

* Escape larger set of characters in Lua output

Started with a minimum of replacements, this should be more complete,
tho not all substitutions are strictly required in Lua.

* Print simple keys unquoted in Lua output

String keys that satisfy the requirements for variable names can be used
as keys without quotes in tables.

* Quote Lua keywords in table keys

Keywords are not valid as unquoted keys, thus must be quoted

* Make output of unquoted Lua table keys optional

Generally safer and simpler to not do it.

* Hook up settings for Lua output

* Allow special characters in Lua prefix and suffix

--lua-suffix='});^M' didn't work, so taking this approach instead

* Panic on unhandled YAML Kind in Lua encoder

* Handle YAML case varied booleans in Lua encoder

* Handle special-case numbers in Lua encoder

* Reject unhandled scalar Tags in Lua encoder

* Add note about how Lua nil is unsuitable as table key

Could add some context tracking in the future to allow rejecting nil in
a table key context.

* Return error instead of panic in Lua encoder

* Add initial test for Lua encoder

Boilerplate mostly copied from toml_test.go

* Additional Lua output tests

* Generate Lua encoder documentation

Mostly just for the boilerplate

* Convert octal for Lua output

Lua doesn't have the 0oNNN syntax for octal integers, only decimal and
hexadecimal, hence those can be passed trough as is while octal needs
special treatment.

* Implement indentation in in Lua output

* Respect string Style in Lua encoder

Lua has 'single', "double" and [[ long ]] strings.

* Expand Lua examples

* Output line comments in Lua output

* Implement Lua globals output mode
tearmoon pushed a commit to taamarin/yq that referenced this issue Aug 11, 2023
* Implement basic Lua output support

Ref mikefarah#1700

Basic but working serialization to Lua tables.

* Escape larger set of characters in Lua output

Started with a minimum of replacements, this should be more complete,
tho not all substitutions are strictly required in Lua.

* Print simple keys unquoted in Lua output

String keys that satisfy the requirements for variable names can be used
as keys without quotes in tables.

* Quote Lua keywords in table keys

Keywords are not valid as unquoted keys, thus must be quoted

* Make output of unquoted Lua table keys optional

Generally safer and simpler to not do it.

* Hook up settings for Lua output

* Allow special characters in Lua prefix and suffix

--lua-suffix='});^M' didn't work, so taking this approach instead

* Panic on unhandled YAML Kind in Lua encoder

* Handle YAML case varied booleans in Lua encoder

* Handle special-case numbers in Lua encoder

* Reject unhandled scalar Tags in Lua encoder

* Add note about how Lua nil is unsuitable as table key

Could add some context tracking in the future to allow rejecting nil in
a table key context.

* Return error instead of panic in Lua encoder

* Add initial test for Lua encoder

Boilerplate mostly copied from toml_test.go

* Additional Lua output tests

* Generate Lua encoder documentation

Mostly just for the boilerplate

* Convert octal for Lua output

Lua doesn't have the 0oNNN syntax for octal integers, only decimal and
hexadecimal, hence those can be passed trough as is while octal needs
special treatment.

* Implement indentation in in Lua output

* Respect string Style in Lua encoder

Lua has 'single', "double" and [[ long ]] strings.

* Expand Lua examples

* Output line comments in Lua output

* Implement Lua globals output mode

(cherry picked from commit d302d75)
tearmoon pushed a commit to taamarin/yq that referenced this issue Aug 11, 2023
* Implement basic Lua output support

Ref mikefarah#1700

Basic but working serialization to Lua tables.

* Escape larger set of characters in Lua output

Started with a minimum of replacements, this should be more complete,
tho not all substitutions are strictly required in Lua.

* Print simple keys unquoted in Lua output

String keys that satisfy the requirements for variable names can be used
as keys without quotes in tables.

* Quote Lua keywords in table keys

Keywords are not valid as unquoted keys, thus must be quoted

* Make output of unquoted Lua table keys optional

Generally safer and simpler to not do it.

* Hook up settings for Lua output

* Allow special characters in Lua prefix and suffix

--lua-suffix='});^M' didn't work, so taking this approach instead

* Panic on unhandled YAML Kind in Lua encoder

* Handle YAML case varied booleans in Lua encoder

* Handle special-case numbers in Lua encoder

* Reject unhandled scalar Tags in Lua encoder

* Add note about how Lua nil is unsuitable as table key

Could add some context tracking in the future to allow rejecting nil in
a table key context.

* Return error instead of panic in Lua encoder

* Add initial test for Lua encoder

Boilerplate mostly copied from toml_test.go

* Additional Lua output tests

* Generate Lua encoder documentation

Mostly just for the boilerplate

* Convert octal for Lua output

Lua doesn't have the 0oNNN syntax for octal integers, only decimal and
hexadecimal, hence those can be passed trough as is while octal needs
special treatment.

* Implement indentation in in Lua output

* Respect string Style in Lua encoder

Lua has 'single', "double" and [[ long ]] strings.

* Expand Lua examples

* Output line comments in Lua output

* Implement Lua globals output mode

(cherry picked from commit d302d75)
tearmoon pushed a commit to taamarin/yq that referenced this issue Aug 11, 2023
* Implement basic Lua output support

Ref mikefarah#1700

Basic but working serialization to Lua tables.

* Escape larger set of characters in Lua output

Started with a minimum of replacements, this should be more complete,
tho not all substitutions are strictly required in Lua.

* Print simple keys unquoted in Lua output

String keys that satisfy the requirements for variable names can be used
as keys without quotes in tables.

* Quote Lua keywords in table keys

Keywords are not valid as unquoted keys, thus must be quoted

* Make output of unquoted Lua table keys optional

Generally safer and simpler to not do it.

* Hook up settings for Lua output

* Allow special characters in Lua prefix and suffix

--lua-suffix='});^M' didn't work, so taking this approach instead

* Panic on unhandled YAML Kind in Lua encoder

* Handle YAML case varied booleans in Lua encoder

* Handle special-case numbers in Lua encoder

* Reject unhandled scalar Tags in Lua encoder

* Add note about how Lua nil is unsuitable as table key

Could add some context tracking in the future to allow rejecting nil in
a table key context.

* Return error instead of panic in Lua encoder

* Add initial test for Lua encoder

Boilerplate mostly copied from toml_test.go

* Additional Lua output tests

* Generate Lua encoder documentation

Mostly just for the boilerplate

* Convert octal for Lua output

Lua doesn't have the 0oNNN syntax for octal integers, only decimal and
hexadecimal, hence those can be passed trough as is while octal needs
special treatment.

* Implement indentation in in Lua output

* Respect string Style in Lua encoder

Lua has 'single', "double" and [[ long ]] strings.

* Expand Lua examples

* Output line comments in Lua output

* Implement Lua globals output mode

(cherry picked from commit d302d75)
@mikefarah
Copy link
Owner

Fixed in 4.35.1

@Zash
Copy link
Contributor Author

Zash commented Aug 30, 2023

Input support would be cool too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants