Is the build step necessary?

[4O4]

Hi, I've got to ask if the build step which concatenates everything to a single file is really necessary? I'm just finishing a quite big refactoring of this library in order to add proper automatic unsubscription (this also required to fix some of other reported issues) and now I'm realizing that the concatenation is gonna be pain in the ass.

When dependencies between classes become more complicated it's all fine until you use standard modular approach with require() but once you put everything into single file, the local ClassName declarations introduce problems with variables scoping. If one class is defined above the other one which it depends on, the script will simply fail. Reordering will not help with that, because the dependency tree is actually more complicated, a bit circular even and more classes are in play (kind of unavoidable).

So... should I worry about this? Can we simply skip building single-file rx.lua lib and replace it with standard init.lua file with all needed exports required into it? Or do you actually need it for some reason and if yes, then if you're willing to adjust the build script by yourself to handle the new layout. Personally I'd prefer the first option because it is cheap, easy and compatible with Lua packaging standards established by the community.

[bjornbytes]

Lua does not have "relative require" functionality, so it is harder for users to consume libraries that consist of multiple files.

If I place the rx folder at lib/rx in my app, all the files in the library need to be changed to require lib.rx.observer, etc. There are some tricks RxLua could use to work around this, like using ... in the main chunk to figure out what the module was required as and chopping off bits of the path to construct a relative path. However, to keep things simple and to make the library easier to use, it was decided to use an "amalgamation" script that concatenates the library into a single file.

If there are circular dependencies between the objects, I think these could be resolved by placing a line like this at the top of the single rx.lua file:

local Observer, Observable, Subscription -- etc.

This kinda "forward declares" the different objects so that they can all know about each other, regardless of which order they're defined in.

The concatenate script could be changed to add this line (and maybe remove the leading local that appears as the first word in each file, so that there aren't 2 definitions of local).

[4O4]

Thanks for fast response! Usually adjusting lua_path or package.path is the way to go, isn't it? Luarocks handles this for you but it's of course not always available or usable in all environments and I get that. Out of curiosity... What kind of app is the one you're talking about? And yep, forward decls would do the job, but then you also have to remove leading local keyword from all places where the definition happens. Don't know how much of a big deal it is though.

…

On Sat, Oct 5, 2019, 21:58 Bjorn ***@***.***> wrote: Lua does not have "relative require" functionality, so it is harder for users to consume libraries that consist of multiple files. If I place the rx folder at lib/rx in my app, all the files in the library need to be changed to require lib.rx.observer, etc. There are some tricks RxLua could use to work around this, like using ... in the main chunk to figure out what the module was required as and chopping off bits of the path to construct a relative path. However, to keep things simple and to make the library easier to use, it was decided to use an "amalgamation" script that concatenates the library into a single file. If there are circular dependencies between the objects, I think these could be resolved by placing a line like this at the top of the single rx.lua file: local Observer, Observable, Subscription -- etc. This kinda "forward declares" the different objects so that they can all know about each other, regardless of which order they're defined in. The concatenate script could be changed to add this line (and maybe remove the leading local that appears as the first word in each file, so that there aren't 2 definitions of local). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#46?email_source=notifications&email_token=ABATUA4Q2XDYN7SOQPZ42QDQNDWXLA5CNFSM4I5YU7H2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAN2Q5Q#issuecomment-538683510>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABATUA6ZTBHFUFCTRQXLQ5DQNDWXLANCNFSM4I5YU7HQ> .

[4O4]

Oh It seems I didn't read till the very end, because you mentioned the leading local already. Sorry.

…

On Sat, Oct 5, 2019, 23:20 Paweł K ***@***.***> wrote: Thanks for fast response! Usually adjusting lua_path or package.path is the way to go, isn't it? Luarocks handles this for you but it's of course not always available or usable in all environments and I get that. Out of curiosity... What kind of app is the one you're talking about? And yep, forward decls would do the job, but then you also have to remove leading local keyword from all places where the definition happens. Don't know how much of a big deal it is though. On Sat, Oct 5, 2019, 21:58 Bjorn ***@***.***> wrote: > Lua does not have "relative require" functionality, so it is harder for > users to consume libraries that consist of multiple files. > > If I place the rx folder at lib/rx in my app, all the files in the > library need to be changed to require lib.rx.observer, etc. There are > some tricks RxLua could use to work around this, like using ... in the > main chunk to figure out what the module was required as and chopping off > bits of the path to construct a relative path. However, to keep things > simple and to make the library easier to use, it was decided to use an > "amalgamation" script that concatenates the library into a single file. > > If there are circular dependencies between the objects, I think these > could be resolved by placing a line like this at the top of the single > rx.lua file: > > local Observer, Observable, Subscription -- etc. > > This kinda "forward declares" the different objects so that they can all > know about each other, regardless of which order they're defined in. > > The concatenate script could be changed to add this line (and maybe > remove the leading local that appears as the first word in each file, so > that there aren't 2 definitions of local). > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#46?email_source=notifications&email_token=ABATUA4Q2XDYN7SOQPZ42QDQNDWXLA5CNFSM4I5YU7H2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAN2Q5Q#issuecomment-538683510>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ABATUA6ZTBHFUFCTRQXLQ5DQNDWXLANCNFSM4I5YU7HQ> > . >

[bjornbytes]

One example of an environment that uses Lua but doesn't use luarocks/LUA_PATH is the LÖVE game engine and RxLove. In that situation games are distributed as standalone packages where the configuration of the target machine is unknown.

[4O4]

Okay, thanks for the clarification. I know LOVE is a thing but have never been developing anything with it and I wouldn't have thought it has such issues. In that case I will try to adjust the build script along with all other things. Wouldn't it be better however to publish rx.lua file as an artifact via GitHub releases instead of commiting it directly to repo all the time?

…

On Sun, Oct 6, 2019, 01:25 Bjorn ***@***.***> wrote: One example of an environment that uses Lua but doesn't use luarocks/LUA_PATH is the LÖVE game engine and RxLove <https://github.com/bjornbytes/rxlove>. In that situation games are distributed as standalone packages where the configuration of the target machine is unknown. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#46?email_source=notifications&email_token=ABATUAY34ZNAN5U5GK6UGHTQNEO6XA5CNFSM4I5YU7H2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAN56MY#issuecomment-538697523>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABATUA7ZRU3Y7VVB5UTNAK3QNEO6XANCNFSM4I5YU7HQ> .

[bjornbytes]

I think it's a matter of preference. An artifact keeps the history clean, whereas committing the file makes it easier to use the library. My preference is to commit the file but I wouldn't really have strong objections to doing it a different way.

[4O4]

Small update: I found that squish util (https://github.com/LuaDist/squish) could be handy and replace current build script. It produces single lua file with all specified modules rendered into it. I've made a test run and all modules are actually being preloaded to package.preload table so it should work natively with require() in most environments, because preload is the first thing being scanned during modules loading at runtime. Will that work in LOVE too?

[4O4]

I've created #47 which addresses the issue with circular dependencies by using lua-amalg to create single-file lib as it seemed easier than maintaining the build script in the current form and adding handling of forward declarations to it.