Motivation for this change
Tivoli Storage Manager is IBM's backup solution. This pull request brings the client software and a NixOS module to use the client for regular system backups. Note that the client also provides an API to be linked against.
The text was updated successfully, but these errors were encountered:
I did some research on the syntax of
First off, some notes on server options:
I also tried to feed logging into syslog/journal, ultimately to no avail:
Further notable changes:
Thanks for your suggestions, @spacefrogg. It is really good to see that I will not be the sole user of that package! I applied most of your requested changes.
Considering your suggestion to create a wrapper script in a NixOS module, I beg to differ. Since creating that wrapper concerns several of your per-line requests, I will concentrate my response to that concept here.
I am convinced that the symlink to
On a sidenote: I used to have a similar setup on my machines -- I reassembled the
@Yarny0 Thank you for your effort! I understand your reasoning and will address each of your remarks in turn. Please note, that I am in no authority over merging this PR or demanding any changes of you. My code reviews are based on my experience and current understanding of how nixpkgs PRs are supposed to be designed to acceptable to people that actually do have the authority. So, feel free to ignore me.
Dear @spacefrogg, I will concentrate on the question of the (im)possibility of the symlink into
I'm not sure if your argument goes against a symlinks in particular or against any kind of reference that points outside of the Nix store. Please forgive if I'm bringing forward unnecessary arguments. Anyway, here's my stance on references that point out of the Nix store:
Is there any documentation or otherwise documented consensus that explicitely forbids -- or frowns upon -- references pointing out of the Nix store? What is the rational to avoid such references in general? Although I still consider myself a novice w.r.t. Nix packaging, the ban on outside references stands in stark contrast to everything in nixpkgs I've seen so far. On the other hand, not using the symlink would hamstring many of my use cases. So please understand my skepticism and excuse my insistence.
While looking for statements about the (in)validity of references pointing out of
The safety and comfort of a NixOS module is independent of the usage of
Do you refer to Apache's httpd? I'm not sure it fully compares to the problem at hand: httpd accepts a config file on its command line, and it is a server process not ment to be invoked by ordinary users. However, I tried it, and httpd from nixpkgs runs fine without a wrapper script -- even when called by non-root (on a port beyond 1024) and even on a non-NixOS machine. I'd like to reach a similar solution with
The software would need it as TSM's API needs the
Why would I add
Let me add and emphasize that I'm very greatful of this discussion, @spacefrogg. Irrespective of the outcome of this particular question, I'm aiming at a solution that covers as much use cases as possible, in a way that is as user/admin friendly as possible. So I would be very inclined if the version that is finally merged covers your use cases as well as mine. I'm still convinced that the symlink to
My argument goes against what I wrote just above. File-based references, i.e., softlinks and hardlinks.
These are not references of the same kind. The main difference between the two is in expectations. Resolving a configuration file from inside a running program is in the responsibility of the program itself. Providing a filesystem without broken links is not. You may not like this distinction, but I, myself, regard this as common consensus (by mere observation) among Linux distribution maintaining folks.
Maybe, but this similarity is not relevant. See my previous remark.
True, and the term you are looking for is purity, impure package and descendants. We try to keep packages pure as much and as often as possible. A package should not reference anything outside the known scope of the nix package manager with very few exceptions, e.g., runtime data (usually databases of any kind). And even that is a concession to practicality. I give you the very idea of NixOS:
The consequence from 3. is, that applications cannot have state, which is impractical, which is why local application state is treated separately. But even then, look at the postgres NixOS module and you will find that it uses versioned directories for its application state. This way, you at least have the possibility to downgrade again, should the database upgrade mess up your end-user application. On a 'normal' Linux distribution you would have literally messed up when you upgraded without making backups first.
As a sidenote: With
Okay, now to the other side of the story... As mentioned, NixOS allows application state, although it strives for purity through statelessness. If it would have been practically achievable and sustainable (i.e., maintainable), there would be no
We even support copying the file from the
So... stepping back, looking at the infrastructure NixOS actually provides you with to avoid statefulness. No, breaking this rule lightly and uncalled for is not acceptable.
I can only refer you to IRC or the discourse forum for further guidance. But anyway, could you please concisely list which of your use cases it would hinder to not set up the symlink but to follow the approach I showed you? I think, I haven't understood your problem domain, yet.
I hope, I could prove to you, that your conclusions are wrong, here. Sidestepping
But it is the very responsibility of a package maintainer to correctly instate the configuration of a package that depends on another one. If we could automate that task, we would do so. If it is as easy as providing every descendent with the right DSMI_DIR environment variable, it's even a no-brainer (maybe not trivial to write, just the architectural idea is). You could write a package setup hook in
Because it adds
I am not, but I think, it would cover my use cases.
I am always willing to share my insights.
Thanks for your response that helped me understand your objections more clearly. I updated the request in order to tweak some minor details (explained below). However, irrespective of other packaging details, I am still confused by the interplay of symlinks and purity:
This leaves me with the impression that it is considered alright when a program looks into
These definitions are independent of the usage of symlinks/hardlinks. In particular, by this definition, the programs listed above (and many more) suffer impurity. E.g. the command
I still can't grasp why dependence on outside state is acceptable unless a symlink is involved. While a filesystem with a "broken symlink" (as criticised above) isn't nice, it won't do any additional harm when compared to a missing config file that is referenced without passing through a symlink. For instance:
All those behaviours are certainly impure to the degree that local configuration is not included in the package itself. Yet, it matches most users' expectations, and it works flawlessly on NixOS and on non-NixOS systems. How the config file is discovered in the end is an implementation detail that is of no interest to the admin as long as where the file is to be placed is documented.
To aid the admin, I added a hint about the configuration file's placement in the
This leaves me confused. This pull request uses the method advertised in the third bullet point of yours to spare the admin the hassle of having to prepare the config file by hand. So, if you want to ensure that TSM's config file is in the Nix store, this pull request satisfies your needs. By using the NixOS module in this pull request, you will have a config file that is generated in the store and then referenced by a symlink in
As I'm still not sure how to balance practicability and purity here, I will consult Discourse.
Regarding the use cases I'd like to cover, they represent most combinations of
in addition to
The combination 2b+3a is of no real interest to me, the combination 2a+3b is used occasionally only. All other combinations are very important.
Talking about expectations, non-NixOS systems are a concern for me in particular, as admins are used to just modifying the config file and expecting the change to be active immediately (like adding a certificate to
When trying what you suggested,
Yet I want to thank you for reminding me of the
Thanks for you offer, and moreover thanks for the assessment of your use cases. This feeds me with optimism regarding this pull request.
This pull request has been mentioned on Nix community. There might be relevant details there:
Executive summary, see below for details:
This should suite both, NixOS and non-NixOS cases without relying on this broken symlink.
It is considered acceptable, because it is not sustainable to make all applications work without hardcoded configuration files. Maybe it helps to look at it from a different angle. You actually don't care where a configuration file is stored on the harddrive. What you do care about is, making the application access the right configuration when it is run. Having a hardcoded location for a configuration file, is a broken, old-school way of providing this functionality. Whenever you can tell your application where to find the right configuration at calltime, you don't need a hardcoded location. This is what we do on NixOS, if possible. There are usage scenarios, where it is important for a user to know what's inside a configuration file. But this is an abuse of hardcoded paths for providing a stable interface altogether. Ideally you'd have a way to ask your application or operating system for providing this information.
Again, the question is, why are hardcoded paths a broken interface? Answer, they lack the notion of multiple (and different) consistent application states. This cannot be done using hardcoded paths. This is the reason why there is a separate
Yes, they are independent of symlinks/hardlinks. On NixOS, programs generally do not suffer from impurity, because, to stay with you example, the administrator cannot write into
What I wanted to convey with my previous remarks, was the following: Nothing you say is plain wrong, but on a NixOS system, we want more guarantees. You are arguing towards not providing these guarantees in the way that I seem acceptable (aka. what I think the NixOS community finds acceptable). Nothing more, nothing less.
You're comparing apples and oranges, here. On a NixOS system this is exactly not how you use
This said, if you want a PR to be merged into NixOS, you should at least accept that this is the way software is used. No ad hoc hacking in
This is not good enough. If possible and sustainable,
True, not good enough. NixOS provides functionality, not inconsistent stuff that must be made consistent as an afterthought. This is where almost all other operating systems fail today.
True, I never argued against this part of your implementation. I merely shared my insights into the architecture of NixOS with you. I argued against putting a broken symlink into the nix store.
Thanks for sharing.
The expectation of ad hoc hacking configuration is not the focus of NixOS. If possible without violating purity, you may write the package in a way to also provide useful non-NixOS behaviour. If not, non-NixOS is not the focus, and folks there have to find a way around the issues. For your particular case, there is no such problem if you follow my suggestions. Non-NixOS systems can happily use
This is true only for non-NixOS systems. On NixOS, both, the application and the configuration are provided consistently at the same time. It is, thus, in the responsibility of the package maintainer to provide the necessary nixos module mechanics. Other distributions like Debian also expect from you to provide sane default configuration for a service you want to integrate in their system.
True. The decision has been made, wrap the binaries. No symlinks pointing outside the store. This decision hinges on filesystems not providing multiple consistent states on hardcoded paths.
Good, good! Do this! I was not sure myself. :)
The pull request received a major overhaul:
Details of changes in the package:
Details of changes in the module:
first of all, thanks again for your hints towards a better implementation. I revised the pull request based on ideas I received on discourse and on what I distilled from your latest response. Could you have a look at the current revision to judge whether the design satisfies purity requirements?
I tried to understand and follow your suggestions, but I'm still not sure if I graps your goals correctly. In case the current design is not acceptable, this is, essentially, where I'm struggling to follow your response:
Just to be sure so we're not talking cross purposes: In my initial design the files
Dear @spacefrogg, I updated the pull request and applied your requests (and responded to non-request comments). Two further remarks are in order:
IBM Spectrum Protect (former name: Tivoli Storage Manager) provides a single point of control for backup and recovery. This package contains the client software, that is, a command line client and linkable libraries. This commit adds two packages to nixpkgs: The TSM client software contains a Java GUI that, naturally, requires Java to be installed. To keep the closure size low, we provide the packages `tsm-client` and `tsm-client-withGUI`. The former comes without the Java GUI. While the product has been renamed, its old name is still alive in filenames and as package name used by other distros.
It seems thaht a subset of the XML is supported for options: https://github.com/NixOS/nixos-homepage/blob/master/nixos/options.tt#L279 but is not applied to packages. It'd be nice to port this over and support it in both!