New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

the fish uvar file (fishd.$macaddr) should not be machine specific or at least the name should be configurable #1912

Closed
pwr22 opened this Issue Jan 21, 2015 · 61 comments

Comments

Projects
None yet
@pwr22
Copy link

pwr22 commented Jan 21, 2015

I use LXC containers a lot and am often snapshotting, restoring or cloning them. When this happens naturally they get a new MAC address and then I have to fiddle the fishd files around each time

Previously when the hostname was used this wasn't a problem with snapshotting but there were still issues with cloning a machine in which it will get a new hostname

I would like it if there were a way to configure a constant to use as the MACHINE_ID

@pwr22 pwr22 changed the title [fishd] Please give a way to configure the MACHINE_ID [fishd] Configurable MACHINE_ID Jan 21, 2015

@ridiculousfish ridiculousfish added this to the next-major milestone Jan 21, 2015

@ridiculousfish

This comment has been minimized.

Copy link
Member

ridiculousfish commented Jan 21, 2015

Seems reasonable. How would you like to see the MACHINE_ID specified?

@pwr22

This comment has been minimized.

Copy link

pwr22 commented Jan 21, 2015

Hmm, it can't be in a variable because that relies on fishd already running and having read the file. Perhaps the default behaviour could be overridden if ~/.config/fish/fishd.MACHINE_ID exists?

@zanchey

This comment has been minimized.

Copy link
Member

zanchey commented Jan 22, 2015

As you say, an environment variable would require editing of the system login configuration, so that's probably out. What about the existence of a file in .config/fish... .fishd.use.hostname, or even a link fishd.OVERRIDE or something?

@pwr22

This comment has been minimized.

Copy link

pwr22 commented Jan 22, 2015

I'd rather not the hostname, because when cloning a container this also changes, the the other sounds fine. Though I think it might be good to have this to allow people to specify the hostname behaviour

At the moment I have a file called fishd.master and am manually symlinking to this whilst the container isn't running to prevent fishd overwriting once I do it

@100ideas

This comment has been minimized.

Copy link

100ideas commented Nov 13, 2015

Sometimes my machine's MAC address changes, and consequently I have a variety of fishd. files in my .config/fish folder. Is there a way to manually set an ID string?

@ridiculousfish

This comment has been minimized.

Copy link
Member

ridiculousfish commented Nov 13, 2015

Not yet. How would you like this to work?

@pickfire

This comment has been minimized.

Copy link
Contributor

pickfire commented Nov 13, 2015

Yeah, the MAC address can be changed. I think the MAC address is not reliable.

https://wiki.archlinux.org/index.php/MAC_address_spoofing

@zanchey

This comment has been minimized.

Copy link
Member

zanchey commented Nov 13, 2015

Someone (I can't find who) suggested using /etc/machine-id if it exists, although care would need to be taken with upgrading users.

@jrobeson

This comment has been minimized.

Copy link

jrobeson commented Nov 16, 2015

yes please for /etc/machine-id (if it exists).

@pickfire

This comment has been minimized.

Copy link
Contributor

pickfire commented Nov 16, 2015

@zanchey Yeah, it exists for my system but there may be systems that doesn't have it. And it might be linux specific.

@jrobeson

This comment has been minimized.

Copy link

jrobeson commented Nov 16, 2015

@pickfire: I'm pretty sure it's systemd specific, not just Linux specific.

@olbrew

This comment has been minimized.

Copy link

olbrew commented Dec 14, 2015

I recently got bitten by fish's use of the MAC address as an identifier as well. I wrote a script to spoof my Mac address while I'm on some unsafe public networks and then noticed that fish didn't load my environment. Considering how easy and widespread (For example this popular spoofing program currently has 1500 stars.) the practice is I would also very much like another option to manually set a reliable identifier.

I've searched around a bit but I'm afraid that to automatically find a truly unique identifier is going to be different for every OS.

@pickfire

This comment has been minimized.

Copy link
Contributor

pickfire commented Dec 15, 2015

@olbrew Here is a one-liner:

ip link set eth0 down; and ip link set eth0 address c8:a7:ed:e2:1f:1d; and ip link set eth0 up
@olbrew

This comment has been minimized.

Copy link

olbrew commented Dec 21, 2015

@pickfire Thanks, that is a handy command. But I already knew how to change (or reset) the MAC address.

That's actually the problem, lots of people do, and for all those people who change it for testing, developing, privacy reasons or whatsoever the use of it as a stable identifier stops.

So that's why I would really like the option to manually send an identifier or another, more stable one, used for the Fish-shell daemon.

@ridiculousfish

This comment has been minimized.

Copy link
Member

ridiculousfish commented Dec 22, 2015

@olbrew how would you like to manually set it? What should the UI be for that?

@faho

This comment has been minimized.

Copy link
Member

faho commented Dec 22, 2015

@olbrew: Also, where should it be set? Currently, we load the fishd file before reading config.fish, so making it configurable there would be a (slight) compatibility break.

I'd rather we use a better source for the identifier (like /etc/machine-id, but I believe there's no equivalent on BSD or OSX?). Alternatively, does anyone actually use multiple fishd files on purpose? We could just drop it altogether and just use a file called "fishd" (or something better, since fishd actually doesn't exist anymore).

@zanchey

This comment has been minimized.

Copy link
Member

zanchey commented Dec 22, 2015

Solaris and FreeBSD have /etc/hostid, although notably on Solaris this starts with a #-delimited comment so you can't just read the file in.

@olbrew

This comment has been minimized.

Copy link

olbrew commented Dec 22, 2015

@faho I was thinking about what the reason behind the identifier was but couldn't come up with a use case that wouldn't be fixed by just placing it in the user's home folder. So yes, your suggestion of just dropping it (if possible?, you guys are much better placed than me to decide this) would absolutely be the best option I think.

@ridiculousfish If you need a random ID (which apparently could not be necessary? see above) I think generating one as explained in this gist under Random IDs from /dev/urandom at install time would be the best.
Maybe it's better to not manually set it after all and just replace the MAC-address with that . The best UI is no UI after all! 😉
Btw thank you all so much for your work! I'm absolutely loving Fish-shell.

@krader1961

This comment has been minimized.

Copy link
Contributor

krader1961 commented Dec 31, 2015

Note that multiple people have reported unexpected problems with their fish prompt in issue #1335 due to the universal var fishd file name changing.

Note that @olbrew's suggestion to just put the file in the "user's home folder" doesn't solve the problem. I'm assuming the suggestion is simply creating a $HOME/.fishd file. Consider NFS mounted home directories. Plus, the ~/.config/fish directory would normally be considered a subdirectory of the user's home folder and for that reason also doesn't change anything.

Honestly, I like @faho's suggestion nine days ago to just drop the concept of per-machine universal vars. The concept appears to be due to nothing more than a need to simplify the mechanism for setting and querying universal vars in a manner that can be easily coordinated by cooperating processes on a single machine. I'm pretty confident we can design an implementation that handles updates from multiple machines. At least assuming sane semantics for shared storage such as that provided by NFS or even cloud storage like Dropbox. The latter would likely require modification timestamps and related logic such as would be found in web browsers (e.g., Chrome) that support a unified config store.

@ridiculousfish

This comment has been minimized.

Copy link
Member

ridiculousfish commented Dec 31, 2015

My claim is that synchronization should already work except perhaps on lockless NFS - see this comment for the design. Of course synchronization only happens when a user changes a uvar, so races are rare anyways.

fish 1.x had per-hostname universal variables, and I think that's because axel had a shared home directory and wanted different uvars on different machines. There is some discussion in #183. Re-reading that thread, it doesn't look like anyone really wants the per-machine uvars - maybe it's time to nix it.

@krader1961

This comment has been minimized.

Copy link
Contributor

krader1961 commented Dec 31, 2015

My claim is that synchronization should already work except perhaps on lockless NFS...

What about lockless cloud storage like Dropbox or Google Drive or Amazon Cloud Drive? I use the first for my config files as a cheap and easy way to keep various "dot" files in sync across my systems. I suspect an increasing number of people are likely to do so as well in the future.

I do agree that per-machine uvars are far less useful than truly universal uvars and thus should be dropped. Whether or not we can achieve truly universal uvars is TBD. But I think we can greatly improve the current situation and make more people happy than sad by at least dropping the problematic per-machine qualification from the file name.

@ridiculousfish

This comment has been minimized.

Copy link
Member

ridiculousfish commented Dec 31, 2015

In the worst case, I think all that can happen is that if you make two simultaneous universal variable updates, one of the changes might get dropped. We should never get a corrupt file. At least that's the claim :)

@krader1961

This comment has been minimized.

Copy link
Contributor

krader1961 commented Dec 31, 2015

This comment by @kballard in issue #183 confuses me:

I don't mind per-machine "universal" variables, that makes sense.

That doesn't make sense to me. It seems to me the whole point of "universal" variables is that they are in fact universal and not per-machine. So the discussion in that issue that resulted in replacing the hostname qualifier with a Mac ID just replaced one failure mode with another. As far as I'm concerned the only vars that should be marked as universal are those that do not depend on the specific host (e.g., fish prompt colors). If the value depends on the current host then it should be a global rather than universal var (e.g., PATH).

@kballard

This comment has been minimized.

Copy link
Contributor

kballard commented Dec 31, 2015

@krader1961 There's two possible meanings for the word "universal":

  1. Variables that are shared among all instances of the shell on a given machine, and
  2. Variables that are shared among all instances of the shell everywhere (assuming some sort of shared home folder).

I only care about the first one. Of course, I don't actually share my home folder, but I do sync my dotfiles across machines. And I have to go out of my way to get per-machine customization in many cases.

It's simply not true to say that per-machine universal variables are less useful than "truly universal" variables. That depends entirely on the variable. Some variables I can see being useful globally. Others should specifically be per-machine.

@ridiculousfish If you want to have "truly universal" variables, I think fish actually needs to support both modes. Alternatively, we could just have a built-in electric variable that overrides the machine ID. You can then set this to the same value on all your machines to get "truly universal" variables. This could in fact be stored as a per-machine universal variable (that always uses the default machine ID for its storage), or I suppose it could just be a global variable that you set in your config.

@ridiculousfish

This comment has been minimized.

Copy link
Member

ridiculousfish commented Dec 31, 2015

Maybe the machine ID can be a normal global variable. I'm not sure why I thought it couldn't. You can control sharing (or not) via just a set command in config.fish.

@faho

This comment has been minimized.

Copy link
Member

faho commented Dec 31, 2015

Maybe the machine ID can be a normal global variable. I'm not sure why I thought it couldn't.

Currently, because we load the universal variables before we load the config file. Allowing customization via a global var would require that to be changed, and I'm not sure what should then happen when you set (edit: or use) a variable that is in the store in the config file.

@olbrew

This comment has been minimized.

Copy link

olbrew commented Dec 31, 2015

@krader1961 Just to clarify, I didn't want to suggest getting fishd out of $HOME/.config/fish/.
I just thought that the semantics of placing config in a home directory would mean that it's for that specific user and not machine-wide.

@krader1961

This comment has been minimized.

Copy link
Contributor

krader1961 commented Dec 31, 2015

@kballard Obviously you care about a specific use case and there are likely others who want the behavior you prefer. The problem is that it causes a lot of pain because both the host name and network MAC ID can change (especially when dealing with mobile devices like a laptop). And I'd be willing to bet that most people would prefer a "fishd" file whose name was not qualified in any way. That is effectively per-machine without a shared home directory (or at least putting the ~/.config/fish dir on shared storage) and avoids the problems caused when the name changes unexpectedly.

It seems to me that it's better to impose a small amount of pain (e.g., by requiring explicitly excluding the fishd file from those that are rsync'd) on people like yourself who want to sync everything but the universal var file than to impose a large amount of pain on everyone else. Having said that if we can find a robust means by which a user can specify the "machine id" portion of the fishd file name, with the default being a fixed string (e.g., "shared") then we can satisfy both groups.

@floam

This comment has been minimized.

Copy link
Member

floam commented Nov 9, 2016

Obviously, easier to just hardcode it in your personal fish repo - but a good example of why the current way we do this isn't ideal.

@ridiculousfish

This comment has been minimized.

Copy link
Member

ridiculousfish commented Nov 9, 2016

Here is a rough proposal:

  1. Introduce a variable fish_universal_id or similar. This will be set to empty, or perhaps "default", by default.
  2. The fish_universal_id maps to a universal variable file ~/.config/fish/fishd.$fish_universal_id
  3. Changes to fish_universal_id cause fish to dynamically switch to a different universal variable set

This is nice because:

  1. The default behavior is just a name (not machine dependent), so it can be synced, transferred, etc. Most users should be happiest with this default.
  2. Users with shared home directories can make the value per-hostname by simply adding set fish_universal_id (hostname) to their config.fish
  3. Advanced users can swap among different universal variable sets (a la rbenv, etc).

This ought to be straightforward to implement, because universal variables are already instanced (env_universal_t). Naturally, fish_universal_id could not itself be universal.

A tweak to this is to make fish_universal_id into an explicit path (fish_universal_path) but that might overly constrain the implementation, and has security concerns.

Thoughts?

@floam

This comment has been minimized.

Copy link
Member

floam commented Nov 9, 2016

Should history items be keyed by this?

@krader1961

This comment has been minimized.

Copy link
Contributor

krader1961 commented Nov 9, 2016

My preference is to have a single uvar config file with a scheme for distinguishing vars set in different namespaces; e.g., prefix the var name with the namespace. For discussion purposes I'm going to use "local" to mean the namespace qualified instance of the uvar and "global" to mean the non-namespace qualified instance. We may want to use other terms to avoid confusion with non-universal global and local vars.

A simple set -U var val would set the value in the namespace identified by fish_universal_id or global if that var is not set. Adding the -g flag would explicitly set the global, non-machine specific version, without having to unset fish_universal_id; e.g., set -Ug var val. For symmetry set -Ul var val sets the "local" instance although obviously the -l flag is not needed since that would be the default.

When referencing a uvar the local instance is used if fish_universal_id is set and there is a local instance. Else fallback to the global instance if one exists. When querying or listing uvars you can add -g to list just the global, i.e., non-namespace qualified uvars. And add -l to list just the local, i.e., namespace qualified uvars. Without either flag (or if you use both, e.g., set -Ugl) you get the union of the local and global namespaces.

@maletor

This comment has been minimized.

Copy link

maletor commented Nov 10, 2016

My preference is to remove the universal file altogether instead of quibbling over the best hacky way to make it work.

The path we should be designing for is one where a fish user stores things they want to keep in a file checked into source control and shared amongst many machines. The universal file works against that because it would not be checked in and would therefore not be universal.

@shibotto

This comment has been minimized.

Copy link

shibotto commented Nov 10, 2016

In case someone needs this, this is the quite ridiculous situation on my laptop and how I "solved" it:

file .config/fish/fishd*
fishd:              ASCII text
fishd.0024be6f5c9b: symbolic link to fishd
fishd.0024be65f7e4: symbolic link to fishd
fishd.VPCS11E7E:    symbolic link to fishd
@krader1961

This comment has been minimized.

Copy link
Contributor

krader1961 commented Nov 10, 2016

@maletor: You can completely ignore the universal var file. Recent changes allow abbreviations to be set globally rather than as a uvar. Same for fish_key_bindings. Similarly, you can set all the prompt color vars as globals. I've only ever used fish_config while exploring problems reported by others. It's possible, even probable, that when changing the prompt theme it sets uvars. But, again, nothing keeps you from just copying them into your config file and setting them as globals.

Is there some reason you think you can't do it the way you desire?

@maletor

This comment has been minimized.

Copy link

maletor commented Nov 10, 2016

I'm conflicted because I like the concept of universal variables but I dislike the misdirection they create for maintaining the same configurations across all variable scopes and all machines.

If the universal file could be made to omit leaked __internal_fish_variables and lose the MACHINE_ID I think it'd fit a happier path for fish use cases across multiple machines.

The universal file would still be generated, but it could be one that we could check into source control so that laptops, work and home computers can all use the same global and universal variables. If one machine wants to not use specific variables then it only needs to not include the universal file from the source control repository -- as opposed to being a more complicated process of managing machine ids. Altogether, something that moves that burden to the user would be better as opposed to creating it as a restriction to be worked around.

It comes down to this for me: universal means universal to all my (opted in) machines not universal to a specific machine.

@floam

This comment has been minimized.

Copy link
Member

floam commented Nov 10, 2016

I guess I'm the minority on this thread, but I don't attempt to sync my configuration across machines and I think the majority of users are just using fish on their one computer.

@floam

This comment has been minimized.

Copy link
Member

floam commented Nov 10, 2016

I think when we're talking about putting files in version control, hand-editing config.fish files, doing things outside fish_config, we're kind of in the weeds of "administrative debris" which is the kind of thing we should try to make unnecessary through better design.

@krader1961

This comment has been minimized.

Copy link
Contributor

krader1961 commented Nov 10, 2016

It comes down to this for me: universal means universal to all my (opted in) machines not universal to a specific machine.

We seem to be in violent agreement 😄 Which is why I made this comment nearly a year ago soon after starting to use fish.

I'm pretty confident my most recent proposal would make everyone happy. The only thing left to figure out is a transition scheme.

@alphapapa

This comment has been minimized.

Copy link

alphapapa commented Nov 10, 2016

I think when we're talking about putting files in version control, hand-editing config.fish files, doing things outside fish_config, we're kind of in the weeds of "administrative debris" which is the kind of thing we should try to make unnecessary through better design.

@floam I confess that I don't understand this at all. :) Storing configuration in VCS is a great thing for so many reasons. Not only does it make syncing easy, but it makes for an easy kind of backups, easy publishing and sharing, and allows freedom to experiment with configs, break things, etc, since you can easily restore them.

This is the case for every program's configuration, not just Fish. I think we should always be designing in a way that pushes more configuration into plain configuration files that are loaded at runtime, easily reloaded, edited, and shared.

Regarding fish_config, I cringe every time I accidentally invoke it when I just want to make a simple change, having to wait for it to launch in a browser, often having to enable JavaScript in that tab, etc. I don't mind it existing, as I'm sure it's helpful for some users, but I would honestly be glad to disable it completely for myself.

Basically, with few exceptions, I don't think we can make configuration unnecessary through design. Configuration == freedom, and that's a good thing. :)

Personally, I like the sound of @ridiculousfish's idea. It's easy to understand, file-based, straightforward, etc. I like how it makes it easy to switch between sets of variables without overlap. @krader1961's idea sounds powerful, but it could be confusing since its sets overlap, and it's less clear where the different instances come from, vs. the non-overlapping, file-based method @ridiculousfish proposes.

Whatever is chosen, I'll be glad to put this behind us. :)

@floam

This comment has been minimized.

Copy link
Member

floam commented Nov 10, 2016

Regarding fish_config, I cringe every time I accidentally invoke it when I just want to make a simple change, having to wait for it to launch in a browser, often having to enable JavaScript in that tab, etc. I don't mind it existing, as I'm sure it's helpful for some users, but I would honestly be glad to disable it completely for myself.

So, better design would probably mean a fast and efficient CLI interface through fish_config, to me. But this is getting far astray of the topic.

graysonwright added a commit to graysonwright/dev that referenced this issue Dec 3, 2016

Pass PATH environment vars to fish containers
This gives us access to all executables in the `bin/` directory
whenever we're inside a `graysonwright/fish` container.

fish-shell/fish-shell#1912
@krader1961

This comment has been minimized.

Copy link
Contributor

krader1961 commented Feb 21, 2017

Another fish user has been confused by the per-machine uvars. See this StackExchange question posted a few hours ago. They copied their entire ~/.config/fish directory from an old system to a new one and could not understand why the color scheme was wrong.

@krader1961 krader1961 changed the title [fishd] Configurable MACHINE_ID the fish uvar file (*fishd.$macaddr*) should not be machine specific or at least the name should be configurable May 21, 2017

@krader1961 krader1961 changed the title the fish uvar file (*fishd.$macaddr*) should not be machine specific or at least the name should be configurable the fish uvar file (fishd.$macaddr) should not be machine specific or at least the name should be configurable May 21, 2017

@krader1961 krader1961 modified the milestones: fish-3.0, next-major Jun 22, 2017

@ridiculousfish

This comment has been minimized.

Copy link
Member

ridiculousfish commented Feb 28, 2018

The problem I ran into when making this configurable is that fish puts some of its own turds in it (__fish_init_2_39_8, __fish_init_2_3_0, __fish_init_3_x) so if the uvars file is changed fish reruns some initialization. I'm not sure what is the best way to deal with this.

@krader1961

This comment has been minimized.

Copy link
Contributor

krader1961 commented Mar 2, 2018

FWIW, I dropped the specialization of the uvar file name long ago in my private fork and couldn't be happier.

diff --git a/src/env_universal_common.cpp b/src/env_universal_common.cpp
index 880d5c73..c6eab065 100644
--- a/src/env_universal_common.cpp
+++ b/src/env_universal_common.cpp
@@ -89,8 +89,7 @@ static wcstring vars_filename_in_directory(const wcstring &wdir) {
     if (wdir.empty()) return L"";

     wcstring result = wdir;
-    result.append(L"/fishd.");
-    result.append(get_machine_identifier());
+    result.append(L"/fishd");
     return result;
 }

I can now copy my ~/.config/fish directory to a new host and (using fish built from my private fork) have my preferred customizations (in particular the color theme) immediately visible. I no longer have to figure out the macaddr of the source and new hosts in order to get a fish environment that looks familiar on the new host. I appreciate that my blunt tool approach is not appropriate for a fish shipped to thousands of users who are running unknown versions. But it reinforces my opinion that the current per-host uvar var file name mechanism causes more problems than it solves.

@faho

This comment has been minimized.

Copy link
Member

faho commented Mar 7, 2018

fish puts some of its own turds in it (__fish_init_2_39_8, __fish_init_2_3_0, __fish_init_3_x)

It might be possible to remove at least most of those.

Two migrate abbrs - first to the format with " " instead of "=", then to the new variables. We could run these and then delete the old variables, but that would make downgrading annoying. Always running them is probably too slow. So maybe these should be kept.

One guards setting the color variables, but they are all guarded via set -q $var; or set anyway, so I think we can just remove that one.

I have no idea what

        if not set -q __fish_init_2_3_0
            set line2 \n(_ 'Type `help` for instructions on how to use fish')
        end

is supposed to do. Why only add that if 2.3.0 has not been initialized?

Then there's the possibility to move to one variable and set it to the version. I.e. instead of __fish_init_2_3_0 and __fish_init_3_0_0 we'd just have __fish_init and set it to 300.


Anyway, I'm not quite sure this needs to be configurable. I think per-machine variables are quite rare, and if you want those you can just use globals.

@ridiculousfish

This comment has been minimized.

Copy link
Member

ridiculousfish commented Mar 14, 2018

Ok, you've convinced me. Let's simply drop the machine specific portion.

ridiculousfish added a commit that referenced this issue Apr 2, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment