hstr stops working on linux >=6.2.0 (depending on kernel config)

[leapfog]

Just wanted to leave a note, that hstr stops working with linux kernel 6.2.0, when CONFIG_LEGACY_TIOCSTI isn't set. So maybe you want to add that information to a trouble shooting guide or FAQ or something.

(re: "stops working": the history can still be browsed, but cannot be inserted anymore)

[BlueMax]

Edit: Never mind, probably only switchable with CONFIG_LEGACY_TIOCSTI=y.

How is this meant to be done?

This functionality can be changed at runtime with the dev.tty.legacy_tiocsti sysctl. This configuration option sets the default value of the sysctl.
https://cateee.net/lkddb/web-lkddb/LEGACY_TIOCSTI.html

It can't be set though.

# sysctl -w dev.tty.legacy_tiocsti=1
sysctl: setting key "dev.tty.legacy_tiocsti": Invalid argument
# echo 1 > /proc/sys/dev/tty/legacy_tiocsti
bash: echo: write error: Invalid argument

Arch Linux 6.2.1

[sxe]

I came here as well cause hstr stopped working.
Could we get an official reply on how this situation will be handled? Will hstr be adjusted to work without it?
I would rather not enable a legacy option.

Cheers

[rajakesar]

❯ uname -r
6.2.1-arch1-1

❯ date
Thu Mar 2 10:38:30 AM EST 2023

Hstr is confirmed NOT working on the above kernel as of the time above.

[jakedane]

If you run zgrep CONFIG_LEGACY_TIOCSTI /proc/config.gz you can see CONFIG_LEGACY_TIOCSTI is not set for 6.2.1-arch1-1. If indeed hstr insertion is not working because CONFIG_LEGACY_TIOCSTI is not set then this is why for 6.2.1-arch1-1.

[mmeier86]

I can confirm that it's the CONFIG_LEGACY_TIOCSTI option. I had the same problem after I updated my Gentoo to 6.2, and after reenabling that option in my Kernel, it started to work again with 6.2.

[leapfog]

Seems fzf's history search is still working, even without CONFIG_LEGACY_TIOCSTI, so there is a way.

[jakedane]

Seems fzf's history search is still working, even without CONFIG_LEGACY_TIOCSTI, so there is a way.

fzf has a bash function around it for the history function, that sets the readline input buffer with the history line that was selected in fzf. The same can be done with hstr!

What I did:

1. Build hstr with DEBUG_NO_TIOCSTI set. You just need to uncomment this line and recompile hstr: https://github.com/dvorka/hstr/blob/master/src/hstr_utils.c#L29. With DEBUG_NO_TIOCSTI set hstr doesn't use the insecure TIOCSTI and instead prints the selected history line to stderr. On Arch Linux I did this with:

auracle download hstr
cd hstr
makepkg -so
sed -i -r 's|//(#define DEBUG_NO_TIOCSTI)|\1|' src/hstr-*/src/hstr_utils.c
makepkg -e
sudo pacman -U hstr-*.zst

2. We don't want the selected history line on stderr, we want it in the readline input buffer. I couldn't fully follow how fzf was doing that but on the wiki there is an example for bash: https://github.com/junegunn/fzf/wiki/Examples#with-write-to-terminal-capabilities. I used the 3rd example, the one starting with # re-wrote the script above, and adjusted it a bit for hstr. Basically: call hstr, redirect its stderr to a file so that can be read back into a variable and then pass that variable to the function from the wiki that sets the readline input buffer. Add this to your ~/.bashrc:

bind '"\C-r": "\C-x1\e^\er"'
bind -x '"\C-x1": __hstr';

__hstr ()
{
	hstr 2> ~/.hstr.tmp
	hstr_tmp=$(< ~/.hstr.tmp)
	__ehc "$hstr_tmp"
}

__ehc()
{
if
        [[ -n $1 ]]
then
        bind '"\er": redraw-current-line'
        bind '"\e^": magic-space'
        READLINE_LINE=${READLINE_LINE:+${READLINE_LINE:0:READLINE_POINT}}${1}${READLINE_LINE:+${READLINE_LINE:READLINE_POINT}}
        READLINE_POINT=$(( READLINE_POINT + ${#1} ))
else
        bind '"\er":'
        bind '"\e^":'
fi
}

3. And you need to disable the default hstr keybindings in your ~/.bashrc so comment out these lines:

# if this is interactive shell, then bind hstr to Ctrl-r (for Vi mode check doc)
if [[ $- =~ .*i.* ]]; then bind '"\C-r": "\C-a hstr -- \C-j"'; fi
# if this is interactive shell, then bind 'kill last command' to Ctrl-x k
if [[ $- =~ .*i.* ]]; then bind '"\C-xk": "\C-a hstr -k \C-j"'; fi

Now if I open a new terminal, press Ctrl+R, hstr works again and the selected history line is placed on the readline input.

I don't fully understand all of the above — and it can probably be cleaned up and made nicer — but at least I have a working hstr again.

[BlueMax]

CopyQ is another candidate that still works. I'm not sure but i think they load the clipboard and send a keystroke via X11 to paste the text.

[papavlos]

Thanks @jakedane for the workaround. It works!
There is a small issue - pressing Enter on selected command in history places the command into input buffer but without automatic execution. I have to press Enter again, but this is very small issue.

[Mte90]

I am on debian sid and stopped working too with 6.2.2-2-siduction-amd64 #1 SMP PREEMPT_DYNAMIC siduction 6.2-2.1 (2023-03-06) x86_64 GNU/Linux

apt source hstr
sed -i -r 's|//(#define DEBUG_NO_TIOCSTI)|\1|' src/hstr_utils.c
debuild -uc -us

Compiled with the patch, it still not work. I just get the command in the terminal after the selection compared to before.

[jakedane]

Compiled with the patch, it still not work. I just get the command in the terminal after the selection compared to before.

That sounds like you did step 1 from #478 (comment) but didn't proceed with step 2 and 3 to modify your .bashrc file (and open a new terminal after).

[Mte90]

So I tried that bash changes and in this way works only Ctrl+r not hh as example (I tried changing the bash-completion script):

#
# Copyright (C) 2014-2022  Martin Dvorak <martin.dvorak@mindforger.com>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Bash completion support for HSTR

# Source this file or install it to /usr/share/bash-completion/completions or /etc/bash_completion.d/
# See https://iridakos.com/tutorials/2018/03/01/bash-programmable-completion-tutorial.html
# help complete
# complete -W "--favorites --kill-last-command --non-interactive --show-configuration --show-zsd-configuration --show-blacklist --version --help" hstr

bind '"\C-r": "\C-x1\e^\er"'
bind -x '"\C-x1": _hstr';

#_hstr()
#{
#        local cur prev OPTS
#        COMPREPLY=()
#        cur="${COMP_WORDS[COMP_CWORD]}"
#        prev="${COMP_WORDS[COMP_CWORD-1]}"
#        case $prev in
#                '-h'|'--help'|'-v'|'--version')
#                        return 0
#                        ;;
#        esac
#        case $cur in
#                -*)
#                        OPTS="--favorites --kill-last-command --non-interactive --show-configuration --show-zsd-configuration --show-blacklist --version --help"
#                        COMPREPLY=( $(compgen -W "${OPTS[*]}" -- $cur) )
#                        return 0
#                        ;;
#        esac
#        compopt -o bashdefault
#        COMPREPLY=( $(compgen -c -- $cur) )
#        return 0
#}

_hstr ()
{
	hstr 2> /tmp/.hstr.tmp
	hstr_tmp=$(< /tmp/.hstr.tmp)
	__ehc "$hstr_tmp"
}

__ehc()
{
if
        [[ -n $1 ]]
then
        bind '"\er": redraw-current-line'
        bind '"\e^": magic-space'
        READLINE_LINE=${READLINE_LINE:+${READLINE_LINE:0:READLINE_POINT}}${1}${READLINE_LINE:+${READLINE_LINE:READLINE_POINT}}
        READLINE_POINT=$(( READLINE_POINT + ${#1} ))
else
        bind '"\er":'
        bind '"\e^":'
fi
}
complete -F _hstr hstr
complete -F _hstr hh

# eof

[leapfog]

While I still hope for hstr to be modified to work without CONFIG_LEGACY_TIOCSTI, kernel 6.2.x has been fixed, so re-enabling it at runtime is now possible:

# sysctl -w dev.tty.legacy_tiocsti=1 # fix hstr
dev.tty.legacy_tiocsti = 1

I only confirmed that with linux-6.2.6 today, but it might also have worked earlier.

[karlovskiy]

I can confirm that with 6.2.5-arch1-1 re-enabling sysctl -w dev.tty.legacy_tiocsti=1 is now also works.

[Mte90]

I can confirm too on Debian sid (siduction) with 6.2.6-1 the hstr package from debian repository works again.

[chrischmo]

Re-enabling sysctl -w dev.tty.legacy_tiocsti=1 also works on OpenSUSE Tumbleweed (6.2.4-1-default) with hstr from the lemmy04 repo.

[dvorka]

@Mte90 @chrischmo @karlovskiy @leapfog @jakedane @papavlos @mmeier86 thank you all for detailed descriptions of the problem, root cause identification and proposed solutions!

I worked on a version of HSTR for Cygwin and WSL where TIOCSTI is not available in the past. If you are able to build HSTR and you use Bash, can you please try #481?

The PR just enables existing HSTR functionality to work w/o TIOCSTI w/ the new define. I did not used it in the past because it requires shell command (and many users will not (be able to) do such configuration).

Anyway if the PR will work for you, I will fix it also for zsh (which I use on 90% of my machines) and release a new version.

Do you think that it would make sense to release HSTR with a TIOCSTI parameter allowing to run version with or without TIOCSTI use? Is there any way how to safely detect TIOCSTI availability? Will it compile on systems with new kernel?

Thank you all your interest in HSTR and your help!

[jakedane]

@dvorka I built hstr with #481 and with LINUX_KERNEL_6 defined, I put the new config in my .bashrc and with that Ctrl+R works properly in Bash. Thank you!

I'm on Arch Linux with kernel 6.2.6. No issue compiling.

I don't know how to safely detect TIOCSTI is available. Probably too hacky and not portable but on Linux if sysctl -ne dev.tty.legacy_tiocsti prints 0 (zero) that means TIOCSTI is disabled.

[dvorka]

Preliminary TIOCSTI design:

• .bashrc/.zshrc function:
  • detects TIOCSTI and based on that:
    • call the right Bash/zsh function which binds to the shortcut
    • hstr command w/ or w/o --no-tiocsti parameter
• hstr binary:
  • detects TIOCSTI and based on that it either uses it or not
    • hstr binary supports both modes
    • #define is not used to compile only one option

---

.bashrc

• detect whether TIOCSTI is supported by the kernel @ Bash
• based on whether TIOCSTI is available bind keyword shortcuts
  either to hstr command or hstrcygwin Bash function

# detect TIOCSTI
function is_tiocsti {
    if test -w /dev/tty && { stty -echo; echo -n a | tioctl TIOCSTI; stty echo; } >/dev/null 2>&1; then
        echo "TIOCSTI is supported by the kernel."
    else
        echo "TIOCSTI is not supported by the kernel."
    fi
}

# command binding
if is_tiocsti
then
    # if this is interactive shell, then bind hstr to Ctrl-r (for Vi mode check doc)
    if [[ $- =~ .*i.* ]]; then bind '"\C-r": "\C-a hstr -- \C-j"'; fi
    # if this is interactive shell, then bind 'kill last command' to Ctrl-x k
    if [[ $- =~ .*i.* ]]; then bind '"\C-xk": "\C-a hstr -k \C-j"'; fi
else
    if [[ $- =~ .*i.* ]]; then bind -x '"\C-r": "hstrcygwin"'; fi
fi 

---

Detect TIOCSTI from C:

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <termios.h>
#include <sys/ioctl.h>

int main() {
   int fd;
   struct termios t;
   fd = open("/dev/tty", O_RDWR);
   if (fd < 0) {
      perror("open /dev/tty");
      exit(1);
   }
   if (tcgetattr(fd, &t) < 0) {
      perror("tcgetattr");
      exit(1);
   }
   if (!ioctl(fd, TIOCSTI, "a")) {
      printf("TIOCSTI supported\n");
   } else {
      printf("TIOCSTI not supported\n");
   }
   close(fd);
   return 0;
}

[jakedane]

The C code detects TIOCSTI correctly. I tested it both with dev.tty.legacy_tiocsti=0 and dev.tty.legacy_tiocsti=1.

The bash code does not detect it correctly. is_tiocsti always responds "TIOCSTI is supported by the kernel."

Edit: ah, removing the redirect to /dev/null I get tioctl: command not found.

[dvorka]

@jakedane thank you for testing the code! It really helped. As tioctl command does not have to be present on the system, I will always hstr - I will create a hstr parameter which will make the test (and use it in .bashrc/.zshrc).

[dvorka]

FYI working on the fix @ dev-2.7.0 branch.

[dvorka]

Fixed by aa14f03 and released by f370a80. Please report any problems!

[EmJotGeh]

The fix only seems to work to some extent. After entering the hstr_notiocsti() function in my .bashrc, the history can be opened via CTRL+R but the selected command now appears in the console and is not executed until ENTER is pressed. This corresponds more to the function of the right cursor key (->) - to display the command and to be able to change it if necessary.

[cmonty14]

Imo the fix is not working as expected.
This means I can display bash history and select any line item.
However when I hit ENTER the line item will only be copied to console, but not executed.

I'm running Arch Linux with hstr version "3.1.0" (2023-04-18T08:50:00) and Linux Kernel 6.2.12-arch1-1.
TIOCSTI is disabled (dev.tty.legacy_tiocsti = 0).

My understanding was that hstr should work /w and w/o TIOCSTI.

[krossekrabbe]

Yes can confirm on fedora, ENTER writes to console but not executes.

But I can live with that for now, close enough 😂 Thanks for the fix @dvorka

[jakedane]

My error I think. Where I wrote for the earlier patch "with that Ctrl+R works properly in Bash" I meant it works the same as my workaround, which has this behavior (which I actually prefer), and should have clarified with the patch Enter puts the command on the console and Enter is needed again to execute it. Sorry @dvorka if I put you on wrong footing here!

[gowza]

Is the version 3.1.0 supposed to fix selecting command from running hstr command directly as well (not using Ctrl + R)? If so, it is not working for me with bash on Arch Linux with kernel version 6.2.12-arch1-1. The interactive history shows and I can search, but selecting a command using tab or enter returns to the prompt with the selected command on its own line and the prompt line empty.

Enabling TIOCSTI with sysctl -w dev.tty.legacy_tiocsti=1 make it works. Although the selected command still print on its own line as well as populating the prompt line.

Thank you for the great work btw.

[cmonty14]

Hello,
I'm running
Linux Kernel 6.3.1-arch2-1
hstr 3.1-1
and it works only as expected with dev.tty.legacy_tiocsti = 1.

My understanding is that latest hstr release should work w/o TIOCSTI, therefore I don't understand why this issue is closed.

[GiulioCentorame]

The interactive history shows and I can search, but selecting a command using tab or enter returns to the prompt with the selected command on its own line and the prompt line empty.

Same issue on Fedora 38 (running on kernel 6.2.14-300.fc38.x86_64)

[palves]

Should there be a separate bug for this "selecting command does not execute directly" issue? I ask because this bug is closed.

[Gooberpatrol66]

Hello, I'm running Linux Kernel 6.3.1-arch2-1 hstr 3.1-1 and it works only as expected with dev.tty.legacy_tiocsti = 1.

My understanding is that latest hstr release should work w/o TIOCSTI, therefore I don't understand why this issue is closed.

Same here, it was fixed for a while and then it broke again. kernel 6.3.9

[quicktrick]

Void Linux, kernel 6.3.10, hstr does not work.

[smahm006]

Confirming on Debian Sid kernel 6.3.0-1-amd64 has the same issue. sysctl -w dev.tty.legacy_tiocsti=1 does fix the issue.

[rivenirvana]

This needs to be reopened.

[jakedane]

Several comments above saying hstr doesn't work. It's not clear what "doesn't work" means.

Does it mean hstr doesn't work at all without setting dev.tty.legacy_tiocsti=1? And if so with which shell? Or does it mean hstr works but Enter needs to be pressed twice to run a command from history? First Enter puts the command on the prompt, second Enter runs it. In either case I think that should be a new issue.

I'm on Arch Linux with kernel 6.4.4 and hstr works for me in Bash. Enter needs to be pressed twice to run a command from history which is different from with setting dev.tty.legacy_tiocsti=1 but it does work.

[Gooberpatrol66]

Does it mean hstr doesn't work at all without setting dev.tty.legacy_tiocsti=1?

The same behavior as before the fix was applied

And if so with which shell?

bash

Or does it mean hstr works but Enter needs to be pressed twice to run a command from history?

No, that's not it

[jakedane]

@Gooberpatrol66 which distro are you on? Which terminal are you using? Which hstr version and how does your configuration in .bashrc differ from the recommended:

# HSTR configuration - add this to ~/.bashrc
alias hh=hstr                    # hh to be alias for hstr
export HSTR_CONFIG=hicolor       # get more colors
shopt -s histappend              # append new history items to .bash_history
export HISTCONTROL=ignorespace   # leading space hides commands from history
export HISTFILESIZE=10000        # increase history file size (default is 500)
export HISTSIZE=${HISTFILESIZE}  # increase history size (default is 500)
# ensure synchronization between bash memory and history file
export PROMPT_COMMAND="history -a; history -n; ${PROMPT_COMMAND}"
function hstrnotiocsti {
    { READLINE_LINE="$( { </dev/tty hstr ${READLINE_LINE}; } 2>&1 1>&3 3>&- )"; } 3>&1;
    READLINE_POINT=${#READLINE_LINE}
}
# if this is interactive shell, then bind hstr to Ctrl-r (for Vi mode check doc)
if [[ $- =~ .*i.* ]]; then bind -x '"\C-r": "hstrnotiocsti"'; fi
export HSTR_TIOCSTI=n

I'm using hstr 3.1, bash 5.1.016 and Linux 6.4.4 on Arch Linux. That works. I tried with Gnome Terminal, Console and Black Box. My .bashrc uses the recommended configuration.

I also tried hstr 3.0 on Fedora 38 and after manually fixing hstrnotiocsti in .bashrc, that also works.

[Gooberpatrol66]

oh, i didn't see the bashrc changed

[eleius]

I forgot I was still using the sysctl "fix", so I removed it and now hstr doesn't work for me.

I'm using hstr 3.1.0 on arch linux, kernel 6.5.5, and my bashrc has the same lines as in jakedane's comment above.

I can search history but pressing enter doesn't select the entry, it only echo'es it.
If I set dev.tty.legacy_tiocsti=1 it works.

[jakedane]

I can search history but pressing enter doesn't select the entry, it only echo'es it.

You have to press Enter twice. 1st Enter puts the selected history item on the bash prompt, 2nd Enter runs it.

If any expect hstr to work differently (that 1st Enter runs the command directly), I think that needs to be a separate issue.

[jessienab]

I can search history but pressing enter doesn't select the entry, it only echo'es it.

You have to press Enter twice. 1st Enter puts the selected history item on the bash prompt, 2nd Enter runs it.

If any expect hstr to work differently (that 1st Enter runs the command directly), I think that needs to be a separate issue.

@eleius another suggestion is to select your command, use TAB to bring it to your prompt, and then press enter; removing the need to hit Enter twice.

[eleius]

@jakedane @jasonnab I've already tried Tab+Enter and Enter+Enter, but for some reason when I press either Tab or Enter, the selected history item is just displayed as when you type the echo command (hstr doesn't bring it to the bash prompt.)
I've tried deleting my current .bashrc and only placing the hstr lines in it, but still same issue. Don't know why it doesn't work.

[quicktrick]

@eleius, don't be upset. I have hstr behaving exactly the same way on Void Linux 6.3.13.

[Mte90]

any update for this issue?
when it is planned a new release?

[jakedane]

@Mte90 hstr 3.1.0 fixed the issue. Currently with bash 5.2.21 in gnome-terminal 3.50.1 (vte 0.74.1) on Arch Linux, kernel 6.6.3, with the output of hstr --show-configuration in the .bashrc file, that works without problems. The kernel is not compiled with CONFIG_LEGACY_TIOCSTI set and dev.tty.legacy_tiocsti is also not set.

If it doesn't work for you, what specifically are you using?

[Mte90]

I didn't saw the new parameter for bash export HSTR_TIOCSTI=y I think that now it should work.