Creating UEFI Bootable USB stick hangs while reading /proc/mounts #2095

abbbi · 2019-03-21T20:37:27Z

Relax-and-Recover (ReaR) Issue Template

ReaR version ("/usr/sbin/rear -V"):

2.4 Git.

OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"):

Debian buster

ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

USING_UEFI_BOOTLOADER=yes
OUTPUT=USB
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000

UEFI bios
Storage (lokal disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):

USB Stick

Trying to create the USB boot image with UEFI option enabled somehow ends up in hanging while trying to read /proc/mounts via awk, last lines of log:

> 
> + echo '2019-03-21 21:34:55.848881665 Leaving debugscripts mode (back to previous bash flags and options settings).'
> 2019-03-21 21:34:55.848881665 Leaving debugscripts mode (back to previous bash flags and options settings).
> + apply_bash_flags_and_options_commands 'set +o xtrace;set +o vi;set +o verbose;set +o privileged;set +o posix;set +o pipefail;set +o physical;set +o onecmd;set +o nounset;set +o notify;set +o nolog;set +o noglob;set +o noexec;set +o noclobber;set +o monitor;set +o keyword;set -o interactive-comments;set +o ignoreeof;set +o history;set +o histexpand;set -o hashall;set +o functrace;set +o errtrace;set +o errexit;set +o emacs;set -o braceexpand;set +o allexport;shopt -u autocd;shopt -u assoc_expand_once;shopt -u cdable_vars;shopt -u cdspell;shopt -u checkhash;shopt -u checkjobs;shopt -s checkwinsize;shopt -s cmdhist;shopt -u compat31;shopt -u compat32;shopt -u compat40;shopt -u compat41;shopt -u compat42;shopt -u compat43;shopt -u compat44;shopt -s complete_fullquote;shopt -u direxpand;shopt -u dirspell;shopt -u dotglob;shopt -u execfail;shopt -u expand_aliases;shopt -u extdebug;shopt -s extglob;shopt -s extquote;shopt -u failglob;shopt -s force_fignore;shopt -s globasciiranges;shopt -u globstar;shopt -u gnu_errfmt;shopt -u histappend;shopt -u histreedit;shopt -u histverify;shopt -s hostcomplete;shopt -u huponexit;shopt -u inherit_errexit;shopt -s interactive_comments;shopt -u lastpipe;shopt -u lithist;shopt -u localvar_inherit;shopt -u localvar_unset;shopt -u login_shell;shopt -u mailwarn;shopt -u no_empty_cmd_completion;shopt -u nocaseglob;shopt -u nocasematch;shopt -s nullglob;shopt -s progcomp;shopt -u progcomp_alias;shopt -s promptvars;shopt -u restricted_shell;shopt -u shift_verbose;shopt -s sourcepath;shopt -u xpg_echo;'
> + eval 'set +o xtrace;set +o vi;set +o verbose;set +o privileged;set +o posix;set +o pipefail;set +o physical;set +o onecmd;set +o nounset;set +o notify;set +o nolog;set +o noglob;set +o noexec;set +o noclobber;set +o monitor;set +o keyword;set -o interactive-comments;set +o ignoreeof;set +o history;set +o histexpand;set -o hashall;set +o functrace;set +o errtrace;set +o errexit;set +o emacs;set -o braceexpand;set +o allexport;shopt -u autocd;shopt -u assoc_expand_once;shopt -u cdable_vars;shopt -u cdspell;shopt -u checkhash;shopt -u checkjobs;shopt -s checkwinsize;shopt -s cmdhist;shopt -u compat31;shopt -u compat32;shopt -u compat40;shopt -u compat41;shopt -u compat42;shopt -u compat43;shopt -u compat44;shopt -s complete_fullquote;shopt -u direxpand;shopt -u dirspell;shopt -u dotglob;shopt -u execfail;shopt -u expand_aliases;shopt -u extdebug;shopt -s extglob;shopt -s extquote;shopt -u failglob;shopt -s force_fignore;shopt -s globasciiranges;shopt -u globstar;shopt -u gnu_errfmt;shopt -u histappend;shopt -u histreedit;shopt -u histverify;shopt -s hostcomplete;shopt -u huponexit;shopt -u inherit_errexit;shopt -s interactive_comments;shopt -u lastpipe;shopt -u lithist;shopt -u localvar_inherit;shopt -u localvar_unset;shopt -u login_shell;shopt -u mailwarn;shopt -u no_empty_cmd_completion;shopt -u nocaseglob;shopt -u nocasematch;shopt -s nullglob;shopt -s progcomp;shopt -u progcomp_alias;shopt -s promptvars;shopt -u restricted_shell;shopt -u shift_verbose;shopt -s sourcepath;shopt -u xpg_echo;'
> ++ set +o xtrace
> 2019-03-21 21:34:55.859526842 Including prep/default/320_include_uefi_env.sh
> 2019-03-21 21:34:55.862689084 Entering debugscripts mode via 'set -x'.
> + source /usr/share/rear/prep/default/320_include_uefi_env.sh
> ++ grep -qw noefi /proc/cmdline
> ++ is_false yes
> ++ case "$1" in
> ++ return 1
> ++ modprobe -q efivars
> ++ SYSFS_DIR_EFI_VARS=
> ++ [[ -d /sys/firmware/efi/vars ]]
> ++ SYSFS_DIR_EFI_VARS=/sys/firmware/efi/vars
> ++ grep -qw efivars /proc/mounts
> ++ SYSFS_DIR_EFI_VARS=/sys/firmware/efi/efivars
> +++ find /boot -maxdepth 1 -iname efi -type d
> ++ test /boot/efi
> ++ local esp_mount_point=
> ++ esp_mount_point='/\/boot\/efi/'
> +++ awk /proc/mounts
>

The text was updated successfully, but these errors were encountered:

jsmeix · 2019-03-22T09:20:30Z

Perhaps commit
640d3fa
has broken that code in usr/share/rear/prep/default/320_include_uefi_env.sh ?

jsmeix · 2019-03-22T09:23:39Z

On my openSUSE Leap 15.0 system the current code seems to work:

# grep boot/efi /proc/mounts
/dev/sda1 /boot/efi vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro 0 0

# ( set -x ; esp_mount_point='/\/boot\/efi/' ; UEFI_FS_TYPE=$(awk $esp_mount_point' { print $3 }' /proc/mounts) ; echo $UEFI_FS_TYPE )
+ esp_mount_point='/\/boot\/efi/'
++ awk '/\/boot\/efi/ { print $3 }' /proc/mounts
+ UEFI_FS_TYPE=vfat
+ echo vfat
vfat

abbbi · 2019-03-22T16:21:54Z

strange enough those steps work if executed from bash directly:

lap: grep boot/efi /proc/mounts 
/dev/sda1 /boot/efi vfat rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro 0 0

lap: ( set -x ; esp_mount_point='/\/boot\/efi/' ; UEFI_FS_TYPE=$(awk $esp_mount_point' { print $3 }' /proc/mounts) ; echo $UEFI_FS_TYPE )
+ esp_mount_point='/\/boot\/efi/'
++ awk '/\/boot\/efi/ { print $3 }' /proc/mounts
+ UEFI_FS_TYPE=vfat
+ echo vfat
vfat

jsmeix · 2019-03-25T12:01:57Z

@abbbi
a blind shot in the dark:
During "rear mkbackup" the environment could be different
compared to running commands directly on root's command line,
cf. usr/sbin/rear things like export LC_CTYPE=C LC_ALL=C LANG=C
but I cannot imagine how specifically this could make a difference here.

shortydle · 2019-04-10T15:28:20Z

Same issue with debian buster, rear 2.4 /Git

2019-04-10 17:26:24.651807706 Entering debugscripts mode via 'set -x'.
+ source /usr/share/rear/prep/default/320_include_uefi_env.sh
++ grep -qw noefi /proc/cmdline
++ is_false
++ case "$1" in
++ return 1
++ modprobe -q efivars
++ SYSFS_DIR_EFI_VARS=
++ [[ -d /sys/firmware/efi/vars ]]
++ SYSFS_DIR_EFI_VARS=/sys/firmware/efi/vars
++ grep -qw efivars /proc/mounts
++ SYSFS_DIR_EFI_VARS=/sys/firmware/efi/efivars
+++ find /boot -maxdepth 1 -iname efi -type d
++ test /boot/efi
++ local esp_mount_point=
++ esp_mount_point='/\/boot\/efi/'
+++ awk /proc/mounts

root@debian:/tmp# grep boot/efi /proc/mounts
/dev/sda1 /boot/efi vfat rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro 0 0
root@debian:/tmp# ( set -x ; esp_mount_point='/\/boot\/efi/' ; UEFI_FS_TYPE=$(awk $esp_mount_point' { print $3 }' /proc/mounts) ; echo $UEFI_FS_TYPE )
+ esp_mount_point='/\/boot\/efi/'
++ awk '/\/boot\/efi/ { print $3 }' /proc/mounts
+ UEFI_FS_TYPE=vfat
+ echo vfat
vfat
root@debian:/tmp#

gozora · 2019-04-10T19:41:10Z

I wish I could explain better but this have something to do with almighty nullglob which is used by ReaR but not your default shell.

if you do:

# set -x
# shopt -s nullglob
# esp_mount_point='/\/boot\/efi/'
# awk $esp_mount_point' { print $3 }' /proc/mounts
+ awk /proc/mounts
^C

hence you get exact same behavior as seen in ReaR.

If you use double quotes (") for $esp_mount_point variable problem is solved:

# awk "$esp_mount_point"' { print $3 }' /proc/mounts
+ awk '/\/boot\/efi/ { print $3 }' /proc/mounts
vfat

So if you update your current /usr/share/rear/prep/default/320_include_uefi_env.sh with something like this:

@@ -54,11 +54,11 @@ local esp_mount_point=""
 
 # next step, check filesystem partition type (vfat?)
 esp_mount_point='/\/boot\/efi/'
-UEFI_FS_TYPE=$(awk $esp_mount_point' { print $3 }' /proc/mounts)
+UEFI_FS_TYPE=$(awk "$esp_mount_point"' { print $3 }' /proc/mounts)
 # if not mounted at /boot/efi, try /boot
 if [[ -z "$UEFI_FS_TYPE" ]]; then
     esp_mount_point='/\/boot/'
-    UEFI_FS_TYPE=$(awk $esp_mount_point' { print $3 }' /proc/mounts)
+    UEFI_FS_TYPE=$(awk "$esp_mount_point"' { print $3 }' /proc/mounts)
 fi
 
 # ESP must be type vfat (under Linux)
@@ -70,4 +70,4 @@ fi
 USING_UEFI_BOOTLOADER=1
 LogPrint "Using UEFI Boot Loader for Linux (USING_UEFI_BOOTLOADER=1)"
 
-awk $esp_mount_point' { print $1 }' /proc/mounts >$VAR_DIR/recovery/bootdisk 2>/dev/null
+awk "$esp_mount_point"' { print $1 }' /proc/mounts >$VAR_DIR/recovery/bootdisk 2>/dev/null

this problem should be solved.

Maybe @jsmeix can explain why is nullglob behaving like this in this particular case if he have a bit of free time ...

I can prepare PR for this small problem if needed, but since I'm nowadays a bit short on time, I'm not sure when it will be ready ...

Hope it helps.

V.

gozora · 2019-04-10T19:54:10Z

Interestingly enough as @jsmeix pointed out in #2095 (comment), my example from #2095 (comment) works fine on SLE15

sle15:~ # set -x
sle15:~ # shopt -s nullglob
+ shopt -s nullglob
sle15:~ # esp_mount_point='/\/boot\/efi/'
+ esp_mount_point='/\/boot\/efi/'
sle15:~ # awk $esp_mount_point' { print $3 }' /proc/mounts
+ awk '/\/boot\/efi/ { print $3 }' /proc/mounts
vfat

Best guess I can make right now are different awk packages on SuSE and Debian.
On my SLE15 I have gawk-4.2.1-1.41.x86_64 and on my Debian Buster I have mawk-1.3.3-17+b3.

So maybe different implementation ???

V.

shortydle · 2019-04-11T09:13:53Z

Hello

@gozora: Thanks, your workaround worked.
I have both gawk 1:4.2.1+dfsg-1 & mawk 1.3.3-17+b3

update-alternatives --config awk
There are 2 choices for the alternative awk (providing /usr/bin/awk).

  Selection    Path            Priority   Status
------------------------------------------------------------
* 0            /usr/bin/gawk    10        auto mode
  1            /usr/bin/gawk    10        manual mode
  2            /usr/bin/mawk    5         manual mode

After that I found another dependency issue: efibootmgr was missing.
Strangely apt autoremove suggested to remove it after new kernel update :

Start-Date: 2019-04-10  16:32:12
Commandline: apt autoremove
Requested-By: shorty (1000)
Remove: linux-headers-4.19.0-1-amd64:amd64 (4.19.12-1), efibootmgr:amd64 (15-1), libatkmm-1.6-1v5:amd64 (2.28.0-2), linux-headers-4.19.0-1-common:amd64 (4.19.12-1), linux-image-4.19.0-1-amd64:amd64 (4.19.12-1), libcairomm-1.0-1v5:amd64 (1.12.2-4), qpdf:amd64 (8.4.0-2), libmariadbclient18:amd64 (1:10.3.13-1), libglibmm-2.4-1v5:amd64 (2.58.0-2), libpangomm-1.4-1v5:amd64 (2.42.0-2), libgtkmm-3.0-1v5:amd64 (3.24.0-2)
End-Date: 2019-04-10  16:32:29

After reinstall backup went ok.

jsmeix · 2019-04-11T11:10:52Z

I overhauled that code part via
#2115

@gozora
could you have a look at the code changes?

@shortydle
could you test if my changed code in
https://github.com/rear/rear/pull/2115/files
also makes things work for you.
You may use the whole changed script from
https://raw.githubusercontent.com/rear/rear/11045023e916995eb403aca6053f9b5ef98930dc/usr/share/rear/prep/default/320_include_uefi_env.sh

shortydle · 2019-04-11T11:53:06Z

@jsmeix: tested OK with https://raw.githubusercontent.com/rear/rear/11045023e916995eb403aca6053f9b5ef98930dc/usr/share/rear/prep/default/320_include_uefi_env.sh

debug output :

2019-04-11 13:39:18.092371798 Including prep/default/320_include_uefi_env.sh
2019-04-11 13:39:18.094627419 Entering debugscripts mode via 'set -x'.
+ source /usr/share/rear/prep/default/320_include_uefi_env.sh
++ grep -qw noefi /proc/cmdline
++ is_false
++ case "$1" in
++ return 1
++ modprobe -q efivars
++ SYSFS_DIR_EFI_VARS=
++ [[ -d /sys/firmware/efi/vars ]]
++ SYSFS_DIR_EFI_VARS=/sys/firmware/efi/vars
++ grep -qw efivars /proc/mounts
++ SYSFS_DIR_EFI_VARS=/sys/firmware/efi/efivars
+++ find /boot -maxdepth 1 -iname efi -type d
++ test /boot/efi
++ esp_proc_mounts_line=()
++ local esp_proc_mounts_line
++ esp_proc_mounts_line=($( grep ' /boot/efi ' /proc/mounts || echo false ))
+++ grep ' /boot/efi ' /proc/mounts
++ is_false /dev/sda1
++ case "$1" in
++ return 1
++ test vfat = vfat
++ DebugPrint 'Found EFI system partition /dev/sda1 on /boot/efi type vfat'
++ Debug 'Found EFI system partition /dev/sda1 on /boot/efi type vfat'
++ test 1
++ Log 'Found EFI system partition /dev/sda1 on /boot/efi type vfat'
+++ date '+%Y-%m-%d %H:%M:%S.%N '
++ local 'timestamp=2019-04-11 13:39:18.115732367 '
++ test 1 -gt 0
++ echo '2019-04-11 13:39:18.115732367 Found EFI system partition /dev/sda1 on /boot/efi type vfat'
2019-04-11 13:39:18.115732367 Found EFI system partition /dev/sda1 on /boot/efi type vfat
++ test 1
++ Print 'Found EFI system partition /dev/sda1 on /boot/efi type vfat'
++ test 1
++ echo -e 'Found EFI system partition /dev/sda1 on /boot/efi type vfat'
++ USING_UEFI_BOOTLOADER=1
++ LogPrint 'Using UEFI Boot Loader for Linux (USING_UEFI_BOOTLOADER=1)'
++ Log 'Using UEFI Boot Loader for Linux (USING_UEFI_BOOTLOADER=1)'
+++ date '+%Y-%m-%d %H:%M:%S.%N '
++ local 'timestamp=2019-04-11 13:39:18.117191505 '
++ test 1 -gt 0
++ echo '2019-04-11 13:39:18.117191505 Using UEFI Boot Loader for Linux (USING_UEFI_BOOTLOADER=1)'
2019-04-11 13:39:18.117191505 Using UEFI Boot Loader for Linux (USING_UEFI_BOOTLOADER=1)
++ Print 'Using UEFI Boot Loader for Linux (USING_UEFI_BOOTLOADER=1)'
++ test 1
++ echo -e 'Using UEFI Boot Loader for Linux (USING_UEFI_BOOTLOADER=1)'
++ echo /dev/sda1
+ test 1
+ Debug 'Leaving debugscripts mode (back to previous bash flags and options settings).'
+ test 1
+ Log 'Leaving debugscripts mode (back to previous bash flags and options settings).'
++ date '+%Y-%m-%d %H:%M:%S.%N '
+ local 'timestamp=2019-04-11 13:39:18.118946341 '
+ test 1 -gt 0
+ echo '2019-04-11 13:39:18.118946341 Leaving debugscripts mode (back to previous bash flags and options settings).'
2019-04-11 13:39:18.118946341 Leaving debugscripts mode (back to previous bash flags and options settings).
+ apply_bash_flags_and_options_commands 'set +o xtrace;set +o vi;set +o verbose;set +o privileged;set +o posix;set +o pipefail;set +o physical;set +o onecmd;set +o nounset;set +o notify;set +o nolog;set +o noglob;set +o noexec;set +o noclobber;set +o monitor;set +o keyword;set -o interactive-comments;set +o ignoreeof;set +o history;set +o histexpand;set -o hashall;set +o functrace;set +o errtrace;set +o errexit;set +o emacs;set -o braceexpand;set +o allexport;shopt -u autocd;shopt -u assoc_expand_once;shopt -u cdable_vars;shopt -u cdspell;shopt -u checkhash;shopt -u checkjobs;shopt -s checkwinsize;shopt -s cmdhist;shopt -u compat31;shopt -u compat32;shopt -u compat40;shopt -u compat41;shopt -u compat42;shopt -u compat43;shopt -u compat44;shopt -s complete_fullquote;shopt -u direxpand;shopt -u dirspell;shopt -u dotglob;shopt -u execfail;shopt -u expand_aliases;shopt -u extdebug;shopt -s extglob;shopt -s extquote;shopt -u failglob;shopt -s force_fignore;shopt -s globasciiranges;shopt -u globstar;shopt -u gnu_errfmt;shopt -u histappend;shopt -u histreedit;shopt -u histverify;shopt -s hostcomplete;shopt -u huponexit;shopt -u inherit_errexit;shopt -s interactive_comments;shopt -u lastpipe;shopt -u lithist;shopt -u localvar_inherit;shopt -u localvar_unset;shopt -u login_shell;shopt -u mailwarn;shopt -u no_empty_cmd_completion;shopt -u nocaseglob;shopt -u nocasematch;shopt -s nullglob;shopt -s progcomp;shopt -u progcomp_alias;shopt -s promptvars;shopt -u restricted_shell;shopt -u shift_verbose;shopt -s sourcepath;shopt -u xpg_echo;'
+ eval 'set +o xtrace;set +o vi;set +o verbose;set +o privileged;set +o posix;set +o pipefail;set +o physical;set +o onecmd;set +o nounset;set +o notify;set +o nolog;set +o noglob;set +o noexec;set +o noclobber;set +o monitor;set +o keyword;set -o interactive-comments;set +o ignoreeof;set +o history;set +o histexpand;set -o hashall;set +o functrace;set +o errtrace;set +o errexit;set +o emacs;set -o braceexpand;set +o allexport;shopt -u autocd;shopt -u assoc_expand_once;shopt -u cdable_vars;shopt -u cdspell;shopt -u checkhash;shopt -u checkjobs;shopt -s checkwinsize;shopt -s cmdhist;shopt -u compat31;shopt -u compat32;shopt -u compat40;shopt -u compat41;shopt -u compat42;shopt -u compat43;shopt -u compat44;shopt -s complete_fullquote;shopt -u direxpand;shopt -u dirspell;shopt -u dotglob;shopt -u execfail;shopt -u expand_aliases;shopt -u extdebug;shopt -s extglob;shopt -s extquote;shopt -u failglob;shopt -s force_fignore;shopt -s globasciiranges;shopt -u globstar;shopt -u gnu_errfmt;shopt -u histappend;shopt -u histreedit;shopt -u histverify;shopt -s hostcomplete;shopt -u huponexit;shopt -u inherit_errexit;shopt -s interactive_comments;shopt -u lastpipe;shopt -u lithist;shopt -u localvar_inherit;shopt -u localvar_unset;shopt -u login_shell;shopt -u mailwarn;shopt -u no_empty_cmd_completion;shopt -u nocaseglob;shopt -u nocasematch;shopt -s nullglob;shopt -s progcomp;shopt -u progcomp_alias;shopt -s promptvars;shopt -u restricted_shell;shopt -u shift_verbose;shopt -s sourcepath;shopt -u xpg_echo;'
++ set +o xtrace

…nclude_uefi_env_sh_issue2095 Simplify awk constructs in 320_include_uefi_env.sh into using plain grep and a bash array to avoid inexplicable wrong behaviour in some cases that is somehow related to the nullglob bash option together with different kind of awk, see #2095

jsmeix · 2019-04-11T12:37:33Z

@shortydle
thank you for your prompt response!

With #2115 merged
this issue should be fixed.

jsmeix added the support / question label Mar 22, 2019

gozora self-assigned this Apr 10, 2019

gozora added the bug The code does not do what it is meant to do label Apr 10, 2019

gozora added this to the ReaR v2.5 milestone Apr 10, 2019

jsmeix added a commit to jsmeix/rear that referenced this issue Apr 11, 2019

Simplify awk constructs in 320_include_uefi_env.sh (issue rear#2095)

1104502

jsmeix mentioned this issue Apr 11, 2019

Simplify awk constructs in 320_include_uefi_env.sh (issue #2095) #2115

Merged

jsmeix self-assigned this Apr 11, 2019

jsmeix added cleanup fixed / solved / done and removed support / question labels Apr 11, 2019

jsmeix closed this as completed Apr 11, 2019

jsmeix mentioned this issue May 28, 2019

Recovery fails on LUKS-encrypted filesystem using simple password #2151

Closed

jsmeix mentioned this issue Aug 12, 2019

ReaR mkrescue stalls in Fedora 30 VM with no error #2203

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Creating UEFI Bootable USB stick hangs while reading /proc/mounts #2095

Creating UEFI Bootable USB stick hangs while reading /proc/mounts #2095

abbbi commented Mar 21, 2019 •

edited by jsmeix

jsmeix commented Mar 22, 2019

jsmeix commented Mar 22, 2019

abbbi commented Mar 22, 2019 •

edited by jsmeix

jsmeix commented Mar 25, 2019

shortydle commented Apr 10, 2019 •

edited

gozora commented Apr 10, 2019 •

edited

gozora commented Apr 10, 2019

shortydle commented Apr 11, 2019

jsmeix commented Apr 11, 2019 •

edited

shortydle commented Apr 11, 2019

jsmeix commented Apr 11, 2019

Creating UEFI Bootable USB stick hangs while reading /proc/mounts #2095

Creating UEFI Bootable USB stick hangs while reading /proc/mounts #2095

Comments

abbbi commented Mar 21, 2019 • edited by jsmeix

Relax-and-Recover (ReaR) Issue Template

jsmeix commented Mar 22, 2019

jsmeix commented Mar 22, 2019

abbbi commented Mar 22, 2019 • edited by jsmeix

jsmeix commented Mar 25, 2019

shortydle commented Apr 10, 2019 • edited

gozora commented Apr 10, 2019 • edited

gozora commented Apr 10, 2019

shortydle commented Apr 11, 2019

jsmeix commented Apr 11, 2019 • edited

shortydle commented Apr 11, 2019

jsmeix commented Apr 11, 2019

abbbi commented Mar 21, 2019 •

edited by jsmeix

abbbi commented Mar 22, 2019 •

edited by jsmeix

shortydle commented Apr 10, 2019 •

edited

gozora commented Apr 10, 2019 •

edited

jsmeix commented Apr 11, 2019 •

edited