copyfile source file collision

[tim-oe]

just installed the latest version
using 2024-03-15-raspios-bookworm-arm64-lite.img
building image on same version
launched on a pi 3b+

tried with both the following:

separate plugins
--plugin copyfile:"from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinstall|chown=tcronin:tcronin|chmod=600" \ --plugin copyfile:"from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase=postinstall|chown=ansible:ansible|chmod=600" \

and filelist
--plugin copyfile:"filelist=./filelist" \
filelist content
from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|chown=ansible:ansible|chmod=600|mkdirif|runphase=postinstall from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=600|mkdirif|runphase=postinstall
both instances would copy the first from to each to target
even verified reordering the files
also the filelist command required adding the mkdirif even with a preceding mkdir plugin creating the target dir.

full customize command:
sudo sdm --customize \ --plugin user:"adduser=tcronin|password=$TEC_PWD" \ --plugin mkdir:"dir=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=700" \ --plugin copyfile:"from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinstall|chown=tcronin:tcronin|chmod=600" \ --plugin user:"adduser=ansible|password=$ANS_PWD" \ --plugin mkdir:"dir=/home/ansible/.ssh|chown=ansible:ansible|chmod=700" \ --plugin copyfile:"from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase=postinstall|chown=ansible:ansible|chmod=600" \ --plugin user:"deluser=pi" \ --plugin L10n:"keymap=us|locale=en_US.UTF-8|timezone=America/Chicago" \ --regen-ssh-host-keys \ --plugin network:"netman=nm|wifissid=tec-wan|wifipassword=$WIFI_PWD|wificountry=US" \ --restart img/bookworm.img

[gitbls]

Investigating...

[gitbls]

Please provide the redacted-as-necessary /etc/sdm/history from the IMG or console output from the customize command. Thx

[tim-oe]

I think i found the issue from command output:
first copy:
`> Run Plugin 'copyfile' (/usr/local/sdm/plugins/copyfile) Phase 1 with arguments:
'from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinst
all|chown=tcronin:tcronin|chmod=600'

• Plugin copyfile: Start Phase 1

Plugin copyfile: Copy 'authorized_keys' from /etc/sdm/assets/copyfile to '/home/tcronin/.ssh'
Plugin copyfile: Set owner 'tcronin:tcronin' on file '/home/tcronin/.ssh/authorized_keys'
Plugin copyfile: Set mode '600' on file '/home/tcronin/.ssh/authorized_keys'

• Plugin copyfile: Complete Phase 1
  `

second copy:
`> Run Plugin 'copyfile' (/usr/local/sdm/plugins/copyfile) Phase 1 with arguments:
'from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinst
all|chown=tcronin:tcronin|chmod=600'

• Plugin copyfile: Start Phase 1

Plugin copyfile: Copy 'authorized_keys' from /etc/sdm/assets/copyfile to '/home/tcronin/.ssh'
Plugin copyfile: Set owner 'tcronin:tcronin' on file '/home/tcronin/.ssh/authorized_keys'
Plugin copyfile: Set mode '600' on file '/home/tcronin/.ssh/authorized_keys'

• Plugin copyfile: Complete Phase 1

`

key being:
Copy 'authorized_keys' from /etc/sdm/assets/copyfile

below is the full redacted output:

tcronin@tec-pi-mgr:~/src/piImage $ src/sdm/image.sh

• Mount IMG 'img/bookworm.img'
  mount: /dev/loop0 mounted on /mnt/sdm.
  mount: /dev/loop1 mounted on /mnt/sdm/boot/firmware.
• Start Configuration

Command Line: /usr/local/bin/sdm --customize --plugin user:adduser=tcronin|password= --plugin
mkdir:dir=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=700 --plugin
copyfile:from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=
postinstall|chown=tcronin:tcronin|chmod=600 --plugin user:adduser=ansible|password= --plugin
mkdir:dir=/home/ansible/.ssh|chown=ansible:ansible|chmod=700 --plugin
copyfile:from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase=
postinstall|chown=ansible:ansible|chmod=600 --plugin user:deluser=pi --plugin
L10n:keymap=us|locale=en_US.UTF-8|timezone=America/Chicago --plugin disables:piwiz
--regen-ssh-host-keys --plugin network:netman=nm|wifissid=tec-wan|wifipassword=|wificountry=US
--restart img/bookworm.img

• Host Information
  Hostname: tec-pi-mgr
  uname: Linux tec-pi-mgr 6.6.20+rpt-rpi-v8 Welcome to sdm Discussions! #1 SMP PREEMPT Debian 1:6.6.20-1+rpt1 (2024-03-07)
  aarch64 GNU/Linux
  os-release Name: Debian GNU/Linux 12 (bookworm)
  Version: 12 (bookworm)
  sdm Version: V11.6
• IMG Information
  Name: img/bookworm.img
  Date: 2024-03-15
  RasPiOS Version: 12
  RasPiOS Architecture: 64-bit aarch64
  os-release Version: 12 (bookworm)
• Plugins selected:
  • user
    Args: adduser=tcronin|password=
  • mkdir
    Args: dir=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=700
  • copyfile
    Args: from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinstall|chown=tcronin:tcronin|chmod=600
  • user
    Args: adduser=ansible|password=
  • mkdir
    Args: dir=/home/ansible/.ssh|chown=ansible:ansible|chmod=700
  • copyfile
    Args: from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase=postinstall|chown=ansible:ansible|chmod=600
  • user
    Args: deluser=pi
  • L10n
    Args: keymap=us|locale=en_US.UTF-8|timezone=America/Chicago
  • disables
    Args: piwiz
  • network
    Args: netman=nm|wifissid=tec-wan|wifipassword=|wificountry=US

IMG 'img/bookworm.img' has 319788 1K-blocks (0.3GB, 0.3GiB) free
Copy sdm to /usr/local/sdm in the IMG

• Start Phase 0 image customization

Run Plugins Phase '0'
Run Plugin 'user' (/mnt/sdm/usr/local/sdm/plugins/user) Phase 0 with arguments:
'adduser=tcronin|password='

• Plugin user: Start Phase 0

Plugin user: Keys/values found:
adduser: tcronin
password:
Plugin user: Create user 'tcronin' home directory '/home/tcronin' so available for other
plugins

• Plugin user: Complete Phase 0

Run Plugin 'mkdir' (/mnt/sdm/usr/local/sdm/plugins/mkdir) Phase 0 with arguments:
'dir=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=700'

• Plugin mkdir: Start Phase 0

Plugin mkdir: Keys/values found:
dir: /home/tcronin/.ssh
chown: tcronin:tcronin
chmod: 700
Plugin mkdir: Create directory '/home/tcronin/.ssh'

• Plugin mkdir: Complete Phase 0

Run Plugin 'copyfile' (/mnt/sdm/usr/local/sdm/plugins/copyfile) Phase 0 with arguments:
'from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinst
all|chown=tcronin:tcronin|chmod=600'

• Plugin copyfile: Start Phase 0

Plugin copyfile: Keys/values found:
from: /mnt/clones/data/home/tcronin/.ssh/authorized_keys
to: /home/tcronin/.ssh
runphase: postinstall
chown: tcronin:tcronin
chmod: 600
Plugin copyfile: Add '/mnt/clones/data/home/tcronin/.ssh/authorized_keys' to files to copy

• Plugin copyfile: Complete Phase 0

Run Plugin 'user' (/mnt/sdm/usr/local/sdm/plugins/user) Phase 0 with arguments:
'adduser=ansible|password='

• Plugin user: Start Phase 0

Plugin user: Keys/values found:
adduser: ansible
password:
Plugin user: Create user 'ansible' home directory '/home/ansible' so available for other
plugins

• Plugin user: Complete Phase 0

Run Plugin 'mkdir' (/mnt/sdm/usr/local/sdm/plugins/mkdir) Phase 0 with arguments:
'dir=/home/ansible/.ssh|chown=ansible:ansible|chmod=700'

• Plugin mkdir: Start Phase 0

Plugin mkdir: Keys/values found:
dir: /home/ansible/.ssh
chown: ansible:ansible
chmod: 700
Plugin mkdir: Create directory '/home/ansible/.ssh'

• Plugin mkdir: Complete Phase 0

Run Plugin 'copyfile' (/mnt/sdm/usr/local/sdm/plugins/copyfile) Phase 0 with arguments:
'from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase=postinst
all|chown=ansible:ansible|chmod=600'

• Plugin copyfile: Start Phase 0

Plugin copyfile: Keys/values found:
from: /mnt/clones/data/home/ansible/.ssh/authorized_keys
to: /home/ansible/.ssh
runphase: postinstall
chown: ansible:ansible
chmod: 600
Plugin copyfile: Add '/mnt/clones/data/home/ansible/.ssh/authorized_keys' to files to copy

• Plugin copyfile: Complete Phase 0

Run Plugin 'user' (/mnt/sdm/usr/local/sdm/plugins/user) Phase 0 with arguments: 'deluser=pi'

• Plugin user: Start Phase 0

Plugin user: Keys/values found:
deluser: pi

• Plugin user: Complete Phase 0

Run Plugin 'L10n' (/mnt/sdm/usr/local/sdm/plugins/L10n) Phase 0 with arguments:
'keymap=us|locale=en_US.UTF-8|timezone=America/Chicago'

• Plugin L10n: Start Phase 0

Plugin L10n: Keys/values found:
keymap: us
locale: en_US.UTF-8
timezone: America/Chicago
Plugin L10n: Save L10n configuration

• Plugin L10n: Complete Phase 0

Run Plugin 'disables' (/mnt/sdm/usr/local/sdm/plugins/disables) Phase 0 with arguments:
'piwiz'

• Plugin disables: Start Phase 0

Plugin disables: Keys/values found:
piwiz:

• Plugin disables: Complete Phase 0

Run Plugin 'network' (/mnt/sdm/usr/local/sdm/plugins/network) Phase 0 with arguments:
'netman=nm|wifissid=tec-wan|wifipassword=|wificountry=US'
Plugin network: Configuring 'network-manager' for network configuration management
% Plugin network: Network Manager requires either a .nmconnection file for WiFi
or explicit WiFi configuration via arguments: wifissid wifipassword wificountry
Found neither; WiFi likely will not connect

• Phase 0 Completed
• Enter image 'img/bookworm.img' for Phase 1
• Start Phase 1 image customization

IMG 'img/bookworm.img' has 319696 1K-blocks (0.3GB, 0.3GiB) free at start of Phase 1 image customization
Configure and enable sdm FirstBoot service (sdm-firstboot)
Start 'apt update'
IMG 'img/bookworm.img' has 319692 1K-blocks (0.3GB, 0.3GiB) free at start of 'apt update'
IMG 'img/bookworm.img' has 205272 1K-blocks (0.2GB, 0.2GiB) free at end of 'apt update'
Run Plugins Phase '1'
Run Plugin 'user' (/usr/local/sdm/plugins/user) Phase 1 with arguments:
'adduser=tcronin|password='

• Plugin user: Start Phase 1

Plugin user: Add user 'tcronin'
Plugin user: Add groups to user 'tcronin':
dialout,cdrom,floppy,audio,video,plugdev,users,adm,sudo,users,input,netdev,spi,i2c,gpio
Plugin user: Enable sudo for user 'tcronin'
Plugin user: Copy skel files for user 'tcronin'
Plugin user: Set user 'tcronin' home directory '/home/tcronin' owner to 'tcronin:tcronin'

• Plugin user: Complete Phase 1

Run Plugin 'mkdir' (/usr/local/sdm/plugins/mkdir) Phase 1 with arguments:
'dir=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=700'

• Plugin mkdir: Start Phase 1
• Plugin mkdir: Complete Phase 1

Run Plugin 'copyfile' (/usr/local/sdm/plugins/copyfile) Phase 1 with arguments:
'from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinst
all|chown=tcronin:tcronin|chmod=600'

• Plugin copyfile: Start Phase 1

Plugin copyfile: Copy 'authorized_keys' from /etc/sdm/assets/copyfile to '/home/tcronin/.ssh'
Plugin copyfile: Set owner 'tcronin:tcronin' on file '/home/tcronin/.ssh/authorized_keys'
Plugin copyfile: Set mode '600' on file '/home/tcronin/.ssh/authorized_keys'

• Plugin copyfile: Complete Phase 1

Run Plugin 'user' (/usr/local/sdm/plugins/user) Phase 1 with arguments:
'adduser=ansible|password='

• Plugin user: Start Phase 1

Plugin user: Add user 'ansible'
Plugin user: Add groups to user 'ansible':
dialout,cdrom,floppy,audio,video,plugdev,users,adm,sudo,users,input,netdev,spi,i2c,gpio
Plugin user: Enable sudo for user 'ansible'
Plugin user: Copy skel files for user 'ansible'
Plugin user: Set user 'ansible' home directory '/home/ansible' owner to 'ansible:ansible'

• Plugin user: Complete Phase 1

Run Plugin 'mkdir' (/usr/local/sdm/plugins/mkdir) Phase 1 with arguments:
'dir=/home/ansible/.ssh|chown=ansible:ansible|chmod=700'

• Plugin mkdir: Start Phase 1
• Plugin mkdir: Complete Phase 1

Run Plugin 'copyfile' (/usr/local/sdm/plugins/copyfile) Phase 1 with arguments:
'from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase=postinst
all|chown=ansible:ansible|chmod=600'

• Plugin copyfile: Start Phase 1

Plugin copyfile: Copy 'authorized_keys' from /etc/sdm/assets/copyfile to '/home/ansible/.ssh'
Plugin copyfile: Set owner 'ansible:ansible' on file '/home/ansible/.ssh/authorized_keys'
Plugin copyfile: Set mode '600' on file '/home/ansible/.ssh/authorized_keys'

• Plugin copyfile: Complete Phase 1

Run Plugin 'user' (/usr/local/sdm/plugins/user) Phase 1 with arguments: 'deluser=pi'

• Plugin user: Start Phase 1

Plugin user: Delete user 'pi'
userdel: pi mail spool (/var/mail/pi) not found

• Plugin user: Complete Phase 1

Run Plugin 'L10n' (/usr/local/sdm/plugins/L10n) Phase 1 with arguments:
'keymap=us|locale=en_US.UTF-8|timezone=America/Chicago'

• Plugin L10n: Start Phase 1
  % Plugin L10n: Keymap 'us' will be set during sdm FirstBoot

Plugin L10n: Disable keyboard-setup service; will be re-enabled during sdm FirstBoot
Removed "/etc/systemd/system/sysinit.target.wants/keyboard-setup.service".
Plugin L10n: Set locale to 'en_US.UTF-8'
Generating locales (this might take a while)...
en_US.UTF-8... done
Generation complete.
Plugin L10n: Set timezone to 'America/Chicago'

• Plugin L10n: Complete Phase 1

Run Plugin 'disables' (/usr/local/sdm/plugins/disables) Phase 1 with arguments: 'piwiz'

• Plugin disables: Start Phase 1
• Plugin disables: Complete Phase 1

Run Plugin 'network' (/usr/local/sdm/plugins/network) Phase 1 with arguments:
'netman=nm|wifissid=tec-wan|wifipassword=|wificountry=US'

• Plugin network: Start Phase 1
• Plugin network: Complete Phase 1
• Phase 1 post-app installation/configuration

Start 'apt upgrade'
IMG 'img/bookworm.img' has 200380 1K-blocks (0.2GB, 0.2GiB) free at start of 'apt upgrade'
IMG 'img/bookworm.img' has 196856 1K-blocks (0.2GB, 0.2GiB) free at end of 'apt upgrade'
Start 'apt autoremove'
IMG 'img/bookworm.img' has 196856 1K-blocks (0.2GB, 0.2GiB) free at start of 'apt autoremove'
IMG 'img/bookworm.img' has 196856 1K-blocks (0.2GB, 0.2GiB) free at end of 'apt autoremove'
IMG 'img/bookworm.img' has 196856 1K-blocks (0.2GB, 0.2GiB) free at end of Phase 1 image customization

• Phase 1 Completed

Run Plugins Phase 'post-install'
Run Plugin 'user' (/usr/local/sdm/plugins/user) Phase post-install with arguments:
'adduser=tcronin|password='

• Plugin user: Start Phase post-install
• Plugin user: Complete Phase post-install

Run Plugin 'mkdir' (/usr/local/sdm/plugins/mkdir) Phase post-install with arguments:
'dir=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=700'

• Plugin mkdir: Start Phase post-install

Plugin mkdir: Change owner on '/home/tcronin/.ssh' to 'tcronin:tcronin'
Plugin mkdir: Change directory protection for '/home/tcronin/.ssh' to '700'

• Plugin mkdir: Complete Phase post-install

Run Plugin 'copyfile' (/usr/local/sdm/plugins/copyfile) Phase post-install with arguments:
'from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinst
all|chown=tcronin:tcronin|chmod=600'

• Plugin copyfile: Start Phase post-install
• Plugin copyfile: Complete Phase post-install

Run Plugin 'user' (/usr/local/sdm/plugins/user) Phase post-install with arguments:
'adduser=ansible|password='

• Plugin user: Start Phase post-install
• Plugin user: Complete Phase post-install

Run Plugin 'mkdir' (/usr/local/sdm/plugins/mkdir) Phase post-install with arguments:
'dir=/home/ansible/.ssh|chown=ansible:ansible|chmod=700'

• Plugin mkdir: Start Phase post-install

Plugin mkdir: Change owner on '/home/ansible/.ssh' to 'ansible:ansible'
Plugin mkdir: Change directory protection for '/home/ansible/.ssh' to '700'

• Plugin mkdir: Complete Phase post-install

Run Plugin 'copyfile' (/usr/local/sdm/plugins/copyfile) Phase post-install with arguments:
'from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase=postinst
all|chown=ansible:ansible|chmod=600'

• Plugin copyfile: Start Phase post-install
• Plugin copyfile: Complete Phase post-install

Run Plugin 'user' (/usr/local/sdm/plugins/user) Phase post-install with arguments:
'deluser=pi'

• Plugin user: Start Phase post-install
• Plugin user: Complete Phase post-install

Run Plugin 'L10n' (/usr/local/sdm/plugins/L10n) Phase post-install with arguments:
'keymap=us|locale=en_US.UTF-8|timezone=America/Chicago'
Run Plugin 'disables' (/usr/local/sdm/plugins/disables) Phase post-install with arguments:
'piwiz'

• Plugin disables: Start Phase post-install

Plugin disables: Disable running piwiz and userconfig on first system boot

• Plugin disables: Complete Phase post-install

Run Plugin 'network' (/usr/local/sdm/plugins/network) Phase post-install with arguments:
'netman=nm|wifissid=tec-wan|wifipassword=|wificountry=US'

• Plugin network: Start Phase post-install

Plugin network: Keys/values found:
netman: network-manager
wifissid: tec-wan
wifipassword:
wificountry: US
Plugin network: Set WiFi Country 'US' for sdm FirstBoot
Plugin network: Enable SSH service via /boot/ssh

• Plugin network: Complete Phase post-install

Run graphics post-install configuration
Plugin sdm-phase1: No known Display Manager found
Plugin sdm-phase1: Set RasPiOS to B1:'console no login' for sdm FirstBoot
IMG 'img/bookworm.img' has 196848 1K-blocks (0.2GB, 0.2GiB) free at end of image customization

• Enter Shell Command Prompt
  'exit' to exit back to host system

[gitbls]

Thanks! Will correct this for next release.

[gitbls]

This is fixed in v11.7. Please test/verify. Thx

[tim-oe]

works! thanks for the quick turn around.

[gitbls]

Perfect. And thanks for finding this rather nasty bug. Using the same source filename in two different directories never occurred to me. 😵