Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[warm boot] move warm reboot/fast reboot knowledge to syncd service script #372

Merged
merged 5 commits into from
Nov 15, 2018
Merged

[warm boot] move warm reboot/fast reboot knowledge to syncd service script #372

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7b3fddf
[warm boot] set warm boot flag before killing swss docker
yxieca Nov 7, 2018
1588815
[fast/warm reboot] let syncd service handle different types of shutdowns
yxieca Nov 8, 2018
d44bf0b
[warm/fast reboot] stop execution after error
yxieca Nov 13, 2018
42204d5
[warm boot] set warm boot flag earlier and clear it on error
yxieca Nov 13, 2018
3085bbb
[warm boot] stop syncd before backing up database
yxieca Nov 13, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 17 additions & 21 deletions scripts/fast-reboot
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,20 @@
#!/bin/bash
#!/bin/bash -e

REBOOT_USER=$(logname)
REBOOT_TIME=$(date)
REBOOT_CAUSE_FILE="/var/cache/sonic/reboot-cause.txt"
REBOOT_TYPE=$(basename $0)
WARM_DIR=/host/warmboot

function clear_warm_boot()
{
config warm_restart disable || /bin/true

TIMESTAMP=`date +%Y%m%d-%H%M%S`
if [[ -f ${WARM_DIR}/config_db.json ]]; then
mv -f ${WARM_DIR}/config_db.json ${WARM_DIR}/config_db-${TIMESTAMP}.json || /bin/true
fi
}

# Check reboot type supported
BOOT_TYPE_ARG="cold"
Expand All @@ -13,6 +24,8 @@ case "$REBOOT_TYPE" in
;;
"warm-reboot")
BOOT_TYPE_ARG="warm"
trap clear_warm_boot EXIT HUP INT QUIT TERM KILL ABRT ALRM
config warm_restart enable system
;;
*)
echo "Not supported reboot type: $REBOOT_TYPE" >&2
Expand Down Expand Up @@ -119,15 +132,15 @@ fi
# Kill swss dockers
docker kill swss

# syncd service stop is capable of handling both warm/fast/cold shutdown
systemctl stop syncd
Copy link
Contributor

@qiluo-msft qiluo-msft Nov 14, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible risk with these 2 lines.

docker kill swss will trigger swss service stop(), and will docker stop syncd. Then it will trigger syncd service stop().

But you manually systemctl stop syncd again. #Closed

Copy link
Contributor Author

@yxieca yxieca Nov 15, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested with following code:

#!/bin/bash

docker kill swss
echo $?

sudo systemctl stop syncd
echo $?

sonic prompt # ./test.sh
swss
0
0

Stopping a service after it has been stopped is no-op with success result. Does this address your concern? or are you thinking about something else?


# Warm reboot: dump state to host disk
# Note: even if syncd changed ASIC_DB before killed, we don't care
if [[ "$REBOOT_TYPE" = "warm-reboot" ]]; then
# Set whole system warm reboot flag
config warm_restart enable system
# Dump redis content to directory
# Note: don't use pretty mode redis-dump (1.1) since there is a known bug with key pattern
DUMP_CMD="redis-dump -s /var/run/redis/redis.sock"
WARM_DIR=/host/warmboot
mkdir -p $WARM_DIR
# Note: requiring redis-dump-load
# Save applDB in /host/warm-reboot/appl_db.json
Expand All @@ -144,23 +157,6 @@ if [[ "$REBOOT_TYPE" = "warm-reboot" ]]; then
$DUMP_CMD -d 3 -o $WARM_DIR/loglevel_db.json
fi

if [[ "$REBOOT_TYPE" = "warm-reboot" ]]; then
# Gracefully stop syncd for warm-reboot
systemctl stop syncd
elif [[ "$REBOOT_TYPE" = "fast-reboot" ]]; then
# syncd graceful stop for fast-reboot is supported only for Broadcom platforms only for now
if [[ "$sonic_asic_type" = 'broadcom' ]]; then
# Gracefully stop syncd
docker exec -i syncd /usr/bin/syncd_request_shutdown --cold > /dev/null

# Check that syncd was stopped
while docker top syncd | grep -q /usr/bin/syncd
do
sleep 0.1
done
fi
fi

# Kill other containers to make the reboot faster
docker ps -q | xargs docker kill > /dev/null

Expand Down