Reboots

 

Overview

Periodically reboot mission critical Solaris and Linux servers, verifying that all environments and applications restart correctly.

Configure cron jobs to send email and page reminders to perform reboots on a regularly scheduled basis, say once every 4 months.

 

Reboot Audit

Before rebooting, perform an audit to determine what is running...

  1. Get a listing of all processes and process owners on all boxes in a distributed system.

    ssh -f hostname ps -ef  |  cut -c1-8,47-120  |  sort  |  uniq

  2. Review scripts in the /etc/init.d directory, grep'ing for processes found by the above ps command. Review the start/kill scripts in the linked /etc/rc?d directories. Find out what is being stopped, and what is being started, during the reboot process.

    Note that K* scripts are executed first, followed by the S* scripts. Scripts ending in .sh are executed in the same shell and can be used to set environment variables used further on in the same directory.

    A basic startup script looks like this: 

        #!/bin/sh"
    # Sample /tech/sun/commands/init.d.html">init.d script.
    # Install a copy under /etc/init.d/your-daemon
    # make links to /etc/rc2.d/Sxxyour-daemon (or rc3.d)
    # and /etc/rc[01].d/Kxxyour-daemon.
    # Scripts ending in .sh are executed with the sh "." command.
    # Scripts not ending in .sh are executed as "sh script"

    case "$1" in
    start)
        #... commands to start daemon ....
        ;;
    stop)
        #... commands to stop daemon ....
        ;;
    esac

    To stop/start these scripts from the command...

    service scriptname stop
    service scriptname start
    service scriptname restart

  3. Review other startup files, including /etc/inittab and /etc/fstab.

 

Routing

If you get "network unreachable" messages, review routing

 

NFS

Before reboots, unmount any remote NFS filesystems. Clients that mount NFS filesystems, but never unmount them before shutting down, leave stale information in the server's rmtab file.

To remount remote NFS filesystems, put share commands in the /etc/dfs/dfstab file. Here is a sample dfstab file containing our two share commands: 

% cat /etc/dfs/dfstab 
#   place share(1M) commands here for automatic execution 
#   on entering init state 3. 
# 
#   share [-F fstype] [ -o options] [-d ""]  [resource] 
#   .e.g., 
#   share  -F nfs  -o rw=engineering  -d "home dirs"  /export/home2 
share -F nfs -o rw=crab:horseshoe ro  /usr/man
share -F nfs -o rw=rodent:crab:horseshoe:jerboas  /export/home/research

 

/var/tmp

The /var/tmp directory is made available for programs that require temporary files or directories that are preserved between system reboots. Therefore, data stored in /var/tmp is more persistent than data in /tmp. Files and directories located in /var/tmp must not be deleted when the system is booted. Although data stored in /var/tmp is typically deleted in a site-specific manner, it is recommended that deletions occur at a less frequent interval than /tmp.

 

See Also

  1.   last
  2.   reboot (Sun)
  3.   shutdown (Sun)
  4.   reboot (Linux)
  5.   shutdown (Linux)
  6.   Solaris Multihomed Hosts
  7.   DNS Disaster Planning
  8.   Checking Routing
  9.   Startup Files
  10.   Auditing and Logging
  11.   TCP Basic Configuration
  12.   consadm
  13.   keyserv
  14.   halt.sh

 

Home