Debian mirror script “ftpsync” on OmniOS

by

The ftpsync script from the Debian archive mirror page is used to mirror the packages from the ftp.at.debian.org and security.debian.org archives.

The script shows some errors on OmniOS due to differences in options to the hostname command.

The blog post OmniOS lipkg zone as debian package mirror describes the setup of the zone on OmniOS thats used as a Debian package mirror.

The ftpsync script makes use of hostname -s (short hostname), hostname -f (full-qualified hostname) and hostname -d (domainname). These options work on Debian / Linux but not on OmniOS (and Illumos / Solaris).

The commands and the used equivalents:

  • hostname -s gives the same output as hostname without the -s
  • hostname -d equivalent is domainname. Add the domainname to /etc/defaultdomain
  • hostname -f from the suggestions on ServerFault i used check-hostname | awk '{ print $NF }'

Four files need to be changed in my setup and diffs are shown in the order the files are listed here.

  • etc/ftpsync.conf
  • etc/ftpsync-security.conf
  • etc/common
  • bin/ftpsync

The diffs can be applied from the base directory ftpsync with patch -p1 < patchfile.

The configuration files etc/ftpsync.conf and etc/ftpsync-security.conf set the variables MIRRORNAME and TRACEHOST that hold the full-qualified hostname of the machine they are running on. The variable MIRRORNAME is set once and then used instead of hostname -f:

  • for MIRRORNAME replace hostname -f with check-hostname | awk '{ print $NF }'
  • for TRACEHOST replace $(hostname -f) with ${MIRRORNAME}
diff -u archvsync/etc/ftpsync.conf ftpsync/etc/ftpsync.conf
--- archvsync/etc/ftpsync.conf   Fri Oct  7 21:25:11 2016
+++ ftpsync/etc/ftpsync.conf    Sat Oct  8 08:20:06 2016
@@ -8,12 +8,12 @@

 ## Mirrorname. This is used for things like the trace file name and should always
 ## be the full hostname of the mirror.
-MIRRORNAME=`hostname -f`
+MIRRORNAME=`/usr/sbin/check-hostname | awk '{ print $NF }'`

 ## Destination of the mirrored files. Should be an empty directory.
 ## CAREFUL, this directory will contain the mirror. Everything else
@@ -157,7 +157,7 @@
 #EXTENDEDTRACE="full"

 ## The local hostname to be written to the trace file.
-TRACEHOST="$(hostname -f)"
+TRACEHOST="${MIRRORNAME}"

 ## We sync our mirror using rsync (everything else would be insane), so
 ## we need a few options set.

In the file etc/common a few uses of hostname -s need to be replaced by hostname:

diff -u ftpsync/etc/common archvsync/etc/common
diff -u archvsync/etc/common ftpsync/etc/common
--- archvsync/etc/common        Wed Oct 12 13:47:07 2016
+++ ftpsync/etc/common  Sat Oct  8 08:20:06 2016
@@ -188,9 +188,9 @@
 # Set $PROGRAM to a string to have it added to the output.
 log () {
     if [[ -z "${PROGRAM}" ]]; then
-        echo "$(date +"%b %d %H:%M:%S") $(hostname -s) [$] $@"
+        echo "$(date +"%b %d %H:%M:%S") $(hostname) [$] $@"
     else
-        echo "$(date +"%b %d %H:%M:%S") $(hostname -s) ${PROGRAM}[$]: $@"
+        echo "$(date +"%b %d %H:%M:%S") $(hostname) ${PROGRAM}[$]: $@"
     fi
 }

@@ -199,7 +199,7 @@
 error () {
     log "$@"
     if [[ -n "${MAILTO}" ]]; then
-        echo "$@" | mail -e -s "[$PROGRAM@$(hostname -s)] ERROR [$]" ${MAILTO}
+        echo "$@" | mail -e -s "[$PROGRAM@$(hostname)] ERROR [$]" ${MAILTO}
     fi
 }

The script bin/ftpsync has all three options to hostname.
The variable substitutions in line 26/27 and 44/45 could be omitted completely as the variables are already set in the config file.

diff -u archvsync/bin/ftpsync ftpsync/bin/ftpsync
--- archvsync/bin/ftpsync       Wed Oct 12 13:47:07 2016
+++ ftpsync/bin/ftpsync Sat Oct  8 08:20:06 2016
@@ -116,7 +116,7 @@
     if [[ -n ${MAILTO} ]]; then
         # In case rsync had something on stderr
         if [[ -s ${LOGDIR}/rsync-${NAME}.error ]]; then
-            mail -E -s "[${PROGRAM}@$(hostname -s)] ($) rsync ERROR on $(date +"%Y.%m.%d-%H:%M:%S")" ${MAILTO} < "${LOGDIR}/rsync-${NAME}.error"
+            mail -E -s "[${PROGRAM}@$(hostname)] ($) rsync ERROR on $(date +"%Y.%m.%d-%H:%M:%S")" ${MAILTO} < "${LOGDIR}/rsync-${NAME}.error"
         fi
         if [[ ${ERRORSONLY} = false ]]; then
             # And the normal log
@@ -125,7 +125,7 @@
                 # Someone wants full logs including rsync
                 MAILFILES="${MAILFILES} ${LOGDIR}/rsync-${NAME}.log"
             fi
-            cat ${MAILFILES} | mail -E -s "[${PROGRAM}@$(hostname -s)] archive sync finished on $(date +"%Y.%m.%d-%H:%M:%S")" ${MAILTO}
+            cat ${MAILFILES} | mail -E -s "[${PROGRAM}@$(hostname)] archive sync finished on $(date +"%Y.%m.%d-%H:%M:%S")" ${MAILTO}
         fi
     fi

@@ -365,7 +365,7 @@
 ########################################################################
 # There should be nothing to edit here, use the config file            #
 ########################################################################
-MIRRORNAME=${MIRRORNAME:-$(hostname -f)}
+MIRRORNAME=${MIRRORNAME}
 # Where to put logfiles in
 LOGDIR=${LOGDIR:-"${BASEDIR}/log"}
 # Our own logfile
@@ -378,7 +378,7 @@
 PROGRAM=${PROGRAM:-"${NAME}"}

 # Where to send mails about mirroring to?
-if [[ $(hostname -d) != debian.org ]]; then
+if [[ $(domainname) != debian.org ]]; then
     # We are not on a debian.org host
     MAILTO=${MAILTO:-"root"}
 else
@@ -420,7 +420,7 @@
 EXTENDEDTRACE=${EXTENDEDTRACE:-"full"}

 # The local hostname to be written to the trace file.
-TRACEHOST=${TRACEHOST:-$(hostname -f)}
+TRACEHOST=${TRACEHOST:-${MIRRORNAME}}

 # rsync program
 RSYNC=${RSYNC:-rsync}
@@ -795,7 +795,7 @@
             fi
             if [[ ${difference} -ge 259200 ]]; then
                 # Only warn every third day
-                mail -s "[$(hostname -s)] Update for ftpsync available" ${MAILTO} <<EOF
+                mail -s "[$(hostname)] Update for ftpsync available" ${MAILTO} <<EOF
 Hello admin,

 i found that there is a new version of me available.
@@ -804,7 +804,7 @@

 Me, myself and I - and the Debian mirroradmins - would be very grateful
 if you could update me. You can find the latest version on your mirror,
-check $(hostname -s):${TO}/project/ftpsync/ftpsync-${LATEST}.tar.gz
+check $(hostname):${TO}/project/ftpsync/ftpsync-${LATEST}.tar.gz

 You can ensure the validity of that file by using sha512sum or md5sum
 against the available checksum files secured with a signature from the

With these changes in place the script runs on OmniOS.

There are still things to fix like the invokation of the mail command which probably doesn’t work on OmniOS but thats currently not so important to me.

Update 2016-11-21:

I found a problem with extended regex’s usage with grep when updating the trace file, the /usr/bin/grep on OmniOS only supports a subset of  the options. The /usr/xpg4/bin/grep support the -E switch

diff -u archvsync/bin/ftpsync ftpsync/bin/ftpsync
--- archvsync/bin/ftpsync       Mon Nov 21 08:00:47 2016
+++ ftpsync/bin/ftpsync Sat Oct  8 08:21:28 2016
@@ -242,7 +242,7 @@ tracefile() {
                     mv "${TO}/${TRACEHIERARCHY}".new "${TO}/${TRACEHIERARCHY}"
                     cp "${TO}/${TRACEHIERARCHY}" "${TO}/${TRACEHIERARCHY}.mirror"
 
-                    (cd ${tracedir} && ls -1rt) | grep -E -v '^_|^master$' > "${TO}/${TRACELIST}"
+                    (cd ${tracedir} && ls -1rt) | /usr/xpg4/bin/grep -E -v '^_|^master$' > "${TO}/${TRACELIST}"
                 else
                     rm -f "${TO}/${TRACEHIERARCHY}" "${TO}/${TRACELIST}"
                 fi

1 thought on “Debian mirror script “ftpsync” on OmniOS”

  1. Pingback: OmniOS lipkg zone as debian package mirror – valo.at

Leave a Comment