Configuring chroot(2) environment for named on Oracle Solaris. v1.2

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.

Stacey Marshall, Principal Software Engineer for Oracle Global Services Limited.

Introduction

The following is provided as a proof of concept so that engineering and services may test the feature and observe the difficulties and requirements of such a configuration. It does not constitute support for, or official documentation, it is subject to change.

A chroot(2) environment is a directory in which all necessary libraries, devices and required files are available for a process to execute following a call to chroot(). A chroot environment should be designed to be the absolute minimum required to support the particular application that runs in it.

The -t option to named(1M) causes it to call chroot() with the directory specified before reading the configuration file. The chroot directory has to be setup and maintained in accordance with the requirements of BIND and any subsystem it then utilizes.

Oracle does not provide a configuration program to establish the chroot directory, or documentation on how to create the chroot directory environment manually. Such a configuration depends on the base OS, its libraries, version of named and the named.conf configuration. Those requirements may so easily change and hence the recommendation is to use Oracle Solaris zones instead.

The use of named ‘-t’ option is at the risk of the user, it is not an option that can be simply used, it requires the given directory to be created following any system or configuration change. Furthermore it may be removed in a future release of BIND and therefore within a patch update.

RECOMMENDATION is to use zones

Oracle Solaris zones provide the infrastructure to manage the zone.

By running dns server in its own zone any security flaw would then be limited to that zone. Furthermore running dns server as a non-root user is recommend, see SMF property start/user as documented in dns-server(8s).

Running a branded zone (BrandZ) is not necessary as Solaris provides dns server BIND in all current supported releases.

WARNING and REQUIREMENT NOTICE

Configuring a chroot directory is complicated and subject to change due to patching and or changes in configuration.

The configuration of a new chroot directory SHOULD be carried out following any updates of the server to ensure the chroot directory is up to date.

If named fails to run within a chroot configuration, but runs within a non-chroot configuration than the issue is a configuration issue, not a bug.

Failures following updates will be because the chroot directory needs some additional driver or library updated or added to it. The onus is on the system administrator to address that mismatch.

• IF USING CHROOT WITHIN A ZONE

The method of using cpio or mknod to create special devices will not work within a non-global zone. An alternative is to use LOopback virtual File System, lofs(7FS), mounted into the chroot directory.

Configuring a chroot environment for BIND named.

The following was completed using BIND 9.11.8 on Solaris 11.4

Step 1. Create a suitable unique user and group.

When creating a suitably named unique user and group make sure that both names and id numbers are unique (getent(1) should not match any existing names):

• Example create-named-user:

  getent group dns || getent group 200 || groupadd -g 200 dns
  getent passwd named || getent passwd 200 ||
      useradd -u 200 -G 200 -c "DNS server" -d /var/named/chroot \
          -s /bin/false named
  usermod -A solaris.smf.manage.bind named
  

Step 2. Initialize suitable root directory environment

Create the necessary directory structure and copy the required BIND executables and libraries (see ldd(1)) to the new root filesystem.

Note: variable chroot will be used through this document as the name of the chroot directory. Recommend that in a production it be named to identify the configuration it represents, for example bind-s11.4sru16.

• Example initial-chroot-env:

  chroot=/var/named/chroot
  mkdir -p $chroot
  cd $chroot
  mkdir -p lib usr/dns usr/sfw dev etc var/run
  chown named:dns $chroot/var/run
  chown named:dns $chroot
  cp /lib/libc.so.1 /lib/ld.so.1 ./lib/.
  OPS2=$PS2
  PS2=""
  ldd /usr/sbin/named | while read l a f
  do
     echo copying lib $f
     if [ -f "$f" ]; then
        subdir=`dirname $f`
        [ ! -d "$chroot$subdir" ] && mkdir -p "$chroot$subdir"
        cp "$f" "$chroot$f"
      fi
  done
  

Step 3. Create necessary devices

In addition to the files some devices may be needed by BIND and its support libraries (for example openSSL). Note additional drivers may be needed by other libraries too!

The devices to create are found by looking for /dev within the BIND binaries. Additional devices may be needed, for example for cryptography!

Here I have used lofs to loop-back-mount the devices.

Note, remember to update vfstab(5) to reflect this requirement.

• Example device-loopback.

  echo Creating chroot devices under $chroot
  mkdir -p $chroot/dev
  # devices="random null poll zero tty"
  devices=`strings /usr/sbin/named /usr/lib/dns/lib*.so /usr/lib/dns/*/lib*.so | \
    awk '/^\/dev\//{print $1}' | sort -u`
  echo discovered $devices
  for dev in $devices; do
      echo creating $chroot$dev
      touch $chroot$dev;
      mount -F lofs $dev $chroot$dev;
  done
  mount | grep $chroot
  

The above for BIND 9.11.8 mounted the following devices:

• Example lofs-mounts

  # mount | grep $chroot
  /var/named/chroot/dev/null on /dev/null read/write/setuid/devices/rstchown/dev=fff40000 on Fri Jan  3 09:47:45 2020
  /var/named/chroot/dev/poll on /dev/poll read/write/setuid/devices/rstchown/dev=fff40000 on Fri Jan  3 09:47:45 2020
  /var/named/chroot/dev/random on /dev/random read/write/setuid/devices/rstchown/dev=fff40000 on Fri Jan  3 09:47:45 2020
  /var/named/chroot/dev/tty on /dev/tty read/write/setuid/devices/rstchown/dev=fff40000 on Fri Jan  3 09:47:45 2020
  

Update vfstab to include the loop-back mount director`ies.

NOTE: remove older loopback mounts as necessary (the old chroot directory from previous maintenance updates).

• Example Update-vfstab

  mount | nawk -v dir=$chroot '$0 ~ dir {printf "\%s\t-\t%s\tlofs\t-\tyes\t-\n", $3, $1;}' >> /etc/vfstab
  grep $chroot /etc/vfstab
  

Using older mknod and cpio method (For historical reference only)

WARNING: Not suitable within non-global zones (bug 19076171).

mknod (For historical reference only)

Each identified device would need to be created. The following creates the poll device. Similar commands would be run to create the other devices.

• Example mknod.

  # ls -ld /devices/pseudo/poll@0:poll
  crw-rw-rw-  1 root  sys  138, 0 Mar 3 15:25 /devices/pseudo/poll@0:poll
  # cd $chroot/dev
  # ls
  null    random
  # mknod poll c 138 0
  # chmod 666 poll
  # ls -ld /var/named/chroot/dev/poll
  crw-rw-rw-   1 root     root     138,  0 Mar  3 15:33 /var/named/chroot/dev/poll
  

cpio (For historical reference only)

The following cpio example creates the devices as listed within the here-document. This is an older example, not applicable with the current release.

• Example cpio.

  cpio -pduL /var/named/chroot >/dev/null 2>&1 <<EOF
        /dev/null
        /dev/random
        /dev/poll
        /dev/crypto
        EOF
  chmod 666 /var/named/chroot/dev/*
  

Step 4. Create BIND configuration

Create rndc key files.

• Example rndc-confgen

  rndc-confgen -t $chroot -a
  mv /etc/named.conf $chroot/etc/named.conf
  

Copy any zone files and or duplicate any directory structure required by named, as defined in named.conf.

• Example move-named-config:

  # grep directory $chroot/etc/named.conf
    	      directory "/var/named/config"
  # mkdir -p $chroot/var/named
  # cp -r /var/named/config $chroot/var/named/.
  

Step 5. Test, update and re-test as necessary

To test the configuration run named from root command line specifying the user with ‘-u’ option, the chroot directory with the ‘-t’ option and running it in the foreground with the ‘-g’ option:

• Example named-test-command

  named -u named -t $chroot -c /etc/named.conf -g
  

And, if necessary, use truss to discover other missing requirements.

BIND 9.11.8 required a fair number of openssl files discovered when run under truss, output showed “Err#2 ENOENT”:

• Example truss-named-command

  truss named -u named -t $chroot -c /etc/named.conf -g
  

• Example truss-named-output: with truss output sent to file /tmp/truss.

  root@solaris:/# truss -o /tmp/truss named -u named -t $chroot -c /etc/named.conf -g
  03-Jan-2020 15:26:51.768 starting BIND 9.11.8 (Extended Support Version)
  03-Jan-2020 15:26:51.770 running on SunOS i86pc 5.11 11.4.15.5.0
  03-Jan-2020 15:26:51.772 built with '--prefix=/usr' '--mandir=/usr/share/man' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--libdir=/usr/lib/dns/amd64' '--enable-full-report' '--with-python=/usr/bin/python3.4' '--with-libtool' '--with-openssl=/usr' '--with-pkcs11=/usr/lib/amd64/libpkcs11.so.1' '--with-libxml2=/usr' '--enable-threads' '--enable-devpoll' '--enable-fixed-rrset' '--with-tuning=large' '--enable-largefile' '--sysconfdir=/etc' '--localstatedir=/var' '--with-randomdev=/dev/random' '--with-gssapi=krb5-config' '--with-docbook-xsl=/usr/share/sgml/docbook' '--with-python-install-dir=/usr/lib/python3.4/vendor-packages' 'CC=/usr/gcc/7/bin/gcc' 'CFLAGS=-m64 -O3' 'LDFLAGS=' 'CPPFLAGS=-m64' 'PKG_CONFIG_PATH=/usr/lib/amd64/pkgconfig'
  03-Jan-2020 15:26:51.773 running as: named -u named -t /var/named/chroot -c /etc/named.conf -g
  03-Jan-2020 15:26:51.774 compiled by GCC 7.3.0
  03-Jan-2020 15:26:51.775 compiled with OpenSSL version: OpenSSL 1.0.2o  27 Mar 2018
  03-Jan-2020 15:26:51.788 linked to OpenSSL version: OpenSSL 1.0.2r  26 Feb 2019
  03-Jan-2020 15:26:51.790 compiled with libxml2 version: 2.9.5
  03-Jan-2020 15:26:51.792 linked to libxml2 version: 20909
  03-Jan-2020 15:26:51.793 compiled with libjson-c version: 0.12
  03-Jan-2020 15:26:51.798 linked to libjson-c version: 0.12
  03-Jan-2020 15:26:51.800 compiled with zlib version: 1.2.11
  03-Jan-2020 15:26:51.805 linked to zlib version: 1.2.11
  03-Jan-2020 15:26:51.807 threads support is enabled
  03-Jan-2020 15:26:51.808 ----------------------------------------------------
  03-Jan-2020 15:26:51.809 BIND 9 is maintained by Internet Systems Consortium,
  03-Jan-2020 15:26:51.811 Inc. (ISC), a non-profit 501(c)(3) public-benefit
  03-Jan-2020 15:26:51.812 corporation.  Support and training for BIND 9 are
  03-Jan-2020 15:26:51.814 available at https://www.isc.org/support
  03-Jan-2020 15:26:51.815 ----------------------------------------------------
  03-Jan-2020 15:26:51.818 found 2 CPUs, using 2 worker threads
  03-Jan-2020 15:26:51.820 using 1 UDP listener per interface
  03-Jan-2020 15:26:51.845 using up to 21000 sockets
  03-Jan-2020 15:26:51.881 initializing DST: no engine
  03-Jan-2020 15:26:51.883 exiting (due to fatal error)
  

  From the above we can see that named exited, no engine for DST was found…

• Example examine-truss-file

  root@solaris:/# egrep -n 'chroot\(|ENOENT' /tmp/truss
  9:openat(AT_FDCWD, "/var/ld/64/ld.config", O_RDONLY) Err#2 ENOENT
  10:fstatat(AT_FDCWD, "/usr/lib/dns/amd64/libc.so.1", 0x7FD6382090D0, 0) Err#2 ENOENT
  35:fstatat(AT_FDCWD, "/usr/lib/dns/amd64/libxml2.so.2", 0x7FD638208C30, 0) Err#2 ENOENT
  59:openat(AT_FDCWD, "/usr/lib/locale/en_US.UTF-8/LC_MESSAGES/libdns.cat", O_RDONLY) Err#2 ENOENT
  60:openat(AT_FDCWD, "/usr/lib/locale/en_US.UTF-8/LC_MESSAGES/libisc.cat", O_RDONLY) Err#2 ENOENT
  61:openat(AT_FDCWD, "/usr/lib/locale/en_US.UTF-8/LC_MESSAGES/libdst.cat", O_RDONLY) Err#2 ENOENT
  68:openat(AT_FDCWD, "/usr/lib/locale/en_US.UTF-8/LC_MESSAGES/libisccc.cat", O_RDONLY) Err#2 ENOENT
  114:fstatat(AT_FDCWD, "/usr/lib/dns/amd64/libscf.so.1", 0x7FD638209330, 0) Err#2 ENOENT
  179:chroot("/var/named/chroot")                 = 0
  214:fstatat(AT_FDCWD, "/usr/lib/dns/amd64/libcrypto.so.1.0.0", 0x7FD638209370, 0) Err#2 ENOENT
  237:fstatat(AT_FDCWD, "/usr/lib/dns/amd64/libjson-c.so.2", 0x7FD638209370, 0) Err#2 ENOENT
  238:fstatat(AT_FDCWD, "/lib/64/libjson-c.so.2", 0x7FD638209370, 0) Err#2 ENOENT
  253:fstatat(AT_FDCWD, "/usr/lib/dns/amd64/libz.so.1", 0x7FD638209370, 0) Err#2 ENOENT
  391:/1: openat(AT_FDCWD, "/etc/openssl/openssl.cnf", O_RDONLY) Err#2 ENOENT
  396:/1: fstatat(AT_FDCWD, "/lib/openssl/engines/64/libpk11.so", 0x7FD638209750, 0) Err#2 ENOENT
  397:/1: openat(AT_FDCWD, "/usr/lib/locale/en_US.UTF-8/LC_MESSAGES/solaris_linkers.mo", O_RDONLY) Err#2 ENOENT
  398:/1: openat(AT_FDCWD, "/usr/lib/locale/en_US.UTF-8/LC_MESSAGES/solaris_lib_libc.mo", O_RDONLY) Err#2 ENOENT
  

  Looking at the truss file the chroot() call occurred at line 179, files after this point need to come from the chroot directory. Some of the above entries were subsequently found else where (not shown here), the main fault occurred in openssl and so those files needed to be included.

The following was needed for the simple named configuration I used, a different configuration may need other files! Again, this list is most certainly going to change.

• Example additional-chroot-files-subject-to-change

  mkdir -p $chroot/etc/openssl
  cp /etc/openssl/openssl.cnf $chroot/etc/openssl/openssl.cnf
  cd /lib
  tar cf - openssl | (cd $chroot/lib; tar xf -)
  cp /usr/lib/64/libpkcs11.so.1 $chroot//usr/lib/64/libpkcs11.so.1
  
  cp /usr/lib/64/libucrypto.so.1 $chroot/lib/64/libucrypto.so.1
  
  mkdir $chroot/etc/crypto
  cp /etc/crypto/ucrypto.conf $chroot/etc/crypto/.
  
  cp /etc/crypto/pkcs11.conf $chroot/etc/crypto/pkcs11.conf
  

Step 6. Update SMF to start named using chroot env.

Lastly, modify smf(5) service instance properties to use chroot directory, the newly added user and restart named.

Only do this once testing has proved the chroot configuration works.

• Example update-SMF

  svccfg -s dns/server:default setprop \
        options/chroot_dir=$chroot
  svccfg -s dns/server:default setprop \
        start/user=named
  svcadm refresh dns/server:default
  svcadm restart dns/server:default
  

Final WARNING:

As you can see steps 2 through 5 have copied or created a fair number of files including executable, configuration, devices and libraries. Hence this chroot environment should be recreated each time the configuration or system is updated to make sure it has the latest versions. This is a complicated procedure and this is why I recommend using Zones instead of chroot environments.

---