syzkaller is a fuzzer that test system calls. Therefore, it starts one or more VMs in order to test system calls on the OS (target) inside the VM.

syzkaller consists of three main components. syz-manager runs on the host. It starts the VMs and connects to them via SSH. syz-fuzzer runs inside a VM. It maintains one or more syz-executor, syz-executor actually executes the system call on the target.

To be able to start a VM, syzkaller uses a virtualization software. syzkaller best supports qemu.

The first task is to create VM with target OS. The second one is to build syzkaller (requires go) and the third and last one is to create a config in order to run syzkaller.

My environment:

• Host: Ubuntu 18.04.1 LTS (amd64)
• Target: FreeBSD-13.0-CURRENT-amd64-20190110
• QEMU emulator version 2.11.1

create directory and qemu hard disk image

mkdir -p ~/qemu/freebsd
cd ~/qemu/freebsd
sudo qemu-img create freebsd.img 16G

start vm and install FreeBSD (download iso from freebsd.org). Make sure FreeBSD source files are copied to /usr/src during installation.

sudo qemu-system-x86_64 -enable-kvm -hda freebsd.img -m 2G -net user,hostfwd=tcp::10021-:22 -net nic -curses -cdrom FreeBSD.iso -boot d

after installation let vm print to console → add in /boot/loader.conf

console="comconsole"

start vm without graphic and with network

sudo qemu-system-x86_64 -hda freebsd.img -enable-kvm -m 2G -net user,hostfwd=tcp::10021-:22 -net nic -nographic

• -net user let the vm uses a user mode network stack. Since hostfwd=tcp::10021-:22 is added, a connection to localhost:10021 (on host) is forwarded to the vm.

     guest (10.0.2.15)  <------>  Firewall/DHCP server <-----> Internet
                           |          (10.0.2.2)
                           |
                           ---->  DNS server (10.0.2.3)
                           |
                           ---->  SMB server (10.0.2.4)

• -nographic prints the console output of the vm directly into the console on the host

By now, KCOV support is not enabled by default. Create a own kernel config file in the VM (/usr/src should contain the FreeBSD source files).

cd /usr/src/sys/amd64/conf
cp GENERIC KCOV

In file KCOV, uncomment the lines

options        COVERAGE
options        KCOV

Build and install FreeBSD kernel as root, and restart

cd /usr/src
make -j4 kernel KERNCONF=KCOV
shutdown -r now

syzkaller needs root access to the OS in the VM via SSH.

create ssh keys on the host

ssh-keygen -f freebsd_id_rsa

add public key of the host (freebsd_id_rsa.pub) into /root/.ssh/authorized_keys in the vm

allow ssh as root in vm. In /etc/ssh/sshd_config set

#PermitRootLogin no

to

PermitRootLogin yes

after that, restart sshd

sudo service sshd restart

Test ssh from host to vm

ssh -i ~/qemu/freebsd/freebsd_id_rsa -p 10021 -o "StrictHostKeyChecking no" root@localhost

This must work without entering a password.

syzkaller uses Go → Install Go

wget https://storage.googleapis.com/golang/go1.10.4.linux-amd64.tar.gz
tar -xf go1.10.4.linux-amd64.tar.gz
mv go goroot
export GOROOT=`pwd`/goroot
export PATH=$GOROOT/bin:$PATH
mkdir gopath
export GOPATH=`pwd`/gopath

get syzkaller

go get -u -d github.com/google/syzkaller/...

build syz-manager, syz-fuzzer on host

cd gopath/src/github.com/google/syzkaller/
mkdir workdir
make manager fuzzer execprog TARGETOS=freebsd

note the used GitRevision, for me it was

GitRevision=5b11ac2c91d99eababa8f04c7e2f9484c0763b53

build syz-executor on target (inside the VM)

• create folder in VM

mkdir -p ~/syzkaller/executor

• use scp on host to copy source files from host to VM

scp -P 10021 ~/gopath/github.com/google/syzkaller/executor/* localhost:~/src/syzkaller/executor/

• build syz-executor in vm using the GitRevision as noted before.

cd ~/syzkaller
c++ -o syz-executor executor/executor.cc -pthread -Wall -Wframe-larger-than=8192 -Wparentheses -Werror -O2 -m64 -static -DGOOS_freebsd=1 -DGOARCH_amd64=1 -DGIT_REVISION=\"5b11ac2c91d99eababa8f04c7e2f9484c0763b53\"

• use scp to copy built executable from VM to host.

scp -P 10021 localhost:~/syzkaller/syz-executor ~/gopath/src/github.com/google/syzkaller/bin/freebsd_amd64/

To present code coverage, syzkaller needs FreeBSD source and object files. Start the VM with qemu and use scp on host to copy source and object files from target to host.

mkdir -p ~/freebsd/usr
scp -r -o "StrictHostKeyChecking no" -P 10021 localhost:/usr/src ~/freebsd/usr/
scp -r -o "StrictHostKeyChecking no" -P 10021 localhost:/usr/obj ~/freebsd/usr/

create syz-manager config file (on host) ~/gopath/src/github.com/google/syzkaller/freebsd.cfg

{
  "target": "freebsd/amd64",
  "http": "127.0.0.1:56741",
  "workdir": "/home/msvoelker/gopath/src/github.com/google/syzkaller/workdir",
  "image": "/home/msvoelker/qemu/freebsd/freebsd.img",
  "sshkey": "/home/msvoelker/qemu/freebsd/freebsd_id_rsa",
  "syzkaller": "/home/msvoelker/gopath/src/github.com/google/syzkaller",
  "kernel_obj": "/home/msvoelker/freebsd/usr/obj/usr/src/amd64.amd64/sys/KCOV",
  "kernel_src": "/home/msvoelker/freebsd",
  "enable_syscalls": [
    "socket$inet_sctp",
    "bind$inet",
    "listen",
    "connect$inet",
    "accept$inet",
    "read",
    "write",
    "close"
  ],
  "procs": 1,
  "type": "qemu",
  "cover": true,
  "vm": {
    "count": 16,
    "cpu": 1,
    "mem": 1024
  }
}

start syz-manager on host

cd ~/gopath/src/github.com/google/syzkaller
sudo bin/syz-manager -config freebsd.cfg

github repository for syzkaller (contains documentation): https://github.com/google/syzkaller