Ubuntu/Debian: Remove prompts from commands

This should facilitate easier copy-and-paste.
Richard Laager 2019-11-04 21:20:16 -06:00
parent a0b47d0b57
commit 0ea96f54fb
3 changed files with 512 additions and 474 deletions

@ -2,11 +2,11 @@ This experimental guide has been made official at [[Debian Buster Root on ZFS]].
If you have an existing system installed from the experimental guide, adjust your sources:
# vi /etc/apt/sources.list.d/buster-backports.list
vi /etc/apt/sources.list.d/buster-backports.list
deb http://deb.debian.org/debian buster-backports main contrib
deb-src http://deb.debian.org/debian buster-backports main contrib
# vi /etc/apt/preferences.d/90_zfs
vi /etc/apt/preferences.d/90_zfs
Package: libnvpair1linux libuutil1linux libzfs2linux libzpool2linux zfs-dkms zfs-initramfs zfs-test zfsutils-linux zfs-zed
Pin: release n=buster-backports
Pin-Priority: 990
@ -15,19 +15,19 @@ This will allow you to upgrade from the locally-built packages to the official b
You should set a root password before upgrading:
# passwd
passwd
Apply updates:
# apt update
# apt dist-upgrade
apt update
apt dist-upgrade
Reboot:
# reboot
reboot
If the bpool fails to import, then enter the rescue shell (which requires a root password) and run:
# zpool import -f bpool
# zpool export bpool
# reboot
zpool import -f bpool
zpool export bpool
reboot

@ -41,27 +41,27 @@ ZFS native encryption encrypts the data and most metadata in the root pool. It d
If you have a second system, using SSH to access the target system can be convenient.
$ sudo apt update
$ sudo apt install --yes openssh-server
$ sudo systemctl restart ssh
sudo apt update
sudo apt install --yes openssh-server
sudo systemctl restart ssh
**Hint:** You can find your IP address with `ip addr show scope global | grep inet`. Then, from your main machine, connect with `ssh user@IP`.
1.3 Become root:
$ sudo -i
sudo -i
1.4 Setup and update the repositories:
# echo deb http://deb.debian.org/debian buster contrib >> /etc/apt/sources.list
# echo deb http://deb.debian.org/debian buster-backports main contrib >> /etc/apt/sources.list
# apt update
echo deb http://deb.debian.org/debian buster contrib >> /etc/apt/sources.list
echo deb http://deb.debian.org/debian buster-backports main contrib >> /etc/apt/sources.list
apt update
1.5 Install ZFS in the Live CD environment:
# apt install --yes debootstrap gdisk dkms dpkg-dev linux-headers-$(uname -r)
# apt install --yes -t buster-backports zfs-dkms
# modprobe zfs
apt install --yes debootstrap gdisk dkms dpkg-dev linux-headers-$(uname -r)
apt install --yes -t buster-backports zfs-dkms
modprobe zfs
* The dkms dependency is installed manually just so it comes from buster and not buster-backports. This is not critical.
@ -70,32 +70,37 @@ If you have a second system, using SSH to access the target system can be conven
2.1 If you are re-using a disk, clear it as necessary:
If the disk was previously used in an MD array, zero the superblock:
# apt install --yes mdadm
# mdadm --zero-superblock --force /dev/disk/by-id/scsi-SATA_disk1
apt install --yes mdadm
mdadm --zero-superblock --force /dev/disk/by-id/scsi-SATA_disk1
Clear the partition table:
# sgdisk --zap-all /dev/disk/by-id/scsi-SATA_disk1
sgdisk --zap-all /dev/disk/by-id/scsi-SATA_disk1
2.2 Partition your disk(s):
Run this if you need legacy (BIOS) booting:
# sgdisk -a1 -n1:24K:+1000K -t1:EF02 /dev/disk/by-id/scsi-SATA_disk1
sgdisk -a1 -n1:24K:+1000K -t1:EF02 /dev/disk/by-id/scsi-SATA_disk1
Run this for UEFI booting (for use now or in the future):
# sgdisk -n2:1M:+512M -t2:EF00 /dev/disk/by-id/scsi-SATA_disk1
sgdisk -n2:1M:+512M -t2:EF00 /dev/disk/by-id/scsi-SATA_disk1
Run this for the boot pool:
# sgdisk -n3:0:+1G -t3:BF01 /dev/disk/by-id/scsi-SATA_disk1
sgdisk -n3:0:+1G -t3:BF01 /dev/disk/by-id/scsi-SATA_disk1
Choose one of the following options:
2.2a Unencrypted or ZFS native encryption:
# sgdisk -n4:0:0 -t4:BF01 /dev/disk/by-id/scsi-SATA_disk1
sgdisk -n4:0:0 -t4:BF01 /dev/disk/by-id/scsi-SATA_disk1
2.2b LUKS:
# sgdisk -n4:0:0 -t4:8300 /dev/disk/by-id/scsi-SATA_disk1
sgdisk -n4:0:0 -t4:8300 /dev/disk/by-id/scsi-SATA_disk1
Always use the long `/dev/disk/by-id/*` aliases with ZFS. Using the `/dev/sd*` device nodes directly can cause sporadic import failures, especially on systems that have more than one storage pool.
@ -106,7 +111,7 @@ Always use the long `/dev/disk/by-id/*` aliases with ZFS. Using the `/dev/sd*`
2.3 Create the boot pool:
# zpool create -o ashift=12 -d \
zpool create -o ashift=12 -d \
-o feature@async_destroy=enabled \
-o feature@bookmarks=enabled \
-o feature@embedded_data=enabled \
@ -143,7 +148,7 @@ Choose one of the following options:
2.4a Unencrypted:
# zpool create -o ashift=12 \
zpool create -o ashift=12 \
-O acltype=posixacl -O canmount=off -O compression=lz4 \
-O dnodesize=auto -O normalization=formD -O relatime=on -O xattr=sa \
-O mountpoint=/ -R /mnt \
@ -151,11 +156,11 @@ Choose one of the following options:
2.4b LUKS:
# apt install --yes cryptsetup
# cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha256 \
apt install --yes cryptsetup
cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha256 \
/dev/disk/by-id/scsi-SATA_disk1-part4
# cryptsetup luksOpen /dev/disk/by-id/scsi-SATA_disk1-part4 luks1
# zpool create -o ashift=12 \
cryptsetup luksOpen /dev/disk/by-id/scsi-SATA_disk1-part4 luks1
zpool create -o ashift=12 \
-O acltype=posixacl -O canmount=off -O compression=lz4 \
-O dnodesize=auto -O normalization=formD -O relatime=on -O xattr=sa \
-O mountpoint=/ -R /mnt \
@ -163,7 +168,7 @@ Choose one of the following options:
2.4c ZFS native encryption:
# zpool create -o ashift=12 \
zpool create -o ashift=12 \
-O acltype=posixacl -O canmount=off -O compression=lz4 \
-O dnodesize=auto -O normalization=formD -O relatime=on -O xattr=sa \
-O encryption=aes-256-gcm -O keylocation=prompt -O keyformat=passphrase \
@ -188,72 +193,84 @@ Choose one of the following options:
3.1 Create filesystem datasets to act as containers:
# zfs create -o canmount=off -o mountpoint=none rpool/ROOT
# zfs create -o canmount=off -o mountpoint=none bpool/BOOT
zfs create -o canmount=off -o mountpoint=none rpool/ROOT
zfs create -o canmount=off -o mountpoint=none bpool/BOOT
On Solaris systems, the root filesystem is cloned and the suffix is incremented for major system changes through `pkg image-update` or `beadm`. Similar functionality for APT is possible but currently unimplemented. Even without such a tool, it can still be used for manually created clones.
3.2 Create filesystem datasets for the root and boot filesystems:
# zfs create -o canmount=noauto -o mountpoint=/ rpool/ROOT/debian
# zfs mount rpool/ROOT/debian
zfs create -o canmount=noauto -o mountpoint=/ rpool/ROOT/debian
zfs mount rpool/ROOT/debian
# zfs create -o canmount=noauto -o mountpoint=/boot bpool/BOOT/debian
# zfs mount bpool/BOOT/debian
zfs create -o canmount=noauto -o mountpoint=/boot bpool/BOOT/debian
zfs mount bpool/BOOT/debian
With ZFS, it is not normally necessary to use a mount command (either `mount` or `zfs mount`). This situation is an exception because of `canmount=noauto`.
3.3 Create datasets:
# zfs create rpool/home
# zfs create -o mountpoint=/root rpool/home/root
# zfs create -o canmount=off rpool/var
# zfs create -o canmount=off rpool/var/lib
# zfs create rpool/var/log
# zfs create rpool/var/spool
zfs create rpool/home
zfs create -o mountpoint=/root rpool/home/root
zfs create -o canmount=off rpool/var
zfs create -o canmount=off rpool/var/lib
zfs create rpool/var/log
zfs create rpool/var/spool
The datasets below are optional, depending on your preferences and/or
software choices:
The datasets below are optional, depending on your preferences and/or software
choices.
If you wish to exclude these from snapshots:
# zfs create -o com.sun:auto-snapshot=false rpool/var/cache
# zfs create -o com.sun:auto-snapshot=false rpool/var/tmp
# chmod 1777 /mnt/var/tmp
zfs create -o com.sun:auto-snapshot=false rpool/var/cache
zfs create -o com.sun:auto-snapshot=false rpool/var/tmp
chmod 1777 /mnt/var/tmp
If you use /opt on this system:
# zfs create rpool/opt
zfs create rpool/opt
If you use /srv on this system:
# zfs create rpool/srv
zfs create rpool/srv
If you use /usr/local on this system:
# zfs create -o canmount=off rpool/usr
# zfs create rpool/usr/local
zfs create -o canmount=off rpool/usr
zfs create rpool/usr/local
If this system will have games installed:
# zfs create rpool/var/games
zfs create rpool/var/games
If this system will store local email in /var/mail:
# zfs create rpool/var/mail
zfs create rpool/var/mail
If this system will use Snap packages:
# zfs create rpool/var/snap
zfs create rpool/var/snap
If you use /var/www on this system:
# zfs create rpool/var/www
zfs create rpool/var/www
If this system will use GNOME:
# zfs create rpool/var/lib/AccountsService
zfs create rpool/var/lib/AccountsService
If this system will use Docker (which manages its own datasets & snapshots):
# zfs create -o com.sun:auto-snapshot=false rpool/var/lib/docker
zfs create -o com.sun:auto-snapshot=false rpool/var/lib/docker
If this system will use NFS (locking):
# zfs create -o com.sun:auto-snapshot=false rpool/var/lib/nfs
zfs create -o com.sun:auto-snapshot=false rpool/var/lib/nfs
A tmpfs is recommended later, but if you want a separate dataset for /tmp:
# zfs create -o com.sun:auto-snapshot=false rpool/tmp
# chmod 1777 /mnt/tmp
zfs create -o com.sun:auto-snapshot=false rpool/tmp
chmod 1777 /mnt/tmp
The primary goal of this dataset layout is to separate the OS from user data. This allows the root filesystem to be rolled back without rolling back user data such as logs (in `/var/log`). This will be especially important if/when a `beadm` or similar utility is integrated. The `com.sun.auto-snapshot` setting is used by some ZFS snapshot utilities to exclude transient data.
@ -261,8 +278,8 @@ If you do nothing extra, `/tmp` will be stored as part of the root filesystem. A
3.4 Install the minimal system:
# debootstrap buster /mnt
# zfs set devices=off rpool
debootstrap buster /mnt
zfs set devices=off rpool
The `debootstrap` command leaves the new system in an unconfigured state. An alternative to using `debootstrap` is to copy the entirety of a working system into the new ZFS root.
@ -270,9 +287,9 @@ The `debootstrap` command leaves the new system in an unconfigured state. An al
4.1 Configure the hostname (change `HOSTNAME` to the desired hostname).
# echo HOSTNAME > /mnt/etc/hostname
echo HOSTNAME > /mnt/etc/hostname
# vi /mnt/etc/hosts
vi /mnt/etc/hosts
Add a line:
127.0.1.1 HOSTNAME
or if the system has a real name in DNS:
@ -283,9 +300,12 @@ The `debootstrap` command leaves the new system in an unconfigured state. An al
4.2 Configure the network interface:
Find the interface name:
# ip addr show
# vi /mnt/etc/network/interfaces.d/NAME
ip addr show
Adjust NAME below to match your interface name:
vi /mnt/etc/network/interfaces.d/NAME
auto NAME
iface NAME inet dhcp
@ -293,50 +313,50 @@ Customize this file if the system is not a DHCP client.
4.3 Configure the package sources:
# vi /mnt/etc/apt/sources.list
vi /mnt/etc/apt/sources.list
deb http://deb.debian.org/debian buster main contrib
deb-src http://deb.debian.org/debian buster main contrib
# vi /mnt/etc/apt/sources.list.d/buster-backports.list
vi /mnt/etc/apt/sources.list.d/buster-backports.list
deb http://deb.debian.org/debian buster-backports main contrib
deb-src http://deb.debian.org/debian buster-backports main contrib
# vi /mnt/etc/apt/preferences.d/90_zfs
vi /mnt/etc/apt/preferences.d/90_zfs
Package: libnvpair1linux libuutil1linux libzfs2linux libzpool2linux zfs-dkms zfs-initramfs zfs-test zfsutils-linux zfsutils-linux-dev zfs-zed
Pin: release n=buster-backports
Pin-Priority: 990
4.4 Bind the virtual filesystems from the LiveCD environment to the new system and `chroot` into it:
# mount --rbind /dev /mnt/dev
# mount --rbind /proc /mnt/proc
# mount --rbind /sys /mnt/sys
# chroot /mnt /bin/bash --login
mount --rbind /dev /mnt/dev
mount --rbind /proc /mnt/proc
mount --rbind /sys /mnt/sys
chroot /mnt /bin/bash --login
**Note:** This is using `--rbind`, not `--bind`.
4.5 Configure a basic system environment:
# ln -s /proc/self/mounts /etc/mtab
# apt update
ln -s /proc/self/mounts /etc/mtab
apt update
# apt install --yes locales
# dpkg-reconfigure locales
apt install --yes locales
dpkg-reconfigure locales
Even if you prefer a non-English system language, always ensure that `en_US.UTF-8` is available.
# dpkg-reconfigure tzdata
dpkg-reconfigure tzdata
4.6 Install ZFS in the chroot environment for the new system:
# apt install --yes dpkg-dev linux-headers-amd64 linux-image-amd64
# apt install --yes zfs-initramfs
apt install --yes dpkg-dev linux-headers-amd64 linux-image-amd64
apt install --yes zfs-initramfs
4.7 For LUKS installs only, setup crypttab:
# apt install --yes cryptsetup
apt install --yes cryptsetup
# echo luks1 UUID=$(blkid -s UUID -o value \
echo luks1 UUID=$(blkid -s UUID -o value \
/dev/disk/by-id/scsi-SATA_disk1-part4) none \
luks,discard,initramfs > /etc/crypttab
@ -350,20 +370,20 @@ Choose one of the following options:
4.8a Install GRUB for legacy (BIOS) booting
# apt install --yes grub-pc
apt install --yes grub-pc
Install GRUB to the disk(s), not the partition(s).
4.8b Install GRUB for UEFI booting
# apt install dosfstools
# mkdosfs -F 32 -s 1 -n EFI /dev/disk/by-id/scsi-SATA_disk1-part2
# mkdir /boot/efi
# echo PARTUUID=$(blkid -s PARTUUID -o value \
apt install dosfstools
mkdosfs -F 32 -s 1 -n EFI /dev/disk/by-id/scsi-SATA_disk1-part2
mkdir /boot/efi
echo PARTUUID=$(blkid -s PARTUUID -o value \
/dev/disk/by-id/scsi-SATA_disk1-part2) \
/boot/efi vfat nofail,x-systemd.device-timeout=1 0 1 >> /etc/fstab
# mount /boot/efi
# apt install --yes grub-efi-amd64 shim-signed
mount /boot/efi
apt install --yes grub-efi-amd64 shim-signed
* The `-s 1` for `mkdosfs` is only necessary for drives which present 4 KiB logical sectors (“4Kn” drives) to meet the minimum cluster size (given the partition size of 512 MiB) for FAT32. It also works fine on drives which present 512 B sectors.
@ -371,13 +391,13 @@ Install GRUB to the disk(s), not the partition(s).
4.9 Set a root password
# passwd
passwd
4.10 Enable importing bpool
This ensures that `bpool` is always imported, regardless of whether `/etc/zfs/zpool.cache` exists, whether it is in the cachefile or not, or whether `zfs-import-scan.service` is enabled.
```
# vi /etc/systemd/system/zfs-import-bpool.service
vi /etc/systemd/system/zfs-import-bpool.service
[Unit]
DefaultDependencies=no
Before=zfs-import-scan.service
@ -391,21 +411,21 @@ This ensures that `bpool` is always imported, regardless of whether `/etc/zfs/zp
[Install]
WantedBy=zfs-import.target
# systemctl enable zfs-import-bpool.service
systemctl enable zfs-import-bpool.service
```
4.11 Optional (but recommended): Mount a tmpfs to /tmp
If you chose to create a `/tmp` dataset above, skip this step, as they are mutually exclusive choices. Otherwise, you can put `/tmp` on a tmpfs (RAM filesystem) by enabling the `tmp.mount` unit.
# cp /usr/share/systemd/tmp.mount /etc/systemd/system/
# systemctl enable tmp.mount
cp /usr/share/systemd/tmp.mount /etc/systemd/system/
systemctl enable tmp.mount
4.12 Optional (but kindly requested): Install popcon
The `popularity-contest` package reports the list of packages install on your system. Showing that ZFS is popular may be helpful in terms of long-term attention from the distro.
# apt install --yes popularity-contest
apt install --yes popularity-contest
Choose Yes at the prompt.
@ -413,24 +433,22 @@ Choose Yes at the prompt.
5.1 Verify that the ZFS boot filesystem is recognized:
# grub-probe /boot
zfs
grub-probe /boot
5.2 Refresh the initrd files:
# update-initramfs -u -k all
update-initramfs: Generating /boot/initrd.img-4.19.0-6-amd64
update-initramfs -u -k all
**Note:** When using LUKS, this will print "WARNING could not determine root device from /etc/fstab". This is because [cryptsetup does not support ZFS](https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1612906).
5.3 Workaround GRUB's missing zpool-features support:
# vi /etc/default/grub
vi /etc/default/grub
Set: GRUB_CMDLINE_LINUX="root=ZFS=rpool/ROOT/debian"
5.4 Optional (but highly recommended): Make debugging GRUB easier:
# vi /etc/default/grub
vi /etc/default/grub
Remove quiet from: GRUB_CMDLINE_LINUX_DEFAULT
Uncomment: GRUB_TERMINAL=console
Save and quit.
@ -439,11 +457,7 @@ Later, once the system has rebooted twice and you are sure everything is working
5.5 Update the boot configuration:
# update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.19.0-6-amd64
Found initrd image: /boot/initrd.img-4.19.0-6-amd64
done
update-grub
**Note:** Ignore errors from `osprober`, if present.
@ -451,22 +465,20 @@ Later, once the system has rebooted twice and you are sure everything is working
5.6a For legacy (BIOS) booting, install GRUB to the MBR:
# grub-install /dev/disk/by-id/scsi-SATA_disk1
Installing for i386-pc platform.
Installation finished. No error reported.
grub-install /dev/disk/by-id/scsi-SATA_disk1
Do not reboot the computer until you get exactly that result message. Note that you are installing GRUB to the whole disk, not a partition.
Note that you are installing GRUB to the whole disk, not a partition.
If you are creating a mirror or raidz topology, repeat the `grub-install` command for each disk in the pool.
5.6b For UEFI booting, install GRUB:
# grub-install --target=x86_64-efi --efi-directory=/boot/efi \
grub-install --target=x86_64-efi --efi-directory=/boot/efi \
--bootloader-id=debian --recheck --no-floppy
5.7 Verify that the ZFS module is installed:
# ls /boot/grub/*/zfs.mod
ls /boot/grub/*/zfs.mod
5.8 Fix filesystem mount ordering
@ -475,66 +487,71 @@ Until there is support for mounting `/boot` in the initramfs, we also need to mo
We need to activate `zfs-mount-generator`. This makes systemd aware of the separate mountpoints, which is important for things like `/var/log` and `/var/tmp`. In turn, `rsyslog.service` depends on `var-log.mount` by way of `local-fs.target` and services using the `PrivateTmp` feature of systemd automatically use `After=var-tmp.mount`.
For UEFI booting, unmount /boot/efi first:
# umount /boot/efi
umount /boot/efi
Everything else applies to both BIOS and UEFI booting:
# zfs set mountpoint=legacy bpool/BOOT/debian
# echo bpool/BOOT/debian /boot zfs \
zfs set mountpoint=legacy bpool/BOOT/debian
echo bpool/BOOT/debian /boot zfs \
nodev,relatime,x-systemd.requires=zfs-import-bpool.service 0 0 >> /etc/fstab
# mkdir /etc/zfs/zfs-list.cache
# touch /etc/zfs/zfs-list.cache/rpool
# ln -s /usr/lib/zfs-linux/zed.d/history_event-zfs-list-cacher.sh /etc/zfs/zed.d
# zed -F &
mkdir /etc/zfs/zfs-list.cache
touch /etc/zfs/zfs-list.cache/rpool
ln -s /usr/lib/zfs-linux/zed.d/history_event-zfs-list-cacher.sh /etc/zfs/zed.d
zed -F &
Verify that zed updated the cache by making sure this is not empty:
# cat /etc/zfs/zfs-list.cache/rpool
cat /etc/zfs/zfs-list.cache/rpool
If it is empty, force a cache update and check again:
# zfs set canmount=noauto rpool/ROOT/debian
zfs set canmount=noauto rpool/ROOT/debian
Stop zed:
# fg
fg
Press Ctrl-C.
Fix the paths to eliminate /mnt:
# sed -Ei "s|/mnt/?|/|" /etc/zfs/zfs-list.cache/rpool
sed -Ei "s|/mnt/?|/|" /etc/zfs/zfs-list.cache/rpool
## Step 6: First Boot
6.1 Snapshot the initial installation:
# zfs snapshot bpool/BOOT/debian@install
# zfs snapshot rpool/ROOT/debian@install
zfs snapshot bpool/BOOT/debian@install
zfs snapshot rpool/ROOT/debian@install
In the future, you will likely want to take snapshots before each upgrade, and remove old snapshots (including this one) at some point to save space.
6.2 Exit from the `chroot` environment back to the LiveCD environment:
# exit
exit
6.3 Run these commands in the LiveCD environment to unmount all filesystems:
# mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
# zpool export -a
mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
zpool export -a
6.4 Reboot:
# reboot
reboot
6.5 Wait for the newly installed system to boot normally. Login as root.
6.6 Create a user account:
# zfs create rpool/home/YOURUSERNAME
# adduser YOURUSERNAME
# cp -a /etc/skel/. /home/YOURUSERNAME
# chown -R YOURUSERNAME:YOURUSERNAME /home/YOURUSERNAME
zfs create rpool/home/YOURUSERNAME
adduser YOURUSERNAME
cp -a /etc/skel/. /home/YOURUSERNAME
chown -R YOURUSERNAME:YOURUSERNAME /home/YOURUSERNAME
6.7 Add your user account to the default set of groups for an administrator:
# usermod -a -G audio,cdrom,dip,floppy,netdev,plugdev,sudo,video YOURUSERNAME
usermod -a -G audio,cdrom,dip,floppy,netdev,plugdev,sudo,video YOURUSERNAME
6.8 Mirror GRUB
@ -542,21 +559,22 @@ If you installed to multiple disks, install GRUB on the additional disks:
6.8a For legacy (BIOS) booting:
# dpkg-reconfigure grub-pc
dpkg-reconfigure grub-pc
Hit enter until you get to the device selection screen.
Select (using the space bar) all of the disks (not partitions) in your pool.
6.8b UEFI
# umount /boot/efi
umount /boot/efi
For the second and subsequent disks (increment debian-2 to -3, etc.):
# dd if=/dev/disk/by-id/scsi-SATA_disk1-part2 \
dd if=/dev/disk/by-id/scsi-SATA_disk1-part2 \
of=/dev/disk/by-id/scsi-SATA_disk2-part2
# efibootmgr -c -g -d /dev/disk/by-id/scsi-SATA_disk2 \
efibootmgr -c -g -d /dev/disk/by-id/scsi-SATA_disk2 \
-p 3 -L "debian-2" -l '\EFI\debian\grubx64.efi'
# mount /boot/efi
mount /boot/efi
## Step 7: (Optional) Configure Swap
@ -564,7 +582,7 @@ If you installed to multiple disks, install GRUB on the additional disks:
7.1 Create a volume dataset (zvol) for use as a swap device:
# zfs create -V 4G -b $(getconf PAGESIZE) -o compression=zle \
zfs create -V 4G -b $(getconf PAGESIZE) -o compression=zle \
-o logbias=throughput -o sync=always \
-o primarycache=metadata -o secondarycache=none \
-o com.sun:auto-snapshot=false rpool/swap
@ -577,31 +595,31 @@ The compression algorithm is set to `zle` because it is the cheapest available a
**Caution**: Always use long `/dev/zvol` aliases in configuration files. Never use a short `/dev/zdX` device name.
# mkswap -f /dev/zvol/rpool/swap
# echo /dev/zvol/rpool/swap none swap discard 0 0 >> /etc/fstab
# echo RESUME=none > /etc/initramfs-tools/conf.d/resume
mkswap -f /dev/zvol/rpool/swap
echo /dev/zvol/rpool/swap none swap discard 0 0 >> /etc/fstab
echo RESUME=none > /etc/initramfs-tools/conf.d/resume
The `RESUME=none` is necessary to disable resuming from hibernation. This does not work, as the zvol is not present (because the pool has not yet been imported) at the time the resume script runs. If it is not disabled, the boot process hangs for 30 seconds waiting for the swap zvol to appear.
7.3 Enable the swap device:
# swapon -av
swapon -av
## Step 8: Full Software Installation
8.1 Upgrade the minimal system:
# apt dist-upgrade --yes
apt dist-upgrade --yes
8.2 Install a regular set of software:
# tasksel
tasksel
8.3 Optional: Disable log compression:
As `/var/log` is already compressed by ZFS, logrotates compression is going to burn CPU and disk I/O for (in most cases) very little gain. Also, if you are making snapshots of `/var/log`, logrotates compression will actually waste space, as the uncompressed data will live on in the snapshot. You can edit the files in `/etc/logrotate.d` by hand to comment out `compress`, or use this loop (copy-and-paste highly recommended):
# for file in /etc/logrotate.d/* ; do
for file in /etc/logrotate.d/* ; do
if grep -Eq "(^|[^#y])compress" "$file" ; then
sed -i -r "s/(^|[^#y])(compress)/\1#\2/" "$file"
fi
@ -609,7 +627,7 @@ As `/var/log` is already compressed by ZFS, logrotates compression is going t
8.4 Reboot:
# reboot
reboot
### Step 9: Final Cleanup
@ -617,29 +635,29 @@ As `/var/log` is already compressed by ZFS, logrotates compression is going t
9.2 Optional: Delete the snapshots of the initial installation:
$ sudo zfs destroy bpool/BOOT/debian@install
$ sudo zfs destroy rpool/ROOT/debian@install
sudo zfs destroy bpool/BOOT/debian@install
sudo zfs destroy rpool/ROOT/debian@install
9.3 Optional: Disable the root password
$ sudo usermod -p '*' root
sudo usermod -p '*' root
9.4 Optional: Re-enable the graphical boot process:
If you prefer the graphical boot process, you can re-enable it now. If you are using LUKS, it makes the prompt look nicer.
$ sudo vi /etc/default/grub
sudo vi /etc/default/grub
Add quiet to GRUB_CMDLINE_LINUX_DEFAULT
Comment out GRUB_TERMINAL=console
Save and quit.
$ sudo update-grub
sudo update-grub
**Note:** Ignore errors from `osprober`, if present.
9.5 Optional: For LUKS installs only, backup the LUKS header:
$ sudo cryptsetup luksHeaderBackup /dev/disk/by-id/scsi-SATA_disk1-part4 \
sudo cryptsetup luksHeaderBackup /dev/disk/by-id/scsi-SATA_disk1-part4 \
--header-backup-file luks1-header.dat
Store that backup somewhere safe (e.g. cloud storage). It is protected by your LUKS passphrase, but you may wish to use additional encryption.
@ -654,36 +672,36 @@ Go through [Step 1: Prepare The Install Environment](#step-1-prepare-the-install
For LUKS, first unlock the disk(s):
# apt install --yes cryptsetup
# cryptsetup luksOpen /dev/disk/by-id/scsi-SATA_disk1-part4 luks1
apt install --yes cryptsetup
cryptsetup luksOpen /dev/disk/by-id/scsi-SATA_disk1-part4 luks1
Repeat for additional disks, if this is a mirror or raidz topology.
Mount everything correctly:
# zpool export -a
# zpool import -N -R /mnt rpool
# zpool import -N -R /mnt bpool
# zfs load-key -a
# zfs mount rpool/ROOT/debian
# zfs mount -a
zpool export -a
zpool import -N -R /mnt rpool
zpool import -N -R /mnt bpool
zfs load-key -a
zfs mount rpool/ROOT/debian
zfs mount -a
If needed, you can chroot into your installed environment:
# mount --rbind /dev /mnt/dev
# mount --rbind /proc /mnt/proc
# mount --rbind /sys /mnt/sys
# chroot /mnt /bin/bash --login
# mount /boot
# mount -a
mount --rbind /dev /mnt/dev
mount --rbind /proc /mnt/proc
mount --rbind /sys /mnt/sys
chroot /mnt /bin/bash --login
mount /boot
mount -a
Do whatever you need to do to fix your system.
When done, cleanup:
# exit
# mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
# zpool export -a
# reboot
exit
mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
zpool export -a
reboot
### MPT2SAS
@ -709,11 +727,13 @@ Set a unique serial number on each virtual disk using libvirt or qemu (e.g. `-dr
To be able to use UEFI in guests (instead of only BIOS booting), run this on the host:
$ sudo apt install ovmf
$ sudo vi /etc/libvirt/qemu.conf
sudo apt install ovmf
sudo vi /etc/libvirt/qemu.conf
Uncomment these lines:
nvram = [
"/usr/share/OVMF/OVMF_CODE.fd:/usr/share/OVMF/OVMF_VARS.fd",
"/usr/share/AAVMF/AAVMF_CODE.fd:/usr/share/AAVMF/AAVMF_VARS.fd"
]
$ sudo service libvirt-bin restart
sudo service libvirt-bin restart

@ -36,58 +36,63 @@ LUKS encrypts almost everything: the OS, swap, home directories, and anything el
1.2 Setup and update the repositories:
$ sudo apt-add-repository universe
$ sudo apt update
sudo apt-add-repository universe
sudo apt update
1.3 Optional: Install and start the OpenSSH server in the Live CD environment:
If you have a second system, using SSH to access the target system can be convenient.
$ passwd
passwd
There is no current password; hit enter at that prompt.
$ sudo apt install --yes openssh-server
sudo apt install --yes openssh-server
**Hint:** You can find your IP address with `ip addr show scope global | grep inet`. Then, from your main machine, connect with `ssh ubuntu@IP`.
1.4 Become root:
$ sudo -i
sudo -i
1.5 Install ZFS in the Live CD environment:
# apt install --yes debootstrap gdisk zfs-initramfs
apt install --yes debootstrap gdisk zfs-initramfs
## Step 2: Disk Formatting
2.1 If you are re-using a disk, clear it as necessary:
If the disk was previously used in an MD array, zero the superblock:
# apt install --yes mdadm
# mdadm --zero-superblock --force /dev/disk/by-id/scsi-SATA_disk1
apt install --yes mdadm
mdadm --zero-superblock --force /dev/disk/by-id/scsi-SATA_disk1
Clear the partition table:
# sgdisk --zap-all /dev/disk/by-id/scsi-SATA_disk1
sgdisk --zap-all /dev/disk/by-id/scsi-SATA_disk1
2.2 Partition your disk(s):
Run this if you need legacy (BIOS) booting:
# sgdisk -a1 -n1:24K:+1000K -t1:EF02 /dev/disk/by-id/scsi-SATA_disk1
sgdisk -a1 -n1:24K:+1000K -t1:EF02 /dev/disk/by-id/scsi-SATA_disk1
Run this for UEFI booting (for use now or in the future):
# sgdisk -n2:1M:+512M -t2:EF00 /dev/disk/by-id/scsi-SATA_disk1
sgdisk -n2:1M:+512M -t2:EF00 /dev/disk/by-id/scsi-SATA_disk1
Run this for the boot pool:
# sgdisk -n3:0:+1G -t3:BF01 /dev/disk/by-id/scsi-SATA_disk1
sgdisk -n3:0:+1G -t3:BF01 /dev/disk/by-id/scsi-SATA_disk1
Choose one of the following options:
2.2a Unencrypted:
# sgdisk -n4:0:0 -t4:BF01 /dev/disk/by-id/scsi-SATA_disk1
sgdisk -n4:0:0 -t4:BF01 /dev/disk/by-id/scsi-SATA_disk1
2.2b LUKS:
# sgdisk -n4:0:0 -t4:8300 /dev/disk/by-id/scsi-SATA_disk1
sgdisk -n4:0:0 -t4:8300 /dev/disk/by-id/scsi-SATA_disk1
Always use the long `/dev/disk/by-id/*` aliases with ZFS. Using the `/dev/sd*` device nodes directly can cause sporadic import failures, especially on systems that have more than one storage pool.
@ -98,7 +103,7 @@ Always use the long `/dev/disk/by-id/*` aliases with ZFS. Using the `/dev/sd*`
2.3 Create the boot pool:
# zpool create -o ashift=12 -d \
zpool create -o ashift=12 -d \
-o feature@async_destroy=enabled \
-o feature@bookmarks=enabled \
-o feature@embedded_data=enabled \
@ -130,7 +135,7 @@ Choose one of the following options:
2.4a Unencrypted:
# zpool create -o ashift=12 \
zpool create -o ashift=12 \
-O acltype=posixacl -O canmount=off -O compression=lz4 \
-O dnodesize=auto -O normalization=formD -O relatime=on -O xattr=sa \
-O mountpoint=/ -R /mnt \
@ -138,10 +143,10 @@ Choose one of the following options:
2.4b LUKS:
# cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha256 \
cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha256 \
/dev/disk/by-id/scsi-SATA_disk1-part4
# cryptsetup luksOpen /dev/disk/by-id/scsi-SATA_disk1-part4 luks1
# zpool create -o ashift=12 \
cryptsetup luksOpen /dev/disk/by-id/scsi-SATA_disk1-part4 luks1
zpool create -o ashift=12 \
-O acltype=posixacl -O canmount=off -O compression=lz4 \
-O dnodesize=auto -O normalization=formD -O relatime=on -O xattr=sa \
-O mountpoint=/ -R /mnt \
@ -164,72 +169,84 @@ Choose one of the following options:
3.1 Create filesystem datasets to act as containers:
# zfs create -o canmount=off -o mountpoint=none rpool/ROOT
# zfs create -o canmount=off -o mountpoint=none bpool/BOOT
zfs create -o canmount=off -o mountpoint=none rpool/ROOT
zfs create -o canmount=off -o mountpoint=none bpool/BOOT
On Solaris systems, the root filesystem is cloned and the suffix is incremented for major system changes through `pkg image-update` or `beadm`. Similar functionality for APT is possible but currently unimplemented. Even without such a tool, it can still be used for manually created clones.
3.2 Create filesystem datasets for the root and boot filesystems:
# zfs create -o canmount=noauto -o mountpoint=/ rpool/ROOT/ubuntu
# zfs mount rpool/ROOT/ubuntu
zfs create -o canmount=noauto -o mountpoint=/ rpool/ROOT/ubuntu
zfs mount rpool/ROOT/ubuntu
# zfs create -o canmount=noauto -o mountpoint=/boot bpool/BOOT/ubuntu
# zfs mount bpool/BOOT/ubuntu
zfs create -o canmount=noauto -o mountpoint=/boot bpool/BOOT/ubuntu
zfs mount bpool/BOOT/ubuntu
With ZFS, it is not normally necessary to use a mount command (either `mount` or `zfs mount`). This situation is an exception because of `canmount=noauto`.
3.3 Create datasets:
# zfs create rpool/home
# zfs create -o mountpoint=/root rpool/home/root
# zfs create -o canmount=off rpool/var
# zfs create -o canmount=off rpool/var/lib
# zfs create rpool/var/log
# zfs create rpool/var/spool
zfs create rpool/home
zfs create -o mountpoint=/root rpool/home/root
zfs create -o canmount=off rpool/var
zfs create -o canmount=off rpool/var/lib
zfs create rpool/var/log
zfs create rpool/var/spool
The datasets below are optional, depending on your preferences and/or
software choices:
The datasets below are optional, depending on your preferences and/or software
choices.
If you wish to exclude these from snapshots:
# zfs create -o com.sun:auto-snapshot=false rpool/var/cache
# zfs create -o com.sun:auto-snapshot=false rpool/var/tmp
# chmod 1777 /mnt/var/tmp
zfs create -o com.sun:auto-snapshot=false rpool/var/cache
zfs create -o com.sun:auto-snapshot=false rpool/var/tmp
chmod 1777 /mnt/var/tmp
If you use /opt on this system:
# zfs create rpool/opt
zfs create rpool/opt
If you use /srv on this system:
# zfs create rpool/srv
zfs create rpool/srv
If you use /usr/local on this system:
# zfs create -o canmount=off rpool/usr
# zfs create rpool/usr/local
zfs create -o canmount=off rpool/usr
zfs create rpool/usr/local
If this system will have games installed:
# zfs create rpool/var/games
zfs create rpool/var/games
If this system will store local email in /var/mail:
# zfs create rpool/var/mail
zfs create rpool/var/mail
If this system will use Snap packages:
# zfs create rpool/var/snap
zfs create rpool/var/snap
If you use /var/www on this system:
# zfs create rpool/var/www
zfs create rpool/var/www
If this system will use GNOME:
# zfs create rpool/var/lib/AccountsService
zfs create rpool/var/lib/AccountsService
If this system will use Docker (which manages its own datasets & snapshots):
# zfs create -o com.sun:auto-snapshot=false rpool/var/lib/docker
zfs create -o com.sun:auto-snapshot=false rpool/var/lib/docker
If this system will use NFS (locking):
# zfs create -o com.sun:auto-snapshot=false rpool/var/lib/nfs
zfs create -o com.sun:auto-snapshot=false rpool/var/lib/nfs
A tmpfs is recommended later, but if you want a separate dataset for /tmp:
# zfs create -o com.sun:auto-snapshot=false rpool/tmp
# chmod 1777 /mnt/tmp
zfs create -o com.sun:auto-snapshot=false rpool/tmp
chmod 1777 /mnt/tmp
The primary goal of this dataset layout is to separate the OS from user data. This allows the root filesystem to be rolled back without rolling back user data such as logs (in `/var/log`). This will be especially important if/when a `beadm` or similar utility is integrated. The `com.sun.auto-snapshot` setting is used by some ZFS snapshot utilities to exclude transient data.
@ -237,8 +254,8 @@ If you do nothing extra, `/tmp` will be stored as part of the root filesystem. A
3.4 Install the minimal system:
# debootstrap bionic /mnt
# zfs set devices=off rpool
debootstrap bionic /mnt
zfs set devices=off rpool
The `debootstrap` command leaves the new system in an unconfigured state. An alternative to using `debootstrap` is to copy the entirety of a working system into the new ZFS root.
@ -246,9 +263,9 @@ The `debootstrap` command leaves the new system in an unconfigured state. An al
4.1 Configure the hostname (change `HOSTNAME` to the desired hostname).
# echo HOSTNAME > /mnt/etc/hostname
echo HOSTNAME > /mnt/etc/hostname
# vi /mnt/etc/hosts
vi /mnt/etc/hosts
Add a line:
127.0.1.1 HOSTNAME
or if the system has a real name in DNS:
@ -259,10 +276,12 @@ The `debootstrap` command leaves the new system in an unconfigured state. An al
4.2 Configure the network interface:
Find the interface name:
# ip addr show
ip addr show
Adjust NAME below to match your interface name:
# vi /mnt/etc/netplan/01-netcfg.yaml
vi /mnt/etc/netplan/01-netcfg.yaml
network:
version: 2
ethernets:
@ -273,7 +292,7 @@ Customize this file if the system is not a DHCP client.
4.3 Configure the package sources:
# vi /mnt/etc/apt/sources.list
vi /mnt/etc/apt/sources.list
deb http://archive.ubuntu.com/ubuntu bionic main universe
deb-src http://archive.ubuntu.com/ubuntu bionic main universe
@ -285,39 +304,40 @@ Customize this file if the system is not a DHCP client.
4.4 Bind the virtual filesystems from the LiveCD environment to the new system and `chroot` into it:
# mount --rbind /dev /mnt/dev
# mount --rbind /proc /mnt/proc
# mount --rbind /sys /mnt/sys
# chroot /mnt /bin/bash --login
mount --rbind /dev /mnt/dev
mount --rbind /proc /mnt/proc
mount --rbind /sys /mnt/sys
chroot /mnt /bin/bash --login
**Note:** This is using `--rbind`, not `--bind`.
4.5 Configure a basic system environment:
# ln -s /proc/self/mounts /etc/mtab
# apt update
ln -s /proc/self/mounts /etc/mtab
apt update
# dpkg-reconfigure locales
dpkg-reconfigure locales
Even if you prefer a non-English system language, always ensure that `en_US.UTF-8` is available.
# dpkg-reconfigure tzdata
dpkg-reconfigure tzdata
If you prefer nano over vi, install it:
# apt install --yes nano
apt install --yes nano
4.6 Install ZFS in the chroot environment for the new system:
# apt install --yes --no-install-recommends linux-image-generic
# apt install --yes zfs-initramfs
apt install --yes --no-install-recommends linux-image-generic
apt install --yes zfs-initramfs
**Hint:** For the HWE kernel, install `linux-image-generic-hwe-18.04` instead of `linux-image-generic`.
4.7 For LUKS installs only, setup crypttab:
# apt install --yes cryptsetup
apt install --yes cryptsetup
# echo luks1 UUID=$(blkid -s UUID -o value \
echo luks1 UUID=$(blkid -s UUID -o value \
/dev/disk/by-id/scsi-SATA_disk1-part4) none \
luks,discard,initramfs > /etc/crypttab
@ -331,20 +351,20 @@ Choose one of the following options:
4.8a Install GRUB for legacy (BIOS) booting
# apt install --yes grub-pc
apt install --yes grub-pc
Install GRUB to the disk(s), not the partition(s).
4.8b Install GRUB for UEFI booting
# apt install dosfstools
# mkdosfs -F 32 -s 1 -n EFI /dev/disk/by-id/scsi-SATA_disk1-part2
# mkdir /boot/efi
# echo PARTUUID=$(blkid -s PARTUUID -o value \
apt install dosfstools
mkdosfs -F 32 -s 1 -n EFI /dev/disk/by-id/scsi-SATA_disk1-part2
mkdir /boot/efi
echo PARTUUID=$(blkid -s PARTUUID -o value \
/dev/disk/by-id/scsi-SATA_disk1-part2) \
/boot/efi vfat nofail,x-systemd.device-timeout=1 0 1 >> /etc/fstab
# mount /boot/efi
# apt install --yes grub-efi-amd64-signed shim-signed
mount /boot/efi
apt install --yes grub-efi-amd64-signed shim-signed
* The `-s 1` for `mkdosfs` is only necessary for drives which present 4 KiB logical sectors (“4Kn” drives) to meet the minimum cluster size (given the partition size of 512 MiB) for FAT32. It also works fine on drives which present 512 B sectors.
@ -352,13 +372,13 @@ Install GRUB to the disk(s), not the partition(s).
4.9 Set a root password
# passwd
passwd
4.10 Enable importing bpool
This ensures that `bpool` is always imported, regardless of whether `/etc/zfs/zpool.cache` exists, whether it is in the cachefile or not, or whether `zfs-import-scan.service` is enabled.
```
# vi /etc/systemd/system/zfs-import-bpool.service
vi /etc/systemd/system/zfs-import-bpool.service
[Unit]
DefaultDependencies=no
Before=zfs-import-scan.service
@ -372,43 +392,41 @@ This ensures that `bpool` is always imported, regardless of whether `/etc/zfs/zp
[Install]
WantedBy=zfs-import.target
# systemctl enable zfs-import-bpool.service
systemctl enable zfs-import-bpool.service
```
4.11 Optional (but recommended): Mount a tmpfs to /tmp
If you chose to create a `/tmp` dataset above, skip this step, as they are mutually exclusive choices. Otherwise, you can put `/tmp` on a tmpfs (RAM filesystem) by enabling the `tmp.mount` unit.
# cp /usr/share/systemd/tmp.mount /etc/systemd/system/
# systemctl enable tmp.mount
cp /usr/share/systemd/tmp.mount /etc/systemd/system/
systemctl enable tmp.mount
4.12 Setup system groups:
# addgroup --system lpadmin
# addgroup --system sambashare
addgroup --system lpadmin
addgroup --system sambashare
## Step 5: GRUB Installation
5.1 Verify that the ZFS boot filesystem is recognized:
# grub-probe /boot
zfs
grub-probe /boot
5.2 Refresh the initrd files:
# update-initramfs -u -k all
update-initramfs: Generating /boot/initrd.img-4.15.0-46-generic
update-initramfs -u -k all
**Note:** When using LUKS, this will print "WARNING could not determine root device from /etc/fstab". This is because [cryptsetup does not support ZFS](https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1612906).
5.3 Workaround GRUB's missing zpool-features support:
# vi /etc/default/grub
vi /etc/default/grub
Set: GRUB_CMDLINE_LINUX="root=ZFS=rpool/ROOT/ubuntu"
5.4 Optional (but highly recommended): Make debugging GRUB easier:
# vi /etc/default/grub
vi /etc/default/grub
Comment out: GRUB_TIMEOUT_STYLE=hidden
Set: GRUB_TIMEOUT=5
Below GRUB_TIMEOUT, add: GRUB_RECORDFAIL_TIMEOUT=5
@ -420,11 +438,7 @@ Later, once the system has rebooted twice and you are sure everything is working
5.5 Update the boot configuration:
# update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.15.0-46-generic
Found initrd image: /boot/initrd.img-4.15.0-46-generic
done
update-grub
**Note:** Ignore errors from `osprober`, if present.
@ -432,22 +446,20 @@ Later, once the system has rebooted twice and you are sure everything is working
5.6a For legacy (BIOS) booting, install GRUB to the MBR:
# grub-install /dev/disk/by-id/scsi-SATA_disk1
Installing for i386-pc platform.
Installation finished. No error reported.
grub-install /dev/disk/by-id/scsi-SATA_disk1
Do not reboot the computer until you get exactly that result message. Note that you are installing GRUB to the whole disk, not a partition.
Note that you are installing GRUB to the whole disk, not a partition.
If you are creating a mirror or raidz topology, repeat the `grub-install` command for each disk in the pool.
5.6b For UEFI booting, install GRUB:
# grub-install --target=x86_64-efi --efi-directory=/boot/efi \
grub-install --target=x86_64-efi --efi-directory=/boot/efi \
--bootloader-id=ubuntu --recheck --no-floppy
5.7 Verify that the ZFS module is installed:
# ls /boot/grub/*/zfs.mod
ls /boot/grub/*/zfs.mod
5.8 Fix filesystem mount ordering
@ -459,62 +471,65 @@ Until there is support for mounting `/boot` in the initramfs, we also need to mo
For UEFI booting, unmount /boot/efi first:
# umount /boot/efi
umount /boot/efi
Everything else applies to both BIOS and UEFI booting:
# zfs set mountpoint=legacy bpool/BOOT/ubuntu
# echo bpool/BOOT/ubuntu /boot zfs \
zfs set mountpoint=legacy bpool/BOOT/ubuntu
echo bpool/BOOT/ubuntu /boot zfs \
nodev,relatime,x-systemd.requires=zfs-import-bpool.service 0 0 >> /etc/fstab
# zfs set mountpoint=legacy rpool/var/log
# echo rpool/var/log /var/log zfs nodev,relatime 0 0 >> /etc/fstab
zfs set mountpoint=legacy rpool/var/log
echo rpool/var/log /var/log zfs nodev,relatime 0 0 >> /etc/fstab
# zfs set mountpoint=legacy rpool/var/spool
# echo rpool/var/spool /var/spool zfs nodev,relatime 0 0 >> /etc/fstab
zfs set mountpoint=legacy rpool/var/spool
echo rpool/var/spool /var/spool zfs nodev,relatime 0 0 >> /etc/fstab
If you created a /var/tmp dataset:
# zfs set mountpoint=legacy rpool/var/tmp
# echo rpool/var/tmp /var/tmp zfs nodev,relatime 0 0 >> /etc/fstab
zfs set mountpoint=legacy rpool/var/tmp
echo rpool/var/tmp /var/tmp zfs nodev,relatime 0 0 >> /etc/fstab
If you created a /tmp dataset:
# zfs set mountpoint=legacy rpool/tmp
# echo rpool/tmp /tmp zfs nodev,relatime 0 0 >> /etc/fstab
zfs set mountpoint=legacy rpool/tmp
echo rpool/tmp /tmp zfs nodev,relatime 0 0 >> /etc/fstab
## Step 6: First Boot
6.1 Snapshot the initial installation:
# zfs snapshot bpool/BOOT/ubuntu@install
# zfs snapshot rpool/ROOT/ubuntu@install
zfs snapshot bpool/BOOT/ubuntu@install
zfs snapshot rpool/ROOT/ubuntu@install
In the future, you will likely want to take snapshots before each upgrade, and remove old snapshots (including this one) at some point to save space.
6.2 Exit from the `chroot` environment back to the LiveCD environment:
# exit
exit
6.3 Run these commands in the LiveCD environment to unmount all filesystems:
# mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
# zpool export -a
mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
zpool export -a
6.4 Reboot:
# reboot
reboot
6.5 Wait for the newly installed system to boot normally. Login as root.
6.6 Create a user account:
# zfs create rpool/home/YOURUSERNAME
# adduser YOURUSERNAME
# cp -a /etc/skel/. /home/YOURUSERNAME
# chown -R YOURUSERNAME:YOURUSERNAME /home/YOURUSERNAME
zfs create rpool/home/YOURUSERNAME
adduser YOURUSERNAME
cp -a /etc/skel/. /home/YOURUSERNAME
chown -R YOURUSERNAME:YOURUSERNAME /home/YOURUSERNAME
6.7 Add your user account to the default set of groups for an administrator:
# usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sambashare,sudo YOURUSERNAME
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sambashare,sudo YOURUSERNAME
6.8 Mirror GRUB
@ -522,21 +537,22 @@ If you installed to multiple disks, install GRUB on the additional disks:
6.8a For legacy (BIOS) booting:
# dpkg-reconfigure grub-pc
dpkg-reconfigure grub-pc
Hit enter until you get to the device selection screen.
Select (using the space bar) all of the disks (not partitions) in your pool.
6.8b UEFI
# umount /boot/efi
umount /boot/efi
For the second and subsequent disks (increment ubuntu-2 to -3, etc.):
# dd if=/dev/disk/by-id/scsi-SATA_disk1-part2 \
dd if=/dev/disk/by-id/scsi-SATA_disk1-part2 \
of=/dev/disk/by-id/scsi-SATA_disk2-part2
# efibootmgr -c -g -d /dev/disk/by-id/scsi-SATA_disk2 \
efibootmgr -c -g -d /dev/disk/by-id/scsi-SATA_disk2 \
-p 3 -L "ubuntu-2" -l '\EFI\ubuntu\grubx64.efi'
# mount /boot/efi
mount /boot/efi
## Step 7: (Optional) Configure Swap
@ -544,7 +560,7 @@ If you installed to multiple disks, install GRUB on the additional disks:
7.1 Create a volume dataset (zvol) for use as a swap device:
# zfs create -V 4G -b $(getconf PAGESIZE) -o compression=zle \
zfs create -V 4G -b $(getconf PAGESIZE) -o compression=zle \
-o logbias=throughput -o sync=always \
-o primarycache=metadata -o secondarycache=none \
-o com.sun:auto-snapshot=false rpool/swap
@ -557,21 +573,21 @@ The compression algorithm is set to `zle` because it is the cheapest available a
**Caution**: Always use long `/dev/zvol` aliases in configuration files. Never use a short `/dev/zdX` device name.
# mkswap -f /dev/zvol/rpool/swap
# echo /dev/zvol/rpool/swap none swap discard 0 0 >> /etc/fstab
# echo RESUME=none > /etc/initramfs-tools/conf.d/resume
mkswap -f /dev/zvol/rpool/swap
echo /dev/zvol/rpool/swap none swap discard 0 0 >> /etc/fstab
echo RESUME=none > /etc/initramfs-tools/conf.d/resume
The `RESUME=none` is necessary to disable resuming from hibernation. This does not work, as the zvol is not present (because the pool has not yet been imported) at the time the resume script runs. If it is not disabled, the boot process hangs for 30 seconds waiting for the swap zvol to appear.
7.3 Enable the swap device:
# swapon -av
swapon -av
## Step 8: Full Software Installation
8.1 Upgrade the minimal system:
# apt dist-upgrade --yes
apt dist-upgrade --yes
8.2 Install a regular set of software:
@ -579,15 +595,15 @@ Choose one of the following options:
8.2a Install a command-line environment only:
# apt install --yes ubuntu-standard
apt install --yes ubuntu-standard
8.2b Install a full GUI environment:
# apt install --yes ubuntu-desktop
apt install --yes ubuntu-desktop
**Hint**: If you are installing a full GUI environment, you will likely want to manage your network with NetworkManager:
# vi /etc/netplan/01-netcfg.yaml
vi /etc/netplan/01-netcfg.yaml
network:
version: 2
renderer: NetworkManager
@ -596,7 +612,7 @@ Choose one of the following options:
As `/var/log` is already compressed by ZFS, logrotates compression is going to burn CPU and disk I/O for (in most cases) very little gain. Also, if you are making snapshots of `/var/log`, logrotates compression will actually waste space, as the uncompressed data will live on in the snapshot. You can edit the files in `/etc/logrotate.d` by hand to comment out `compress`, or use this loop (copy-and-paste highly recommended):
# for file in /etc/logrotate.d/* ; do
for file in /etc/logrotate.d/* ; do
if grep -Eq "(^|[^#y])compress" "$file" ; then
sed -i -r "s/(^|[^#y])(compress)/\1#\2/" "$file"
fi
@ -604,7 +620,7 @@ As `/var/log` is already compressed by ZFS, logrotates compression is going t
8.4 Reboot:
# reboot
reboot
### Step 9: Final Cleanup
@ -612,30 +628,30 @@ As `/var/log` is already compressed by ZFS, logrotates compression is going t
9.2 Optional: Delete the snapshots of the initial installation:
$ sudo zfs destroy bpool/BOOT/ubuntu@install
$ sudo zfs destroy rpool/ROOT/ubuntu@install
sudo zfs destroy bpool/BOOT/ubuntu@install
sudo zfs destroy rpool/ROOT/ubuntu@install
9.3 Optional: Disable the root password
$ sudo usermod -p '*' root
sudo usermod -p '*' root
9.4 Optional: Re-enable the graphical boot process:
If you prefer the graphical boot process, you can re-enable it now. If you are using LUKS, it makes the prompt look nicer.
$ sudo vi /etc/default/grub
sudo vi /etc/default/grub
Uncomment: GRUB_TIMEOUT_STYLE=hidden
Add quiet and splash to: GRUB_CMDLINE_LINUX_DEFAULT
Comment out: GRUB_TERMINAL=console
Save and quit.
$ sudo update-grub
sudo update-grub
**Note:** Ignore errors from `osprober`, if present.
9.5 Optional: For LUKS installs only, backup the LUKS header:
$ sudo cryptsetup luksHeaderBackup /dev/disk/by-id/scsi-SATA_disk1-part4 \
sudo cryptsetup luksHeaderBackup /dev/disk/by-id/scsi-SATA_disk1-part4 \
--header-backup-file luks1-header.dat
Store that backup somewhere safe (e.g. cloud storage). It is protected by your LUKS passphrase, but you may wish to use additional encryption.
@ -650,34 +666,34 @@ Go through [Step 1: Prepare The Install Environment](#step-1-prepare-the-install
For LUKS, first unlock the disk(s):
# cryptsetup luksOpen /dev/disk/by-id/scsi-SATA_disk1-part4 luks1
cryptsetup luksOpen /dev/disk/by-id/scsi-SATA_disk1-part4 luks1
Repeat for additional disks, if this is a mirror or raidz topology.
Mount everything correctly:
# zpool export -a
# zpool import -N -R /mnt rpool
# zpool import -N -R /mnt bpool
# zfs mount rpool/ROOT/ubuntu
# zfs mount -a
zpool export -a
zpool import -N -R /mnt rpool
zpool import -N -R /mnt bpool
zfs mount rpool/ROOT/ubuntu
zfs mount -a
If needed, you can chroot into your installed environment:
# mount --rbind /dev /mnt/dev
# mount --rbind /proc /mnt/proc
# mount --rbind /sys /mnt/sys
# chroot /mnt /bin/bash --login
# mount /boot
# mount -a
mount --rbind /dev /mnt/dev
mount --rbind /proc /mnt/proc
mount --rbind /sys /mnt/sys
chroot /mnt /bin/bash --login
mount /boot
mount -a
Do whatever you need to do to fix your system.
When done, cleanup:
# exit
# mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
# zpool export -a
# reboot
exit
mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
zpool export -a
reboot
### MPT2SAS
@ -703,11 +719,13 @@ Set a unique serial number on each virtual disk using libvirt or qemu (e.g. `-dr
To be able to use UEFI in guests (instead of only BIOS booting), run this on the host:
$ sudo apt install ovmf
$ sudo vi /etc/libvirt/qemu.conf
sudo apt install ovmf
sudo vi /etc/libvirt/qemu.conf
Uncomment these lines:
nvram = [
"/usr/share/OVMF/OVMF_CODE.fd:/usr/share/OVMF/OVMF_VARS.fd",
"/usr/share/AAVMF/AAVMF_CODE.fd:/usr/share/AAVMF/AAVMF_VARS.fd"
]
$ sudo service libvirt-bin restart
sudo service libvirt-bin restart