Bootable Linux USB stick

We regularly and successfully use USB sticks for courses where participants bring their own notebooks. The big benefit is that students learn Linux, and realize that they can easily use the hardware they already own. Current notebook hardware is by far fast enough for data processing, structure solution and coot visualization. What we found:

1. the sticks should be fast USB3 (very good results with SANdisk Extreme 32GB; physically small sticks like Kingston DataTraveler look nicer but are slow). The computers do not have to be recent, nor do they have to have USB3 ports. USB2 ports support up to 30MB/s, but only USB3 sticks deliver this! With a bit of tuning (below) the stick feels as fast as a local harddisk.
2. we use Fedora 23 because its hardware support is very good. We always use the 64bit distro.
3. The stick can be booted on MacBooks as well (press the alt key at the boot sound); their hardware works well with Fedora 23. For Windows clients (press F11 or F12 or sometimes F9 or F10 for the boot menu; if that does not work press F2 or DEL for the BIOS menu and change the boot order), one has to make sure that "fast boot" (or "fast startup") is disabled (or Shift is pressed while shutting Windows down), and sometimes powercfg -H off (as Administrator in a console window) is additionally required; otherwise the USB stick may not boot. Occasionally we find a computer that does not boot from the stick because the BIOS screen can not be reached (due to unknown BIOS password; happens with machines belonging to institutions which administer them centrally) or some such, but 19 out of 20 work as they should.
4. we just install CCP4 and whatever else we need (XDS, Phenix, Chimera, ..), and then dd or ddrescue (on a machine with USB3 ports) an image of that stick to all other sticks.
5. any number of bells and whistles could be added to this, like clients sending their hostnames to a server after booting, and accepting updates by rsync.

To make students familiar with the sticks and how to boot them, one needs 30+ minutes and a few tutors.

Some more details

1. we always create a few-GB FAT32 partition because that makes file exchange with Windows and Macs very simple. The FAT32 partition should be the first partition on the stick; 2GB is enough for us. Then comes a biosboot partition (1MB; required for booting bios system from a GPT disk), an efi partition (e.g. 250MB; required for UEFI/EFI boot on PC's/MAC's) and the Linux partition, with 13.9GB, so that the sum of the four partitions is slightly below 16GB. The third partition is then a 16.1GB /data partition. This scheme has the advantage that the image of the stick can just as well be copied (with dd or better ddrescue) to a 16GB stick; the /data partition then does not fit and cannot be used on that stick, but the operating system will then work on the small stick just as well. This requires that the /data partition is not fsck'ed automatically from /etc/fstab (0 in the 6th field).
2. it is a good idea to give easy root access to the one user you create because certainly some packages will have to be installed or updated when the stick is in use
3. it is a good idea to save an image of the stick whenever you made a successful change; otherwise you might need to start from scratch if you mess something up.
4. (if the installation not already does it for you) you should label the partitions and use the labels for mounting the partitions in /etc/fstab, alternatively use UUID's; don't use /dev/sda1 or the like because depending on the hardware it is booted on, after booting the stick, and depending on the actual hardware, the stick may be /dev/sdb or /dev/sdc or ... !

How to create a bootabled GPT-partitioned USB-stick with Fedora 23

This will create a USB stick capable to

• EFI boot on Macs
• UEFI boot on PCs
• BIOS boot on newer hardware (legacy boot / CSM enabled in BIOS)
• BIOS boot on medium aged hardware
• not boot on really old BIOS hardware which doesn't support booting from GPT (see below)

GPT partition the stick using gdisk

The example shows a 32 GB Sandisk Extreme:

Part 1: +1G VFAT            (0700) (Windows needs it to be the first partition) 
Part 2: +1M BIOSBOOT        (ef02)
Part 3: +250M EFI           (ef00)
Part 4: +13500M /           (0083)
Part 5: +15000M /mnt/data   (0083)

For performance, make sure that the big partitions are aligned to 8192-sector boundaries; gdisk default is 2048-sector boundaries - the default can be adjusted. If you use the above sizes and the + sign for specifying them, this happens to work out automatically.

install Fedora23 on an UEFI machine (UEFI enabled ; LegacyBoot / CSM disabled)

Whenever you see a ?, you must fill in the appropriate drive letter (e.g. c or d).

sd?3: efi	        /boot/efi
sd?4: ext4             / 
sd?5: ext4             /mnt/data

adjust the USB-stick for UEFI/EFI boot (before reboot from chroot environment)

chroot to the installed system:

chroot /mnt/sysimage

install Fedora updates:

dnf -y update    # dnf on FC23 is successor to yum

configure grub2 for (U)EFI systems:

• disable auto recognition of other installed Operating systems (specific to current computer), and
• update grub2-efi.cfg

echo 'GRUB_DISABLE_OS_PROBER=”true”' >> /etc/default/grub
grub2-mkconfig -o /etc/grub2-efi.cfg

(last step automatically puts UUID's to recognise the boot device in grub2-efi.cfg ; default after fresh Fedora installation is the device name which might be different on another computer)

Performance tuning (not strictly required):

We use ext4 filesystems and data=writeback,nobarrier in /etc/fstab. To be able to set these options also on the / filesystem, we use tune2fs to set both as default mount options on a Linux machine where the stick (dev/sd?) is just plugged in (i.e. not booted from):

tune2fs -o journal_data_writeback,nobarrier /dev/sd?4
tune2fs -o journal_data_writeback,nobarrier /dev/sd?5

create & label vfat filesystem:

mkfs.vfat /dev/sd?1
dosfslabel /dev/sd?1 VFAT

shutdown or reboot:

exit
systemctl poweroff (or reboot for testing)

Install grub2 for BIOS boot (from chroot environment)

boot Fedora Live DVD on a BIOS machine and chroot to the stick:

mount /dev/sd?4 /mnt
mount -o bind /dev /mnt/dev
mount -t proc /proc /mnt/proc
mount -t sysfs /sys /mnt/sys
chroot /mnt

install grub2 for BIOS boot and configure it:

rm /boot/grub2/grubenv
grub2-install /dev/sd?
grub2-mkconfig -o /etc/grub2.cfg

(on UEFI systems /boot/grub2/grubenv is a symlink to /boot/efi/EFI/fedora/grubenv (from package grub2-efi.rpm). The symlink has to be removed before grub2-install can finish successfully, it automatically creates a new file /boot/grub2/grubenv (as in package grub2.rpm)

check /etc/grub2.cfg and if needed, change at all positions: “linuxefi” to “linux16”, and “intrdefi” to “intrd16”

Problems on old BIOS hardware

Some old computers still might not boot since they don't support booting from GPT. This can be fixed to boot on such old hardware, however the stick afterwards cannot boot on EFI/UEFI systems anymore:

• set the boot flag (active) on the single 0xEE partition in the protective MBR by using fdisk version <= 2.22 (versions 1.23 and newer have GPT support and thus don't make changes to the protective MBR but to the GPT) (I used fdisk (util-linux-ng 2.17.2) on a ScientificLinux 6.7 machine)

fdisk /dev/sd? 
a
part 1
w

• recompute CHS values using option “h” in gdisk's expert menu:

gdisk /dev/sd?
x
h
m
q

this can be reversed: - toggle bootflag off in old fdisk - recompute CHS values in gdisk

(using a sgdisk backup of the GPT should also be fine)

helpful links:

http://www.rodsbooks.com/gdisk/bios.html#bios --> option h of gdisks experts menu did the trick!

https://fedoraproject.org/wiki/GRUB_2?rd=Grub2#Updating_GRUB_2_configuration_on_BIOS_systems

generating images and copies of the stick

Save a compressed disk image - we use the parallel gzip program called pigz:

dd if=/dev/sd? bs=4096 | pigz -c usbstick.img.gz

(time: ~180 secs speed: ~175MB/s, size of image: 1.8 GB) write compressed image back to a stick:

unpigz -c usbstick.img.gz | dd of=/dev/sd? bs=4096

(speed should be >100MB/s)

for techies: taking care of the stick

The following is about performance and durability of the USB stick, and reading it is not required for the functionality described above. If you don't know what TRIM is and how it relates to flash media, this section is probably not for you.

To fill empty part of all partitions on the stick with “zeros”: for each partition, do (as root)

dd if=/dev/zero of=/mountpoint_of_partition/delete.me bs=10M  # this will stop when filesystem is full
rm /mountpoint_of_partition/delete.me

This is sensible to do after deleting large amounts of data from the USB stick, and before saving and compressing an image of it. It also seems to result in faster writes afterwards i.e. it may be a way to restore the write speed of the stick.

hdparm's (ENHANCED) SECURITY ERASE initializes the whole stick in 12.5 seconds - I checked the result.

All other methods, like the fstrim command and the discard mount option do not work for USB sticks, presumably because the usb-storage kernel module does not pass the ATA trim command through the USB bridge and controller to the device. The same goes for blkdiscard. The wiper.sh script (which is part of hdparm) gives no error message, but does not reproducibly fill the empty space of the filesystem with zeroes (it sometimes does).