1,330
edits
Bootable Linux USB stick: Difference between revisions
Jump to navigation
Jump to search
→for techies: taking care of the stick
| Line 144: | Line 144: | ||
For creation/maintainance a bunch of sticks, a multiple port USB-HUB is a very usefull tool. E.g. the Raid Sonic Icy Box IB-AC6113 accepts 12 Sandisk Extreme sticks at once (port 7 has to stay empty due to spatial restrictions) and all 12 stick are recognized by the OS (CentOS7). | For creation/maintainance a bunch of sticks, a multiple port USB-HUB is a very usefull tool. E.g. the Raid Sonic Icy Box IB-AC6113 accepts 12 Sandisk Extreme sticks at once (port 7 has to stay empty due to spatial restrictions) and all 12 stick are recognized by the OS (CentOS7). | ||
== | == USB stick: hardware considerations == | ||
Cheap USB sticks can hold a lot of data, but when it comes to writing small files (which happens when used as the operating system disk), they are either slow from the beginning, or their write performance quickly degrades as soon as free space becomes low for the first time. We have been using [https://crystalmark.info/software/CrystalDiskMark/index-e.html CrystalDiskMark] (Windows) for characterizing USB media: as a rule of thumb, the "Random write 4k blocks" disciplines should not show less than 1 MB/s for the media to be of acceptable performance. However, CrystalDiskMark does not capture the effect of degrading write performance, which can only be mitigated by TRIMming the media - a feature that few USB sticks support. | |||
The following is about performance and durability of the USB media. If you don't know what TRIM is and how it relates to flash media, read [https://en.wikipedia.org/wiki/Trim_(computing) this]. | |||
=== types of USB media === | |||
# very good: USB sticks that support TRIM. The ones known to us are: Sandisk Extreme USB sticks bought before 2017 (e.g. the 32GB model for ~22€ or 64GB for ~40€; unfortunately, these are no longer available), and some high-quality Mushkin models. These sticks are very fast (CrystalDiskMark scores >10MB/s for random 4K writes). Same holds for current (2018) Sandisk Extreme Pro USB sticks (only 128 and 256 GB models exist). | |||
# good: USB3 adapter with (micro)SD card (separate items). USB3 adapters are cheap (around 5€ at Amazon, like https://www.amazon.de/EX1-Kartenleser-Micro-Karte-Adapter/dp/B06XX3T219; cheaper at Ebay). microSD cards usually come with a SD adapter, and can be TRIMmed in SD slots from time to time. microSD cards ideally should be in the [https://en.wikipedia.org/wiki/Secure_Digital#Speed_class_rating Class 1 (A1) application performance class] since this is what is defined as requirement for use as extended operating system disk in Android smartphones. In practice, non-A1 microSD cards also seem to work well, but may require more often TRIMming. | |||
# acceptable: USB sticks that do not support TRIM but are fast and high quality, e.g. Sandisk Extreme Go or Sandisk Ultra bought 2017 and later. However, these may degrade with time. | |||
# inacceptable: e.g. Sandisk Ultra Fit become hot, degrade fast and are unreliable. For cheap sticks to be useful, one must be lucky. | |||
=== fill empty space of all partitions with “zeroes” === | |||
This could be done after deleting large amounts of data from the USB stick, and before saving and compressing an image of it, or before re-using the media with a fresh install. | |||
USB sticks that support “Deterministic read data after TRIM” according to <code>hdparm</code>, and microSD cards (which always have zeroes in free space after TRIM in a SD slot) should be TRIMmed because this does not wear out the storage cells. | |||
Other media: for each partition, do (as root) | |||
dd if=/dev/zero of=/mountpoint_of_partition/delete.me bs=10M # this will stop when filesystem is full | dd if=/dev/zero of=/mountpoint_of_partition/delete.me bs=10M # this will stop when filesystem is full | ||
rm /mountpoint_of_partition/delete.me | rm /mountpoint_of_partition/delete.me | ||
However, this stresses the flash cells; they sustain only a limited number of write cycles. | |||
=== initializing the media === | |||
Low level formatting can restore the write performance. | |||
==== | ==== "very good" USB sticks ==== | ||
<code>hdparm</code>'s (ENHANCED) SECURITY ERASE initializes the whole stick in a few seconds, and restores it to an almost factory-fresh condition, including zeroing the device. This works on all sticks where <code>hdparm -I /dev/sdX</code> reports "supported: enhanced erase" but if used wrongly (e.g. on the wrong device, or with the wrong options, or ...) it may brick your device, or delete valuable data! So use at your own risk: | <code>hdparm</code>'s (ENHANCED) SECURITY ERASE initializes the whole stick in a few seconds, and restores it to an almost factory-fresh condition, including zeroing the device. This works on all sticks where <code>hdparm -I /dev/sdX</code> reports "supported: enhanced erase" but if used wrongly (e.g. on the wrong device, or with the wrong options, or ...) it may brick your device, or delete valuable data! So use at your own risk: | ||
* check if <code>hdparm -I /dev/sdX</code> reports "supported: enhanced erase" and "not frozen" (to unfreeze a frozen diks, suspend your system with <code>pm-suspend</code> and wake it up again) | * check if <code>hdparm -I /dev/sdX</code> reports "supported: enhanced erase" and "not frozen" (to unfreeze a frozen diks, suspend your system with <code>pm-suspend</code> and wake it up again) | ||
| Line 163: | Line 177: | ||
hdparm --user-master u --security-unlock Eins /dev/sdX</code> | hdparm --user-master u --security-unlock Eins /dev/sdX</code> | ||
==== | ==== (micro)SD cards ==== | ||
TRIMming informs the | In a SD slot, <code>blkdiscard /dev/mmcXXXXXXX</code> initializes the card. | ||
==== other USB sticks ==== | |||
One may try to write zeroes on the whole stick | |||
dd if=/dev/zero of=/dev/sdX bs=8192k | |||
hoping that the firmware will understand this as a request for low-level formatting. This seems to work with some sticks, but takes a long time and may stress the flash cells. | |||
=== TRIM === | |||
==== USB sticks ==== | |||
TRIMming informs the firmware about blocks of the filesystem that do not contain file data. This is available for USB sticks for which <code>hdparm -I /dev/sdX</code> returns "Data Set Management TRIM supported"; if that command also returns "Deterministic read data after TRIM" then zeros are returned upon reads of TRIMmed blocks. | |||
The <code>wiper.sh</code> script (part of the <code>hdparm</code> source distribution) TRIMs ext3/ext4/xfs filesystems, but does not reproducibly seem to work for the SanDisk Extreme 32GB stick if the filesystem is mounted. This is probably due to the fact that this stick has a limitation of max 65536 blocks in one TRIM command (https://sourceforge.net/p/hdparm/bugs/63/). Since <code>wiper.sh</code> also works for unmounted filesystems, and the mapping of unused space is then obtained with a different tool, one should try with an unmounted filesystem as well - this worked for us in all cases tried so far (tested with [http://lightrush.ndoytchev.com/random-1/checkiftrimonext4isenabledandworking this script]). | |||
All other methods, like the <code>fstrim</code> command and the <code>discard</code> mount option '''do not work for USB sticks''', because the usb-storage kernel module does not pass the ATA TRIM command through the USB bridge and controller to the device. The same goes for [http://unix.stackexchange.com/questions/97143/utility-to-trim-unallocated-space-on-drive <code>blkdiscard</code>]. | |||
==== (micro)SD cards ==== | |||
These support TRIM when in a SD slot, but not when inserted in a USB adapter. When in a SD slot, the <code>lsblk -D</code> command reveals that the card can be TRIMmed, and <code>fstrim</code> works. | |||
== installation of crystallographic software == | == installation of crystallographic software == | ||