When we delete a file from a filesystem, the data is not physically removed: the operating system simply marks the area previously occupied by the file, as free and makes it available to store new information. The only way to make sure data is actually removed from a device is to override it with other data. We may want to perform such operation for privacy reasons (maybe we plan to sell the device and we want to be sure the new owner cannot access our data), or maybe to prepare a device for encryption. In this tutorial we will see some tools we can use to completely wipe data on a device
In this tutorial you will learn:
- How to shred data using dd
- How to secure-erase files and devices using the shred utility
- How to overwrite data using badblocks
Software Requirements and Conventions Used
| Category | Requirements, Conventions or Software Version Used |
|---|---|
| System | Distribution-independent |
| Software | Dd, shred or badblocks |
| Other |
|
| Conventions | # – requires given linux commands to be executed with root privileges either directly as a root user or by use of sudo command$ – requires given linux commands to be executed as a regular non-privileged user |
Erasing data using dd
Dd is a very powerful program included by default in all the major Linux distributions. In a previous article we saw how to use dd in detail; in this case, all we want to do is to override the content of our hypothetical block device with zeros or random data. In both cases, we can use data generated by “special” files: /dev/zero and dev/urandom (or /dev/random) respectively. The former returns zeroes each time a read operation is performed on it; the latter returns random bytes using the Linux kernel random number generator.
To fill the disk with zeroes we can run:
$ sudo dd if=/dev/zero of=/dev/sdx
To use random data, instead:
$ sudo dd if=/dev/urandom of=/dev/sdx
Using a LUKS container as random data generator
Overriding a device with random data is a time consuming operation, but can be useful especially if we plan to use full disk encryption, in order to make the used and unused part of the disks indistinguishable. To speedup the process we can use a little “trick”: we can create a LUKS(Linux Unified Key Setup) container on the device or the partition we want to fill with random data, and write zeros to it. Thanks to encryption, the data will be transparently written on the underlying device as random.
First of all we create the LUKS container:
$ sudo cryptsetup luksFormat /dev/sdx WARNING! ======== This will overwrite data on /dev/sdx irrevocably. Are you sure? (Type uppercase yes): YES Enter passphrase for /dev/sdx: Verify passphrase:
In this case is not really necessary to use a strong password, since we are using the container as a random data generator, and we will wipe it out after the operation is complete. Once the container is ready, we open it by running the following command:
$ sudo cryptsetup luksOpen /dev/sdx crypted Enter passphrase for /dev/sdx:
Now that the container is opened we can use dd and fill it with zeros. Very important: we write to the LUKS container mapped as /dev/mapper/crypted, not on the underlying /dev/sdx device directly:
$ sudo dd if=/dev/zero of=/dev/mapper/crypted bs=1M
Once all the data has been written, we close the container, and override the luks header with random data. The size of the header depends on the format of LUKS in use: it is 2MiB for the legacy LUKS format, and 16MiB for the LUKS2 format, which has become the default in recent versions of cryptsetup. Just to be sure, we can override the first 20MiB of the disk:
$ sudo cryptsetup luksClose /dev/mapper/crypted $ sudo dd if=/dev/urandom of=/dev/sdx bs=1M count=20
Erasing data using shred
The name of this utility its pretty self explanatory: its main goal, as stated in the manual, is to overwrite files and optionally delete it. The shred utility relies on the assumption that the filesystem overwrites data in place. The application may not let us achieve the expected result, for example, on journaled filesystems, like ext4 (probably the most used Linux filesystem), if it is mounted with the data=journal option.
When mounting an ext4 filesystem, with the data=ordered or data=writeback options (the former is the default), the data is written to the main filesystem after the metadata is committed to the journal. In both cases, shred works fine, producing the expected results.
When using the data=journal option, instead, not only the metadata, but the data itself is written to the filesystem journal, before being written to the main filesystem. It’s easy to see why this can cause problems.
Let’s see some examples of the application usage. Suppose we want to secure-delete a file named “test”. All we have to do is to run the following command (here we use the -v option to make the program more verbose):
$ shred -v test shred: test: pass 1/3 (random)... shred: test: pass 2/3 (random)... shred: test: pass 3/3 (random)...
By default the application overrides the specified file 3 times with random data. The number of passes can be changed using the -n (short for --iterations) option. To override the file 6 times we would run:
shred -v -n 6 test shred: test: pass 1/6 (random)... shred: test: pass 2/6 (000000)... shred: test: pass 3/6 (555555)... shred: test: pass 4/6 (ffffff)... shred: test: pass 5/6 (aaaaaa)... shred: test: pass 6/6 (random)...
There are some cases in which we may want to hide the fact that a shredding operation was performed on a file or device. In those situations we can use the program -z (short for --zero) option to make the program perform an additional pass with zeroes after the shredding:
$ shred -v -n 6 -z test shred: test: pass 1/7 (random)... shred: test: pass 2/7 (ffffff)... shred: test: pass 3/7 (aaaaaa)... shred: test: pass 4/7 (555555)... shred: test: pass 5/7 (000000)... shred: test: pass 6/7 (random)... shred: test: pass 7/7 (000000)...
From the verbose output of the command, we can indeed notice how the last pass is performed by writing zeroes (000000). We can verify it by running the hexdump program on the file:
$ hexdump test 0000000 0000 0000 0000 0000 0000 0000 0000 0000 * 0008000
Deleting the file
If we take a look at the filesystem after running one of the commands in examples above, we can notice that although overwritten with random data, the file itself has not been deleted: this happens because the command can also be used on files that represent entire block devices or partitions (e.g /dev/sda), and those should not be deleted.
When operating on common files, however, we may want to also deallocate a file from the filesystem after overriding it. To achieve this behavior, we can use the -u or the --remove options. Both options cause a file to be deleted, but with the latter we can also specify how the deletion should be performed. We can hoose between:
- unlink: the file is removed by using a standard
unlinksystem call; - wipe: bytes in the file name are obfuscated before the deletion;
- wipesync: the obfuscated bytes are also synced to the disk;
The wipesync mode is the default.
Erasing data using badblocks
Although the badblocks utility main goal is to search for bad blocks, by using a write-mode disruptive test we can effectively overwrite and secure-erase existing data on a device. All we have to do is to launch the command and specify the -w option: the test will be performed by first writing and then reading the 0xaa, 0x55, 0xff and 0x00 data patterns on every block and compare the contents.
We can use the -s and -v options, to respectively make the program display progress information and the number of read and write errors encountered. To wipe our device we would therefore run:
$ sudo badblocks -wsv /dev/sdx Checking for bad blocks in read-write mode From block 0 to 3870719 Testing with pattern 0xaa: ^C6.30% done, 0:41 elapsed. (0/0/0 errors)
To run the command above the device should be unmounted, otherwise badblocks will refuse to run unless the operation is forced with the -f option. The default number of blocks tested at a time is 64; we can, however, change this parameter using the -c option.
Conclusions
In this article we saw three utilities we can use to shred data on a device, and some examples of their usage. Dd and shred are part of the GNU core utils, so they are almost surely already installed on your system. Badblocks is a software used to test for the existence of bad blocks: when performing a read-write test with it, we can override data on a device. Please notice that the effectiveness of the data shredding depends also on the type of the device in use: solid state drives, for example, have to deal with phenomenons like write amplification.
