Fat

FAT16, FAT32, NTFS, exFAT - USB Micro SD Flash Drives

If you're reading/writing a single file, NTFS seems to win hands down. But in real life situations where multiple files being read and write, then it's another story, NTFS was more than half a minute slower when writing just 50 MB of multiple files and directories. For some reason, the reading times from Disk Bench seem to be pretty much the same between the file systems, I'm not sure if they're accurate, but many other test results on the Internet (e.g. Irongeek.com, AnandTech.com) have shown that FAT16 to be the quickest as well, although the difference becomes less significant for bigger files. If you have done some tests with your UFD, please feel free to post them. Flash drive fragmentation can have a serious impact on flash drive performance, especially during write operations, which can take several milliseconds per data block. It might not be so important for a USB drive that is used to transfer a couple of files from one computer to the next, but if an SSD is used as a hard disk replacement, it can certainly be relevant. A standard hard disk defragmentation tool might not be the ideal tool to defragment a flash drive, since it is not aware of the allocation block size inside the flash drive. This may slow down the defragmentation procedure.







FAT16, FAT32, NTFS, or exFAT on USB Flash Drives?



If you don't feel like reading this boring guide and your thumb drive or partition is 2 GB* or smaller, then stick with the default FAT16 for best performance and cross-platform compatibility. There is a reason why most UFDs in these sizes, including my 4 GB stick, come pre-formatted with FAT16.
*Windows NT 4.0, 2000, XP, and Vista can support FAT16 up to 4 GB using 64K cluster size. However, it may create compatibility issues with some applications. But for storage purposes, it shouldn't cause any problems.



Question:
I just bought a USB thumb drive, which format should I use?

This question has been asked many times on NBR, many people are not sure about which one to use or suits their needs. In this guide, I will help you to understand the benefits and drawbacks for each of them.
Tools used for this guide:
FAT16 (a.k.a. FAT)
Pros:
Cons:
FAT32
Pros:
Cons:
NTFS
Pros:
Cons:
exFAT (a.k.a. FAT64)
Pros:
Cons:
Some test results with Nodesoft Disk Bench
Multiple tests were done for better accuracy, they were all done with the default allocation size, optimize for performance enabled, and antivirus disabled.

1 MB file | Read (MB/s) | Write (MB/s)
FAT16 - 32.393 - 2.063
FAT32 - 32.393 - 1.339
NTFS - 32.393 - 2.797
exFAT - 32.393 - 1.464
10 MB file | Read (MB/s) | Write (MB/s)
FAT16 - 129.334 - 4.645
FAT32 - 129.334 - 3.943
NTFS - 129.334 - 29.326
exFAT - 129.334 - 4.703
100 MB file | Read (MB/s) | Write (MB/s)
FAT16 - 306.212 - 5.106
FAT32 - 306.212 - 5.065
NTFS - 321.915 - 4.952
exFAT - 379.010 - 5.188
Writing 50 MB of 712 files and 95 folders
FAT16 - 1 min 12 sec
FAT32 - 1 min 19 sec
NTFS - 1 min 50 sec
exFAT - 1 min 55 sec

Conclusions
If you're reading/writing a single file, NTFS seems to win hands down. But in real life situations where multiple files being read and write, then it's another story, NTFS was more than half a minute slower when writing just 50 MB of multiple files and directories. For some reason, the reading times from Disk Bench seem to be pretty much the same between the file systems, I'm not sure if they're accurate, but many other test results on the Internet (e.g. Irongeek.com, AnandTech.com) have shown that FAT16 to be the quickest as well, although the difference becomes less significant for bigger files. If you have done some tests with your UFD, please feel free to post them.




Flash drive fragmentation and performance




Is it necessary to defragment flash storage devices, such as a USB flash drives, SD cards, and newer SSD hard-disk replacements? Does it get faster if you do? There are arguments for and against defragmenting. On hard disk media, reading or writing fragmented files involves the overhead of the read heads having to jump from one place on the platter to the other, which can take as much as 8 milliseconds on a typical hard disk drive. However, hard disks are very fast when it comes to reading sequential data, which is on the order of 100 MB per second on a 3.5 inch consumer-grade hard disk. Obviously, allocating file data sequentially on a hard disk will save time on accessing the file.




Since a flash drive does not have moving parts, its access time is independent on where the data is stored, which is why supposedly flash drives don't need defragmenting. A quick Google search gives us plenty of pages where people suggest that there should be no difference in performance, while defragmentation only increases wear of the flash drive since flash memory can be rewritten only about 10 to 100 thousand times. Wear is usually not a big issue since most flash drives will use wear leveling to spread out the write operations, though. I found two pages where some empirical data is discussed, rather than theoretical considerations. One suggests that there is a small difference, which can be due to a flash device that contains both slow and fast flash memory, and due to I/O operations being needed for accessing non-contiguous data. [Yegulalp 2006] Another web page claims that there is a clear difference upon writing fragmented files. [Diskeeper 2007].

What these sources do not seem to realize is that reading flash memory can be done by random access, or on a sector (or page) level (512 to 2048 bytes at a time), but writing is done in much larger blocks of 16 to 256 kilobytes.[Wikipedia 2008]. Writing a single 512-byte sector of data can involve reading a full 128 kB block, erasing it, and then rewriting the original data with only 512 bytes changed.
I did a test on a Kingston 512 MB SD-card I bought in 2006 for my digital camera and tested it from a computer with SD card reader, running Linux. I created a highly fragmented file by reformatting the flash card (in FAT-32 format, 4 kB cluster size), and then sequentially writing thousands of files with sizes of 124 and 4 kB, interleaved, until the drive was full. Then I deleted all the small 4 kB files and replaced them by a single 15 MB file, that was spread out over about 4000 little fragments. I measured the time it took to write all this data, carefully flushing the disk cache every time with a sync command. Here are the results:
Description
Data size
Time
Transfer speed
Initial write
486 MB
291 s
1.67 MB/s
Fragmented write
15.2 MB
492.6 s
0.031 MB/s
That's pretty shocking: a factor 50 decrease in speed due to fragmentation! Apparently, a single write operation takes up to 120 milliseconds, even if it is only 4 kB of data.
After unmounting and remounting the flash drive (to get rid of data stored in the cache), I tested the read speeds:
Description
Data size
Time
Transfer speed
Contiguous read
12.1 MB
1.7 s
7.2 MB/s
Fragmented read
15.2 MB
4.6 s
3.3 MB/s
Clearly, there is some extra overhead for read operations on a fragmented file, but much, much less than for writing data.




Conclusions

Fragmentation can have a serious impact on flash drive performance, especially during write operations, which can take several milliseconds per data block. It might not be so important for a USB drive that is used to transfer a couple of files from one computer to the next, but if an SSD is used as a hard disk replacement, it can certainly be relevant. A standard hard disk defragmentation tool might not be the ideal tool to defragment a flash drive, since it is not aware of the allocation block size inside the flash drive. This may slow down the defragmentation procedure.

I will repeat the experiment later with other types of flash memory, and try to figure out the internal block size which is not specified anywhere.

Marcadores: , , , ,


# 11/05/2009 09:29:00 AM, Comentários, Links para esta postagem,