Directory performance with a large number of files

From Glitchdata
Jump to navigation Jump to search

How do file directories manage with a high number of files.


Linux Directory Performance

  • Anything > 10000 files in a single directory will result in performance degradation.

ext2 Directory Performance

Maximum number of files: 10^18 Maximum number of files per directory: ~1.3 × 1020 (performance issues past 10,000) Maximum file size 16 GiB (block size of 1 KiB) 256 GiB (block size of 2 KiB) 2 TiB (block size of 4 KiB) 2 TiB (block size of 8 KiB) Maximum volume size 4 TiB (block size of 1 KiB) 8 TiB (block size of 2 KiB) 16 TiB (block size of 4 KiB) 32 TiB (block size of 8 KiB)

ext3 Directory Performance

Maximum number of files: min(volumeSize / 213, numberOfBlocks) Maximum file size: same as ext2 Maximum volume size: same as ext2

ext4 Directory Performance

Maximum number of files: 232 - 1 (4,294,967,295) Maximum number of files per directory: unlimited Maximum file size: 244 - 1 bytes (16 TiB - 1) Maximum volume size: 248 - 1 bytes (256 TiB - 1)


NTFS Directory Performance

Maximum number of files: 232 - 1 (4,294,967,295) Maximum file size Implementation: 244 - 26 bytes (16 TiB - 64 KiB) Theoretical: 264 - 26 bytes (16 EiB - 64 KiB) Maximum volume size Implementation: 232 - 1 clusters (256 TiB - 64 KiB) Theoretical: 264 - 1 clusters


FAT32 Directory Performance

Maximum number of files: 268,173,300 Maximum number of files per directory: 216 - 1 (65,535) Maximum file size: 2 GiB - 1 without LFS, 4 GiB - 1 with


Links