One Billion Dollars! Wait… I Mean One Billion Files!!!

@HK thats about how my layout is falling too, portage + sunrise overlay then add in sabayons entropy package manager and portage overlay.

281207

Running Lucid on a touchscreen netbook...

Results are more interesting then I thought, ranging from 0.18M to 1M. Which begs the question, why do some distro's need ~3 times as much files as others?

Mine is back to 586000 now, after I deleted obsolete sourcetrees (KDE 4.5.1 is in the package manager now, and there are binaries of Firefox with JaegerMonkey enabled now).

Interesting BTW I (still) have more files than a Sabayon desktop install, I think Sabayon came with everything including the kitchensink already installed.

I'm also very curious to how many files are on a typical workstation / server, but that probably depends much more on its task than for desktops.

> ...ranging from 0.18M to 1M.

Well, my count includes all of my video, musc, and picture files; so it's probably quite a few more than you would find in a default Slackware installation.

A quck wc -l on the MANIFEST included with Slackware 13.0 shows 344240 lines, which includes the directories created by aaa-base.

I have lots of -dev and -dbg packages for development.

Quoting:Well, my count includes all of my video, musc, and picture files; so it's probably quite a few more than you would find in a default Slackware installation.

Most of those live on an external drive for me. I have a 1 TB drive and a 320GB drive that were not included in that 180K count.

If I did my greps correctly, and assuming the MANIFEST.bz2 matches reality, there are 299,953 regular files installed from a full Slackware-current package set.

> ...there are 299,953 regular files installed from a full Slackware-current package set.

That sounds about right.

This is why I can't use full fisk LVM encryption on my system, the performance suffers working with too many small files.

@tmx:

Buffer caching mitigates that a lot, and also helps with name/inum resolution.

Reading files that fill the buffer cache (e.g. playing movies) will hurt it.

I just did an rsync of /home, with both source and destinations on encrypted volumes. The activity on the backup volume far outstripped the activity on the source volume, because the source volume is regularly accessed. "df -m" shows:

/dev/mapper/home 348200 72524 275676 21% /home /dev/mapper/rsync 142923 72711 70213 51% /rsync

And the time report on the rsync run:

9.00user 14.21system 4:22.35elapsed 8%CPU (0avgtext+0avgdata 64672maxresident)k 2036056inputs+1943792outputs (3major+14535minor)pagefaults 0swaps

That's on a dual-core Athlon 64, with 4G RAM and two SATA drives. /home's volume is on sda, and /rsync's volume is on sdb.

Oh, I should mention that about 45 seconds of that elapsed time was reading from an NFS mount, rather than the local volume. Still, 8% CPU and only 14.21 seconds system time for NFS networking access and encrypted volume access... that's not bad.

The security I get against a physical attack outweighs any disk slowdown that can be measured, but not felt.

Tmx,

I'll agree with Gus3. I compiled 2.6.35 on the same 1.6GHz dual core laptop one day to the next, after reinstalling with encrypted LVM. There is no greater use of lots of little disk files for me than that.

There was, maybe, 2 minutes of difference in time over the course of an hour between the two compiles, which might very well be caused by being the difference between the .6 to .7 kernels.

And in using it, I too do not perceive any slowdown.

And you have to use the i586-optimized version instead of the generic (when using AES), seems to matter as well.

How many files on my systems? Probably too many.

642475

PCBSD 8.1 + Ports

Which reminds me, counting on OpenBSD or such might be interesting as well.

Base OpenBSD 4.7 32-bit, with no SMP kernel stuff or ports, has 26,539 regular files.

Arch Hurd from the CD has 12,844 files.

(Wow, this is really easy when the disk image is actually a sparse file...)

And FreeBSD 8.1-RELEASE with no ports, has 77,082 files.

10,004,461 files, ext3, on a 24TB raid 6 array

> 10,004,461

I think we have a winner, folks.

i hope it isn't all porn, that would be a lot.

tmx,

At first I thought it was a major storm and pressure drop approaching

Quoting:i hope it isn't all porn, that would be a lot.

that was causing the excruciating pain in my toes*.

YBT

* Aren't you aware this niche is taken?