Swap file optimizing: larger clusters better or not?

[mantis]

Hi, all -- in one section of PC Guide (ref. below), a scheme to optimize swap file performance (by placing it on its own partition at the beginning of a second hard drive) includes using 32K cluster sizes, since this reduces FAT lookups. But I just read an interesting Usenet posting by an MS MVP (ref. below) stating that 4K clusters are best for a FAT32 swap file, for the following reason:

"FAT32 4KB clusters in ... the Swap file ... can be accessed directly, without having to load them back into RAM first. If they aren't FAT32 *and* 4KB clusters, this performance enhancement is nullified."

Wow, two awesomely respected sources (this particular poster is one of the pillars of the newsgroup), two apparently different views. Anyone know the definitive answer on this? Looking forward to learning something new, and it's not just theoretical as I'm planning to try it out (Explorer on my 500 Mhz/256 MB memory/27G HDD Win98 machine is extremely slow and I'm thinking the hard drive's single large partition may be a contributing factor). Many thanks!

PC Guide: "Special-Purpose Partitions and Other Partitioning Issues"
Newsgroup: microsoft.public.win98.gen_discussion, MVP-DTS Gary Terhune

[tdewolf]

I can't see how you could access the swap file without loading data in to RAM first. The CPU gets the stuff in RAM from the disk and the stuff in the CPU registers from the RAM.
But this doesn't matter. With 256 MB of RAM running Win98 and Explorer, you aren't even going to be using the swap file. Explorer is always slow. Especially after using it extensively for a period of time. Lots of time I have to restart....must be another Windows bug.

[xor_chad]

Howdy-Do,
I cant see any logic to this guy's...logic.
The article in PC Guide takes advantage of two things
[list=1]
Location
Cluster size VS File Size
[/list=a]

The first meaning that since you pack more data into the outer
cylinders of platters, and the drive rotates at a constant speed,
then per revolution you would get more data(given its sequential.)

The second was that depending on the type of files you are storing,
you may benefit from allocating relative sized clusters.

If you had a bunch of huge multimedia files, then it would make more sense
to allocate large cluster sizes. If you had many small text files then it would make little sense to waste space(slack) for 1K files saved to 32K clusters and probably exhaust your clusters if you had many many small files.

This person's assumption that a swap file could be directly accesed is not logical. The addresses of sectors are not available in the same way as on memory chips (On boot, all avaliable memory is assigned an address) The data is stored in a register space called heap and must be sent via the memory bus.

I am interested in reading the entire posting from this person.
Maybe you misunderstood what he was saying...or maybe i am wrong.
I will check it out and get back on this. I am interested in hearing
Charles give his opinion. Laters...

------------------
Chad Wilson
C++/ASM Programmer
PC Support Technician

[This message has been edited by xor_chad (edited 11-27-2000).]

[Charles Kozierok]

That section was written some time ago, and is in the process of being revised. As I do this I will probably re-examine this issue.
I suspect that I will end up recommending that people not bother. With the way hard disks have sped up recently, I suspect that it doesn't make a measurable difference. Just a hunch at this point.

------------------
Charles M. Kozierok
Webslave, The PC Guide (http://www.PCGuide.com)
Comprehensive PC Reference, Troubleshooting, Optimization and Buyer's Guides...
Note: Please reply to my forum postings here on the forums. Thanks.

[scada]

I don't see how that would be possible. Since he must be saying that a page could be pulled from the hard drive and sent right to a processor's cache. However, a processor's cache is "mapped" or set associative to main memory not the hard drive's sectors.


That's not to say that his method doesn't optimize the swap file. I just think he wrong as to HOW it's optimized.


Just my 2cents. http://www.PCGuide.com/ubb/smile.gif

[This message has been edited by scada (edited 11-29-2000).]

[Son of Zeus]

mantis,

u raise some interesting issues & quoted MVP-DTS Gary Terhune as stating that "4K clusters are best for a FAT32 swap file, for the following reason...". Huh? Unless u are the possessor of HDs in terabyte sizes I would suggest that u have no choice but to use anything other than 4KB clusters for swap files if u use FAT32. Certainly u would never even consider using FAT32 32KB clusters in a swap file. Charles was ofcourse referring to FAT16 32KB cluster swap files.

A glance at my Win98SE User Manual shows 512MB to be the smallest possible FAT32 partition for 4KB clusters. To use FAT32 32KB clusters u need A PARTITION LARGER THAN 32GB. For the record under FAT32 the cluster sizes start at 4KB for 512MB-8GB partitions, then 8KB for 8GB-16GB partitions, 16KB for 16GB-32GB partitions & finally 32KB for larger than 32GB partitions. So realistically, if u use FAT32 for a swap file u must use 4KB clusters, irrespective of whether the technical whys & wherefores used to justify it are correct or not. An area that I have no intention to get dragged into as it is far too complex (see below).

I am sure that u do not seriously intend to dedicate more than 32GBs to a swap file. If you were I would suggest that u would be far better served buying an extra 256MB RAM rather than buying a 75GB (or more) HD. Even on a 75GB HD u would be pushing your subsequent OS/applications/data so far in to your HD that you would soon be into the deadly 2nd half of the drive-the part I attempt to partition off & never use (except for backups) as it is normally twice as slow (half as fast) as the first half of a drive.

Let’s give Charles his due. I am currently (yes still) running a very ancient i486DX4/100MHz PC with only 32MB FPM RAM under Win98SE I therefore use a very large swap file. It was my interest in swap files & the best way to optimise them that led me here to "The PC Guide" in the first place some 9 or so months ago.

At the time it must have been a major topic of interest on the Net as I had by that stage read quite a number of articles on this issue online & can honestly say that "The PC Guides" summation was far & away the best. I would go so far as to say that some of the other articles were little more than lazy, poorly researched & rather pathetic attempts to gain hits on their site. They made little or no genuine attempt to understand a technically complex issue.

I followed Charles’ recommendations & optimised my swap file as he outlined & found quite an improvement in my PC's performance. I even went so far as to purchase PartitionMagic to experiment with different cluster sizes for different applications & created my own little benchmarks to test them by. I then published these escapades on another site for a number of reasons. First, to show others better ways of optimising their swap files. Second, to direct people to a very good computer reference site in "The PC Guide". Third, to counter some of the other inept attempts in this area & expose their lack of insight.

Remember that Charles’ aim has been to create a general reference site for PCs. Giving his readers plenty of breadth & enough depth on which they can build a good overall understanding of personal computers. He never intended to provide a detailed examination of each & every area. If he did the site would need to hold more than the contents of your average State library & he would need the IQ of 10 or 12 Einsteins rolled into one.

Swap files are a highly complex technical area. It is a point at which both software & hardware, & essentially cache, RAM & HDs converge. Hopefully Microsoft might take up the challenge & issue a White Paper on the subject that will give us some definitive answers once & for all. It is after all their OS that creates & uses the swap file, because of this they are the only ones with the resources & inside knowledge from which to espouse a complete solution.

I would encourage u not to seek answers to all things technical online. Rather do a little research, then try it, & benchmark it yourself. This is the only real way to find out if something works or not. These things (like Intel V’s AMD processors) should come under the area of "Application Specific" ie. they largely depend on the OS & applications that u are running & the way that u run them. As others have noted online the way each person uses their PC is unique. Because of this, what works for one may not work for another. Try, experiment & be practical. What works in an online, ideal, theoretical environment often does not reflect anyone’s PC usage in the real world.

Well that’s my 2 cents worth for today.

Cheers.......Son of Zeus.

P.S.
Charles re: "That section was written some time ago, and is in the process of being revised. As I do this I will probably re-examine this issue. I suspect that I will end up recommending that people not bother. With the way hard disks have sped up recently, I suspect that it doesn't make a measurable difference. Just a hunch at this point."

Not sure if you’re aware that programs such as Norton System Works (?) deliberately use swap files due to the fact that when working at such an imtimate level with a system’s data it is possible for many reasons ie. systems failures due to power outages, software/hardware crashes etc. that this data, if not stored in a non-volatile memory environment like the HD, could be lost. Swap files are not on the way out.

As u know I am intending to undertake studies towards becoming a programmer. I can assure u that any programs I ever write will use swap files. I am confident many others in this field feel the same way. For this reason alone swap files will always be with us. Therefore any ways of improving them will remain of value to PC users for as long as I can see into the future, irrespective of how fast, or how large HDs eventually become or how much RAM we install.