Vista SP1 and the Red Herring (+ breaking the 32bit 4Gb limit)
We all knew it was looming, the mathematical limit to address referencing in 32bit computing. A 32Bit number can only be between 0 and 4,294,967,295 which neatly adds up to 4Gb and what this means is, using existing architectures, a program (or Operating System) will not be able to address more than this number of bytes of system RAM via the existing system called byte addressed memory allocation.
What this means for those among us who do not speak geek, is a system which is built or shipped with 4Gb of RAM (and some other cases*) will not be able to fully utilise all of that space.
Lets take a trip back in history and imagine a room with a cupboard containing 256 drawers. Each drawer could hold one bit of binary information and was administered by a librarian. Anytime anyone wanted a piece (or pieces) of information, they had to ask the librarian. What I am describing here, is the era of 8bit computing circa late 1970/ early 80s with the cupboard representing system memory and the librarian representing the Operating System’s memory management system. During day to day running of the system the librarian takes data in and returns data to people (program threads) from the corresponding drawers where the information is stored. Everything works, everyone is happy.
Now what happens if we introduce a second cupboard containing another 128 or 256 drawers? The librarian can only keep track of information stored in the first 256 drawers and as a result, nothing can be stored or retrieved from the newly added cupboards; in effect, they do not exist. Time to get a new secretary i.e. goto 64bit computing (or in this example, replace the 8bit librarian with a swanky 16bit one – who will even ever use 65536bits of RAM? 😀 )
But wait, there is more… I read today that Windows Vista SP1 changes (depending on hardware configuration) the total amount of displayed RAM from 3.5 Gb (current the RTM limit when 4Gb is put in the machine) to the full 4Gb, although this still does not help, given the limitation previously discussed. But this made me curious, if the Operating System could see RAM, then surely it was not a BIOS / mathematical fundamental limitation. Turns out I was at least half right …
You see, although the fundamental mathematical limitation can not be breached, there is a rather interesting technique called Physical Address Extension. Using this process, a 32bit Windows system can address more than 4Gb of RAM upto a (present) maximum of 128Gb. To explain what Physical Address Extension (PAE) is, lets go back to the previous example and introduce a new figure – an administrator.
The role of this new entity, is to allocate and manage the time of their underling. Lets also assume we are still running a 8bit system (with the 256bit limit) and have 1024bits of memory i.e. four times the mathematical limit. On the face of it, the extra memory is invisible to the librarian however the administrator is smart enough to both know about the extra memory and who (i.e. what program) is currently using what amount of it. As such, any person (program) can request the full mathematical limit 256 drawers for their own use at the same time as another person (and another …etc) requests more memory.The administrator can instruct the librarian which series of drawers to use per person (program).
This is loosely referred to as 36bit computing and, as the non power of 2 number suggests it is a bit of a tweak. The physical address size was increased (on a 32bit processor) from 32 to 36bits back during the days of Pentium Pro (circa 1997) and most modern CPUs have maintained this legacy. It is important to point out, this does not make all 32bit processors 36bit processors as the change happened in the MMU (memory management unit). Modern Operating systems use page tables to store information about the Virtual Memory system and allocate it based on processes requirements. In effect they act like the administrator from my trivialised example and allow multiple processes to benefit from a pool of memory which traditional 32bit systems (without PAE) would not.
I know what you are thinking, you are rejoicing at being able to avoid the negative aspects of migrating to 64bit computing, but hang on, there are a couple of important caveats. Firstly, each thread (person in our example) can only access a maximum of the mathematical limit of RAM. That means, in a system with 16Gb of RAM, you could quite easily have 3 or 4 processes each taking up 4Gb, but no one process taking up 8 or 16Gb. The other bad point is, it is not supported** in Vista or XP. In-fact, to use such a feature, you would need to be running a Server Operating System from Microsoft or a Linux equivalent. Interestingly enough, Linux contains support for PAE since kernel version 2.6 although I will not discuss it further in this post.
Presently, the only Operating Systems with suitable (or rumoured) PAE support are :
Windows 2000: Datacenter Server and Advanced Server Editions
Windows Server 2003: Enterprise and Datacenter Editions
Windows Server 2008: Enterprise and Datacenter Editions
As you can see, non are particularly home desktop friendly. So, despite Vista displaying the correct amount of RAM in Service Pack 1, it is still fundamentally limited to the 32bit mathematical limit despite Microsoft having the technology to at least improve on the functionality of such systems.
On a side note, I brought this up with a few people at my head office. I work for a large UK retail company that sells PCs and Laptops. I was surprised to see when our first 4Gb models came into the stores a few months ago that they were running Vista 32bit Editions. The UK is not a litigious as the United States, but I can’t help wondering how long it will be before the lawsuits start flying. After all, it is misrepresentation in my book to sell something that, due to a software shortcoming, can never be fully utilised to the specification it was advertised at. Particularly since an alternative is available to OEMs and yet, all retailers not just the one I work for seem to be taking a cavalier attitude towards this.
*The total amount of addressable space inside a 32bit system must add up to 4096Mb, this includes system and Video RAM, so if you have an all singing, all dancing SLI graphics card with 2Gb of Graphical RAM, the total amount of system RAM you will be able to address is around 2Gb.
**Actually this is not true, ever since Windows XP Service Pack 2, Microsoft has used PAE for security purposes coupled with the NX bit. This is a hardware security feature built into a processor which allows program and system developers greater control over what they designate to be executable and non-executable user/memory space. Microsoft has set a fundamental limitation of the amount of RAM being used by home versions of 32bit Operating Systems to 4Gb regardless of the fact the technology to increase this is in place.