We all knew it was looming, the mathematical limit to address referencing in 32bit computing. A 32Bit number can only be between 0 and 4,294,967,295 which neatly adds up to 4Gb and what this means is, using existing architectures, a program (or Operating System) will not be able to address more than this number of bytes of system RAM via the existing system called byte addressed memory allocation.
What this means for those among us who do not speak geek, is a system which is built or shipped with 4Gb of RAM (and some other cases*) will not be able to fully utilise all of that space.
Lets take a trip back in history and imagine a room with a cupboard containing 256 drawers. Each drawer could hold one bit of binary information and was administered by a librarian. Anytime anyone wanted a piece (or pieces) of information, they had to ask the librarian. What I am describing here, is the era of 8bit computing circa late 1970/ early 80s with the cupboard representing system memory and the librarian representing the Operating System’s memory management system. During day to day running of the system the librarian takes data in and returns data to people (program threads) from the corresponding drawers where the information is stored. Everything works, everyone is happy.
Now what happens if we introduce a second cupboard containing another 128 or 256 drawers? The librarian can only keep track of information stored in the first 256 drawers and as a result, nothing can be stored or retrieved from the newly added cupboards; in effect, they do not exist. Time to get a new secretary i.e. goto 64bit computing (or in this example, replace the 8bit librarian with a swanky 16bit one – who will even ever use 65536bits of RAM? :D )
But wait, there is more… I read today that Windows Vista SP1 changes (depending on hardware configuration) the total amount of displayed RAM from 3.5 Gb (current the RTM limit when 4Gb is put in the machine) to the full 4Gb, although this still does not help, given the limitation previously discussed. But this made me curious, if the Operating System could see RAM, then surely it was not a BIOS / mathematical fundamental limitation. Turns out I was at least half right …
You see, although the fundamental mathematical limitation can not be breached, there is a rather interesting technique called Physical Address Extension. Using this process, a 32bit Windows system can address more than 4Gb of RAM upto a (present) maximum of 128Gb. To explain what Physical Address Extension (PAE) is, lets go back to the previous example and introduce a new figure – an administrator.
The role of this new entity, is to allocate and manage the time of their underling. Lets also assume we are still running a 8bit system (with the 256bit limit) and have 1024bits of memory i.e. four times the mathematical limit. On the face of it, the extra memory is invisible to the librarian however the administrator is smart enough to both know about the extra memory and who (i.e. what program) is currently using what amount of it. As such, any person (program) can request the full mathematical limit 256 drawers for their own use at the same time as another person (and another …etc) requests more memory.The administrator can instruct the librarian which series of drawers to use per person (program).
This is loosely referred to as 36bit computing and, as the non power of 2 number suggests it is a bit of a tweak. The physical address size was increased (on a 32bit processor) from 32 to 36bits back during the days of Pentium Pro (circa 1997) and most modern CPUs have maintained this legacy. It is important to point out, this does not make all 32bit processors 36bit processors as the change happened in the MMU (memory management unit). Modern Operating systems use page tables to store information about the Virtual Memory system and allocate it based on processes requirements. In effect they act like the administrator from my trivialised example and allow multiple processes to benefit from a pool of memory which traditional 32bit systems (without PAE) would not.
I know what you are thinking, you are rejoicing at being able to avoid the negative aspects of migrating to 64bit computing, but hang on, there are a couple of important caveats. Firstly, each thread (person in our example) can only access a maximum of the mathematical limit of RAM. That means, in a system with 16Gb of RAM, you could quite easily have 3 or 4 processes each taking up 4Gb, but no one process taking up 8 or 16Gb. The other bad point is, it is not supported** in Vista or XP. In-fact, to use such a feature, you would need to be running a Server Operating System from Microsoft or a Linux equivalent. Interestingly enough, Linux contains support for PAE since kernel version 2.6 although I will not discuss it further in this post.
Presently, the only Operating Systems with suitable (or rumoured) PAE support are :
Windows 2000: Datacenter Server and Advanced Server Editions
Windows Server 2003: Enterprise and Datacenter Editions
Windows Server 2008: Enterprise and Datacenter Editions
As you can see, non are particularly home desktop friendly. So, despite Vista displaying the correct amount of RAM in Service Pack 1, it is still fundamentally limited to the 32bit mathematical limit despite Microsoft having the technology to at least improve on the functionality of such systems.
On a side note, I brought this up with a few people at my head office. I work for a large UK retail company that sells PCs and Laptops. I was surprised to see when our first 4Gb models came into the stores a few months ago that they were running Vista 32bit Editions. The UK is not a litigious as the United States, but I can’t help wondering how long it will be before the lawsuits start flying. After all, it is misrepresentation in my book to sell something that, due to a software shortcoming, can never be fully utilised to the specification it was advertised at. Particularly since an alternative is available to OEMs and yet, all retailers not just the one I work for seem to be taking a cavalier attitude towards this.
*The total amount of addressable space inside a 32bit system must add up to 4096Mb, this includes system and Video RAM, so if you have an all singing, all dancing SLI graphics card with 2Gb of Graphical RAM, the total amount of system RAM you will be able to address is around 2Gb.
**Actually this is not true, ever since Windows XP Service Pack 2, Microsoft has used PAE for security purposes coupled with the NX bit. This is a hardware security feature built into a processor which allows program and system developers greater control over what they designate to be executable and non-executable user/memory space. Microsoft has set a fundamental limitation of the amount of RAM being used by home versions of 32bit Operating Systems to 4Gb regardless of the fact the technology to increase this is in place.
It is interesting how perusing or glancing at the popular tech topic currently doing the rounds on wordpress can give an insight into the impact such announcements (or software/game/hardware/press releases etc) are having on the general public. Its all well and good reading about something (in this case Vista Service Pack 1) from recognised tech insiders such as Paul Thurrott, it is far more telling to read about the experiences everyone else is having.
Here are a selection of headlines from the last few days,
Vista wreaks havok on some PCs, users complain (anti Vista blog), My Nightmare trying to upgrade to SP1 (Insightful look into incompatible drivers), Vista SP1 update not showing up is for your own good (Reasons why SP1 may not be available yet for some people), SP1 Now available, Delayed, Delayed, Delayed, SP1 Day two (interesting positive feedback from a user), Hell has frozen over (overexcited user).
And guess what? Its not (entirely) the usual doom and gloom and has become almost ubiquitous when it comes to reports about Vista. Vista Service Pack 1 has come a long way since internal betas handed out to the Microsoft beta testers. These poor guys must have been feeling particularly abused this time round if the early write ups are anything to go by. With several restarts required to complete the process (and a few hours) these so-called tech elite reported back their thoughts on the process and as you can imagine, even the most staunchly pro-Microsoft of them has a few ‘choice’ comments to make.
But anyway, fast forward to now and you will see in your Windows Updates Vista Service Pack 1 waiting patiently for you to let it into your digital home. Software and hardware compatibility is good and the lengthy installation process has been slimmed down dramatically to a single reboot after completion. Inevitably it won’t go that way for everyone with some users reporting issues with certain drivers. To Microsoft’s credit, a fairly comprehensive list of drivers that have issues has been published and I have reproduced the list below.
For x86-based computers: Alcxwdm.sys – version 18.104.22.16842 or earlier
For x64-based computers: Alcwdm64.sys – version 22.214.171.12442 or earlier
For x86-based computers: Sthda.sys – version 5.10.5762.0 or earlier
For x64-based computers: Sthda64.sys – version 5.10.5762.0 or earlier
For x86-based computers: Stwrt.sys – version 6.10.5511.0 or earlier
For x64-based computers: Stwrt64.sys – version 6.10.5511.0 or earlier
For x86-based and x64-based computers: Ctaud2k.sys – version 126.96.36.1992 or earlier
For x86-based computers: P17.sys – all versions (This was originally a Windows XP-based driver.)
Conexant HD Audio
For x86-based computers: Chdart.sys – version 188.8.131.52 or earlier
For x64-based computers: Chdart64.sys – version 184.108.40.206 or earlier
For x86-based computers: Igdkmd32.sys – versions between and including driver 220.127.116.112 and 18.104.22.1683
For x64-based computers: Igdkmd64.sys – versions between and including driver 22.214.171.1242 and 126.96.36.1993
Unfortunately, I am the (not-so) proud owner of a Ac’97 soundcard in my primary laptop so it looks like I may have to fish around for drivers (AGAIN!!) to get my laptop to work properly with Redmond’s latest offering. The issues here are not Microsoft’s fault. Infact, as several tech insiders have noted, Microsoft was beating the drum about drivers to ODM/OEMs for months prior to Vista’s (and SP1’s) release but when the moment came to deliver, most manufacturers did not come to the party.
The reason is simply, it is not really cost effective. Take a computer you bought in the last few years (or Motherboard) and goto the manufacturer’s website and check the date of the ‘latest’ drivers (or BIOS.) Whilst these companies are fairly diligent during the product’s lifecycle, when they move onto something else, they stop putting out bugfixes or updates because it no longer makes commercial sense for them to pay their software engineers to do so. When Vista came out, many people had equipment (like me) that was designed for XP but could, with a fair amount of tweaking, run Vista very comfortably. The problem I (and many others) faced was a complete lack of native driver support for this hardware. I understand the problem, but I still think it is ridiculous. Microsoft did try to smooth this over by building in a compatibility layer into Vista to allow the loading of some XP drivers and while this helped a lot, there were performance penalties.
For now, I am not particularly fussed about SP1 so I will be sticking to vanilla Vista until either these driver issues are resolved (unlikely) or I get the time to find replacement drivers for my laptop.