Jesper's Blog

What do you think, should I do it?

2008-11-16T16:44:00Z

I get a fair bit of blog spam - comments advertising everything from sexual enhancers to fake anti-malware. This one just came in this morning:

Sweet! I can turn off all the blog spam just by e-mailing the criminals? Or, could it possibly be that this is a clever ruse find out what my e-mail address is so they can send their junk there too? Hmm. I think I'll just forward this to abuse@gmail.com.

Fun Experiences at Airport Security

2008-11-15T18:13:00Z

For a while I've been thinking about writing something about interesting times I've had at various airport security checkpoints; security theater, as they have come to be known. There is the obvious shoe removal arguments and the ill-defined rules on electronics (my camera is larger and has more electronics than most laptops, but that can stay in the bag, laptops can't), but there have been more interesting stories. Got any of your own? Share them!

Around November 2001 a colleague of mine and I flew to New York on business. On the way back we went through Kennedy airport. I was wearing a pair of boots, which the TSA (was it even TSA then?) required me to remove, even though shoes were not normally removed at the time as airport security hadn't yet figured out that you could bomb a plane with them. The lady scanned them for explosives and then handed them back saying "these are OK." I was so relieved because I had explicitly asked for the non-exploding boots when I bought them.

Not TSA related, but still: the same year I was traveling through Boston with my competition shotgun. It was broken down into three pieces and stuffed into a very solid, and quite short, aluminum case. When I went to check in I told the check in agent that it needed special screening. She asked me to open it and then asked what it was. I responded that it was a shotgun. She took two steps back from the counter, threw her hands up in the air, and exclaimed "Is it unloaded?" I felt like answering "What? It has to be unloaded? But what if I want to use it during the flight?" Fortunately for me, I didn't.

Several years later I was flying from Seattle, this time with a rifle. Firearms require special screening so after checking in they called a sky cap to carry it for me over to the TSA because I am no longer allowed to touch it at that point at Seattle Tacoma International Airport. Note that at other airports I am perfectly well allowed to touch it as they usually make me hand carry it to the checkpoint. Once I got there the Transportation Security Officer (TSO) asked me for the keys and then struggled with the case for a while before opening it. I offered to help, but he refused as I were not allowed to touch it. He poked around the foam in the case for a while, but all the while refused to lift the rifle. I informed him that the foam is removable and he was welcome to do so as it would make it far less likely I would try to sneak a bomb on the plane. He ignored me. When he was done with that I asked if he was finished and he said "not quite," which turned out to be nearly the only two words this friendly gentleman said to me the entire time. He then turned around, grabbed the explosives swab - and proceeded to swab my rifle down for explosives! I tried asking him how he thought the bullets come out of it! Unfortunately, the airline agent that was with me was laughing so hard I couldn't make myself heard. We both stopped laughing when the TSO explained that he did not find any explosives. It turns out that the Explosives Trace Detection (ETD) units used for explosives swabbing can evidently only detect ammonia-based explosives. Lesson: I wonder when the TSA will realize the giant hole in failing to detect smokeless gun powder?

This year, again with a rifle, I asked why the TSO was so careful not to touch the rifle. Apparently, they are not trained in handling firearms and are afraid they will explode if they touch them. Silly me, I thought they were federal law enforcement officers. Now I realize they are not. They're mostly just people like you and me, except they save lives; and I work in real security.

Shoes again: apparently kid shoes are no threat. I travelled with my three-year old a few years ago. As we went through the check-point they made me remove my shoes for screening, but she could keep hers on. I'm not sure if they were too small to pose a threat (presumably if they were actually bombs there may not have been enough explosives in them to blow a hole in the plane?) or whether they just figured I would be willing to blow myself up but not to sacrifice her. I asked them what size shoes must be to pose a threat, but they refused to answer, citing national security concerns.

A year or so after September 11, I went through Minneapolis airport. Going through the security checkpoint I asked the TSO if he wanted me to put my clothes and underwear in a separate bin or whether I could put them in the same bin. He went beet red and disappeared. The replacement officer told me to take this very seriously and make sure I remove even the smallest piece of metal, like my neck chain, because the scanner was so sensitive this time. I went through without incident. When I got comfortably ensconced in seat 47 E I stuck my hand in my pocket and discovered the three-inch pocket knife I had forgotten to remove. I contemplated briefly calling the TSA and asking if the machine was actually plugged in but decided that would just cause them to empty the whole airport and then arrest me so I figured I'd better let sleeping dogs lie. Amazingly, even with this incredible breach of security, I got home safely.

Right after September 11, 33 days in fact, we were moving from the Boston area to Seattle. Consequently, we had a one-way ticket. When we got to the airport everyone except I received boarding passes stamped with "SSSS". The Secondary Supplemental Security Screening (SSSS) was new at the time so we did not know what that meant. Now we know that it involves getting roughly patted down, your privates squeezed by an inconsiderate TSO, and having your bag torn open, the contents spilled all over the filthy floor, and left to somehow repack your dirty underwear, in the jetway, while the rest of the plane boards, gloating at your misfortune. The selection criteria for being singled out for SSSS are top-secret for national security reasons. There is no apparently truth whatsoever that you are subjected to it if you have a one-way ticket, bought your ticket with cash, changed it the day of the flight, wear a Sikh turban, or have a last name of "Hussein." At any rate, back in October 2001, the system was implemented by airline personnel, who informed us politely (remember when anyone at the airport was polite?) that we would receive the extra screening. I asked them what that entailed and they informed me that they had to look inside our carry-ons, and pat us down; all except me because I was apparently left out due to my advanced frequent flier status. The follow-up question was obvious: what if you have no carry-ons? Then there is no extra screening of those. Consequently, I was left holding six carry-ons and a diaper bag while the bemused gate agent patted down my four-week old daughter for any firearms she may have slipped through the metal detectors in her diaper.

There are probably more stories. What's your most outrageous one? I've heard of many, like the federal marshal who was permitted to fly with a loaded hand gun but had his nail clippers confiscated, and the TSO that held a leatherman knife and failed to recognize it. If you just want to read some others, read Jeffrey Goldberg's article in the Atlantic Monthly.

XP Antivirus in the News

2008-11-07T10:05:00Z

Several helpful people just pointed me to some articles on XP Antivirus and its various variants. In case you do not remember, XP Antivirus was the subject of an article I wrote for The Register a few months back.

It turns out that the scammers got hacked, and the hacker posted some internal accounting details on the web. As suspected, this is a sophisticated business making millions of dollars. It even appears to have an affiliate program.

In case you have not seen the articles yet, here are a few:

http://www.iht.com/articles/2008/10/30/technology/virus.php
http://www.smh.com.au/news/technology/security/russian-scammers-cash-in-on-popup-menace/2008/11/04/1225560814202.html
http://www.scmagazineuk.com/Hacker-reveals-Russian-software-company-behind-anti-virus-scam/article/120152/

Thanks to Marc Michault, Phillippe Jan, and Jason Grubè for all pointing me to articles on this topic.

Is MS08-067 Wormable?

2008-11-04T12:14:00Z

A couple of weeks ago Microsoft released an out-of-band security update in bulletin MS08-067. Looking at the type of vulnerability and the fact that the issue was already being exploited in the wild at the time, this was a good decision. If you have not already installed this security update, you should stop reading this right now and return after you have installed the update.

The problem fixed in MS08-067 is eerily reminiscent of the vulnerabilities that resulted in the Blaster and Sasser worms. Therefore, for obvious reasons, the question arises whether MS08-067 is wormable or not. Microsoft claimed in various outlets that it was wormable "on older systems." Michael Howard backs that up with some interesting analysis on the SDL blog. The Secure Windows Initiative (SWI) blog also discusses the issue and points to a number of mitigations designed to reduce the "wormability" on newer operating systems. By "older systems" Microsoft really means "not Vista and Server 2008." This leads to the question of why the vulnerability cannot be used to create a worm on Windows Vista and Server 2008, and whether the claim is correct or not.

The claim that MS08-067 cannot be used to create a worm on Vista and Server 2008 is based largely on two defenses used on those operating systems. The first is that the vulnerable end-point is not anonymously accessible on those operating systems. That's a pretty good defense out on the general Internet. However, on a corporate network it provides little defense. Anyone with user-level credentials on a host can exploit the vulnerability. Thus, if a single computer gets infected and then is brought inside the corporate network, it can infect any other computers on the corporate network by authenticating to them. It would take a little more coding to write an exploit that does that, but it is certainly not an impossibility.

The second defense is Address-Space Layout Randomization (ASLR). ASLR causes the addresses used for code in memory to change from execution to execution. Each time you execute a program it will be loaded into a portion of memory; but, under ASLR, that memory is offset at one of 256 possible memory locations. Many exploits rely on knowing where in memory certain structures are. Prior to ASLR those locations were deterministic within an Operating System, Serice Pack, and Patch Level combination. However, under ASLR, they are, as I mentioned, no longer deterministic. This makes exploitation much more difficult.

However, do these defenses, and specifically, ASLR, really make a vulnerability "not wormable?" I would argue that the answer is "we do not know" but that it is tending toward "no." The problem is that we really do not understand the spreading patterns of worms well enough to make a claim one way or the other. Let us take a neutral scientific approach to understanding this claim.

Worms rely on spreading from computer to computer. Each computer that is infected with the worm can infect countless additional computers. The only thing that moderates it is time. The spread, however, is exponential. The more infected computers there are, the more computers there are that can spread the infection. Eventually, some form of critical mass is reached at which point the spread turns uncontrollable. Unfortunately, we do not know where that inflection point is.

To see how this works, let us take a hypothetical worm, and let us assume that ASLR is not used. Let's say the infection takes 1/8th of a second per computer. In other words, if computer A is infected and targets the worm at computer B, 1/8th of a second later, computer B is ready to start infecting computer C. In one second, a single computer, computer A, can spread the infection, directly or indirectly, to 64 other computers. The total impact of the worm is t/r^2, where t is the time and r is the rate of spread measured in the time it takes to infect an additional computer. Using that formula, we can see that after 1 second 64 computers could be infected. After 2 seconds, 256 computers can be infected, and so on.

Now let's apply ASLR to this. Using ASLR, the memory address space is allocated over 256 possible addresses. In other words, under a very tight assumption the infection will fail in all but 1/256 cases. The assumption is that we cannot predict where the locations are, and that the randomization will actually cause the infection to succeed in only one case of 256. Let us just say this assumption holds because it lets us analyze a worst-case scenario for the worm. Under ASLR then, we can consider the rate of spread to be 1/256th that of the non-ASLR worm. In other words, rather than infecting the next computer in 1/8th of a second, computer A can only infect one new computer in 32 seconds. This, obviously, slows down the spread of the worm, but is it enough? The spread is still exponential. It just takes longer to spread. Consider this chart:

This chart maps the number of infected computers over a 24-minute period, assuming there is an infinite number of computers to infect, and ASLR is in use on all of them. It is clear from this graph that the spread is exponential. After 24 seconds, 2,025 computers are infected. By contrast, without ASLR, it would take less than 6 seconds to infect that many computers. The point, however, is that ASLR would not stop a worm, it would only slow it down. What we do not know is whether slowing down a worm is effectively enough to stop it. My inclination would be to say that it is probably not enough unless we can slow it down by many orders of magnitude.

In addition to ASLR, the affected service on Windows Vista and Server 2008 would only restart twice before staying down indefinitely. This is important because unsuccessful exploitation would almost certainly cause the service to crash. However, I do not consider that as a defense against worms, because more than likely, the user would at that point either restart the computer or just the service. Given that the restart behavior would only serve to further slow the spreading rate. It would not change the exponential nature of the spread. Again, we arrive at the same conclusion: none of the defenses make a vulnerability non-wormable. They merely slow the spread down.

This is important because there is a risk that people will avoid patching because a vulnerability is not wormable. Make no mistake, remotely exploitable vulnerabilities are still wormable, and within an hour, you could easily have your entire corporate network infected. As if that weren't bad enough, using a remotely exploitable vulnerability, someone with far worse intentions could take over your computers and use them as an entry point into your network. For that the criminal needs only one computer, not a whole network of them. Wormability, or lack thereof, is irrelevant against a targeted attack, which means that ASLR is essentially irrelevant against a targeted attack. in most cases the attacker needs a computer, not a particular computer. Being able to only gain a foot hold on one computer in 256 is likely to be enough because after the initial entry, the vulnerability plays no further part in the compromise of your network. In other words, do not consider ASLR to be a reason not to patch some particular vulnerability.

Now, do I think we will see a worm for MS08-067? No. Not in the traditional sense of Blaster. The time of worms, like Blaster, that are inherently non-destructive, has passed. At this point, criminals are not interested in simply writing worms that self-replicate. They are interested in one of the three big things: money, ideology, or national supremacy. While we may still see massive worms, they will be fundamentally different than the ones of old, and they will probably take a bit longer to write. The new breed will be more targeted, more silent, more deliberate, and more dangerous. Once the objectives change, so do the attack patterns.

In short, please do not use wormability, or lack thereof, as a decision factor in deciding whether to patch a vulnerability or not. Wormability is an irrelevant and potentially dangerously misleading metric.

Need a spare Windows box?

2008-10-24T07:00:00Z

Have you ever found yourself in urgent need of another Windows box? Or, have you wanted to build a web application on Windows, but without having to buy servers? Or maybe you just want to have a network of Windows machines that you can test your new Server Isolation strategy on? You're in luck! Amazon yesterday launched its new Windows on EC2 service. Inside of five minutes you can be ready to log on to your very own Windows on EC2 instance and get started on all those projects!

EC2 is Amazon's Elastic Compute Cloud, a network of virtual servers where you pay only for what you use. Use it for two hours and you get charged for two hours. Use it for a month and you get charged only for a month. It's an eat-all-you-want server where you pay only for what you eat. You can even get it with SQL Server pre-installed.

As if having the ability to build your very own virtual network of Windows computers at minimal cost were not enough, there is even a security whitepaper on how to do it safely. Maybe you will even find some comfort in the familiar name involved in the project?

Revisiting the Immutable Laws

2008-09-23T06:00:00Z

For many years I, and many others, have been referring to the immutable laws of security when trying to explain why something works, or does not work, a particular way. However, I've always wondered how immutable the laws really are? I finally sat down and went through them. The result is a three-piece article series in TechNet Magazine. The first installment just hit your favorite newsstand, or web browser, as the case may be. The second and third pieces will be in the November and December issues of TechNet Magazine.

Anatomy of a Hack 2008

2008-08-22T21:46:00Z

A few years ago I delivered a very popular presentation I called "Anatomy of a Hack." Well, actually, I called it "How to Get Your Network Hacked in 10 Easy Steps" but the marketing department at my previous employer thought that title was a bit, edgy, so they renamed it. The Chinese called it "Anatomy of a Hacker" at TechEd China in 2005, but that's another story altogether. The presentation, which is actually documented in Protect Your Windows Network, had me wandering through an entire network once I got a foothold on one computer.

For the past couple of years I've been telling people that the future of attacks are against people, not networks. In June I got further confirmation of that. A notification came in from my blog that I had a new comment to approve. The comment was just a link, looking like this one:

A Comment has been posted to Jesper's Blog: Hey, Mozilla: Quotes Are Not Legal in a URL by Google Images:
images.google-us.info/index.html Google Images

This looked suspicious enough so I started investigating a bit. What I found just hit the net on The Register. I thought it made an interesting tale of how the bad guys are trying to monetize their handiwork. Sandi has also written about this on her blog here, and here, and here...

On a very much related note, I will actually do a live walkthrough of this type of attack at TechEd EMEA ITPro in Barcelona this coming November. Yes, that's right, I'm going back to TechEd. Hope to see you there!

Security is About Passwords and Credit Cards, Part 3

2008-08-10T06:14:00Z

The final installment in my series called "Security is About Passwords and Credit Cards" is now up on TechNet Magazine. This part of the series discusses updating technologies, including how not to abuse them, messaging about security, and the checkbox syndrome. It ends with the final comments about what we, as an industry, need to do better on to improve our users' ability to protect themselves.

Buy the original Olympic Torch from Beijing

2008-08-09T03:38:00Z

"Buy the original Olympic Torch from Beijing"

That was one of the fake headlines in the latest "CNN.com Daily Top 10" malware spam I've been getting lately. This particular spam is a fake newsfeed which redirects you to one of many sites. All the sites have the same thing in common: they are designed to trick you into installing fake anti-malware software.

I sent some screenshots I took to Sandi, and she wrote up a nice warning about it.

How Not To Build a Highly Available Web Site

2008-07-23T04:45:00Z

Here's what I just got when I went to http://www.technetmagazine.com:

Here's the kicker: it's not TechNet Magazine that is down, nor even TechNet. It is Microsoft Live Sign-in, nee Passport. To get to TechNet it attempts to sign you in to Passport/Live sign-in. Accounts are apparently distributed across servers, and the one holding my account is down, so I can't get to anything that uses it, including the Microsoft.com homepage!

If you want to decrease the uptime on your web site, take a run-time dependency on an unreliable and unnecessary service.

Security is About Passwords and Credit Cards Part 2

2008-07-03T21:32:00Z

The second part of my "Security is About Passwords and Credit Cards" article just hit the web. This installment looks at logon processes, misleading security eye candy, and insecure communications with customers. As always, I'd love your thoughts on it.

Security is About Passwords and Credit Cards

2008-06-20T21:27:00Z

Security is About Passwords and Credit Cards. That's what a very nice lady told me a few months ago. At first I shrugged it off. Of course security is so much more than that. As I started to process it though I realized that is exactly what it is about to end-users. They don't care about the LMCompatibilityLevel, renaming admin accounts, UAC, SafeDllSearchMode, restricted tokens, or IDM. All they care about is to keep their credit cards safe, and the way they do that is by using a password. In the end, I started writing an article on it. When I was done, it was a three-installment piece. The first one just hit the web in the July issue of TechNet Magazine. Let me know what you think.

Thoughts on Security by Obscurity

2008-05-13T17:46:00Z

This has not really been that normal a week for me, but at least another article made it into print. The June 2008 issue of TechNet Magazine is headlined by an article I wrote with my friend Roger Grimes, Security Adviser for Infoworld, on Security by Obscurity. It is another one of those point-counterpoint pieces like we did in the Vista Security book where Roger argues one side of the issue, and I explain why he is wrong; or, rather, argue the other.

Does your AMD-based computer boot after installing XP SP3?

2008-05-07T22:29:00Z

Updates

Updated May 8 to add information on a second issue.
Updated May 9 to add information on possible additional issues as well as instructions for using the recovery console.
Updated May 10 with some clarifications, a possible video driver problem causing other STOP errors, and an additional work-around for the ASUS motherboard.
Updated May 11 with a pointer to a Microsoft article on removing SP3, and added some information on a possible version for the faulting ATI Catalyst driver.
Updated May 12: Added information on free support, and a note on Media Center Digital Rights Management problems.
Updated May 13: Added some information on how to determine which control set to modify for the intelppm workaround. Also added a pointer to an HP support article on the problem and a request to verify a claim made in that article
Updated May 14: Received confirmation about how HP configures its computers. Added an explanation to how the problem occurs.
Updated Again on May 14: Not sure why I didn't think of this until now, but I wrote a small tool that will detect the IntelPPM problem and mitigate it before installing the service pack.
Updated May 20: Fixed the description of the intelppm.sys problem to more accurately represent how the problem occurs.
Updated May 22: Added a note on how to properly download the tool using Safari.
Updated May 24: Added information on conflicts with anti-malware software, including Symantec's suites. The short version is: you MUST disable any security software before installing SP3.
Updated June 4: Added information on a conflict with certain wireless cards.

Before you read on, read this!

There are several issues that can cause a Windows XP computer to not reboot properly after installing Service Pack 3. Most of them affect relatively specific configurations, and most appear to have relatively simple work-arounds. Please: do not do anything rash. I have seen a lot of reports of people who reformat and reinstall when they run into this problem, losing all their data in the process. There is often no need to do anything that drastic. First read this post, and see if anything here helps you. If not, call Microsoft's technical support line and see if they can't help you.

If you have not yet installed SP3, make sure you disable, or better yet, remove, any anti-malware suite before doing so. If you do not, it is possible that you will get various kinds of corruption during the installation.

Free SP3 Support from Microsoft

EmilySc, a Microsoft employee, posted in the newsgroups yesterday that there is now free installation and troubleshooting support for SP3. This may be a real help to those who need interactive help solving the problem.

You can find all the support options on the Microsoft Support Website. In North America, free telephone support is available by calling (866) 234-6020.

The Problem

Last night WSUS deployed XP Service Pack 3 (SP3) to the sole remaining computer running XP that I have. This morning, I came down and was greeted with incessant reboots. The computer booted, apologized for not being able to boot properly, asked if I wanted to boot into safe mode, defaulted to normal boot, rebooted, and so on and so on. At this point, I want to clarify that the endless rebooting is not at all related to SP3 per se. The problem is that with some configurations, SP3 causes the computer to crash during boot, and Windows XP, by default, is set up to automatically reboot when it crashes. That is why you end up in the endless rebooting scenario.

There are many possible reasons why a computer may crash at boot time. SP3 seems to introduce two that are related to AMD-based computers, and, possibly, one or two more that appear to affect Intel-based computers. Which one it is impacts which work-around you use. At this point, the information is still trickling in. If you have a crash on boot problem that does not match what I describe below, and it happened as soon as you installed SP3, I'm sure others would like to know as well, including as much detail as you can give us.

First problem, affecting AMD-based computers with OEM images, primarily HP Desktops

NEW: Use this tool to mitigate the problem

If you have an AMD-based computer, and all you want to do is prevent the problem before installing Service Pack 3, then try the new tool I just wrote. It will first check whether you have an AMD-based computer. If you do it will check whether the IntelPPM driver is set to load. If it is it will offer you an option to disable it. The tool works by simply double-clicking it. If you need to check many computers on a network you can do that by running it from the command line, using this command:

removeIntelPPMonAMD.vbs <computer 1> <computer 2> <computer 3>...

It will take an arbitrary number of computers. The only caveat is that the tool will prompt you several times for each computer. If you really need a silent version, I can probably be persuaded to write one for you.

Note that if you are downloading the tool on Safari there is a bug in how Safari handles these types of downloads. If you just click on the link Safari will save the tool with a .txt extension instead and open it. You can remove that extension and then double-click the tool to run it. If you right-click the link and select "Download link as..." Safari will put the name on the containing page on the tool, not the name of the tool itself. You would need to rename it to something with a .vbs extension first to use it. Neither Firefox nor Internet Explorer makes it this difficult to download that tool, although Firefox does not properly handle right-clicking and selecting "Save link as..."

Disclaimer: the tool is provided "as is" with no warranty express or implied. It is designed to make changes to your system and those changes always carries a risk. Even though I have tested it as much as I can, I cannot guarantee that it will work for you. By running the tool you agree to hold me harmless for any damage it may cause to your computer.

Problem Details

In my case, the computer would boot into safe mode fine, so I did that. Not knowing what it was, I ran a disk check, which turned out to be a real mistake. Once I configured the computer to run a disk check at startup it would not even boot into safe mode.

Fortunately, I know Bill Castner, another Microsoft MVP, and he pointed me to a solution. It turns out that this computer is running an OEM OS image from HP. If you have an HP computer with a part number that ends with a 'z' you have an AMD-based computer. Other manufacturers have also shipped AMD-based computers, but it is unclear whether they have built their images the same way HP did.

The problem is that HP, and possibly other OEMs, deploy the same image to Intel-based desktops that they do to AMD-based desktops. It also appears that this is unique to their desktop image, and any HP AMD-based laptops are unaffected by the problem. Because the image for both Intel and AMD is the same all have the intelppm.sys driver installed and running. That driver provides power management on Intel-based computers. On an AMD-based computer, amdk8.sys provides the same functionality. Microsoft points out in a Knowledge Base article that installing both drivers on the same computer is an unsupported configuration, putting the blame on the OEM that deploys the image. The article in question was written when the same problem occurred after installing Service Pack 2 for Windows XP.

Ordinarily, having intelppm.sys listed in the registry on an AMD-based computer appears to cause no problems, so long as the binary does not actually run. On HPs images, the driver is not installed, even though the driver is listed in the registry and supposed to load. However, on the first reboot after a service pack installation, it causes a big problem. The computer either fails to boot, as in my case, or crashes with a STOP error code of 0x0000007e. If you see that error code you almost certainly have this problem. The computer will boot into safe mode because the drivers are disabled there. Please note here that simply having the intelppm.sys file on your computer is not the problem so searching for it in the Windows directory is not relevant. Nor is only having a directive in the registry to load it a problem. It must be running to cause a problem, which means the file has to both exist on the disk, and the registry has to be configured to load it. Therein lies the problem. HPs images have the registry key set but no driver on disk. When the service pack is installed the pre-existing directive in the registry is read, the installer lays down the driver on the disk, and on the next reboot it launches, causing the crash.

You may not see the error code because the computer reboots too fast. To force the computer to stop when it crashes, you need to set an option during startup. To do so, hit the F8 key during restart right when you see the black Windows XP screen come up. Then select the "Disable automatic restart on system failure" option, as shown below:

To fix the problem, boot into safe mode, or boot to a WinPE disk, or into the recovery console, and disable the intelppm.sys driver.

WARNING: Do NOT under any circumstance disable the intelppm driver on an Intel-based computer. It will make your computer not boot! If your computer will not boot because you disabled the intelppm driver on an Intel-based computer, follow the directions in the Recovery Console section below.

If you have an AMD-based computer, however, you do not need the intelppm driver and can disable it. Boot into Safe Mode by hitting the F8 key as above, but select Safe Mode instead. You will need your Administrator account to log on in safe mode. To disable the driver, take the following steps:

If you booted into the recovery console, from a command prompt, run "disable intelppm"

If you booted into safe mode you can run "sc config intelppm start= disabled"

If you booted into WinPE, you have to manually edit the registry. Do this:

Run regedit
Click on HKEY_LOCAL_MACHINE
From the File menu, select "Load hive"
Navigate to %systemdriver%\Windows\System32\Config on the dead system and select the file name System
Name it something you can remember, such as "horked"
Navigate to horked\<the current control set>\Services\IntelPPM. See below for how to determine which one is the current control set.
Double click the Start value and set it to 4
If you did what I did and completely destroyed things by running a disk check, navigate to <the current control set>\Control\SessionManager. Open the BootExecute value and clear out the autochk entries
Reboot

Step 6 asks you to navigate to <the current control set>. Under HKEY_LOCAL_MACHINE\SYSTEM there are typically at least two numbered control sets, and sometimes there are up to four. They are called ControlSet001, ControlSet002, and so on. Control sets hold all the configuration data for the computer, including all drivers that load. One of them is designated the current one, and the others are backups of previous configurations that worked. The control set that is currently used as the current one is the one listed in the "Current" value under HKEY_LOCAL_MACHINE\SYSTEM\Select. That is the control set that you need to modify in step 6. If you modify one of the other control sets it will not solve the problem. You need to modify the current one. If you manage to boot the computer, CurrentControlSet will be a pointer to the current one and you can modify that one. If you boot from the recovery disk you have to figure out which one is the current to modify the proper one. It will not always be ControlSet001.

If this was your problem, the computer should now reboot just fine.

HPs Response

On May 13, 2008 HP posted a support article on this problem. In that article they claim that the Service Pack copies the intelppm.sys driver to the computer even though it was not there before the Service Pack was deployed.

HP is partially correct. On their desktop images the intelppm.sys file does not exist in the %systemroot%\system32\drivers directory prior to installing the service pack. However, on its laptop images the file does exist there. By contrast, on the HP desktop images the intelppm registry key does exist under HKML\SYSTEM\CurrentControlSet, and it directs the driver to start. On the laptop images, the registry key does not exist. This is why HPs desktop images exhibit the problem and the laptop images are fine. It is not the presence of the driver on disk that is the problem. It is the instruction to load it that HP put into the registry that causes the problem.

That would also explain why the SP3 installer lays down the driver file on disk even though it did not previously exist. I would expect that the installer looks at all the drivers listed in the registry and simply makes sure that there are updated versions of all of them, without checking first whether they existed prior to installing the service pack. After all, if a driver is listed in the registry, and the operating system is instructed to load it, developers could very easily make the assumption that the driver is present on the computer and actually does load.

Regardless of whether the driver file is there or not, I still have to say that the problem is that the registry key should not exist on an AMD-based computer, regardless of what files are laid down on disk. It is not the presence of a file that causes a problem, but the instruction to load that file on boot, and that instruction is represented by the registry key. It is perfectly legitimate to lay down all kinds of files on disk during installation but not load them. In fact, HP itself lays down the intelppm.sys file in the i386 directory - the on-disk cache directory of operating system files. This strategy is also used successfully by Microsoft Office, Windows Vista, Windows Server 2008, and several Adobe Products. It prevents the user from needing access to the original disks to update, repair, or modify an installation.

What this means is that if you have one of the affected HP desktop computers you can prevent the problem before it even starts. Before installing the service pack go to a command prompt and run either of these commands:

reg add HKLM\System\CurrentControlSet\Services\Intelppm /v Start /d 4 /t REG_DWORD

sc config intelppm start= disabled

Both commands will disable the driver before you install the service pack and will prevent the problem from ever occuring.

Potential impact on Media Center

Two separate posters have reported problems with Windows Media Center after this work-around. At this point, I am not ready to say that this is caused by using the work-around, but if you have this problem, I would appreciate a note to confirm it.

Logically, it could be related. This is pure speculation, but based on what I know about the Digital Rights Management (DRM) in Media Center it may detect the change in hardware, disabling the intelppm driver, as a hostile action and disables viewing DRM protected content. Unfortunately, Comcast Cable puts a DRM signal into some of their cable channels, which means you can no longer watch those channels. You would also be unable to watch previously recorded content. The content provided by Comcast is not actually encrypted, but Windows enforces the DRM nevertheless.

I had a very similar problem with Media Center last year. At the time I was unable to resolve it. However, I would encourage anyone who has this problem to try resetting the DRM components in Media Center. If that does not work, try re-enabling the intelppm driver and see if that helps. It should be safe to do so if the intelppm.sys file is not present in the %systemroot%\system32\drivers directory (check first), and once the computer has booted properly after the service pack installation.

Bill Castner, who is rapidly becoming my new hero, also posted a solid work-around for Media Center problems over in the AumHa forums. Try that one as well, it may solve your problems too.

Second problem, affecting certain AMD motherboards

The second problem type manifests itself in a different error code during boot, and also seems to affect only AMD-based computers. The error code will say something similar to:

Problem was detected and windows has been shut down to protect your computer from damage.

The BIOS in this system is not fully ACPI compliant

You will then get some information about how to update your BIOS. The BIOS is the basic operating system built into the computer that handles reading and writing from disk and memory, as well as some other devices. That is most likely not your problem. The screen ends with the tell-tale error code: STOP: 0x000000A5. If you have that error code, and you just installed SP3, this is most likely your problem.

At the moment, I do not know for sure why this is happening, and I have not personally seen it. The problem appears to be the ASUS A8N32-SLI Deluxe motherboard, also with an AMD processor. Several different AMD processors have been fitted on that board, however, so it seems more likely to be the board than the processor.

The solution is simplicity itself: insert a USB flash drive, or some other form of secondary storage mechanism, before booting the computer. The people have that have seen this problem report that it goes away when they do. The catch is that the computer will only boot with a secondary drive attached. If you remove the secondary drive it will no longer boot.

It also appears that this could be related to using a USB mouse. If you have a USB mouse, try moving it to the PS/2 port instead (the little round port, you should have received an adapter with your mouse). That seems to resolve the problem without the use of an external USB flash drive.

If you have this problem, and either solution helps, or even if they do not help, I'd appreciate a comment on the blog so we can figure out what is going on here.

Other STOP Errors

Every time a service pack is installed, or any major maintenance like it is performed, a certain, very small, number of computers seem to not come back up. The reasons could range from malware on them that is conflicting with the installation or the new files, to bad hardware that somehow failed at that very moment.

For that reason, there may be other STOP errors involved in this problem. Due to the default settings in XP, all of them would result in an endless reboot cycle. Only if there are many of them does it usually indicate a problem with the service pack. A fair number of people are reporting an error code 0x00000024. It usually means either that the file system driver, ntfs.sys, has been corrupted, or you have a hard disk with bad blocks in bad places. It could be totally unrelated to the service pack. At this point, I just do not have enough details to tell. This one seems to be more related to Intel-based computers though.

It is also possible that 0x00000024 has to do with a faulty video driver. I have seen a couple of reports of crashes caused by the ATI Catalyst 8.4 drivers, and one of a crash involving an nVidia driver of some kind, but I do not know which one. To see if that is your problem, try booting into Safe Mode or VGA mode. If VGA mode works you very likely have a video driver issue. Gary Barclay, in a comment below, pointed out that the 8.432 version of the driver may be the one that is faulting, and that version 8.467 appears to work properly. If anyone else can confirm that I'm sure may others will be happy about it.

If you are getting the 0x00000024 error, there are some things to try:

There is some good information in the Microsoft knowledge base on how to trouble-shoot STOP errors. Try following that.
If you have multiple drives in the computer, disconnect them one by one and try booting. The problem may not be on your primary drive and this could let you isolate which one has the problem.
Run chkdsk /r. The problem could be file system related, and chkdsk could fix it. However, to do that you have to boot the computer successfully. If you have a 0x00000024 error, it will not boot even into safe mode. You will need to follow the instructions in the Recovery Console or WinPE sections below to boot the computer.
Replace the ntfs.sys driver. If the driver file itself has become corrupted there is a backup copy in the %windir%\system32\dllcache folder. If nothing else helps, you could try replacing the version in %windir%\system32\drivers folder with the one from dllcache and see if maybe it was a corrupted file problem.
If you have an ATI or nVidia driver for for your graphics card, notably the ATI Catalyst 8.4, and your computer will not boot, try booting into VGA mode and see if that works. If it does, you almost certainly have a video driver problem. Uninstall the driver and see if Windows will find a better one. If this works for you, please either contact me using the contact link, or post a comment, so others can learn what is really happening here.

There have also been sporadic reports of video driver problems as well as other issues, like the VPN issues. Most of those have to do with some form of third-party software that does not work with SP3. If you have a problem that is not covered here, it would be good if you could let us know. It may be related to SP3, in which case others may have it too. The VPN issue mentioned by one of the posters has me very interested, for example.

Other people are reporting that the computer is complaining that a particular file is corrupted. Sometimes the corruption results in a blue screen, other times something does not work right after the computer reboots. At this point I am not sure what could be causing this, and I would encourage anyone who runs into that problem to call the Microsoft support line listed above. If they manage to figure out what the problem is, please post back here so the rest of us can find out.

Conflicts with Certain Wireless Card Drivers

Tim Steele read the blog and found that his problem was not solved. After doing some more research he discovered a conflict with certain wireless cards. I asked if I could post his discovery. This is what he wrote:

Some 802.11b wireless cards cause XP to blue screen after installing SP3

If you have any of the following 802.11b wireless cards you'll see a blue screen after installing SP3:

SMC 2635W, Belkin F5D6001, Linksys WPC11 v1, Blitz NetWave Point PC, Xterasys Cardbus XN-2411b, D-Link DWL-520 Revision C, Xterasys Cardbus XN-2411b, Fiberline FL-WL-200X, 3com Office Connect 3CRSHPW796, Corega WLPCIB-11, SMC 2602W V2, and D-Link DWL-520 Revision C.

These cards all use the adm8211 chipset. The driver was provided by ADMtek and badged by the vendors. The last version on the net seems to be 1.80. The D-Link driver is WHQL certified and signed.

There are plenty of adm8211 cards out there inside machines which are about to update to SP3, Windows Update doesn't check whether you have one of these cards before automatically installing SP3, so the effect for many users will be a mysterious blue screen and no obvious cause.

It's not clear whether the vendors or Microsoft should be responsible for fixing this, but surely as a minimum SP3 should not install on machines with this hardware.

Conflicts with Anti-Malware Software

Gregg Keizer wrote an interesting couple of articles in Computer World (second piece is here) about conflicts between Symantec's anti-malware suites and SP3. It appears all but certain that the anti-malware suites cause registry corruption, failures in device manager, and other problems, when you install SP3. An interesting thread on Symantec's support forums documents some of the problems. There are directions for how to disable Symantec's software in another thread.

The security suites add significant hooks into the operating systems. It is quite possible that they will prevent a major installation, such as a service pack, from completing properly. For that reason, you should at the very least disable any anti-malware or security software you have installed prior to installing the service pack. If you can uninstall it, install the service pack, and then reinstall the anti-malware software, you will probably have even greater chance of success.

Using the Recovery Console in XP

If you cannot boot into safe mode you can try using the Recovery Console in Windows XP. This requires you to have a Windows XP CD. Knowledge Base Article 307654 has directions on how to use it. You do not need to follow the instructions for how to install it. In fact, if you have a problem like the 0x00000024 issue above, you probably can not boot from an installed recovery console anyway.

In brief, to boot from the recovery console in XP, do this:

Insert your Windows XP CD
Boot the computer
Select to boot from the CD. On many computers you have to hit a button to do that. On Dell computers the button is usually F12. On HP it is usually ESC.
The computer will work for a while and eventually you get a screen that says "Welcome to Setup". Hit the R key here
If will ask you which installation you want to boot. If you have several XP installations on this computer, select the one you want. Of course, if you have several installations, and one still works, you would not need these steps.
Type the administrator password for the installation you need to repair.

At this point, you should be at a command prompt. The commands you can run are very limited and they are often different from what you are used to. If you have disabled the intelppm driver on an Intel-based computer and need to re-enable it, run "enable intelppm SERVICE_SYSTEM_START".

If you need to run chkdsk you can do it from the recovery console window as well. The C: drive is the boot volume in your Windows XP installation. To run the full check run "chkdsk c: /p /r"

Build a WinPE Disk on a Flash Drive

Another option, recommended for advanced users, is to have a Windows PE disk handy. Windows PE is a miniature version of Window that can boot from a CD, and starting with Windows Vista, a USB Flash Drive. I wrote up directions on how to build a Flash Drive with Windows PE in the Vista book, and there are now also directions on TechNet. You need to have access to a computer that boots, and you need a copy of the Automated Installation Kit (WAIK). Once you burn the AIK image to a disk you can install it and start building your Win PE disk.

Using a Windows PE disk you get access to all the normal tools, like regedit. It has far more features than what you have with the recovery console, but requires a lot more prep work to get started.

Removing SP3

A few people decided the problems were sufficient to just remove SP3 altogether. If you have a problem that is not covered above, that may be your best option for the moment. Microsoft just published an article on how to remove the service pack. It includes information on how to remove it even from the Recovery Console, so even if your computer will not boot you should be able to do it.

Phishing for a Tax Refund

2008-05-05T04:30:00Z

What's wrong with this picture?

If you answered "why would the IRS use a web server in Korea to ask for information about my tax refund" you are a winner!

This is a phishing site preying on people who do not know that all you need to do to get your tax rebate is to file a tax return this year. Apparently, this is the hot new phishing scam, and the IRS has instructions for how to handle it.

The e-mail came in at 21:07 PDT today. By 21:30 PDT it was not recognized as a phishing site by either Internet Explorer or Firefox. By 21:35 Firefox had it marked. Impressive. By 21:40 IE did not have it marked, which I found interesting.