IDS and HTTP decoding

I’ve recently been doing some work on intrusion detection systems (IDS). As anyone who has ever discussed the subject with me will know, I am somewhat sceptical about the value they add to protecting an application, particularly when HTTP is involved. Part of the reason for the sceptism is the complexity of many of the tasks an IDS needs to carry out. Take for example decoding a URL. It’s claimed in a paper by Daniel Roelker at IDSResearch.org that there are over 8 different types of encoding possible for HTTP despite only two being defined in the relevant RFCs. An IDS needs to be able to understand each of these methods before it can hope to identify a malicious request. The task is complicated further by different products supporting different methods, with IIS perhaps being the worst offender. Whether such deviance from the standards is due to irresponsible software manufacturers or due to limitations or ambiguities in the standards, it is hard to tell. Note IIS7 now seems to disable many of the encoding techniques although they can easily be reactivated. IDS Research also has some useful tools for testing which encoding methods are supported by your web server and to allow you to see if your IDS can pick up the various types of encoding. It’s well worth testing your systems. You might be surprised what shows up.