bjm battle room

The only winning move is not to play

2010-02-14T05:42:00.000-08:00

Did you ever see the movie WarGames? This featured a computer called the WOPR. I really liked this movie. I thought there was a good balance between the human story and the computer. I liked most of the movie Twister up until the end. Both movies climaxed with flashing computer lights. In Twister it was just numbers scrolling on a laptop, which was lame. In WarGames the flashing lights showed simulated attacks demonstrating the computer learning, and worked well.

Two recent things reminded me of the movie WarGames. One item was an article in the Washington Post about students hacking into the school computer and changing grades. The other was the new release of Oracle WebCache that includes a learning rule capability for request filtering. In the movie, they use Tic-Tac-Toe to help the computer learn. To try out the new WebCache learning mode I chose Nikto. This is a web security scanner that sends thousands of requests to test the security of a web server. This tool is easy to setup and use, and it rhymes with Tic-Tac-Toe. Let's try it against the Webcache Oracle Process for Request Filtering (WOPRF), and see how it does.

I fired up the new WebCache 11g and chose the new Request Filters menu option. I looked to see what rules were provided out of the box. There are less than 10 rules defined! I checked Metalink for any additional information about the learned rules feature and did not find anything. I read the documentation, however it does not explain how this feature works. I looked at the rules that were defined and it included one for the Trace method and another for checking for null bytes. Those are keepers. I tried adding a rule and then checked the configuration to see the rule format. The new rule was added to the webcache.xml file. It's too bad the rules were not split out into a separate XML file. Most of the WebCache configuration does not change, but the rules are likely to be updated more regularly. Managing these in an independent file would have been much better. If you add rules to block bots and spiders, and other rules for cross site scripting and SQL injection then this will add a lot of entries to the file.

There are two categories for learned rules, method and URL. I ran Nikto and it sent thousands of requests. After the test was complete I reviewed the learned rules. The Method category showed learned rules for all the HTTP methods (OPTIONS, PUT, etc.) with a suggested action of Allow. The URL category showed 25 learned rules. This included a number of useful Deny rules for prefixes like cgi-bin and scripts. The following screen shot shows the URL category with some of the learned rules.

Overall WOPRF was more like Twister rather than WarGames. If you use mod_security it's easy to find a large number of rules that can be plugged right in and used. You would either have to create these rules in WebCache or convert the mod_security rules to WebCache format. Another limitation is the filtering is limited to requests. This means the response cannot be filtered. Filtering responses to prevent information leaks with error messages is a basic feature that needs to be there.

If you have a strong filtering mechanism in place to protect the web site, then you can achieve your goal of reaching a draw between people trying to attack the web site and your defenses. This may convince them that the only winning move is not to play.

Artificial Ignorance - Elementary my dear Watson

2010-01-24T13:23:00.000-08:00

Marcus Ranum applied the term artificial ignorance to the process of monitoring log files. You build a filter of events to ignore, and then look at everything else. All the items that you consider normal get filtered out. This reminds me of the Sherlock Holmes quote “Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth.” - Arthur Conan Doyle. It’s impossible to identify all the potential attacks that might be launched against a web site. New attacks surface every day. While part of the strategy should be to look for specific types of attacks, an equally important strategy is to look for the unusual.

One of the web sites we support is frequently bookmarked by end users. When a user bookmarks a site, the browser sends a request to retrieve the favicon.ico file from the web server. This is the image that will be used for the bookmark. We currently do not have a bookmark image for the web site. This means a 404 Not Found error is returned to the browser. The user never sees this error, the browser simply continues to bookmark the site using a generic image. Web site reports always show the file favicon.ico as the top Not Found document. In our situation, this is normal. We considered modifying the response for this request to eliminate the error, but the 404 error actually serves a purpose. If we review the web site reports, we always expect to see this file with the most 404 errors. If this file is not at the top, then something unusual happened. This is a clue to do more investigation. It may mean that a popular document has been removed, either by mistake or in an attempt to deface the site.

We keep a close watch on HTTP 500 Internal Server errors because they always indicate some problem on the site. Unfortunately, some of the products that we run trap errors and return a 200 Successful code to the browser, which does not indicate any error in the logs. The message on the screen back to the user might say something like “fatal error”, but the logs don’t show this. One way to look for these situations is to inspect the number of bytes sent. The number of bytes for a good result frequently falls in one range, while the fatal errors tend to fall in a different range. This has helped us spot problems when all the other logs did not contain any errors.

It’s important to look at other errors even if the volume is very low. If you see requests for technology that you do not support or types of content that you do not serve, then these are red flags. In our case we do not use php on the site. A request for a php file is a strong indication of some unusual condition. We block these requests and return an error code. Even though you may only see one or two requests that fall in this category it is important to investigate these further because they may mean that someone is probing the site. As Holmes said "You know my method. It is founded upon the observation of trifles." Use this method when setting up your log analysis.

Is your web server an all inclusive resort?

2010-01-17T10:12:00.000-08:00

The Apache web server is a versatile product with lots of options to configure and support a wide variety of web applications. It can act as a proxy server, directly run applications such as Perl and PHP, front-end a Java application server, or just serve up content.

This reminds me of the all inclusive resorts like Club Med. All the activities, food, and drink are available in one place. However, even these resorts modify their model to appeal to specific clientele. The resort locations and activities are designed to fit the groups they cater to. You can go to a resort that is setup for families, for couples, or for singles. While the overall experience is consistent with the resort's philosophy, the activities available at individual clubs can vary widely.

When you are configuring the web server to support applications are you setting the options to cater to your clientele? This can improve the overall security and performance of your site.

The Apache server includes a large number of modules in the core distribution, and vendors that distribute Apache add even more. If you are running a large site, then you probably have many instances of Apache running supporting different needs. This would include different environments, prod and test, different access points, Internet and Intranet, and different functions like proxy and application front-end.

Managing different configurations for each use case would be time consuming and error prone. Apache includes a core feature called IfDefine to mark conditional directives. This can be used to control which directives apply based on a runtime parameter. Using this option will allow you to maintain a single configuration supporting multiple uses while improving the security of the site. Here are some examples of using this capability.

Set a command line option, RUNPROXY, to control enabling proxy LoadModules if this instance is used as a proxy. This technique is typically used for loading the SSL module.

&ltIfDefine RUNPROXY>
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
&lt/IfDefine>

Set a command line option, TESTENV, to allow display of Apache server status if this is a test environment:

&ltIfDefine TESTENV>
&ltLocation /server-status>
SetHandler server-status
    Order deny,allow
Deny from all
    Allow from .your_domain.com
&lt/Location>
/IfDefine

Set a command line option, INTRANET, to enable a specific virtual host on the Intranet:

&ltIfDefine INTRANET>
&ltVirtualHost intranet.your_domain.com:80>
Port 80
    DocumentRoot /intranet/docs
TransferLog /intanet/logs/access.log
    ErrorLog /intranet/logs/error.log
&lt/VirtualHost>
&lt/IfDefine>

The command line options you use can be tailored to fit your environment and requirements. You can maintain a master configuration file and still support a custom setup that is appropriate for the clientele you serve.

Keep a tight lid on HTTP headers

2010-01-10T20:01:00.000-08:00

The web server is a common infrastructure component supporting a wide range of applications. If you manage the web server, then you are the community manager, and you have a responsibility to maintain the reputation of the community.

The weather has been very windy lately. Some people in the neighborhood put out their garbage and don't put a tight lid on the trash can. Of course, the trash blows down the street and into other yards. Now a few of the people in the neighborhood do not see this as a big problem. After all their yard looks fine, and their trash is gone. The president of the homeowners association has been sending out emails to remind people to put a tight lid on their trash cans.

Software vendors or application providers sometimes have the same attitude when you report a cross site scripting or HTTP response splitting issue. The reaction seems to be "How does this affect me"?

These types of vulnerabilities do not directly affect the application that is creating the issue, but they impact the reputation of the whole community, because a person is launching an attack based on the reputation of the web site. The manager of the web server needs to operate as a community manager and police the site.

If you look at the SANS top cyber security risks for 2009, you will see cross-site scripting at the top of the list. The WASC Web Application security statistics show HTTP response splitting near the top of the list, along with cross-site scripting and SQL injection.

The root cause of many of these vulnerabilities is due to software developers setting HTTP headers, such as cookies and redirects, from unvalidated user input. Here is an example using a vulnerability that was reported in the Apache mod_status module. A patch is available for this issue. It is considered a moderate level risk because the Apache status page should not be publicly available, and it is disabled by default. This is a link to a description of the issue: CVE-2007-6388 - Cross-site scripting (XSS) vulnerability in mod_status

The mod_status module included a feature to refresh the screen using a user entered value for the number of seconds. The refresh parameter normally contains a number of seconds. However, the parameter allows the specification of a URL, and the browser will refresh to the URL specified. The mod_status module did not check the user input. It simply took the value of the parameter and rewrote it as an HTTP refresh header. This means a user could click a link expecting to go to a trusted site showing server-status, and instead be redirected to another site on the Internet. This is what the request/response looks like:

telnet my-trusted-site.com 80
Trying 127.0.0.1...
Connected my-trusted-site.com.
Escape character is '^]'.
GET /server-status?refresh=0;url=http://untrusted-site.com/ HTTP/1.0

HTTP/1.1 200 OK
Date: Wed, 13 Jan 2010 17:48:03 GMT
Refresh: 0;url=http://untrusted-site.com/
Connection: close
Content-Type: text/html

The value of the refresh parameter that was entered by the user, and not validated, was placed in an HTTP header in the response causing the browser to immediately redirect to a new site. A user clicking this link would think they were going to my-trusted-site, but instead would be redirected to untrusted-site.

An HTTP header is like a trash can lid. If you don't keep a tight lid on HTTP headers, then garbage will be blown all over the community.

Checking out Oracle OHS Apache 11.1.1.2

2010-01-06T19:43:00.000-08:00

Taking a tour of the latest Oracle HTTP Server (OHS) 11.1.1.2 release from a security perspective. This uses a simple red, yellow, green scale to assess how that configuration item was addressed in this release.
It has been a long time since a new version of OHS has come out. The previous OHS versions based on Apache 1.3 and 2.0 were a little tired. The web server is your front door. Let's see how the security of the new OHS version based on Apache 2.2.10 stacks up. This first look was based on installing the web tier configuration on Linux.

OHS 11.1.1.2 Security First Look
Configuration Item	Rating
http://bjm-battle-room.blogspot.com/
Apache Version - The latest release is based on Apache 2.2.10, which is a very recent version. It looks like Oracle included a fix for a vulnerability in mod_status that Apache fixed in 2.2.11.
Where is mod_security? -The doc says version 2.5.9 is included in the distribution. However, if you check you will see the module is not there. The previous versions of OHS distributed an old version of mod_security (1.84), which was still very usable. We used mod_security to setup filtering rules including a rule that stopped an XSS vulnerability in OBIEE. We opened an SR and Oracle replied that mod_security would not be distributed with FWM 11g. Integrating the open source version is not supported, so if you were using this module you are out of luck.
Conf file macros - The httpd.conf file uses macros to set the Oracle Home directory for directives such as the DocumentRoot. This means other tools cannot understand the format. We always used the .apachectl configtest option to verify the config before restarting OHS. This no longer works with the macro format. The macro option is not a bad choice, but why not go with mod_macro?
Compile options - Oracle compiled a minimum number of modules in the distribution. It's basically just the core modules and two Oracle specific modules, odl_log and ora_audit. You can see the list with the command httpd.worker -l. This gives you the option to disable modules that are not needed for the intended use. Oracle also compiled with the threaded MPM worker by default, unlike the previous OHS 2.0 version that required OPMN changes to enable threads.
Wildcard include - The httpd.conf uses a wildcard to include conf files from the moduleconf sub-directory. This is really a bad idea. If someone inserts a bad conf file into the moduleconf sub-directory then the next time the OHS server restarts it will pick up this file and use it. The configuration files that are used should be explicitly named.
Server Signature - The config directives were set more securely with the addition of ServerTokens Prod and ServerSignature off.
cgi-bin - The printenv and test-cgi scripts are still being distributed in the cgi-bin directory. In addition, the cgi-bin directory is uncommented in the httpd.conf file. The only thing preventing execution of these scripts is setting execute permission on the file. These files should not be included in the distribution.
Logging - The default logging is ODL style with an option to switch to standard Apache logging. Using ODL logging can limit the options for using third party analysis tools. This just does not look like it adds real value. The XML style logs just seem to add more overhead.

What do you think?

Origin of Battle Room

2010-01-04T19:03:00.000-08:00

The term Battle Room comes from the book Ender's Game. It refers to a zero gravity environment where battles are fought. Tactics are an important part of winning the battles. If you have not yet read this book, feel free now to go get a copy and read it. You can return to this blog later. Enjoy!

Ender's Game