Log HTTP Headers in Web Requests
URL filtering provides visibility and control over web traffic on your network. For improved visibility into web content, you can now configure the URL Filtering Profile to log HTTP header attributes included in a web request. When a client requests a web page, the HTTP header includes the user agent, referer, and x-forwarded-for fields as attribute-value pairs and forwards them to the web server. When enabled for logging HTTP Headers, the firewall logs the following attribute-value pairs in the URL Filtering logs:
Attribute Description
User-Agent The web browser that the user used to access the URL, for example, Internet Explorer. This information is sent in the HTTP request to the server.
Referer The URL of the web page that linked the user to another web page; it is the source that redirected (referred) the user to the web page that is being requested.
X-Forwarded-For The header field option that preserves the IP address of the user who requested the web page. It allows you to identify the IP address of the user particularly if you have a proxy server on your network, where all requests might seem to originate from the proxy server’s IP address.
To view the HTTP header, check for the HTTP Headers widget in the detailed log view in Monitor > Logs > URL filtering tab. If there are multiple URLs in a single session, each URL has a separate log with its own set of HTTP headers. The associated headers are grouped together and can be viewed as a set of related logs. Further, to aid in correlating data and analyzing web activity across the network, the HTTP Header options are also displayed with the corresponding threat logs and Wildfire logs.
The HTTP Header fields are available for generating custom reports on the firewall and on Panorama; they are also available for custom log-forwarding to an external syslog server.
Obtain and install a URL filtering license on the firewall. This feature also requires that you install content update version 454 or later on the firewall/Panorama.
Enable HTTP Header Logging
Create a URL Filtering profile or select an existing one. Select Objects > Security Profiles > URL Filtering. Select the default profile and then click Clone. The new profile will be named default-1. Select the new profile and rename it.
Define how to control access to web content. In the Categories tab, for each category that you want visibility into or control over, select a value from the Action column as follows: If you do not care about traffic to a particular category (that is you neither want to block it nor log it), select Allow. For visibility into traffic to sites in a category, select Alert. To deny access to traffic that matches the category and to enable logging of the blocked traffic, select Block.
Specify what to log. The Log container page only option is enabled by default so that only the main page that matches the category is logged, not subsequent pages/categories that may be loaded within the container page. In the Settings tab, enable the options for HTTP Header Logging. .
Attach the URL Filtering profile to a policy rule. Select Policies > Security and select the appropriate security policy rule to modify. Select the Actions tab and in the Profile Setting section, select the profile you just created from the URL Filtering drop-down. (If you don’t see drop-downs for selecting profiles, select Profiles from the Profile Type drop-down.) Click OK to save the profile. Commit the configuration.
View the URL filtering logs. Select Monitor > Logs > URL Filtering. Click the details icon next to a specific log to view the HTTP Headers widget in the detailed log view. (Optional) Adjust the log display to include the Referer, User-Agent, and X-Forwarded-For columns.
View related logs for easy data correlation between log entries. You will no longer have to close the URL log and find the corresponding entry in the threat logs. Click into a URL filtering log and scroll/click through the related log entries. When you click a related log, for example a threat log entry, the widget views switch to display the relevant details.

Related Documentation