Automatic Proxy Configuration

Manage proxy settings remotely and dynamically


The problem

When you're dealing with automatic system configuration and hands-free system administration in a managed environment, you may find that there's this one item that's hard to deal with : Proxy servers.

Some web browsers provide tools for hands-free configuration. For Internet Explorer, there are IE Administration Resource Kits. Internet Explorer's proxy settings can also be managed with Policies, usually Group Policy Objects in a Active Directory environment. Firefox can be configured remotely with 'Mission Control", a.k.a. AutoConfig.

However, when you consider the proxy settings as part of your overall network design - e.g. in a DMZ / firewall / access control context, you'd expect proxy settings to be managed through network services such as dns and dhcp. These are, after all, your tools to manage your network : dynamic host configuration. So, how do we include proxy settings here ?

PAC : Proxy Auto Configuration

Proxy Auto Configuration is based on a javascript function that takes the requested URL by the browser, processes it, and returns instructions to the browser on where to get the requested web page (or any other file) : directly from the web server in question, or through a given proxy.

The javascript in question is called the "pac" file, and lives on a web server. Therefore, the browsers need to be configured to use an "automatic proxy configuration script", by providing the url to the proxy script in the web browser's settings (eg Firefox : Preferences : Connection Settings : Automatic Proxy Configuration URL ). Obviously, to implement Proxy Auto Configuration on a local network in order to manage the use of your local proxy server, you need to run a web server alongside your proxy server(s). This server needs to associate the .pac extension with MIME type "application/x-ns-proxy-autoconfig".

Here's a sample pac file. The function FindProxyForURL(url, host) is the main component of the script. It can simply return a proxy server for any and all requests, or you can use variables and the usual control structures (case, if .. then .. else, ...) to diversify and customize.

	function FindProxyForURL(url, host) { return "PROXY proxy.foo.com:8080; DIRECT"; }
	

PAC allows you to centrally manage the proxying (i.e. which proxy to use, and how), but still requires you to set up each client to use the appropriate Automatic Proxy Configuration URL. Once set up, it does allow central management : you can modify the pac script (on the web server) to change all the clients' behavior.

WPAD : Web Proxy Automatic Discovery

WPAD extends the PAC mechanism by providing the Automatic Proxy Configuration URL through standard network services such as dns and dhcp. This means that a client can receive its proxy settings the same way it's getting other network settings (IP address, default gateway, DNS and/or WINS servers, ...) : through dhcp - it's getting interesting now.

Wpad can work through dhcp, which is the recommended solution. As with pac, you need a wpad file on a web server. The client receives the url to this wpad script through dhcp option 252. To set this up, you need :

  1. a wpad file. This is the same javascript file as the pac file, renamed (or symlinked) to wpad.dat
  2. a web server to host the wpad.dat script. The wpad script needs to be located in the document root of the http server
  3. the web server must be configured for .dat files with a MIME type of "application/x-ns-proxy-autoconfig"
  4. a dhcp server, configured with option 252 = "http://your.server.here/wpad.dat"

Your browser needs to be configured to use proxy auto-discovery ! This kinda defeats the purpose : you still need to configure clients (once). This can be done during the initial baseline setup. The advantage over PAC is that with WPAD, the location of the wpad file is transparent to the clients : if you decide to move it to another server (or rename the server), all you need to do is adjust the path in the dhcp option, in stead of setting a new auto-configuration url on all clients. It can also be used to accommodate roaming users / laptops : they will get proxy settings relevant to the network the connect to (and get their dhcp settings from).

One caveat: Microsoft's Internet Explorer version 6.01 expects the string in option 252 to be NUL-terminated. As such, it unconditionally strips off the final character of the string before using it. Earlier versions of Microsoft's Internet Explorer do not do this. To satisfy all versions, simply explicitly include a NUL (\0)as the last octet of the string.

WPAD can work without dhcp. In this case, the client tries to derive the WPAD URL from its own DNS hostname, and just tries any probable solution. Therefore, a client set up to use wpad while no wpad file is available on your network, may end up using just any wpad file (and thus any proxy server) on any network. This opens doors to all sorts of browser hijacks and other security issues (see also wpad.com).

To avoid this attack vector, starting with Windows Server 2008, Microsoft DNS server blocks request for the (address of a) wpad host. You can see this when you run dnscmd dnsservername /info globalqueryblocklist . To undo it, reset de block list to only include istapi : dnscmd dnsservername /config /globalqueryblocklist isatap . This setting doesn't propagate to other dns servers: you need to run this on all your dns servers.

As it happens, Firefox does noet support WPAD via dhcp, only via DNS (apart from plain "PAC"). So on a network with Windows DNS servers, Firefox won't be able to do web proxy auto-discovery without the fixes shown just before.

To set up DNS-based WPAD, your DNS server needs to have a DNS entry for a host named WPAD, which is the web server where the wpad script lives. The other requirements (web server, document root, mime type, ...) still apply - you just don't need the dhcp option 252.

Configuration examples

sample Apache wpad setup

	## ?? /etc/apache2/conf/mime.types
	text/html 				html htm
	application/x-ns-proxy-autoconfig	dat
	

OR

	## ?? /etc/apache2/httpd.conf
	AddType application/x-ns-proxy-autoconfig .dat 
	

sample lighttpd wpad setup

	## /etc/lighttpd/lighttpd.conf
	mimetype.assign = (
        		".html"    =>   "text/html",
       			".txt"     =>   "text/plain",
			".wpad"    =>   "application/x-ns-proxy-autoconfig",
			[...]
	)
	[...]	

sample dhcp3d wpad setup

	## ?? /etc/dhcp3d/dhcp3d.conf
	subnet 123.123.123.123 {
		[...]

		option custom-proxy-server "http://your.server.here/wpad.dat";
		
		}
	

Alternative solutions - see which one works for you ...

		option option-252               "http://wpad.host.co.nz/proxy.pac";
	

or even

	## in the global section of your configuration:
		option wpad-url    code 252 = text;    		##(defines a new option)

	## in either the global or appropriate subnet section(s) of your configuration:

		option wpad-url    "http://wpad.my.domain.tld/proxy.pac\n";    ##(applies new option)
	

set option 252 on Microsoft Windows DHCP server

right click on the server and click "Set predefined options" to add 252 as a String value (it's not in there by default. See MS KB 252898 for more.) Then insert the wpad url value. This adds the option 252 proxy-auto-config to the list of available dhcp options. You can now enable the option in the server or scope option dropdown list.

Note re. dhcp configuration : You might want to try using an IP address instead of hostnames; apparently, DNS lookups don't always work.

Note re Internet Explorer clients : It seems that (some versions of) IE eat the last character of the URL given in option 252, because it assumes the last character is an EOL and can be disposed of. You have to append a space there (or any other character such as \n or \0) to get it working.

sample dnsmasq wpad setup

	## dnsmasq is a combined dns and dhcp server
	## 	/etc/dnsmasq.conf
	dhcp-option=252,http://your.server.here/wpad.dat 
	

Sample DNS config for DNS-based wpad

	## Add all or some of the following entries to your DNS zone file.
	wpad            IN      A       192.168.0.254
 	                IN      TXT     "service: wpad:!http://wpad.host.co.nz:80/wpad.dat"
	wpad.tcp        IN      SRV     0 0 80 wpad.host.co.nz.
	

sample wpad file

	// proxy configuration script for wpad

	function FindProxyForURL(url, host) {
		// variables
	        var ProxyAddress = "192.168.123.123"
	        var ProxyPort = "3128"
	        var strProxy = "PROXY " & ProxyAddress & ":" & ProxyPort
	
	        var localnet =  "192.168.123.0"
	        var localmask = "255.255.255.0"
	        var localdomain = "kicks.xx"
	
	        var dmznet = ""
	        var dmzmask = ""
	
	     	// local addresses / urls with subdomains of localdomain don't need proxy
		if (shExpMatch(url,"*." & localdomain & " /*")) 	{return "DIRECT";}
		if (shExpMatch(url, "*." & localdomain & ":*/*"))       {return "DIRECT";}
	
		// local and DMZ addresses go DIRECTly
		if (isInNet(host, localnet,  localmask) || (isInNet(host, dmznet,  dmzmask)){
	                	return "DIRECT";
		}
	
	       	// All other http requests go through proxy - can be adapted for other protocols
	    	// 	fallback ; go DIRECTly to the WWW if proxy doesn't respond 
		//			(will be limited by firewall web access rules if any)
        	
		if (url.substring(0, 5) == "http:") {return strProxy &  " ; DIRECT"; }
	
	     	// catch all ; go directly to the WWW (will be limited by firewall web access rules if any)
     		return "DIRECT";
  	}
	

Transparent Proxy

pac and wpad only work for clients/applications who are susceptible to it - web browsers, yes, but what about other applications ? Think outside the box : look for alternative solutions that provide equally good opportunities for dynamic, centralized remote/hands-free system and network administration. Eg: for apt, the http/ftp software download and install tool on Debian Linux systems, there's apt-proxy. Network applications started from a Linux console can also get proxy settings from the user profile, eg.

	## /etc/profile  - systemwide user profile

	http_proxy=http://proxy.mydomain.com:8080/
	https_proxy=https://proxy.mydomain.com:8080/

	export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE INPUTRC http_proxy https_proxy
	

Yet another way of dealing with this problem is on the network (TCP/IP) level : using packet filtering and routing, you can intercept packets with a given destination port (eg 80: http) and redirect/reroute them to the ipaddress:portnumber of your proxy server, which then should fetch the required data, and send them back to the client. In this approach, the client is not configured to use a proxy, and is unaware of the redirection. This can be accomplished with eg. iptables (packet filtering and redirection) and squid (proxy server). You need to configure squid to specifically deal with this sort of redirected requests.

Needs a closer look. Start here: Transparent Proxy HOWTO

Socks

Socks is a network/application protocol for a transparent, multi-application proxy server. Most proxy-servers are application-specific, eg Squid : http and ftp proxy, ...; SOCKS can proxy (just about) any application, and can thus be used to manage access to remote servers through a firewall.

A SOCKS server can be used in combination with WPAD : WPAD scripts can return "SOCKS" (in stead of "DIRECT" or a proxy-url).

References

RFC 3040 : Internet Web Replication & Caching Taxonomy
Deals with all sorts of http proxy issues, including pac and wpad
Frequently Given Answers : Web Browser Auto Proxy Configuration
In depth article on pac and wpad, including a discussion of some security issues and other caveats
Wikipedia : Proxy Auto Config
quick start intro
Wikipedia : Web Proxy Auto Discovery
quick start intro
Linux WPAD config : http://www.wlug.org.nz/WPAD
Linux hands-on WPAD configuration, presented by the Waikato Linux User Group. Detailed and accurate.
Microsoft TechnetInternet Explorer Resource Kit
PAC and WPAD setup with focus on Internet Explorer (zero) administration
Navigator Proxy Auto-Config File Format (netscape.org/mozilla)
Describes the FindProxyForURL function, and javascript functions for use in PAC and WPAD scripts
Linux Documentation Project : Transparent Proxy HOWTO
Generic linux howto transparent proxy with Squid

Koen Noens
July 2007