[CTX] [ALL] HTTP proxy/server

---

HTTP proxy/server

Example: DeleGate as an HTTP proxy
firewall% delegated -P8080 SERVER=http

Then use this DeleGate from your client on the internal host, specifying "firewall:8080" as the proxy for HTTP, HTTPS (Security), FTP, Gopher, Wais, and so on.

Example: cascaded DeleGate as an HTTP proxy

firewall% delegated -P8888 SERVER=delegate RELIABLE=internal
internal% delegated -P8080 SERVER=http MASTER=firewall:8888


Run a generalist DeleGate on the firewall which only accepts request from internal host, then run a specialist on internal which use the generalist on firewall host. A generalist can be shared as an upstream DeleGate from multiple DeleGates of arbitrary protocol.

Example: DeleGate as an origin HTTP server

host# delegated -P80 SERVER=http \

MOUNT="/* /path/of/www/*" RELAY=no RELIABLE="*"

A file with a name with ".cgi" extension is treated as a CGI script. Also you can use arbitrary name of CGI scripts under a specified directory with a MOUNT like:
MOUNT="/xxx/* cgi:/path/of/cgi-bin/*"

Example: DeleGate as a CGI program

You can use DeleGate as a CGI program from a HTTP server. For example, specify in the "httpd.conf" file of your HTTP server(A) as follows.
Exec /other/* /path/of/cgi-delegate
Then write the content of the file /path/of/cgi-delegate as follows:
#!/bin/sh


delegated -Fcgi

MOUNT="/-* =" \
MOUNT="/www2/* http://wwwserv2/*" \
MOUNT="/news/* nntp://newsserv/*"


This will add a pseudo sub tree "/other/" onto server(A) including /other/www2/ which is a content of a HTTP server "wwwserv2", and /other/news/ which is a content of a NNTP server "newsserv".

HTTP Transfer Log Format

The format of PROTOLOG for HTTP protocol can be modified with a optional format specification postfixed after the LogFilename as PROTOLOG="LogFilename:format". In this case, default file name can be omitted. For example, PROTOLOG=":%X" specifies making a NCSA like extended common logfile format. The default format is PROTOLOG=":%C %D".

%C	-- common logfile format of CERN-HTTPD
%c	-- same as %C except the date is recorded in millisecond precision
%D	-- extension by DeleGate (connTime+relayTime:status)
%X	== '%C "%r" "%u"' common logfile format with Referer and User-Agent
%r	-- Referer field in the request message
%u	-- User-Agent field in the request message
%S	-- status of CONNECTion to the server (c,m,p,d,s,v)
%s	-- session Id by HTTPCONF=session
%As	-- session Id in Digest Authorization
%{field}	-- arbitrary field in the request message

The client information part in %C, which records hostname, Ident, and username by default, can be modified by AUTH="log:" parameter.

The session identifier put by "%s" is generated by specifying HTTPCONF=session:cookie option while "%As" is generated by AUTHORIZER=-dgauth option. The format of session identifier is as this:

time.pid.proc#+conn#[+req#].reqnum

For example, "031114-173045.1234.5+6+7.9" means that it is the 9th request from the client in the session started at 17:30:45 on November 14 by the process of PID=1234. The start of the session is recorded in LOGFILE as this:

11/14 17:30:45 [1234] 5+6/7: NewSession 031114-173045.1234.5+6+7.0

Note that the reqnum part may not be unique because of parallel or pipelined requests generated by a client as the owner of the session. Note that the reqnum part for "%As" may not be incremented on each request because the container of session identifier, the opaque" parameter in Digest Authentication in this case, may not be updated on each request.

HTTPCONF parameter  ==  HTTPCONF=what:conf

welcome:listOfWelcomeFiles

-- default: welcome.{dgp,shtml,html,cgi},index.{dgp,shtml,html,cgi},-dir.html
A list of index files or CGI scripts which should be used for URLs of directories ending with "/" like "/path/". This is a list of candidate file names. The list may be ended with "-dir.html" which means a built-in index generator. If the list is empty, empty data is substituted for index data.

search:pathOfSearchScript

-- default: none
The path of a CGI script to be applied for URLs with search part like "/path?search". This is a global specification applied for all URLs. Also you can specify a local search script for each MOUNT point like

MOUNT="/path2/* /root/of/path2/* search:script2".

This local specification is prior to the global one. A special local specification "search:-" can be used just to ignore the global specification for the MOUNT point.

nvserv:noauto | auto | alias | gen | none

-- default: HTTPCONF=nvserv:noauto
Automatically detects virtual servers in MOUNT parameters and notate each of them as a MOUNT rule to be applied only to a virtual server. It can be done manually by specifying "nvserv" MountOption for each MOUNT parameter. "nvserv:alias" means detecting target servers of host names with common IP-addresses and notate them as virtual servers. "nvserv:gen" means notating MOUNT parameters with "genvhost" MountOption as virtual servers. "nvserv:auto" is equivalent to "nvserv:alias,gen". The guessing can be overridden by explicitly specifying the avserv MountOption for each MOUNT parameter. "nvserv:none" disables any treatment of virtual servers which are detected automatically or specified explicitly by the "nvserv" MountOption.

methods:listOfAcceptableMethods

-- default: methods:OPTIONS,GET,HEAD,POST,PUT,...
-- See the output to LOGFILE with HTTPCONF=methods:"+"
Limit or add HTTP methods to be accepted.


Example:

HTTPCONF=methods:GET,HEAD -- accept only GET and HEAD
HTTPCONF=methods:-POST,-PUT -- don't accept POST and PUT
HTTPCONF=methods:+,NEWMETHOD1 -- add NEWMETHOD1 to be accepted
HTTPCONF=methods:* -- accept any methods


rvers:listOfAcceptableResponseVersions

-- default: rvers:HTTP
Add versions in response message from HTTP servers.

Example:

HTTPCONF=rvers:+,ICY
HTTPCONF=rvers:* -- accept any response version

post-ccx-type:listOfTypes

-- default: post-ccx-type:x-www-form-urlencoded
Specify the list of names of Content-Type of request message to which conversion by CHARCODE is applied.

Example:

HTTPCONF=post-ccx-type:+,multipart/form-data

cryptCookie:listOfCookies:cryptKey


listOfCookies == attributes[@domains]
attributes == attribute | {attribute,attribute,...}
domains == domain | {domain,domain,...}
domain == [.]domainName
cryptKey == string | %a | %P

encrypt specified attributes in a Set-Cookie response to be stored in a client, then decrypt and forward the Cookie request only to the originator of the Cookie. An attribute in a Cookie is specified as "attribute@host" or "attribute@.domain". In the former case, a cookie generated by a host is encrypted and echoed to host only. In the latter case, a cookie generated by hosts in the domain can be echoed to hosts in the domain. The special string "%a" in cryptKey is substituted by the IP-address of the client. This makes the crypted Cookie be usable only by clients on the host of the IP-address.

Example:

HTTPCONF="cryptCookie:SessionID@host1.dom1,UserID@.dom2:nanjamonja"
HTTPCONF="cryptCookie:UserID@.dom:nanjamonja"
HTTPCONF="cryptCookie:UserID@{host1.dom,host2.dom}:nanjamonja"


kill-[i][qr]head: listOfHeaders

erase header fields listed in listOfHeaders before forwarding a request/response message to server/client. "kill-qhead" is applied only to request message to server and "kill-rhead" is applied only to response message to client. If "i" is prefixed as "kill-iqhead:Pragma,Cache-Control", the specified fields are erased before DeleGate interpret it as the HTTP headers.

Example:

HTTPCONF=kill-qhead:Referer
HTTPCONF=kill-qhead:If-*,Accept-*
HTTPCONF=kill-rhead:Set-Cookie


add-[i][qr]head:name:body

add a header name:body to forwarded request/response message. Client's identity information can be inserted into body string with "%format" notation. If "i" is prefixed as "add-ihead:Pragma:no-cache", then the specified field with the body is added to the input to be interpreted by DeleGate as a HTTP header from the client or the server.

Example:

HTTPCONF="add-qhead:X-Forward-For:%a"


replace-[i][qr]head:name:body

replace headers of "name:" with "name:body" in forwarded request/response message. This is equivalent to the combination of kill-head and add-head.

Example:

HTTPCONF=replace-qhead:Referer:http://x.y.z
HTTPCONF=kill-qhead:Referer HTTPCONF=add-qhead:Referer:http://x.y.z

kill-tag: listOfTags

disable a tag listed in listOfTags when it it used in a text/html response from a server.

Example:

HTTPCONF=kill-tag:SCRIPT,APPLET


ver:1.0

act as a HTTP/1.0 client/server

svver:1.0

act as a HTTP/1.0 client against servers (send request in HTTP/1.0)

clver:1.0

act as a HTTP/1.0 server against clients (do not use chunked encoding in response)

acc-encoding:encoding [-thrugzip]

Accept-Encoding header to be sent to server. This is applied to DeleGate which does some interpretation of content with MOUNT, CHARCODE, CACHE, etc. To do such interpretation, the content (response body) is not to be encoded in a format unknown to DeleGate. "identity" specifies disabling any encoding of content in the server. "-thrugzip" specifies forwarding Accept-Encoding:gzip from client to server. If the program "gzip" is not available on the host of DeleGate (i.e. not found in LIBPATH), "identity" is sent regardlessly.

gen-encoding:encoding [gzip]

The encoding applied to the content sent from DeleGate to client. Only "gzip" is available in the current implementations. "identity" and others disable any encoding.

tout-wait-reqbody:period [30]

max. period to wait the first data in request body.

tout-in-reqbody:period [15]

max. period to wait next data in request body. (adjustable for a slow filter program at the client side)

tout-buff-reqbody:period [5]

max. period to do buffering request message to server

tout-buff-resbody:period [8]

max. period to do buffering response message to client

tout-cka:period [5]

max. period to keep a connection with the client alive. (the timeout is ten times longer for the first connection from each client host)

max-cka:number [20]

max. number of requests to be relayed on a single connection in keep-alive. (ten times larger value is given to the first connection from each client host)
This is applied to the communication over a connection of SSLtunnel by the CONNECT method too. A pair of upstream and downstream data is regarded as a pair of request and response on HTTP, then the maximum number of such pairs is restricted.

max-ckapch:number [8]

max. number of connections in keep-alive at a time for each client host.

max-reqline:length [8k]

max. length of request line to be accepted.

max-reqhead:length [12k]

max. length of request header to be accepted.

max-gw-reqline:length [512]

max. length of request line to be forwarded to another server in another protocol.

max-hops:number [20]

max. number of hops in the chain of HTTP proxies.

tout-resp:period [TIMEOUT=io]

max. period to wait response from server.

tout-pack-intvl

[0.25]

max. interval between packets relayed on SSLtunnel by the CONNECT method.

halfdup

Forbid full-duplex usage of SSLtunnel by the CONNECT method.

urlesc[:escChars] [empty]

the set of characters in request URL to be escaped by "%XX" notation before any processing. As a special case, HTTPCONF="urlesc" means HTTPCONF="urlesc:<>".

proxycontrol[:{on|off}]

consume substring following "?_?" in request URL as control information for the DeleGate; when DeleGate get request URL?_?proxycontrol, only URL part is forwarded to the server and proxycontrol part is (possibly) used by DeleGate.

cka-cfi

make connection with the client keep-alive even with external filter (FCL, FTOCL).

nolog:listOfCodeType[:connMap]

listOfCodeType == [ respCode / ][ Type [ / subType ] ]
specify response code or Content-Type of response message not to be logged in access log (PROTOLOG).

Example:

HTTPCONF="nolog:302,304,image"


xferlog:ftp

record FTP/HTTP transactions in xferlog format too.

bugs:listOfBugs

listOfBugs is a list of features to be disabled to bypass possible bugs as follows:

no-gzip ... disable Content-Encoding:gzip

no-keepalive ... disable Connection:Keep-Alive

no-keepaliveproxy ... disable Connection:Keep-Alive with client side proxy

no-chunked ... disable Transfer-Encoding:chunked

no-flush-chunk ... disable flushing response after each chunk

kill-contleng ... erase original Content-Length in chunked encoding

add-contleng ... add or update Content-Length even in chunked encoding

do-authconv ... enable Authentication conversion from client's Basic to server's Digest

bugs:thru-304

disable the conversion from "304 Not Modified" to "200 Ok" in the message as the response to a conditional request with the "If-Modified-Since" header. DeleGate with external filters tries to return a HTTP response messages with a body (with the code "200 Ok") when it is filtered (and possibly rewritten) by the filters even if the body should be returned as empty (304 Not Modified) based on the modification date (Last-Modified) of the target data. This is necessary to return data rewritten dynamically by filters, but it disables the merit by the conditional request and the "304" response. "thru-304" turns the above conversion off and let DeleGate pass through "If-Modified-Since" request from clients and the "Last-Modified" and "304 Not Modified" response from servers.

svauth:no-basic

Stop forwarding Basic Authorization (which contains cleartext password) from client to server. If the server supports Digest authentication, then the DeleGate do it by proxy of the client.

svauth:less-basic

Postpone forwarding Basic Authorization from client to server until the server requires the Basic Authentication.

session:cookie

Enable session management based on Cookie. The session identifier is passed to CFI/CGI programs in environment variable "X_COOKIE_SESSION", and can be logged into PROTOLOG.

Example:

HTTPCONF=session PROTOLOG=":%s %X"

Example:

HTTPCONF=search:/path/of/searchScript
MOUNT="/bin/* cgi:/path/of/cgi-bin/* search:-"
MOUNT="/* file:/path/of/data/*"

---