|By Janice Hunt and Basant Kukreja, with contributions from Marina Sum, September 10, 2007|
In the HTTP/1.0 protocol, the length of a message body is determined by the
content-length header. A key difference between HTTP/1.0 and HTTP/1.1 is the introduction of chunked transfer coding in the latter. With that coding, clients that understand HTTP/1.1 can break a message body into chunks of arbitrary lengths through the header
Transfer-Encoding: Chunked, which specifies that the message contains a chunked body. HTTP/1.1 clients can take advantage of chunking in these scenarios:
Typically, chunking is used in responses but sometimes in requests, too. This article explains how Sun Java System Web Server 7.0 (henceforth, Web Server) handles chunked requests through Netscape Server API (NSAPI) plug-ins.Contents
|-||Format for Chunked Requests|
|-||Handling of Chunked Requests|
A properly formatted chunked request from a client contains chunked borders (size indicators) a data chunk with the length of the chunk. To mark the end of the message, the last chunk has a zero-length chunked border. Since all HTTP/1.1 requests with a message body must contain either a
content-length or a
transfer-encoding header, a properly formatted chunked request always contains a
Here is an example of a client request with chunked transfer coding:
PUT /dav/file.html HTTP/1.1 Content-type: text/xml; charset="utf-8" Host: host.domain Transfer-encoding: chunked Connection: close 5 abcde 0
For more information on chunked transfer coding, see the section Chunked Transfer Coding in the Hypertext Transfer Protocol--HTTP/1.1 specification.
To handle chunked requests, Web Server allocates a buffer whose size is specified in the
max-unchunk-size parameter of the
http element in the
server.xml file. The default value of that parameter is 8,192 bytes with a maximum of 2 Gbytes.
When it receives a request with a chunked message body, Web Server allocates a buffer whose size is as specified in
max-unchunk-size. If the message body size is less than the limit specified, Web Server performs the following tasks:
sn->inbufdata structureand calculates a content length.
If you send a message whose body size exceeds the limit specified, Web Server cancels the request and throws a
411 error to alert you that it cannot fit the entire request into the buffer. As a resolution, edit
server.xml file and specify a larger limit for
On occasion, some message bodies might exceed the 2-Gbyte maximum limit for
max-unchunk-size. In that case, set
0. Web Server then performs two tasks:
Web Server does not calculate the
content-length header, hence the Server Application Function (SAF) must read the data from
sn->inbuf until the end of the file (EOF).
Web Server 6.1 and later releases support input filtering. While processing the internal filter, Web Server deletes the chunked borders, so the SAF does not see them but sees only the request data.
Note: You can override
max-unchunk-size for a particular SAF with the
ChunkedRequestBufferSize obj.conf parameter.
To verify whether a message body exists, Service SAFs or input filter plug-ins check whether a
transfer-encoding parameter is present in
Note: HTTP header names are case-insensitive. However, header names in NSAPI data structures are case-sensitive and are stored in those structures in lowercase by default.
If either parameter is present, the entity body is available in the
sn->inbuf data structure. If Web Server sets a
content-length header, then the plug-in reads the
content-length bytes from
sn->inbuf. However, if the
transfer-encoding header is set with no content length, the plug-in must read the entity body until EOF.
If a plug-in works as a reverse proxy to an application server and forms a new HTTP message to a back-end application server, then the plug-in can calculate the content length and generate a
content-length header to pass in the request to the application server. Alternatively, the plug-in can choose to apply transfer coding and generate a
transfer-encoding header of its own to pass back to the application server.
transfer-encoding is a hop-by-hop header, any plug-in that serves as a reverse proxy to an application server must not automatically pass that header to the application server.
You can edit the value for the
max-unchunk-size parameter with the
set-http-prop. The following example sets
max-unchunk-size to 16,384 bytes.
wadm> get-http-prop -config=test ... max-unchunk-size=8192 ... wadm> set-http-prop --config=test max-unchunk-size=16384 CLI201 Command 'set-http-prop' ran successfully wadm> deploy-config test CLI201 Command 'deploy-config' ran successfully
To change the chunked request buffer for a particular SAF, edit the
ChunkedRequestBufferSize parameter in the
obj.conf file, for example:
Service fn="read_post" method="(POST)" ChunkedRequestBufferSize="16384"
Afterward, Web Server allocates a 16,384-byte buffer for the requests that are processed by the
read_post. For other Service or Filter SAFs, the request buffer size is determined by the
Take a look at this sample NSAPI plug-in, which processes request data. The plug-in checks for the
transfer-encoding request headers. If the
content-length header exists, the plug-in reads the
content-length bytes from
sn->inbuf. Otherwise, it reads the data from
sn->inbuf until EOF.
To run the sample plug-in:
% cc -G -IWeb_Server_install_dir
Init fn="load-modules" shlib="libreadpost.so" funcs="read_post"
<Object name="default"> Other code NameTrans fn=assign-name name="posttest" from="/posttest/*" Other code </Object> Other code <Object name="posttest"> Service fn="read_post" method="(POST)" </Object>
Finally, test the NSAPI plug-in. Here are the command and example output:
% telnet testhost.domain POST /posttest/ HTTP/1.1 Host: testhost.domain Transfer-Encoding: chunked Connection: close 4 abcd 0 HTTP/1.1 200 OK Server: Sun-Java-System-Web-Server/7.0 Date: Wed, 05 Sep 2007 02:25:08 GMT Content-type: text/html Connection: close rq->headers contains : host="host.domain" transfer-encoding="identity" accept="*/*" connection="close" content-length="4" content-length in rq->headers : 4 transfer-encoding in rq->headers : identity Trying to read content-length(4) bytes from sn->inbuf Number of bytes read successfully = 4