Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
H
httprl
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Labels
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Commits
Open sidebar
drupal.org
httprl
Commits
d481079e
Commit
d481079e
authored
Aug 30, 2011
by
mikeytown2
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Initial commit.
parents
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
486 additions
and
0 deletions
+486
-0
httprl.info
httprl.info
+5
-0
httprl.module
httprl.module
+481
-0
No files found.
httprl.info
0 → 100755
View file @
d481079e
name
=
HTTP
Parallel
Request
Library
description
=
Send
http
requests
out
in
parallel
in
a
blocking
or
non
-
blocking
manner
.
package
=
Performance
and
scalability
core
=
6.
x
httprl.module
0 → 100755
View file @
d481079e
<?php
/**
* @file
* HTTP Parallel Request Library module.
*
*/
/**
* Perform an HTTP request
*
* @see drupal_http_request()
*
* This is a flexible and powerful HTTP client implementation. Correctly
* handles GET, POST, PUT or any other HTTP requests.
*
* @param $url
* A string containing a fully qualified URI.
* @param array $options
* (optional) An array that can have one or more of the following elements:
* - headers: An array containing request headers to send as name/value pairs.
* - method: A string containing the request method. Defaults to 'GET'.
* - data: A string containing the request body, formatted as
* 'param=value¶m=value&...'. Defaults to NULL.
* - max_redirects: An integer representing how many times a redirect
* may be followed. Defaults to 3.
* - timeout: A float representing the maximum number of seconds the function
* call may take. The default is 30 seconds. If a timeout occurs, the error
* code is set to the HTTP_REQUEST_TIMEOUT constant.
* - context: A context resource created with stream_context_create().
* - blocking: set to FALSE to make this not care about the returned data.
* @return bool
* return value from httprl_send_request().
*/
function
httprl_request
(
$url
,
array
$options
=
array
())
{
global
$base_root
;
$result
=
new
stdClass
();
// Parse the URL and make sure we can handle the schema.
$uri
=
@
parse_url
(
$url
);
if
(
empty
(
$uri
))
{
$result
->
error
=
'unable to parse URL'
;
$result
->
code
=
-
1001
;
return
$result
;
}
if
(
!
isset
(
$uri
[
'scheme'
]))
{
$result
->
error
=
'missing schema'
;
$result
->
code
=
-
1002
;
return
$result
;
}
// Merge the default options.
$options
+=
array
(
'headers'
=>
array
(),
'method'
=>
'GET'
,
'data'
=>
NULL
,
'max_redirects'
=>
3
,
'timeout'
=>
10.0
,
'context'
=>
NULL
,
'blocking'
=>
TRUE
,
);
// Using http 1.0 so set the connection to be closed by default.
// Sending referrer as well.
$options
[
'headers'
]
+=
array
(
'Connection'
=>
'close'
,
'Referer'
=>
$base_root
.
request_uri
(),
);
// stream_socket_client() requires timeout to be a float.
$options
[
'timeout'
]
=
(
float
)
$options
[
'timeout'
];
switch
(
$uri
[
'scheme'
])
{
case
'http'
:
case
'feed'
:
$port
=
isset
(
$uri
[
'port'
])
?
$uri
[
'port'
]
:
80
;
$socket
=
'tcp://'
.
$uri
[
'host'
]
.
':'
.
$port
;
// RFC 2616: "non-standard ports MUST, default ports MAY be included".
// We don't add the standard port to prevent from breaking rewrite rules
// checking the host that do not take into account the port number.
if
(
empty
(
$options
[
'headers'
][
'Host'
]))
{
$options
[
'headers'
][
'Host'
]
=
$uri
[
'host'
];
}
if
(
$port
!=
80
)
{
$options
[
'headers'
][
'Host'
]
.
=
':'
.
$port
;
}
break
;
case
'https'
:
// Note: Only works when PHP is compiled with OpenSSL support.
$port
=
isset
(
$uri
[
'port'
])
?
$uri
[
'port'
]
:
443
;
$socket
=
'ssl://'
.
$uri
[
'host'
]
.
':'
.
$port
;
if
(
empty
(
$options
[
'headers'
][
'Host'
]))
{
$options
[
'headers'
][
'Host'
]
=
$uri
[
'host'
];
}
if
(
$port
!=
443
)
{
$options
[
'headers'
][
'Host'
]
.
=
':'
.
$port
;
}
break
;
default
:
$result
->
error
=
'invalid schema '
.
$uri
[
'scheme'
];
$result
->
code
=
-
1003
;
return
$result
;
}
if
(
empty
(
$options
[
'context'
]))
{
$fp
=
@
stream_socket_client
(
$socket
,
$errno
,
$errstr
,
$options
[
'timeout'
],
STREAM_CLIENT_ASYNC_CONNECT
|
STREAM_CLIENT_CONNECT
);
}
else
{
// Create a stream with context. Allows verification of a SSL certificate.
$fp
=
@
stream_socket_client
(
$socket
,
$errno
,
$errstr
,
$options
[
'timeout'
],
STREAM_CLIENT_ASYNC_CONNECT
|
STREAM_CLIENT_CONNECT
,
$options
[
'context'
]);
}
// Make sure the socket opened properly.
if
(
!
$fp
)
{
// When a network error occurs, we use a negative number so it does not
// clash with the HTTP status codes.
$result
->
code
=
-
$errno
;
$result
->
error
=
trim
(
$errstr
)
?
trim
(
$errstr
)
:
t
(
'Error opening socket @socket'
,
array
(
'@socket'
=>
$socket
));
return
$result
;
}
// Non blocking stream.
stream_set_blocking
(
$fp
,
0
);
// Construct the path to act on.
$path
=
isset
(
$uri
[
'path'
])
?
$uri
[
'path'
]
:
'/'
;
if
(
isset
(
$uri
[
'query'
]))
{
$path
.
=
'?'
.
$uri
[
'query'
];
}
// Merge the default headers.
$options
[
'headers'
]
+=
array
(
'User-Agent'
=>
'Drupal (+http://drupal.org/)'
,
);
// Only add Content-Length if we actually have any content or if it is a POST
// or PUT request. Some non-standard servers get confused by Content-Length in
// at least HEAD/GET requests, and Squid always requires Content-Length in
// POST/PUT requests.
$content_length
=
strlen
(
$options
[
'data'
]);
if
(
$content_length
>
0
||
$options
[
'method'
]
==
'POST'
||
$options
[
'method'
]
==
'PUT'
)
{
$options
[
'headers'
][
'Content-Length'
]
=
$content_length
;
}
// If the server URL has a user then attempt to use basic authentication.
if
(
isset
(
$uri
[
'user'
]))
{
$options
[
'headers'
][
'Authorization'
]
=
'Basic '
.
base64_encode
(
$uri
[
'user'
]
.
(
!
empty
(
$uri
[
'pass'
])
?
":"
.
$uri
[
'pass'
]
:
''
));
}
// If the database prefix is being used by SimpleTest to run the tests in a copied
// database then set the user-agent header to the database prefix so that any
// calls to other Drupal pages will run the SimpleTest prefixed database. The
// user-agent is used to ensure that multiple testing sessions running at the
// same time won't interfere with each other as they would if the database
// prefix were stored statically in a file or database variable.
$test_info
=
&
$GLOBALS
[
'drupal_test_info'
];
if
(
!
empty
(
$test_info
[
'test_run_id'
]))
{
$options
[
'headers'
][
'User-Agent'
]
=
drupal_generate_test_ua
(
$test_info
[
'test_run_id'
]);
}
$request
=
$options
[
'method'
]
.
' '
.
$path
.
" HTTP/1.0
\r\n
"
;
foreach
(
$options
[
'headers'
]
as
$name
=>
$value
)
{
$request
.
=
$name
.
': '
.
trim
(
$value
)
.
"
\r\n
"
;
}
$request
.
=
"
\r\n
"
.
$options
[
'data'
];
$result
->
request
=
$request
;
return
httprl_send_request
(
$fp
,
$url
,
$request
,
$options
);
}
/**
* Perform an HTTP request; does not wait for reply & you never will get it
* back.
*
* @see drupal_http_request()
*
* This is a flexible and powerful HTTP client implementation. Correctly
* handles GET, POST, PUT or any other HTTP requests.
*
* @param $fp
* (optional) A file pointer.
* @param $request
* (optional) A string containing the request headers to send to the server.
* @param $timeout
* (optional) An interger holding the stream timeout value.
* @return bool
* TRUE if function worked as planed.
*/
function
httprl_send_request
(
$fp
=
NULL
,
$url
=
''
,
$request
=
''
,
$options
=
''
)
{
static
$responses
=
array
();
static
$streams
=
array
();
static
$timeout
=
0
;
static
$counter
=
0
;
// Store data in a static and exit.
if
(
!
empty
(
$fp
))
{
$streams
[
$counter
]
=
$fp
;
$timeout
=
max
(
$options
[
'timeout'
],
$timeout
);
$result
=
new
stdClass
();
$result
->
url
=
$url
;
$result
->
request
=
$request
;
$result
->
status
=
'in progress'
;
$result
->
data
=
''
;
$result
->
options
=
$options
;
$result
->
chunk_size
=
1024
;
$responses
[
$counter
]
=
$result
;
$counter
++
;
return
TRUE
;
}
// Exit if there is nothing to do.
if
(
empty
(
$responses
))
{
return
FALSE
;
}
timer_start
(
__FUNCTION__
);
// Run the loop as long as we have a stream to write to.
while
(
!
empty
(
$streams
))
{
// Set the read and write vars to the streams var.
$read
=
$write
=
$streams
;
$except
=
array
();
// Do some voodoo and open all streams at once.
$n
=
stream_select
(
$read
,
$write
,
$except
,
$timeout
);
// We have some streams to write to.
if
(
!
empty
(
$n
))
{
// Readable sockets either have data for us, or are failed connection attempts.
foreach
(
$read
as
$r
)
{
$id
=
array_search
(
$r
,
$streams
);
// Do not read from the non blocking sockets.
if
(
empty
(
$responses
[
$id
]
->
options
[
'blocking'
]))
{
continue
;
}
// Read socket.
$chunk
=
fread
(
$r
,
$responses
[
$id
]
->
chunk_size
);
$responses
[
$id
]
->
data
.
=
$chunk
;
// Get stream data.
$info
=
stream_get_meta_data
(
$r
);
// See if the headers have come back yet.
if
(
empty
(
$responses
[
$id
]
->
headers
)
&&
(
strpos
(
$responses
[
$id
]
->
data
,
"
\r\n\r\n
"
)
||
strpos
(
$responses
[
$id
]
->
data
,
"
\n\n
"
)
||
strpos
(
$responses
[
$id
]
->
data
,
"
\r\r
"
)
)
)
{
httprl_parse_data
(
$responses
[
$id
]);
if
(
!
empty
(
$responses
[
$id
]
->
headers
))
{
// Stream timed out, close connection.
if
(
isset
(
$responses
[
$id
]
->
error
)
&&
$responses
[
$id
]
->
error
==
'request timed out'
)
{
fclose
(
$r
);
unset
(
$streams
[
$id
]);
$responses
[
$id
]
->
status
=
'Done.'
;
continue
;
}
// Sream was a redirect, close this connection; redirect is being followed.
if
(
!
empty
(
$responses
[
$id
]
->
redirect_url
))
{
fclose
(
$r
);
unset
(
$streams
[
$id
]);
$responses
[
$id
]
->
status
=
'Done.'
;
continue
;
}
}
// Now that we have the headers, increase the chunk size.
$responses
[
$id
]
->
chunk_size
=
32768
;
}
$alive
=
!
$info
[
'eof'
]
&&
!
$info
[
'timed_out'
]
&&
strlen
(
$chunk
);
if
(
!
$alive
)
{
if
(
$responses
[
$id
]
->
status
==
'in progress'
)
{
$responses
[
$id
]
->
status
=
'Failed to connect.'
;
}
else
{
$responses
[
$id
]
->
status
=
'Done.'
;
}
fclose
(
$r
);
unset
(
$streams
[
$id
]);
}
else
{
$responses
[
$id
]
->
status
=
'Reading data'
;
}
}
// Write to each stream if it is available.
foreach
(
$write
as
$w
)
{
$id
=
array_search
(
$w
,
$streams
);
if
(
isset
(
$streams
[
$id
])
&&
$responses
[
$id
]
->
status
==
'in progress'
)
{
$len
=
strlen
(
$responses
[
$id
]
->
request
);
$bytes
=
fwrite
(
$w
,
$responses
[
$id
]
->
request
,
$len
);
if
(
$bytes
>=
$len
)
{
$responses
[
$id
]
->
status
=
"Request sent, waiting for response."
;
}
else
{
$responses
[
$id
]
->
request
=
substr
(
$responses
[
$id
]
->
request
,
$bytes
);
}
if
(
empty
(
$responses
[
$id
]
->
options
[
'blocking'
]))
{
fclose
(
$w
);
unset
(
$streams
[
$id
]);
$responses
[
$id
]
->
status
=
'Non-Blocking request sent out. Not waiting for the response.'
;
}
}
}
}
else
{
break
;
}
}
// Put redirects into the correct request url.
$redirects
=
array
();
foreach
(
$responses
as
$id
=>
$result
)
{
if
(
!
empty
(
$result
->
redirect_url
))
{
$redirects
[
$result
->
url
]
=
array
(
'redirect_url'
=>
$result
->
redirect_url
,
'redirect_code'
=>
$result
->
redirect_code
,
'id'
=>
$id
,
);
unset
(
$responses
[
$id
]);
}
}
if
(
!
empty
(
$redirects
))
{
foreach
(
$redirects
as
$orginal_url
=>
$info
)
{
foreach
(
$responses
as
$id
=>
&
$result
)
{
if
(
$result
->
url
==
$info
[
'redirect_url'
])
{
$result
->
redirect_url
=
$info
[
'redirect_url'
];
$result
->
redirect_code
=
$info
[
'redirect_code'
];
$result
->
url
=
$orginal_url
;
unset
(
$redirects
[
$orginal_url
]);
unset
(
$info
);
break
;
}
}
}
}
// Set output.
$output
=
array
();
foreach
(
$responses
as
$id
=>
&
$result
)
{
$output
[
$result
->
url
]
=
$result
;
}
// Free memory.
$responses
=
array
();
$streams
=
array
();
$timeout
=
0
;
$counter
=
0
;
return
$output
;
}
/**
* Extract the header and meta data from the http data stream.
*
* @see drupal_http_request()
*
* @param $result
* An object from httprl_send_request.
*/
function
httprl_parse_data
(
&
$result
)
{
// If in non blocking mode, skip.
if
(
empty
(
$result
->
options
[
'blocking'
]))
{
continue
;
}
// If the headers are already parsed, skip.
if
(
!
empty
(
$result
->
headers
))
{
continue
;
}
// Parse response headers from the response body.
// Be tolerant of malformed HTTP responses that separate header and body with
// \n\n or \r\r instead of \r\n\r\n.
$response
=
$result
->
data
;
list
(
$response
,
$result
->
data
)
=
preg_split
(
"/
\r\n\r\n
|
\n\n
|
\r\r
/"
,
$response
,
2
);
$response
=
preg_split
(
"/
\r\n
|
\n
|
\r
/"
,
$response
);
// Parse the response status line.
list
(
$protocol
,
$code
,
$status_message
)
=
explode
(
' '
,
trim
(
array_shift
(
$response
)),
3
);
$result
->
protocol
=
$protocol
;
$result
->
status_message
=
$status_message
;
$result
->
headers
=
array
();
// Parse the response headers.
while
(
$line
=
trim
(
array_shift
(
$response
)))
{
list
(
$name
,
$value
)
=
explode
(
':'
,
$line
,
2
);
$name
=
strtolower
(
$name
);
if
(
isset
(
$result
->
headers
[
$name
])
&&
$name
==
'set-cookie'
)
{
// RFC 2109: the Set-Cookie response header comprises the token Set-
// Cookie:, followed by a comma-separated list of one or more cookies.
$result
->
headers
[
$name
]
.
=
','
.
trim
(
$value
);
}
else
{
$result
->
headers
[
$name
]
=
trim
(
$value
);
}
}
$responses
=
array
(
100
=>
'Continue'
,
101
=>
'Switching Protocols'
,
200
=>
'OK'
,
201
=>
'Created'
,
202
=>
'Accepted'
,
203
=>
'Non-Authoritative Information'
,
204
=>
'No Content'
,
205
=>
'Reset Content'
,
206
=>
'Partial Content'
,
300
=>
'Multiple Choices'
,
301
=>
'Moved Permanently'
,
302
=>
'Found'
,
303
=>
'See Other'
,
304
=>
'Not Modified'
,
305
=>
'Use Proxy'
,
307
=>
'Temporary Redirect'
,
400
=>
'Bad Request'
,
401
=>
'Unauthorized'
,
402
=>
'Payment Required'
,
403
=>
'Forbidden'
,
404
=>
'Not Found'
,
405
=>
'Method Not Allowed'
,
406
=>
'Not Acceptable'
,
407
=>
'Proxy Authentication Required'
,
408
=>
'Request Time-out'
,
409
=>
'Conflict'
,
410
=>
'Gone'
,
411
=>
'Length Required'
,
412
=>
'Precondition Failed'
,
413
=>
'Request Entity Too Large'
,
414
=>
'Request-URI Too Large'
,
415
=>
'Unsupported Media Type'
,
416
=>
'Requested range not satisfiable'
,
417
=>
'Expectation Failed'
,
500
=>
'Internal Server Error'
,
501
=>
'Not Implemented'
,
502
=>
'Bad Gateway'
,
503
=>
'Service Unavailable'
,
504
=>
'Gateway Time-out'
,
505
=>
'HTTP Version not supported'
,
);
// RFC 2616 states that all unknown HTTP codes must be treated the same as the
// base code in their class.
if
(
!
isset
(
$responses
[
$code
]))
{
$code
=
floor
(
$code
/
100
)
*
100
;
}
$result
->
code
=
$code
;
switch
(
$code
)
{
case
200
:
// OK
case
304
:
// Not modified
break
;
case
301
:
// Moved permanently
case
302
:
// Moved temporarily
case
307
:
// Moved temporarily
$location
=
$result
->
headers
[
'location'
];
$result
->
options
[
'timeout'
]
-=
timer_read
(
__FUNCTION__
)
/
1000
;
if
(
$result
->
options
[
'timeout'
]
<=
0
)
{
$result
->
code
=
HTTP_REQUEST_TIMEOUT
;
$result
->
error
=
'request timed out'
;
}
elseif
(
$result
->
options
[
'max_redirects'
])
{
// Redirect to the new location.
// TODO: Parse cookies from header and send them in the redirect request.
$result
->
options
[
'max_redirects'
]
--
;
$result
->
options
[
'headers'
][
'Referer'
]
=
$result
->
url
;
httprl_request
(
$location
,
$result
->
options
);
$result
->
redirect_code
=
$code
;
}
$result
->
redirect_url
=
$location
;
break
;
default
:
$result
->
error
=
$status_message
;
}
}
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment