3 HTTP::AnyUA - An HTTP user agent programming interface unification
12 my $any_ua = HTTP::AnyUA->new(ua => LWP::UserAgent->new);
13 # OR: my $any_ua = HTTP::AnyUA->new(ua => Furl->new);
14 # OR: my $any_ua = HTTP::AnyUA->new(ua => HTTP::Tiny->new);
17 my $response = $any_ua->get('http://www.example.com/');
19 print "$response->{status} $response->{reason}\n";
21 while (my ($k, $v) = each %{$response->{headers}}) {
22 for (ref $v eq 'ARRAY' ? @$v : $v) {
27 print $response->{content} if length $response->{content};
29 ### Non-blocking user agents cause Future objects to be returned:
31 my $any_ua = HTTP::AnyUA->new(ua => HTTP::Tiny->new, response_is_future => 1);
32 # OR: my $any_ua = HTTP::AnyUA->new(ua => 'AnyEvent::HTTP');
33 # OR: my $any_ua = HTTP::AnyUA->new(ua => Mojo::UserAgent->new);
36 my $future = $any_ua->get('http://www.example.com/');
38 $future->on_done(sub {
41 print "$response->{status} $response->{reason}\n";
43 while (my ($k, $v) = each %{$response->{headers}}) {
44 for (ref $v eq 'ARRAY' ? @$v : $v) {
49 print $response->{content} if length $response->{content};
52 $future->on_fail(sub { print STDERR "Oh no!!\n" });
56 This module provides a small wrapper for unifying the programming
57 interfaces of several different actual user agents (HTTP clients) under
58 one familiar interface.
60 Rather than providing yet another programming interface for you to
61 learn, HTTP::AnyUA follows the HTTP::Tiny interface. This also means
62 that you can plug in any supported HTTP client (LWP::UserAgent, Furl,
63 etc.) and use it as if it were HTTP::Tiny.
65 There are a lot of great HTTP clients available for Perl, each with
66 different goals, different feature sets, and of course different
67 programming interfaces! If you're an end user, you can just pick one of
68 these clients according to the needs of your project (or personal
69 preference). But if you're writing a module that needs to interface
70 with a web server (like perhaps a RESTful API wrapper) and you want
71 your users to be able to use whatever HTTP client they want,
72 HTTP::AnyUA can help you support that!
74 It's a good idea to let the end user pick whatever HTTP client they
75 want to use, because they're the one who knows the requirements of
76 their application or script. If you're writing an event-driven
77 application, you'll need to use a non-blocking user agent like
78 Mojo::UserAgent. If you're writing a simple command-line script, you
79 may decide that your priority is to minimize dependencies and so may
80 want to go with HTTP::Tiny.
82 Unfortunately, many modules on CPAN are hardcoded to work with specific
83 HTTP clients, leaving the end user unable to use the HTTP client that
84 would be best for them. Although the end user won't -- or at least
85 doesn't need to -- use HTTP::AnyUA directly, they will benefit from
86 client choice if their third-party modules use HTTP::AnyUA or something
89 The primary goal of HTTP::AnyUA is to make it easy for module
90 developers to write HTTP code once that can work with any HTTP client
91 the end user may decide to plug in. A secondary goal is to make it easy
92 for anyone to add support for new or yet-unsupported user agents.
98 Get the user agent that was passed to "new".
102 Get and set whether or not responses are Future objects.
106 Get the backend instance. You normally shouldn't need this.
112 $any_ua = HTTP::AnyUA->new(ua => $user_agent, %attr);
113 $any_ua = HTTP::AnyUA->new($user_agent, %attr);
115 Construct a new HTTP::AnyUA.
119 $response = $any_ua->request($method, $url);
120 $response = $any_ua->request($method, $url, \%options);
122 Make a request, get a response.
124 Compare to "request" in HTTP::Tiny.
126 get, head, put, post, delete
128 $response = $any_ua->get($url);
129 $response = $any_ua->get($url, \%options);
130 $response = $any_ua->head($url);
131 $response = $any_ua->head($url, \%options);
134 Shortcuts for "request" where the method is the method name rather than
137 Compare to "get|head|put|post|delete" in HTTP::Tiny.
141 $response = $any_ua->post_form($url, $formdata);
142 $response = $any_ua->post_form($url, $formdata, \%options);
144 Does a POST request with the form data encoded and sets the
145 Content-Type header to application/x-www-form-urlencoded.
147 Compare to "post_form" in HTTP::Tiny.
151 $response = $http->mirror($url, $filepath, \%options);
152 if ($response->{success}) {
153 print "$filepath is up to date\n";
156 Does a GET request and saves the downloaded document to a file. If the
157 file already exists, its timestamp will be sent using the
158 If-Modified-Since request header (which you can override). If the
159 server responds with a 304 (Not Modified) status, the success field
160 will be true; this is usually only the case for 2XX statuses. If the
161 server responds with a Last-Modified header, the file will be updated
162 to have the same modification timestamp.
164 Compare to "mirror" in HTTP::Tiny. This version differs slightly in
165 that this returns internal exception responses (for cases like being
166 unable to write the file locally, etc.) rather than actually throwing
167 the exceptions. The reason for this is that exceptions as responses are
168 easier to deal with for non-blocking HTTP clients, and the fact that
169 this method throws exceptions in HTTP::Tiny seems like an inconsistency
174 HTTP::AnyUA->register_backend($user_agent_package => $backend_package);
175 HTTP::AnyUA->register_backend('MyAgent' => 'MyBackend'); # HTTP::AnyUA::Backend::MyBackend
176 HTTP::AnyUA->register_backend('LWP::UserAgent' => '+SpecialBackend'); # SpecialBackend
178 Register a backend for a new user agent type or override a default
179 backend. Backend packages are relative to the HTTP::AnyUA::Backend::
180 namespace unless prefixed with a +.
182 If you only need to set a backend as a one-off thing, you could also
183 pass an instantiated backend to "new".
185 SUPPORTED USER AGENTS
191 * HTTP::AnyUA - a little bit meta, but why not?
201 Any HTTP client that inherits from one of these in a well-behaved
202 manner should also be supported.
204 Of course, there are many other HTTP clients on CPAN that HTTP::AnyUA
205 doesn't yet support. I'm more than happy to help add support for
206 others, so send me a message if you know of an HTTP client that needs
207 support. See HTTP::AnyUA::Backend for how to write support for a new
210 NON-BLOCKING USER AGENTS
212 HTTP::AnyUA tries to target the HTTP::Tiny interface, which is a
213 blocking interface. This means that when you call "request", it is
214 supposed to not return until either the response is received or an
215 error occurs. This doesn't jive well with non-blocking HTTP clients
216 which expect the flow to reenter an event loop so that the request can
217 complete concurrently.
219 In order to reconcile this, a Future will be returned instead of the
220 normal hashref response if the wrapped HTTP client is non-blocking
221 (such as Mojo::UserAgent or AnyEvent::HTTP). This Future object may be
222 used to set up callbacks that will be called when the request is
223 completed. You can call "response_is_future" to know if the response is
226 This is typically okay for the end user; since they're the one who
227 chose which HTTP client to use in the first place, they should know
228 whether they should expect a Future or a direct response when they make
229 an HTTP request, but it does add some burden on you as a module writer
230 because if you ever need to examine the response, you may need to write
233 my $resp = $any_ua->get('http://www.perl.org/');
235 if ($any_ua->response_is_future) {
237 my $real_resp = shift;
238 handle_response($real_resp);
242 handle_response($resp); # response is the real response already
245 This actually isn't too annoying to deal with in practice, but you can
246 avoid it if you like by forcing the response to always be a Future.
247 Just set the "response_is_future" attribute. Then you don't need to do
248 an if-else because the response will always be the same type:
250 $any_ua->response_is_future(1);
252 my $resp = $any_ua->get('http://www.perl.org/');
254 $resp->on_done(sub { # response is always a Future
255 my $real_resp = shift;
256 handle_response($real_resp);
259 Note that this doesn't make a blocking HTTP client magically
260 non-blocking. The call to "request" will still block if the client is
261 blocking, and your "done" callback will simply be fired immediately.
262 But this does let you write the same code in your module and have it
263 work regardless of whether the underlying HTTP client is blocking or
266 The default behavior is to return a direct hashref response if the HTTP
267 client is blocking and a Future if the client is non-blocking. It's up
268 to you to decide whether or not to set response_is_future, and you
269 should also consider whether you want to expose the possibility of
270 either type of response or always returning Future objects to the end
271 user of your module. It doesn't matter for users who choose
272 non-blocking HTTP clients because they will be using Future objects
273 either way, but users who know they are using a blocking HTTP client
274 may appreciate not having to deal with Future objects at all.
276 FREQUENTLY ASKED QUESTIONS
278 How do I set up proxying, SSL, cookies, timeout, etc.?
280 HTTP::AnyUA provides a common interface for using HTTP clients, not for
281 instantiating or configuring them. Proxying, SSL, and other custom
282 settings can be configured directly through the underlying HTTP client;
283 see the documentation for your particular user agent to learn how to
284 configure these things.
286 AnyEvent::HTTP is a bit of a special case because there is no
287 instantiated object representing the client. For this particular user
288 agent, you can configure the backend to pass a default set of options
289 whenever it calls http_request. See "options" in
290 HTTP::AnyUA::Backend::AnyEvent::HTTP:
292 $any_ua->backend->options({recurse => 5, timeout => 15});
294 If you are a module writer, you should probably receive a user agent
295 from your end user and leave this type of configuration up to them.
297 Why use HTTP::AnyUA instead of some other HTTP client?
299 Maybe you shouldn't. If you're an end user writing a script or
300 application, you can just pick the HTTP client that suits you best and
301 use it. For example, if you're writing a Mojolicious app, you're not
302 going wrong by using Mojo::UserAgent; it's loaded with features and is
303 well-integrated with that particular environment.
305 As an end user, you could wrap the HTTP client you pick in an
306 HTTP::AnyUA object, but the only reason to do this is if you prefer
307 using the HTTP::Tiny interface.
309 The real benefit of HTTP::AnyUA (or something like it) is if module
310 writers use it to allow end users of their modules to be able to plug
311 in whatever HTTP client they want. For example, a module that
312 implements an API wrapper that has a hard dependency on LWP::UserAgent
313 or even HTTP::Tiny is essentially useless for non-blocking
314 applications. If the same hypothetical module had been written using
315 HTTP::AnyUA then it would be useful in any scenario.
317 Why use the HTTP::Tiny interface?
319 The HTTP::Tiny interface is simple but provides all the essential
320 functionality needed for a capable HTTP client and little more. That
321 makes it easy to provide an implementation for, and it also makes it
322 straightforward for module authors to use.
324 Marrying the HTTP::Tiny interface with Future gives us these benefits
325 for both blocking and non-blocking modules and applications.
329 This section specifies a standard set of data structures that can be
330 used to make a request and get a response from a user agent. This is
331 the specification HTTP::AnyUA uses for its programming interface. It is
332 heavily based on HTTP::Tiny's interface, and parts of this
333 specification were adapted or copied verbatim from that module's
334 documentation. The intent is for this specification to be written such
335 that HTTP::Tiny is already a compliant implementor of the specification
336 (at least as of the specification's publication date).
340 A request is a tuple of the form (Method, URL) or (Method, URL,
345 Method MUST be a string representing the HTTP verb. This is commonly
346 "GET", "POST", "HEAD", "DELETE", etc.
350 URL MUST be a string representing the remote resource to be acted upon.
351 The URL MUST have unsafe characters escaped and international domain
352 names encoded before being passed to the user agent. A user agent MUST
353 generated a "Host" header based on the URL in accordance with RFC 2616;
354 a user agent MAY throw an error if a "Host" header is given with the
359 Options, if present, MUST be a hash reference containing zero or more
360 of the following keys with appropriate values. A user agent MAY support
361 more options than are specified here.
365 The value for the headers key MUST be a hash reference containing zero
366 or more HTTP header names (as keys) and header values. The value for a
367 header MUST be either a string containing the header value OR an array
368 reference where each item is a string. If the value for a header is an
369 array reference, the user agent MUST output the header multiple times
370 with each value in the array.
372 User agents MAY may add headers, but SHOULD NOT replace user-specified
373 headers unless otherwise documented.
377 The value for the content key MUST be a string OR a code reference. If
378 the value is a string, its contents will be included with the request
379 as the body. If the value is a code reference, the referenced code will
380 be called iteratively to produce the body of the request, and the code
381 MUST return an empty string or undef value to indicate the end of the
382 request body. If the value is a code reference, a user agent SHOULD use
383 chunked transfer encoding if it supports it, otherwise a user agent MAY
384 completely drain the code of content before sending the request.
388 The value for the data_callback key MUST be a code reference that will
389 be called zero or more times, once for each "chunk" of response body
390 received. A user agent MAY send the entire response body in one call.
391 The referenced code MUST be given two arguments; the first is a string
392 containing a chunk of the response body, the second is an in-progress
397 A response MUST be a hash reference containg some required keys and
398 values. A response MAY contain some optional keys and values.
402 A response MUST include a success key, the value of which is a boolean
403 indicating whether or not the request is to be considered a success
404 (true is a success). Unless otherwise documented, a successful result
405 means that the operation returned a 2XX status code.
409 A response MUST include a url key, the value of which is the URL that
410 provided the response. This is the URL used in the request unless there
411 were redirections, in which case it is the last URL queried in a
416 A response MUST include a status key, the value of which is the HTTP
417 status code of the response. If an internal exception occurs (e.g.
418 connection error), then the status code MUST be 599.
422 A response MUST include a reason key, the value of which is the
423 response phrase returned by the server OR "Internal Exception" if an
424 internal exception occurred.
428 A response MAY include a content key, the value of which is the
429 response body returned by the server OR the text of the exception if an
430 internal exception occurred. This field MUST be missing or empty if the
431 server provided no response OR if the body was already provided via
436 A response SHOULD include a headers key, the value of which is a hash
437 reference containing zero or more HTTP header names (as keys) and
438 header values. Keys MUST be lowercased. The value for a header MUST be
439 either a string containing the header value OR an array reference where
440 each item is the value of one of the repeated headers.
444 A response MAY include a redirects key, the value of which is an array
445 reference of one or more responses from redirections that occurred to
446 fulfill the current request, in chronological order.
450 * PERL_HTTP_ANYUA_DEBUG - If 1, print some info useful for debugging
455 Not all HTTP clients implement the same features or in the same ways.
456 While the point of HTTP::AnyUA is to hide those differences, you may
457 notice some (hopefully) insignificant differences when plugging in
458 different clients. For example, LWP::UserAgent sets some headers on the
459 response such as client-date and client-peer that won't appear when
460 using other clients. Little differences like these probably aren't big
461 deal. Other differences may be a bigger deal, depending on what's
462 important to you. For example, some clients (like HTTP::Tiny) may do
463 chunked transfer encoding in situations where other clients won't
464 (probably because they don't support it). It's not a goal of this
465 project to eliminate all of the differences, but if you come across a
466 difference that is significant enough that you think you need to detect
467 the user agent and write special logic, I would like to learn about
472 These modules share similar goals or provide overlapping functionality:
484 Please report any bugs or feature requests on the bugtracker website
485 https://github.com/chazmcgarvey/HTTP-AnyUA/issues
487 When submitting a bug or request, please include a test-file or a patch
488 to an existing test-file that illustrates the bug or desired feature.
492 Charles McGarvey <chazmcgarvey@brokenzipper.com>
494 COPYRIGHT AND LICENSE
496 This software is copyright (c) 2017 by Charles McGarvey.
498 This is free software; you can redistribute it and/or modify it under
499 the same terms as the Perl 5 programming language system itself.