2 <html><head><meta charset=
"utf-8"><title>Bring Your Own User-Agent
</title><link rel=
"stylesheet" href=
"css/common.css"><link rel=
"stylesheet" href=
"css/slides.css"></head><body><textarea id=
"source">
7 # Bring Your Own User-Agent
12 I want to talk to you about HTTP user agents.
14 Some time ago I had an idea and I wrote a module and put it up on CPAN.
16 I don't think anyone knows it exists because I didn't promote it at all. I didn't even really talk to anyone about it.
17 I'm not a very social person.
19 Anyway, I think it *might* actually be a good idea so I'm going to share it now.
27 So here's the problem that I wanted to solve.
30 class: img-map, center, middle
32 ![WebService modules on CPAN](img/webservice-on-cpan.png)
36 There is a whole class of so-called "web service" modules on CPAN that provide a nice perly interface for interacting
39 All kinds of things, from...
42 class: img-map, center, middle
44 ![Twilio module](img/twilio.png)
50 class: img-map, center, middle
52 ![Ontario Power Generation module](img/opg.png)
55 the Ontario Power Generation website.
58 class: img-map, center, middle
60 ![WebService modules on CPAN](img/webservice-on-cpan-circled.png)
63 Most of these modules congregate here.
80 ##
<strike>`Net`
</strike>
81 ##
<strike>`WWW`
</strike>
84 PSA: For new stuff, prefer the `WebService` namespace for such modules.
89 ## `WebService` modules are useful.
92 Even though a lot of APIs nowadays are RESTful which may be easy to use with just your favorite user agent, these
93 modules often take care of some of the tricky or boring details, like:
98 Details that are important but you don't want to read through the API documentation to figure it out.
101 class: center, middle
106 And here's the problem...
109 class: center, middle
111 ## These modules are **tightly-coupled** to specific user agents.
116 class: center, middle
118 ![Dependencies](img/deps1.png)
119 ![Dependencies](img/deps2.png)
120 ![Dependencies](img/deps3.png)
121 ![Dependencies](img/deps4.png)
122 ![Dependencies](img/deps5.png)
125 Most of them use `LWP` or `HTTP::Tiny`.
128 class: center, middle
130 ## This has problems.
133 Now I'm going to try to convince you that this is a problem.
136 class: center, middle
140 ### How to configure the user agent...
143 User agents typically have defaults and so may not need to be configured, but what if the user needs the user agent to
144 support proxying, caching, TLS, or shorter timeouts?
146 If the webservice package is *composing* (or wrapping) a user agent, then the webservice package needs to expose all of
147 the ways that the user agent can be configured.
153 use WebService::Whatever;
155 my $ws = WebService::Whatever-
>new(verify_SSL =
> 1);
159 my $resp = $ws-
>account_info;
165 So, one way this has been solved is for the webservice to expose all the attributes and knobs needed to configure the
175 use WebService::Whatever;
177 my $ua = HTTP::Tiny-
>new(verify_SSL =
> 1);
179 *my $ws = WebService::Whatever-
>new(ua =
> $ua);
183 my $resp = $ws-
>account_info;
189 So someone remembered that dependency injection was a thing, so now we have modules that let you pass in your own user
198 *use Mojo::UserAgent;
199 use WebService::Whatever;
201 *my $ua = Mojo::UserAgent-
>new(insecure =
> 0);
203 my $ws = WebService::Whatever-
>new(ua =
> $ua);
205 $ua-
>connect_timeout(
10);
207 my $resp = $ws-
>account_info; # ;-(
213 But I can't just plug in any user agent I want! If the webservice module was written for `HTTP::Tiny` or any other user
214 agent, it's expecting that I'm going to pass it the kind of user agent it wants.
219 class: center, middle
221 ## Let the user decide.
224 I think the user writing a program should decide which user agent to use.
226 After all, they're the ones who know what the requirements of their app are.
228 If I'm writing a program that needs to use the least amount of resources, and I want to use a webservice that is coupled
229 with a *not* tiny user agent, then I'm out of luck.
231 Or if somebody wrote a great webservice module using a blocking interface like `HTTP::Tiny` or `LWP` that I want to use
232 but my program is event-driven and so can't block, then I'm out of luck again.
234 So then what, are we just going to write separate webservice modules for each user agent?
238 ## [`Mail::SendGrid`](https://metacpan.org/pod/Mail::SendGrid) -
> [`HTTP::Tiny`](https://metacpan.org/pod/HTTP::Tiny)
239 ## [`Mojo::Sendgrid`](https://metacpan.org/pod/Mojo::Sendgrid) -
> [`Mojo::UserAgent`](https://metacpan.org/pod/Mojo::UserAgent)
240 ## [`WebService::SendGrid`](https://metacpan.org/pod/WebService::SendGrid) -
> [`Net::Curl::Simple`](https://metacpan.org/pod/Net::Curl::Simple)
243 Yeah, that's exactly what's up.
245 What we need is user agent adapter (as in the adapter pattern).
247 Something that has an inteface that module writers can code against and then translates the request and response
248 appropriately for whatever real user agent is provided.
252 ## [`HTTP::Any`](https://metacpan.org/pod/HTTP::Any)
255 I searched CPAN and found just such a thing!
258 #### But it has fatal flaws...
261 in my opinion. (No offense to this module's author.)
264 ###
1. It provides its own *new* interface.
267 - And nobody wants to learn yet another user agent interface.
268 - And it's a callback interface in order to support non-blocking user agents,
270 But having to set callback functions if you're not actually using a non-blocking user agent is kinda clunky.
273 ###
2. It doesn't support many user agents.
276 only `LWP`, `AnyEvent`, and `Curl`.
279 ###
3. It doesn't actually provide a common interface.
282 so it's not really usable as an adapter.
285 class: center, middle
287 ## I wrote a module to fix these problems.
290 class: center, middle
292 ## [`HTTP::AnyUA`](https://metacpan.org/pod/HTTP::AnyUA)
295 This one is different because it has a "UA" at the end.
297 It's also a true HTTP user agent adapter providing a common interface.
300 ## [`HTTP::AnyUA`](https://metacpan.org/pod/HTTP::AnyUA)
302 ###
1. Uses the `HTTP::Tiny` interface.
305 - So not much new to learn.
306 - And you don't have to use callbacks if your user agent is non-blocking.
309 ###
2. Supports at least six user agents.
312 - [`AnyEvent::HTTP`](https://metacpan.org/pod/AnyEvent::HTTP)
313 - [`Furl`](https://metacpan.org/pod/Furl)
314 - [`HTTP::Tiny`](https://metacpan.org/pod/HTTP::Tiny)
317 - [`LWP::UserAgent`](https://metacpan.org/pod/LWP::UserAgent)
318 - [`Mojo::UserAgent`](https://metacpan.org/pod/Mojo::UserAgent)
319 - [`Net::Curl::Easy`](https://metacpan.org/pod/Net::Curl::Easy)
323 Plus any user agent that inherits from any of these in a well-behaved manner should also work.
326 ###
3. Provides a *common* interface.
329 which, like I said, is the `HTTP::Tiny` interface.
345 HTTP::AnyUA-
>new(ua =
> $self-
>ua);
351 A webservice module implementing this looks something like this.
352 - Allow the user to pass in a useragent. You could also default to `HTTP::Tiny` or something if you wanted the attribute
354 - Then you construct an `HTTP::AnyUA` instance and pass it the useragent.
363 * my $resp = $self-
>any_ua-
>get(
364 $self-
>base_url . '/account',
367 authorization =
> $self-
>auth,
377 The webservice methods then use the `HTTP::AnyUA` to make requests using the same args and response that `HTTP::Tiny`
384 my $ua = HTTP::Tiny-
>new;
386 my $ws = WebService::Whatever-
>new(ua =
> $ua);
388 my $resp = $ws-
>account_info;
392 A **user** of the webservice module would look like this.
394 You just provide the useragent to the webservice.
400 my $ua = LWP::UserAgent-
>new;
402 my $ws = WebService::Whatever-
>new(ua =
> $ua);
404 my $resp = $ws-
>account_info;
411 my $ua = Mojo::UserAgent-
>new;
413 my $ws = WebService::Whatever-
>new(ua =
> $ua);
415 my $resp = $ws-
>account_info;
422 my $ua = 'AnyEvent::HTTP';
424 my $ws = WebService::Whatever-
>new(ua =
> $ua);
426 my $resp = $ws-
>account_info;
433 my $ua = 'AnyEvent::HTTP';
435 my $ws = WebService::Whatever-
>new(ua =
> $ua);
437 my $resp = $ws-
>account_info;
441 # url =
> "https://whatever/account"
444 # content =
> "{...}",
450 The response from `HTTP::AnyUA` always looks like an `HTTP::Tiny` response, regardless of which user agent the user
453 In this case, my "whatever" webservice is just passing the raw response back to the user, but a more useful service will
454 decode the response content.
456 And, in the case that the user provides a non-blocking user agent, then instead of returning a hashref with the normal
457 `HTTP::Tiny` response, it returns a `Future` object that resolves to a hashref with the normal `HTTP::Tiny` response. So
458 you know what to expect.
461 class: center, middle
463 ![HTTP::AnyUA diagram](img/http-anyua-diagram.svg)
466 I think this is pretty cool already, but I'll show you one more thing before I get kicked off that's even cooler...
469 class: center, middle
471 ![HTTP::AnyUA with middleware diagram](img/http-anyua-middleware-diagram.svg)
474 You can write components that work for any user agent and plug them in. I've written only a couple such components, one
475 to time the request takes and another to ensure a proper 'content-length' header is set.
477 Middleware components can do anything, even short-circuit and not actually call the user agent.
479 I started writing a caching component. This middleware is taking me awhile to write because I want it to be
480 `RFC-
7234`-compliant (and my interest jumps around), but it would be cool because not every user agent has a decent
483 With HTTP::AnyUA, I just need to implement the cache once and it works for all of them.
486 class: center, middle
491 If you're writing a module that needs to *use* an HTTP user agent but otherwise has no reason to prefer one over
492 another, consider using `HTTP::AnyUA` or something like it.
495 class: center, middle
500 </textarea><script src=
"https://gnab.github.io/remark/downloads/remark-latest.min.js"></script><script>var slideshow = remark.create({countIncrementalSlides: true, highlightLanguage: '', highlightLines: true, highlightStyle: 'hybrid', ratio: '
16:
9', /*slideNumberFormat: '',*/ navigation: {scroll: false, touch: false, click: false}})
</script><script src=
"js/common.js"></script><script src=
"js/slides.js"></script></body></html>
501 <!-- vim: set ts=4 sts=4 sw=4 tw=120 et ft=markdown nowrap: -->