Dogcows Code - chaz/talk-bring-your-own-user-agent/blob - slides.html

   1 <!DOCTYPE html>
   2 <html><head><meta charset="utf-8"><title>Bring Your Own User-Agent</title><link rel="stylesheet" href="css/common.css"><link rel="stylesheet" href="css/slides.css"></head><body><textarea id="source">
   3
   4 class:  center, middle
   5 name:   title
   6
   7 # Bring Your Own User-Agent
   8
   9 ???
  10 Hi, I'm Chaz.
  11
  12 I want to talk to you about HTTP user agents.
  13
  14 ---
  15 class: img-map, center, middle
  16
  17 ![WebService modules on CPAN](img/webservice-on-cpan.png)
  18
  19 ???
  20
  21 There is a whole class of so-called "web service" modules on CPAN that provide a nice perly interface for interacting
  22 with... web services.
  23
  24 All kinds of things, from...
  25
  26 ---
  27 class: img-map, center, middle
  28
  29 ![Twilio module](img/twilio.png)
  30
  31 ???
  32 to
  33
  34 ---
  35 class: img-map, center, middle
  36
  37 ![Ontario Power Generation module](img/opg.png)
  38
  39 ???
  40 the Ontario Power Generation website.
  41
  42 Random, but why not?
  43
  44 ---
  45 class: img-map, center, middle
  46
  47 ![WebService modules on CPAN](img/webservice-on-cpan-circled.png)
  48
  49 ???
  50 Most of these modules congregate here (or at least they should).
  51
  52 ---
  53 class: center, middle
  54
  55 ## `WebService`
  56
  57 ???
  58 In the `WebService` namespace.
  59
  60 --
  61 ## `Net`
  62
  63 ???
  64 There are other common namespaces for this sort of thing...
  65
  66 --
  67 ## `WWW`
  68
  69 ---
  70 class: center, middle
  71
  72 ## `WebService`
  73 ## <strike>`Net`</strike>
  74 ## <strike>`WWW`</strike>
  75
  76 ???
  77 PSA: For new stuff, prefer the `WebService` namespace for such modules.
  78
  79 ---
  80 class: center, middle
  81
  82 ## `WebService` modules are useful.
  83
  84 ???
  85 Even though a lot of APIs nowadays are RESTful which may be easy to use with just your favorite user agent, these
  86 modules often take care of some of the tricky or boring details, like:
  87
  88 - authentication
  89 - paging
  90
  91 Details that are important but you don't want to read through the API documentation to figure it out.
  92
  93 ---
  94 name: but
  95 class: center, middle
  96
  97 But
  98
  99 ???
 100 And here's the problem...
 101
 102 ---
 103 class: center, middle
 104
 105 ## These modules are .major-em[**tightly-coupled**] to specific .major-em[user agents].
 106
 107 ### ;-(
 108
 109 ---
 110 class: center, middle
 111
 112 ![Dependencies](img/deps1.png)
 113 ![Dependencies](img/deps2.png)
 114 ![Dependencies](img/deps3.png)
 115 ![Dependencies](img/deps4.png)
 116 ![Dependencies](img/deps5.png)
 117
 118 ???
 119 Most of them use `LWP` or `HTTP::Tiny`.
 120
 121 ---
 122 class: center, middle
 123
 124 ## This... could be better.
 125
 126 ???
 127 Now I'm going to try to convince you that this is a problem.
 128
 129 ---
 130 class: center, middle
 131
 132 ## Problem:
 133
 134 ### How to configure the user agent...
 135
 136 ???
 137 User agents typically have defaults and so may not need to be configured, but what if the user needs the user agent to
 138 support
 139 - proxying,
 140 - caching,
 141 - TLS verification,
 142 - timeouts...
 143
 144 ---
 145 class: ex-code
 146
 147 ```perl
 148 use WebService::Whatever;
 149
 150 my $ws = WebService::Whatever->new(verify_SSL => 1);
 151
 152 $ws->timeout(10);
 153
 154 my $resp = $ws->account_info;
 155
 156 ...
 157 ```
 158
 159 ???
 160 So, one way this has been solved is for the webservice to expose all the attributes and knobs needed to configure the
 161 internal agent and just pass them through.
 162
 163 But this kinda bad because now every webservice module has to do this,
 164
 165 ...and every module will probably do it slightly differently.
 166
 167 ---
 168 class: ex-code
 169
 170 ```perl
 171 use HTTP::Tiny;
 172 use WebService::Whatever;
 173
 174 my $ua = HTTP::Tiny->new(verify_SSL => 1);
 175
 176 *my $ws = WebService::Whatever->new(ua => $ua);
 177
 178 $ua->timeout(10);
 179
 180 my $resp = $ws->account_info;
 181
 182 ...
 183 ```
 184
 185 ???
 186 Then someone remembered that dependency injection was a thing, so now we have modules that let you pass in your own user
 187 agent.
 188
 189 This is great because it lets the user set up the user agent however they want, and webservice modules writers don't
 190 need to do any of that boring work.
 191
 192 Big improvement!
 193
 194 ---
 195 class: ex-code
 196
 197 ```perl
 198 *use Mojo::UserAgent;
 199 use WebService::Whatever;
 200
 201 *my $ua = Mojo::UserAgent->new(insecure => 0);
 202
 203 my $ws = WebService::Whatever->new(ua => $ua);
 204
 205 $ua->connect_timeout(10);
 206
 207 my $resp = $ws->account_info;       # ;-(
 208
 209 ...
 210 ```
 211
 212 ???
 213 But I can't just plug in any user agent I want!
 214
 215 If the webservice module was written for `HTTP::Tiny` or any other user agent, it's expecting that I'm going to pass it
 216 the kind of user agent it wants.
 217
 218 This makes me sad.
 219
 220 ---
 221 class: center, middle
 222
 223 ## Let the user decide.
 224
 225 ???
 226 I think the user writing a program should decide which user agent to use.
 227
 228 After all, they're the ones who know the requirements of their program.
 229
 230 If I'm writing a program that needs to use the least amount of resources, and I want to use a webservice that is coupled
 231 with a *not* tiny user agent, then I'm out of luck.
 232
 233 Or if somebody wrote a great webservice module using a blocking interface like `HTTP::Tiny` or `LWP` that I want to use
 234 but my program is event-driven and so can't block, then I'm out of luck again.
 235
 236 ---
 237 ## [`Mail::SendGrid`](https://metacpan.org/pod/Mail::SendGrid) -> [`HTTP::Tiny`](https://metacpan.org/pod/HTTP::Tiny)
 238 ## [`Mojo::Sendgrid`](https://metacpan.org/pod/Mojo::Sendgrid) -> [`Mojo::UserAgent`](https://metacpan.org/pod/Mojo::UserAgent)
 239 ## [`WebService::SendGrid`](https://metacpan.org/pod/WebService::SendGrid) -> [`Net::Curl::Simple`](https://metacpan.org/pod/Net::Curl::Simple)
 240 ## ...
 241
 242 ???
 243 The solution we've come up with so far is to just implement a new webservice module for each type of user agent that
 244 anyone cares to use.
 245
 246 Wasted effort. :-(
 247
 248 ---
 249 class: center, middle
 250
 251 ## There's a better way.
 252
 253 ???
 254 What we need is user agent **adapter**.
 255
 256 As in, the **adapter pattern**, which is the same pattern we generally use for the myriad "Any" modules.
 257
 258 We need something that has an common inteface that module writers can code against, and then adapters to transform the
 259 request and response appropriately for whatever real user agent is wanted.
 260
 261 So yeah, this isn't an original idea of mine.
 262
 263 ---
 264 ## [`HTTP::Any`](https://metacpan.org/pod/HTTP::Any)
 265
 266 ???
 267 I searched CPAN and found just such a thing!
 268
 269 --
 270 #### But it has some .major-em[fatal flaws]...
 271
 272 ???
 273 in my opinion. (No offense to this module's author.)
 274
 275 --
 276 ### 1. It provides its own *new* interface.
 277
 278 ???
 279 - Okay, this one's not fatal.
 280 - Nobody wants to learn yet another user agent interface.
 281 - And it's a callback interface in order to support non-blocking user agents.
 282
 283 But having to set callback functions if you're not actually using a non-blocking user agent is kinda clunky.
 284
 285 --
 286 ### 2. It doesn't support many user agents.
 287
 288 ???
 289 - `LWP`
 290 - `Curl`
 291 - `AnyEvent`
 292
 293 --
 294 ### 3. It doesn't actually provide a common interface.
 295
 296 ???
 297 so it's not really usable as an adapter.
 298
 299 There were a couple other potential solutions on CPAN I found, but none of them overcome all of these problems.
 300
 301 Some of the modules that look like they might work at face value, actually are aiming to solve the opposite problem;
 302 that is, when the user doesn't care what user agent is used, just find one and use it.
 303
 304 ---
 305 class: center, middle
 306
 307 ## I wrote a module.
 308
 309 ---
 310 class: center, middle
 311
 312 ## [`HTTP::AnyUA`](https://metacpan.org/pod/HTTP::AnyUA)
 313
 314 ???
 315 This one is different because it has a "UA" at the end (for "user agent").
 316
 317 It also solves the problems I had with the other module.
 318
 319 ---
 320 ## [`HTTP::AnyUA`](https://metacpan.org/pod/HTTP::AnyUA)
 321
 322 ### 1. Uses the `HTTP::Tiny` interface.
 323
 324 ???
 325 - So not much new to learn.
 326 - And it doesn't make you use a callback interface if your user agent is non-blocking.
 327
 328 If your webservice module already uses `HTTP::Tiny`, this is *almost* a drop-in replacement.
 329
 330 --
 331 ### 2. Supports at least six user agents.
 332
 333 .col[
 334 - [`AnyEvent::HTTP`](https://metacpan.org/pod/AnyEvent::HTTP)
 335 - [`Furl`](https://metacpan.org/pod/Furl)
 336 - [`HTTP::Tiny`](https://metacpan.org/pod/HTTP::Tiny)
 337 ]
 338 .col[
 339 - [`LWP::UserAgent`](https://metacpan.org/pod/LWP::UserAgent)
 340 - [`Mojo::UserAgent`](https://metacpan.org/pod/Mojo::UserAgent)
 341 - [`Net::Curl::Easy`](https://metacpan.org/pod/Net::Curl::Easy)
 342 ]
 343
 344 ???
 345 Plus any user agent that inherits from any of these in a well-behaved manner should also work.
 346
 347 It's pretty easy to support for other user agents.
 348
 349 --
 350 ### 3. Provides a *common* interface.
 351
 352 ???
 353 The interface, like I said, is the `HTTP::Tiny` interface.
 354
 355 ---
 356 class: ex-code
 357
 358 ```perl
 359 has ua => (                 # <-- user agent
 360     is       => 'ro',
 361     required => 1,
 362 );
 363
 364 has any_ua => (
 365     is       => 'lazy',
 366     default  => sub {
 367         my $self = shift;
 368         require HTTP::AnyUA;
 369         HTTP::AnyUA->new(ua => $self->ua);
 370     },
 371 );
 372 ```
 373
 374 ???
 375 A webservice module implementing this looks something like this.
 376 - Allow the user to pass in a useragent. You could also default to `HTTP::Tiny` or something if you wanted the attribute
 377   to be optional.
 378 - Then you construct an `HTTP::AnyUA` instance and pass it the useragent.
 379
 380 ---
 381 class: ex-code
 382
 383 ```perl
 384 sub account_info {
 385     my $self = shift;
 386
 387 *   my $resp = $self->any_ua->get(
 388         $self->base_url . '/account',
 389         {
 390             headers => {
 391                 authorization => $self->auth,
 392             },
 393         },
 394     );
 395
 396     return $resp;
 397 }
 398 ```
 399
 400 ???
 401 The webservice methods then use the `HTTP::AnyUA` to make requests using the same args and response that `HTTP::Tiny`
 402 has.
 403
 404 ---
 405 class: ex-code
 406
 407 ```perl
 408 my $ua = HTTP::Tiny->new;
 409
 410 my $ws = WebService::Whatever->new(ua => $ua);
 411
 412 my $resp = $ws->account_info;
 413 ```
 414
 415 ???
 416 A **user** of the webservice module would look like this.
 417
 418 You just provide the useragent to the webservice.
 419
 420 ---
 421 class: ex-code
 422
 423 ```perl
 424 my $ua = LWP::UserAgent->new;
 425
 426 my $ws = WebService::Whatever->new(ua => $ua);
 427
 428 my $resp = $ws->account_info;
 429 ```
 430
 431 ---
 432 class: ex-code
 433
 434 ```perl
 435 my $ua = Mojo::UserAgent->new;
 436
 437 my $ws = WebService::Whatever->new(ua => $ua);
 438
 439 my $resp = $ws->account_info;
 440 ```
 441
 442 ---
 443 class: ex-code
 444
 445 ```perl
 446 my $ua = 'AnyEvent::HTTP';
 447
 448 my $ws = WebService::Whatever->new(ua => $ua);
 449
 450 my $resp = $ws->account_info;
 451 ```
 452
 453 ---
 454 class: ex-code
 455
 456 ```perl
 457 my $ua = 'AnyEvent::HTTP';
 458
 459 my $ws = WebService::Whatever->new(ua => $ua);
 460
 461 my $resp = $ws->account_info;
 462
 463 # {
 464 #     success => 1,
 465 #     url     => "https://whatever/account"
 466 #     status  => 200,
 467 #     reason  => "OK",
 468 #     content => "{...}",
 469 #     headers => {...},
 470 # }
 471 ```
 472
 473 ???
 474 The response from `HTTP::AnyUA` always looks like an `HTTP::Tiny` response, regardless of which user agent the user
 475 provides.
 476
 477 In this case, my "whatever" webservice is just passing the raw response back to the user, but a more useful service will
 478 decode the response content.
 479
 480 And, in the case that the user provides a non-blocking user agent, then instead of returning a hashref with the normal
 481 `HTTP::Tiny` response, it returns a `Future` object that resolves to a hashref with the normal `HTTP::Tiny` response. So
 482 you know what to expect.
 483
 484 ---
 485 class: center, middle
 486
 487 ![HTTP::AnyUA diagram](img/http-anyua-diagram.svg)
 488
 489 ???
 490 I think this is pretty cool already, but I'll show you one more thing before I get kicked off that's even cooler...
 491
 492 When you have a request response pipeline that is shaped like this, it begs to have...
 493
 494 ---
 495 class: center, middle
 496
 497 ![HTTP::AnyUA with middleware diagram](img/http-anyua-middleware-diagram.svg)
 498
 499 ???
 500 Just like in PSGI where you can have middleware components between whatever server handler and the app, you can do the
 501 same sort of thing for the client.
 502
 503 You can write components that work for any user agent and plug them in.
 504
 505 I've written only a couple components,
 506 - one to time or profile each request and
 507 - another to ensure a proper 'content-length' header is set.
 508
 509 Middleware components can do anything, even short-circuit and not actually call the user agent.
 510
 511 I started writing a caching component, but it's taking me awhile to write because I do want it to be
 512 `RFC-7234`-compliant (and my interest jumps around), but it will be cool because not every user agent has a decent
 513 cache.
 514
 515 With HTTP::AnyUA, I just need to implement the cache once and it works for all user agents that can be plugged into this
 516 pipeline.
 517
 518 ---
 519 class: center, middle
 520
 521 ## Conclusion
 522
 523 ???
 524 If you're writing a module that needs to *use* an HTTP user agent but otherwise has no reason to care what the user
 525 agent actually is, consider using `HTTP::AnyUA` or something like it (instead of coupling directly with a user agent).
 526
 527 It will make your webservice module usable by more people.
 528
 529 ---
 530 class: center, middle
 531 name:  last
 532
 533 ### Thanks.
 534
 535 </textarea><script src="https://gnab.github.io/remark/downloads/remark-latest.min.js"></script><script>var slideshow = remark.create({countIncrementalSlides: true, highlightLanguage: '', highlightLines: true, highlightStyle: 'hybrid', ratio: '16:9', /*slideNumberFormat: '',*/ navigation: {scroll: false, touch: false, click: false}})</script><script src="js/common.js"></script><script src="js/slides.js"></script></body></html>
 536 <!-- vim: set ts=4 sts=4 sw=4 tw=120 et ft=markdown nowrap: -->