Dogcows Code - chaz/talk-bring-your-own-user-agent/blob - slides.html

   1 <!DOCTYPE html>
   2 <html><head><meta charset="utf-8"><title>Bring Your Own User-Agent</title><link rel="stylesheet" href="css/common.css"><link rel="stylesheet" href="css/slides.css"></head><body><textarea id="source">
   3
   4 class:  center, middle
   5 name:   title
   6
   7 # Bring Your Own User-Agent
   8
   9 Charles McGarvey
  10
  11 ???
  12 Hi, I'm Chaz.
  13
  14 I want to talk to you about HTTP user agents.
  15
  16 ---
  17 class: img-map, center, middle
  18
  19 ![WebService modules on CPAN](img/webservice-on-cpan.png)
  20
  21 ???
  22
  23 There is a whole class of so-called "web service" modules on CPAN that provide a nice perly interface for interacting
  24 with... web services.
  25
  26 All kinds of things, from...
  27
  28 ---
  29 class: img-map, center, middle
  30
  31 ![Twilio module](img/twilio.png)
  32
  33 ???
  34 to
  35
  36 ---
  37 class: img-map, center, middle
  38
  39 ![Ontario Power Generation module](img/opg.png)
  40
  41 ???
  42 the Ontario Power Generation website.
  43
  44 Random, but why not?
  45
  46 ---
  47 class: img-map, center, middle
  48
  49 ![WebService modules on CPAN](img/webservice-on-cpan-circled.png)
  50
  51 ???
  52 Most of these modules congregate here (or at least they should).
  53
  54 ---
  55 class: center, middle
  56
  57 ## `WebService`
  58
  59 ???
  60 In the `WebService` namespace.
  61
  62 --
  63 ## `Net`
  64
  65 ???
  66 There are other common namespaces for this sort of thing...
  67
  68 --
  69 ## `WWW`
  70
  71 ---
  72 class: center, middle
  73
  74 ## `WebService`
  75 ## <strike>`Net`</strike>
  76 ## <strike>`WWW`</strike>
  77
  78 ???
  79 PSA: For new stuff, prefer the `WebService` namespace for such modules.
  80
  81 ---
  82 class: center, middle
  83
  84 ## `WebService` modules are useful.
  85
  86 ???
  87 Even though a lot of APIs nowadays are RESTful which may be easy to use with just your favorite user agent, these
  88 modules often take care of some of the tricky or boring details, like:
  89
  90 - authentication
  91 - paging
  92
  93 Details that are important but you don't want to read through the API documentation to figure it out.
  94
  95 ---
  96 name: but
  97 class: center, middle
  98
  99 But
 100
 101 ???
 102 And here's the problem...
 103
 104 ---
 105 class: center, middle
 106
 107 ## These modules are .major-em[**tightly-coupled**] to specific .major-em[user agents].
 108
 109 ### ;-(
 110
 111 ---
 112 class: center, middle
 113
 114 ![Dependencies](img/deps1.png)
 115 ![Dependencies](img/deps2.png)
 116 ![Dependencies](img/deps3.png)
 117 ![Dependencies](img/deps4.png)
 118 ![Dependencies](img/deps5.png)
 119
 120 ???
 121 Most of them use `LWP` or `HTTP::Tiny`.
 122
 123 ---
 124 class: center, middle
 125
 126 ## This... could be better.
 127
 128 ???
 129 Now I'm going to try to convince you that this is a problem.
 130
 131 ---
 132 class: center, middle
 133
 134 ## Problem:
 135
 136 ### How to configure the user agent...
 137
 138 ???
 139 User agents typically have defaults and so may not need to be configured, but what if the user needs the user agent to
 140 support
 141 - proxying,
 142 - caching,
 143 - TLS verification,
 144 - timeouts...
 145
 146 ---
 147 class: ex-code
 148
 149 ```perl
 150 use WebService::Whatever;
 151
 152 my $ws = WebService::Whatever->new(verify_SSL => 1);
 153
 154 $ws->timeout(10);
 155
 156 my $resp = $ws->account_info;
 157
 158 ...
 159 ```
 160
 161 ???
 162 So, one way this has been solved is for the webservice to expose all the attributes and knobs needed to configure the
 163 internal agent and just pass them through.
 164
 165 But this kinda bad because now every webservice module has to do this,
 166
 167 ...and every module will probably do it slightly differently.
 168
 169 ---
 170 class: ex-code
 171
 172 ```perl
 173 use HTTP::Tiny;
 174 use WebService::Whatever;
 175
 176 my $ua = HTTP::Tiny->new(verify_SSL => 1);
 177
 178 *my $ws = WebService::Whatever->new(ua => $ua);
 179
 180 $ua->timeout(10);
 181
 182 my $resp = $ws->account_info;
 183
 184 ...
 185 ```
 186
 187 ???
 188 Then someone remembered that dependency injection was a thing, so now we have modules that let you pass in your own user
 189 agent.
 190
 191 This is great because it lets the user set up the user agent however they want, and webservice modules writers don't
 192 need to do any of that boring work.
 193
 194 Big improvement!
 195
 196 ---
 197 class: ex-code
 198
 199 ```perl
 200 *use Mojo::UserAgent;
 201 use WebService::Whatever;
 202
 203 *my $ua = Mojo::UserAgent->new(insecure => 0);
 204
 205 my $ws = WebService::Whatever->new(ua => $ua);
 206
 207 $ua->connect_timeout(10);
 208
 209 my $resp = $ws->account_info;       # ;-(
 210
 211 ...
 212 ```
 213
 214 ???
 215 But I can't just plug in any user agent I want!
 216
 217 If the webservice module was written for `HTTP::Tiny` or any other user agent, it's expecting that I'm going to pass it
 218 the kind of user agent it wants.
 219
 220 This makes me sad.
 221
 222 ---
 223 class: center, middle
 224
 225 ## Let the user decide.
 226
 227 ???
 228 I think the user writing a program should decide which user agent to use.
 229
 230 After all, they're the ones who know the requirements of their program.
 231
 232 If I'm writing a program that needs to use the least amount of resources, and I want to use a webservice that is coupled
 233 with a *not* tiny user agent, then I'm out of luck.
 234
 235 Or if somebody wrote a great webservice module using a blocking interface like `HTTP::Tiny` or `LWP` that I want to use
 236 but my program is event-driven and so can't block, then I'm out of luck again.
 237
 238 ---
 239 ## [`Mail::SendGrid`](https://metacpan.org/pod/Mail::SendGrid) -> [`HTTP::Tiny`](https://metacpan.org/pod/HTTP::Tiny)
 240 ## [`Mojo::Sendgrid`](https://metacpan.org/pod/Mojo::Sendgrid) -> [`Mojo::UserAgent`](https://metacpan.org/pod/Mojo::UserAgent)
 241 ## [`WebService::SendGrid`](https://metacpan.org/pod/WebService::SendGrid) -> [`Net::Curl::Simple`](https://metacpan.org/pod/Net::Curl::Simple)
 242 ## ...
 243
 244 ???
 245 The solution we've come up with so far is to just implement a new webservice module for each type of user agent that
 246 anyone cares to use.
 247
 248 Wasted effort. :-(
 249
 250 ---
 251 class: center, middle
 252
 253 ## There's a better way.
 254
 255 ???
 256 What we need is user agent **adapter**.
 257
 258 As in, the **adapter pattern**, which is the same pattern we generally use for the myriad "Any" modules.
 259
 260 We need something that has an common inteface that module writers can code against, and then adapters to transform the
 261 request and response appropriately for whatever real user agent is wanted.
 262
 263 So yeah, this isn't an original idea of mine.
 264
 265 ---
 266 ## [`HTTP::Any`](https://metacpan.org/pod/HTTP::Any)
 267
 268 ???
 269 I searched CPAN and found just such a thing!
 270
 271 --
 272 #### But it has some .major-em[fatal flaws]...
 273
 274 ???
 275 in my opinion. (No offense to this module's author.)
 276
 277 --
 278 ### 1. It provides its own *new* interface.
 279
 280 ???
 281 - Okay, this one's not fatal.
 282 - Nobody wants to learn yet another user agent interface.
 283 - And it's a callback interface in order to support non-blocking user agents.
 284
 285 But having to set callback functions if you're not actually using a non-blocking user agent is kinda clunky.
 286
 287 --
 288 ### 2. It doesn't support many user agents.
 289
 290 ???
 291 - `LWP`
 292 - `Curl`
 293 - `AnyEvent`
 294
 295 --
 296 ### 3. It doesn't actually provide a common interface.
 297
 298 ???
 299 so it's not really usable as an adapter.
 300
 301 There were a couple other potential solutions on CPAN I found, but none of them overcome all of these problems.
 302
 303 Some of the modules that look like they might work at face value, actually are aiming to solve the opposite problem;
 304 that is, when the user doesn't care what user agent is used, just find one and use it.
 305
 306 ---
 307 class: center, middle
 308
 309 ## I wrote a module.
 310
 311 ---
 312 class: center, middle
 313
 314 ## [`HTTP::AnyUA`](https://metacpan.org/pod/HTTP::AnyUA)
 315
 316 ???
 317 This one is different because it has a "UA" at the end (for "user agent").
 318
 319 It also solves the problems I had with the other module.
 320
 321 ---
 322 ## [`HTTP::AnyUA`](https://metacpan.org/pod/HTTP::AnyUA)
 323
 324 ### 1. Uses the `HTTP::Tiny` interface.
 325
 326 ???
 327 - So not much new to learn.
 328 - And it doesn't make you use a callback interface if your user agent is non-blocking.
 329
 330 If your webservice module already uses `HTTP::Tiny`, this is *almost* a drop-in replacement.
 331
 332 --
 333 ### 2. Supports at least six user agents.
 334
 335 .col[
 336 - [`AnyEvent::HTTP`](https://metacpan.org/pod/AnyEvent::HTTP)
 337 - [`Furl`](https://metacpan.org/pod/Furl)
 338 - [`HTTP::Tiny`](https://metacpan.org/pod/HTTP::Tiny)
 339 ]
 340 .col[
 341 - [`LWP::UserAgent`](https://metacpan.org/pod/LWP::UserAgent)
 342 - [`Mojo::UserAgent`](https://metacpan.org/pod/Mojo::UserAgent)
 343 - [`Net::Curl::Easy`](https://metacpan.org/pod/Net::Curl::Easy)
 344 ]
 345
 346 ???
 347 Plus any user agent that inherits from any of these in a well-behaved manner should also work.
 348
 349 It's pretty easy to support for other user agents.
 350
 351 --
 352 ### 3. Provides a *common* interface.
 353
 354 ???
 355 The interface, like I said, is the `HTTP::Tiny` interface.
 356
 357 ---
 358 class: ex-code
 359
 360 ```perl
 361 has ua => (                 # <-- user agent
 362     is       => 'ro',
 363     required => 1,
 364 );
 365
 366 has any_ua => (
 367     is       => 'lazy',
 368     default  => sub {
 369         my $self = shift;
 370         require HTTP::AnyUA;
 371         HTTP::AnyUA->new(ua => $self->ua);
 372     },
 373 );
 374 ```
 375
 376 ???
 377 A webservice module implementing this looks something like this.
 378 - Allow the user to pass in a useragent. You could also default to `HTTP::Tiny` or something if you wanted the attribute
 379   to be optional.
 380 - Then you construct an `HTTP::AnyUA` instance and pass it the useragent.
 381
 382 ---
 383 class: ex-code
 384
 385 ```perl
 386 sub account_info {
 387     my $self = shift;
 388
 389 *   my $resp = $self->any_ua->get(
 390         $self->base_url . '/account',
 391         {
 392             headers => {
 393                 authorization => $self->auth,
 394             },
 395         },
 396     );
 397
 398     return $resp;
 399 }
 400 ```
 401
 402 ???
 403 The webservice methods then use the `HTTP::AnyUA` to make requests using the same args and response that `HTTP::Tiny`
 404 has.
 405
 406 ---
 407 class: ex-code
 408
 409 ```perl
 410 my $ua = HTTP::Tiny->new;
 411
 412 my $ws = WebService::Whatever->new(ua => $ua);
 413
 414 my $resp = $ws->account_info;
 415 ```
 416
 417 ???
 418 A **user** of the webservice module would look like this.
 419
 420 You just provide the useragent to the webservice.
 421
 422 ---
 423 class: ex-code
 424
 425 ```perl
 426 my $ua = LWP::UserAgent->new;
 427
 428 my $ws = WebService::Whatever->new(ua => $ua);
 429
 430 my $resp = $ws->account_info;
 431 ```
 432
 433 ---
 434 class: ex-code
 435
 436 ```perl
 437 my $ua = Mojo::UserAgent->new;
 438
 439 my $ws = WebService::Whatever->new(ua => $ua);
 440
 441 my $resp = $ws->account_info;
 442 ```
 443
 444 ---
 445 class: ex-code
 446
 447 ```perl
 448 my $ua = 'AnyEvent::HTTP';
 449
 450 my $ws = WebService::Whatever->new(ua => $ua);
 451
 452 my $resp = $ws->account_info;
 453 ```
 454
 455 ---
 456 class: ex-code
 457
 458 ```perl
 459 my $ua = 'AnyEvent::HTTP';
 460
 461 my $ws = WebService::Whatever->new(ua => $ua);
 462
 463 my $resp = $ws->account_info;
 464
 465 # {
 466 #     success => 1,
 467 #     url     => "https://whatever/account"
 468 #     status  => 200,
 469 #     reason  => "OK",
 470 #     content => "{...}",
 471 #     headers => {...},
 472 # }
 473 ```
 474
 475 ???
 476 The response from `HTTP::AnyUA` always looks like an `HTTP::Tiny` response, regardless of which user agent the user
 477 provides.
 478
 479 In this case, my "whatever" webservice is just passing the raw response back to the user, but a more useful service will
 480 decode the response content.
 481
 482 And, in the case that the user provides a non-blocking user agent, then instead of returning a hashref with the normal
 483 `HTTP::Tiny` response, it returns a `Future` object that resolves to a hashref with the normal `HTTP::Tiny` response. So
 484 you know what to expect.
 485
 486 ---
 487 class: center, middle
 488
 489 ![HTTP::AnyUA diagram](img/http-anyua-diagram.svg)
 490
 491 ???
 492 I think this is pretty cool already, but I'll show you one more thing before I get kicked off that's even cooler...
 493
 494 When you have a request response pipeline that is shaped like this, it begs to have...
 495
 496 ---
 497 class: center, middle
 498
 499 ![HTTP::AnyUA with middleware diagram](img/http-anyua-middleware-diagram.svg)
 500
 501 ???
 502 Just like in PSGI where you can have middleware components between whatever server handler and the app, you can do the
 503 same sort of thing for the client.
 504
 505 You can write components that work for any user agent and plug them in.
 506
 507 I've written only a couple components,
 508 - one to time or profile each request and
 509 - another to ensure a proper 'content-length' header is set.
 510
 511 Middleware components can do anything, even short-circuit and not actually call the user agent.
 512
 513 I started writing a caching component, but it's taking me awhile to write because I do want it to be
 514 `RFC-7234`-compliant (and my interest jumps around), but it will be cool because not every user agent has a decent
 515 cache.
 516
 517 With HTTP::AnyUA, I just need to implement the cache once and it works for all user agents that can be plugged into this
 518 pipeline.
 519
 520 ---
 521 class: center, middle
 522
 523 ## Conclusion
 524
 525 ???
 526 If you're writing a module that needs to *use* an HTTP user agent but otherwise has no reason to care what the user
 527 agent actually is, consider using `HTTP::AnyUA` or something like it (instead of coupling directly with a user agent).
 528
 529 It will make your webservice module usable by more people.
 530
 531 ---
 532 class: center, middle
 533 name:  last
 534
 535 ### Thanks.
 536
 537 </textarea><script src="https://gnab.github.io/remark/downloads/remark-latest.min.js"></script><script>var slideshow = remark.create({countIncrementalSlides: true, highlightLanguage: '', highlightLines: true, highlightStyle: 'hybrid', ratio: '16:9', /*slideNumberFormat: '',*/ navigation: {scroll: false, touch: false, click: false}})</script><script src="js/common.js"></script><script src="js/slides.js"></script></body></html>
 538 <!-- vim: set ts=4 sts=4 sw=4 tw=120 et ft=markdown nowrap: -->