Plack Advent Calendar

Day 24: Wrap up

24 days have passed so fast and this is the last entry for this Plack advent calendar.

Best Practices

Plack and PSGI are still really young projects but we've already discovered a couple of suggestions and advices to write a new PSGI application or a framework.

When you write a new framework, be sure to have an access to the PSGI environment hash from end users applications or plugin developers, either directly or with an accessor method. This allows your framework to share and extend functionality with middleware components like Debug or Session.

Do not write your application logic in .psgi files using Plack::Request. It's like writing a 1000 lines of CGI script using CGI.pm, so if you think that's your favorite i won't give you any further advice, but usually you want to make your application testable and reusable by making it a class or an object. Then your .psgi code is just a few lines of code to create a PSGI application out of it and apply some middleware components.

Think twice before using Plack::App::* namespace. Plack::App namespace is for middleware components that do not act as a wrapper but rather an endpoint. Proxy, File, Cascade and URLMap are the good examples. If you write a blog application using Plack, Never call it Plack::App::Blog, okay? Name your software by what it does, not how it's written.

Explore more stuff

Most of the Plack gangs use github for the source control and searching for repositories with "Plack" would give you a fresh look of what would look like an interesting idea. You can also search for modules on CPAN with Plack or PSGI. I keep track of good blog posts and stuff on delicious, so you can see them tagged with psgi or Plack.

Getting in touch with the dev team

Again, Plack is a fairly young project. It's just been 3 months since we gave this project a birth. There are many things that could get more improvements, so if you come across one of them, don't stop there. Let us know what you think is a problem, give us an insight how it could be improved, or if you're impatient, fork the project on github and send us patches.

We're chatting on IRC channel #plack on irc.perl.org and there's a mailing list and an issue tracker on github to communicate with us.

On a final note...

It's been an interesting experiment of writing 24 articles for 24 days, and I'm glad that I finished this myself. Next year, i'm looking forward to having your own advent entries to make the community based advent calendar.

I wish you a Very Merry Christmas and a Happy New Year.

Day 23: Write your own middleware

Let's finish up this middleware discovery with "Do It Yourself" tutorial now.

Writing Middleware

PSGI middleware behaves like a normal PSGI application but wraps the original PSGI application, so from the server it looks like an application but from an application it looks like a server (plays both sides).

A simple middleware that fakes HTTP user-agent would be like this:

# Wrapped application
my $app = sub {
    my $env = shift;
    my $who = $env->{HTTP_USER_AGENT} =~ /Mobile Safari/ ? 'iPhone' : 'non-iPhone';
    return [ 200, ['Content-Type','text/html'], ["Hello $who"] ];
};

# Middleware to wrap $app
my $mw = sub {
    my $env = shift;
    $env->{HTTP_USER_AGENT} .= " (Mobile Safari)";
    $app->($env);
};

The app would display "Hello iPhone" only if a request comes with iPhone browser (Mobile Safari), but the middleware adds that phrase to all incoming requests, so if you run this application and open the page with any browsers, you'll always see "Hello iPhone". And the default Access Log would say:

127.0.0.1 - - [23/Dec/2009 12:34:31] "GET / HTTP/1.1" 200 12 "-" "Mozilla/5.0 
(Macintosh; U; Intel Mac OS X 10_6_2; en-us) AppleWebKit/531.21.8 (KHTML, like
Gecko) Version/4.0.4 Safari/531.21.10 (Mobile Safari)"

You can see " (Mobile Safari)" is added to the tail of User-Agent string.

Make it a reusable Middleware

So that was a good example of writing your own middleware in .psgi. If it is one-time middleware that you can quickly whip up then that's great, but you often want to make it generic enough or reusable in other applications too. Then you should use Plack::Middleware.

package Plack::Middleware::FakeUserAgent;
use strict;
use parent qw(Plack::Middleware);
use Plack::Util::Accessors qw(agent);

sub call {
    my($self, $env) = @_;
    $env->{HTTP_USER_AGENT} = $self->agent;
    $self->app->($env);
};

1;

That's it. All you have to do is to inherit from Plack::Middleware and defines options that your middleware would take, and implement call method that would delegate to $self->app which is a wrapped application. This middleware is now compatible to Plack::Builder DSL so you can say:

use Plack::Builder;

builder {
    enable "FakeUserAgent", agent => "Mozilla/3.0 (MSIE 4.0)";
    $app;
};

to fake all incoming requests as it comes with the good old Internet Explorer, and you can also use enable_if to conditionally enable this middleware.

Post process requests

The previous examples does pre-processing of PSGI request $env hash, what to do about the response? It's almost the same:

my $app = sub { ... };

# Middleware to fake status code to 500
my $mw = sub {
    my $env = shift;
    my $res = $app->($env);
    $res->[0] = 500 unless $res->[2] == 200;
    $res;
};

This is an evil middleware component that changes all the status code to 500 unless it's 200 OK. Not sure if there is any use for this but it's simple enough for a quick example.

Because some servers implement special streaming interface to delay HTTP response, this middleware doesn't really work with such an interface. Dealing with this special callback interface in individual middleware is not efficient, so we have a special callback interface in Plack::Middleware to make this easy:

package Plack::Middleware::BadStatusCode;
use strict;
use parent qw(Plack::Middleware);

sub call {
    my($self, $env) = @_;
    my $res = $self->app->($env);
    $self->response_cb($res, sub {
        my $res = shift;
        $res->[0] = 500 unless $res->[0] == 200;
    });
}

1;

Pass the response $res to response_cb and set the callback to wrap the real response, and the method takes care of the direct response and delayed response.

Namespaces

In this example we use Plack::Middleware namespace to make middleware, but it doesn't really have to be. If you think your middleware is generic enough for all PSGI apps can benefit, feel free to use the namespace, but if the middleware is too specific for your own needs, or works only with a particular application framework, then use whatever namespace, like:

package MyFramework::Middleware::Foo;
use parent qw(Plack::Middleware);

and then use the + (plus) sign to indicate the fully qualified namespace,

enable '+MyFramework::Middleware::Foo', ...;

Or use the non-DSL API,

$app = MyFramework::Middleware::Foo->wrap($app, ...);

and they should work just fine.

Day 22: Discover more middleware

Christmas is coming near and there aren't enough days to explore more middleware components. Today I'll show you a quick intro of great middleware components that I haven't had time to show.

ErrorDocument

When you die out from an application or display some "Forbidden" error message when an auth wasn't successful you'll probably want to display a custom error page based on the response status code. ErrorDocument is exactly the middleware that does this, like Apache's ErrorDocument directive.

builder {
    enable "ErrorDocument", 500 => "/path/to/error.html";
    $app;
};

You can just map arbitrary error code to a static file path to be served. You can enable StackTrace middleware during the development and then this ErrorDocument middleware on the production so as to display nicer error pages.

This middleware is included in the Plack core distribution.

Session

Actually this is (again) a steal from Rack. Rack defines rack.session as a standard Rack environment hash and defines the interface as Ruby's built-in Hash object. We didn't define it as part of the standard interface but stole the idea and actual implementation a lot.

 builder {
     enable "Session", store => "File";
     $qpp;
 };

By default Session will save the session in on-memory hash, which wouldn't work with the prefork (or multi process) servers. It's shipped with a couple of default store engines such as CHI, so it's so easy to adapt to other storage engines, exactly like we see with other middleware components such as Auth.

Session object has standard methods like get and set and can be accessed with plack.session key in the PSGI env hash. Application and frameworks with access to PSGI env hash can use this Session freely in the app, like in Tatsumaki:

 # Tatsumaki app
 sub get {
     my $self = shift;
     my $uid = $self->request->session->get('uid');
     $self->request->session->set(last_access => time);
     ...
 }

And the nice thing is that any PSGI apps can share this session data as long as they use the same storage etc. Some existing framework adapters don't have an access to this environment hash from end users application yet, so it should be updated gradually in the near future.

Session middleware is developed by Stevan Little on github and is available on CPAN as well.

Debug

This is a steal from Rack-bug and django debug toolbar. By enabling this middleware you'll see the handy debug "panels" in the right side where you can click and see the detailed data and analysis about the request.

The panels include Timer (the request time), Memory (how is memory increased if there's any leaks), Request (Detailed request headers) and Responses (Response headers etc.) and so on.

 builder {
     enable "Debug";
     $app;
 };

Using it is so easy as this, and you an also pass the list of panels to enable only certain panels or additional non default panels.

More extensions for the panels, such as DBI query profiler or Catalyst log dumper are being developed on github.

Proxy

It's often useful to proxy HTTP requests to another application, either running on the internet or inside the same network. The former would be necessary if you want to proxy long poll or some JSON API from your application that doesn't support JSONP (because of Cross domain origin policy), and the latter would be to run applications on different machine and use your app as a reverse proxy, though chances are you want to use frontend web servers like nginx, lighttpd or perlbal to do the job.

Anyway, Plack::App::Proxy is the middleware to do this:

use Plack::App::Proxy;
use Plack::Builder;

my $app = Plack::App::Proxy->new(host => '192.168.0.2:8080')->to_app;

builder {
    mount "/app" => $app;
};

Proxy middleware is developed by Lee Aylward on github.

More

There are more middleware components available in the Plack distribution, and on CPAN. Not all middleware components are supposed to be great, but certainly they can be shared and used by most frameworks that support PSGI.

Day 21: Lint your application and middleware

We've been talking about adapting existing web frameworks to PSGI and writing a new application using PSGI as an interface, but we haven't talked about error handling.

Handling errors

We have an awesome stack trace middleware enabled by default, so if an end user application throws an error, we can catch them and display a nice error page. But what if there is an error or a bug in one of middleware components, or web application framework adapters themselves?

Try this code:

> plackup -e 'sub { return [ 0, {"Content-Type","text/html"}, "Hello" ] }'

Again, writing a raw PSGI interface is not something end users would do every day, but this could be a good emulation of what would happen if there's a bug in one of middleware components or framework adapters itself.

When you access this application with the browser, the server dies with:

Not an ARRAY reference at lib/Plack/Util.pm line 145.

or something similar. This is because the response format is invalid per the PSGI interface: the status code is not valid, HTTP headers is not an array ref but a hash reference and the response body is a string instead of an array ref.

Lint middleware

Checking them in the individual server for every request at a run time is possible but not ideal: that will be a duplicate of codes, and doing so in every request is not efficient from the performance standpoint. We should better validate if an application, middleware or server backend conforms to the PSGI interface using the test suite during the development and disable that when running on production for the best performance.

Middleware::Lint is the middleware to validate request and response interface. Run the application above with the middleware:

> plackup -e 'enable "Lint"; sub { return [ 0, { "Content-Type"=>"text/html" }, ["Hello"] ] }'

and now requests for the application would give a nice stack trace saying:

status code needs to be an integer greater than or equal to 100 at ...

since now the Lint middleware checks if the response in the valid PSGI format.

When you develop a new framework adapter or a middleware component, be sure to check with Middleware::Lint during the development.

Writing a new PSGI server

Middleware::Lint validates both request and response interface, so this can be used when you develop a new PSGI web server as well. However if you are a server developer there's a more comprehensive testing tool to make sure your server behaves correctly, and that is Plack::Test::Suite.

You can look at the existing tests in the t/Plack-Server directory for how to use this utility, but it defines lots of expected requests and responses pairs to test a new PSGI server backend. Existing Plack::Server backends included in Plack core distribution as well as other CPAN distributions all pass this test suite.

Day 20: Access your local app from the internet

These days laptops with modern operation systems allows you to quickly develop a web application and test it locally with its local IP address. Often you want to test your application with a global access, to show off your work to friends who don't have an access to your local network, or you're writing a web application that works as a webhooks callback.

Reverse HTTP to the rescue

There are many solutions to this problem, but one notable solution is ReverseHTTP. It is a very simple specification of client-server-gateway protocol that uses pure HTTP/1.1 payloads, and what's nice about it is that there's a demo gateway service running on reversehttp.net, so you can actually use it for demo or testing purpose pretty quickly without setting up servers etc.

If you're curious how this really works, take a look at the spec. The reason why it's called Reverse HTTP is that your application (server) acts as a long-poll HTTP client and the gateway server sends back an HTTP request as a response. This might sound complex but well, it's really simple :)

Plack::Server::ReverseHTTP

Plack::Server::ReverseHTTP is a Plack server backend that implements this ReverseHTTP protocol, so your PSGI based application can be accessed from the outside world via this reversehttp.net gateway service.

To use ReverseHTTP, install the required modules and run this:

> plackup -s ReverseHTTP -o yourhostname --token password \
  -e 'sub { [200, ["Content-Type","text/plain"], ["Hello"]] }'
Public Application URL: http://yourhostname.www.reversehttp.net/

-o is an alias for --host for plackup (because -h is taken for --help :)), and you should specify the subdomain (label) you're going to use. You should also supply --token which is like a generic password so nobody else can use your label once registered. You can omit this option if you really want anyone else to take that subdomain over.

The console will display the address (URL) like seen, and open the URL from the browser and viola! You see the "Hello" page, right?

Use with frameworks

Of course because this is a PSGI server backend, you can use with any frameworks. Want to use it with Catalyst application?

> catalyst.pl MyApp
> cd MyApp
> ./scripts/myapp_create.pl PSGI
> plackup -o yourhost --token password ./scripts/myapp.psgi

That's it! The default Catalyst application will now be accessible with the URL http://yourhost.reversehttp.net/ from anywhere in the world.

Notes

ReverseHTTP.net gateway service is an experimental service and there's no SLA or whatever, so I don't really think it's usable for production environment and such. But it's really handy and useful to quickly test your application that needs a global access, or show off your work to friends that don't have an internal access. Much easier than other solutions that requires other software like SSH or VPN tunneling.

Day 19: Cascade multiple applications

Conditional middleware and URLMap app have something in common: they're PSGI applications but both takes PSGI application or middleware and dispatch them. This is the beauty of PSGI application and middleware architecture and today's application is another example of this.

Cascading multiple applications

Cascading can be useful if you have a couple of applications and runs in order, then try until it returns a successful response. This is sometimes called Chain of responsibility design pattern and often used in web applications such as mod_perl handlers.

Cascade Application

Plack::App::Cascade allows you to composite multiple applications in order and runs until it returns non-404 responses.

use Plack::App::Cascade;
use Plack::App::File;
use Plack::App::URLMap;

my @paths = qw(
    /home/www/static
    /virtualhost/example.com/htdocs/static
    /users/miyagawa/public_html/images
);

my $app = Plack::App::Cascade->new;
for my $path (@paths) {
    my $file = Plack::App::File->new(root => $path);
    $app->add($file);
}

my $map = Plack::App::URLMap->new;
$map->mount("/static" => $app);
$map->to_app;

This application is mapped to /static using URLMap, and all requests will try the three directories specified in @paths using App::File application and returns the first found files. It might be useful if you want to serve static files but want to cascade from multiple directories like this.

Cascade different apps

use CatalystApp;
CatalystApp->setup_engine('PSGI');
my $app1 = sub { CatalystApp->run(@_) };

use CGI::Application::PSGI;
use CGIApp;
my $app2 = sub {
    my $app = CGIApp->new({
        QUERY => CGI::PSGI->new($_[0]),
    });
    CGI::Application::PSGI->run($app);
};

use Plack::App::Cascade;
Plack::App::Cascade->new(apps => [ $app1, $app2 ])->to_app;

This will create two applications, one with Catalyst and the other with CGI::Application and runs two applications in order. Suppose you have an overlapping URL structure and /what/ever.cat served with the Catalyst application and /what/ever.cgiapp served with the CGI::Application app.

Well that might sound crazy and i guess it's better to use URLMap to map two applications in different paths, but if you really want to cascade them, this is the way to go :)

Day 18: Load middleware conditionally

I've introduced a couple of middleware components. Some of them are useful and could be enabled globally, while others might be better enabled on certain conditions. Today we'll talk about a solution to this.

Load middleware conditionally

Conditional middleware is a super (or meta) middleware that takes one middleware and enable that middleware based on a runtime condition. Let's take some examples:

• You want to enable JSONP middleware only if the path begins with /public
• You don't want to enable Basic Auth if the request comes from local IP

We investigated how they deal with situations like this in WSGI and Rack, but couldn't find a generic solution, and they mostly just implement options to individual component, which did not look cool for me.

Middleware::Conditional

The Conditional middleware is an ultimate flexible solution to this:

use Plack::Builder;

builder {
    enable_if { $_[0]->{REMOTE_ADDR} !~ /^192\.168\.0\./ }
        "Auth::Basic", authenticator => ...;
    $app;
};

We added a new keyword to Plack::Builder enable_if, which takes a block that gets evaluated in the request time ($_[0] there is the $env hash) and if the block returns true, run the wrapped application but otherwise pass through.

This example code examines if the request comes from a local network and runs a basic authentication otherwise.

Conditional is implemented as a normal piece of middleware, and internally this is equivalent to:

use Plack::Middleware::Conditional;
use Plack::Middleware::Auth::Basic;

my $app = sub { ... };

$app = Plack::Middleware::Conditional->wrap($app,
    builder => sub {
        Plack::Middleware::Auth::Basic->wrap(
            $_[0], authenticator => ...,
        );
    },
    condition => sub {
        my $env = shift;
        $env->{REMOTE_ADDR} !~ /^192\.168\.0\./;
    },
);

But it's a little boring to write, so we added a DSL version, which I recommend to use :)

Day 17: Serving static files from your application

On day 5 we talked about serving files from the current directory using plackup. Now that we've learned how to use middleware and compound multiple applications with URLMap it's extremely trivial to add a functionality you definitely need when developing an application: serving static files.

Serving files from a certain path

Most frameworks come with this feature but with PSGI and Plack, frameworks don't need to implement this feature anymore. Just use the Static middleware.

use Plack::Builder;

my $app = sub { ... };

builder {
    enable "Static", path => qr!^/static!, root => './htdocs';
    $app;
}

This will intercept all requests beginning with "/static" and map that to the root directory "htdocs". So requests to "/static/images/foo.jpg" will result in serving a file "./htdocs/static/images/foo.jpg".

Often you want to overlap or cofigure the directory names, so a request to the URL "/static/index.css" mapped to "./static-files/index.css", here's how to do that:

builder {
    enable "Static", path => sub { s!^/static/!! }, root => './static-files';
    $app;
}

The important thing here is to use a callback and a pattern match sub { s/// } instead of a plain regular expression (qr). The callback is tested against a request path and if it matches, the value of $_ is being used as a request path. So in this example we tested to see if the request begins with "/static/" and in that case, strip off that part, and map the files under "./static-files/".

As a result, "/static/foo.jpg" would become "./static-files/foo.jpg". All requests not matching the pattern match just passes through to the original $app.

Do it your own with URLMap and App::File

Just like Perl there's more than one way to do it. When you grok how to use mount and URLMap then using App::File with mount should be more intuitive. The previous example can be written like this:

use Plack::Builder;

builder {
    mount "/static" => Plack::App::File->new(root => "./static-files");
    mount "/" => $app;
};

Your mileage may vary, but I think this one is more obvious. Static's callback based configuration allows you to write more complex regular expression, which you can't do with URLMap and mount, so choose whichever fits your need.

Day 16: Adding JSONP support to your app

Today we'll see another very simple but useful example of a middleware component, this time to add functionality beyond just basic HTTP functions.

JSONP

JSONP (JSON-Padding) is a technology to wrap JSON in a JavaScript callback function. This is normally useful when you want to allow your JSON-based content included programatically in the third party websites using HTML script tags.

Middleware::JSONP

Assume your web application returns a JSON encoded data with the Content-Type application/json, again with a simple inline PSGI application:

use JSON;
my $app = sub {
    my $env = shift;
    if ($env->{PATH_INFO} eq '/whatever.json') {
        my $body = JSON::encode_json({
            hello => 'world',
        });
        return [ 200, ['Content-Type', 'application/json'], [ $body ] ];
    }
    return [ 404, ['Content-Type', 'text/html'], ['Not Found']];
};

Adding a JSONP support is easy using Middleware::JSONP:

use Plack::Builder;
builder {
    enable "JSONP";
    $app;
};

So it's just one line! The middleware checks if the response content type is application/json and if so, checks if there is a callback parameter in the URL. So a request to "/whatever.json" continues to return the JSON but requests to "/whatever.json?callback=myCallback" would return:

myCallback({"hello":"world"});

with the Content-Type text/javascript. Content-Length is automatically adjusted if there's any.

Works with frameworks

Supporting JSONP in addition to JSON would be fairly trivial for most frameworks to do, but Middleware::JSONP should be an example of the things that could be done in Plack middleware layer with no complexity.

And of course, this JSONP middleware should work with any existing web frameworks that emits JSON output. So with Catalyst:

package MyApp::View::JSON;
use base qw( Catalyst::View::JSON );

package MyApp::Controller::Foo;
sub hello : Local {
    my($self, $c) = @_;
    $c->stash->{message} = 'Hello World!';
    $c->forward('MyApp::View::JSON');
}

And then using Catalyst::Engine::PSGI and Plack::Builder, you can add a JSONP support to this controller.

use MyApp;
MyApp->setup_engine('PSGI');
my $app = sub { MyApp->run(@_) };

use Plack::Builder;
builder {
    enable "JSONP";
    $app;
};

Accidentally this Catalyst::View::JSON is my module :) and supports JSONP callback configuration by default, but there is more than one way to do it!

Day 15: Authenticate your app with Middleware

There're lots of Plack middleware components out there, whether in Plack core distribution as well as separate distributions on CPAN. While I've been writing this Plack advent calendar lots of people have shown their interest and taken ideas out of my wishlist.

From today we'll introduce some of the nice middleware components that you can use quickly to enhance any of your PSGI ready applications.

Basic authentication

Since Plack middleware wraps an application, the best thing it can do is pre-process or post-process to do things around HTTP layers. Today let's talk about the Basic Authentication.

Adding a basic authentication can be done in multiple ways: you can do that in the web application framework layer if it's supported in your framework. In case of Catalyst it's Catalyst::Authentication::Credential::HTTP. Just like other Catalyst tools, it allows you to configure the authentication from simple to very complex, by using credential (how to authenticate: basic and/or digest) and store (how to authorize username and passwords).

Otherwise you can do the authentication in the web server layer. For instance if you run your application with Apache and mod_perl, using Apache's default mod_auth module to authenticate is pretty easy and handy for development, while it limits the ability to share "how to authenticate users" since usually you need to write your custom module to do things like database backed authentication.

Plack middleware allows web application frameworks to share such a functionality, mostly with a pretty simple Perl callback system, and Plack::Middleware::Auth::Basic is to do this for Basic authentication. And this is why most Plack standalone servers do not have an authentication system: it's best implemented as a middleware component.

Using Plack::Middleware::Auth::Basic

Just like other middleware, using Auth::Basic middleware is quite simple:

use Plack::Builder;

my $app = sub { ... };

builder {
    enable "Auth::Basic", authenticator => sub {
        my($username, $password) = @_;
        return $username eq 'admin' && $password eq 'foobar';
    };
    $app;
};

This adds a basic authentication to your application $app, and the user admin can sign in with the password foobar and nobody else. The successful signed-in user gets REMOTE_USER set in PSGI $env hash so it can be used in the applications and is logged using the standard AccessLog middleware.

Since it's a callback based, adding another authentication system such as Kerberos would be pretty trivial and easy with modules such as Authen::Simple:

use Plack::Builder;
use Authen::Simple;
use Authen::Simple::Kerberos;

my $auth = Authen::Simple->new(
    Authen::Simple::Kerberos->new(realm => ...),
);

builder {
    enable "Auth::Basic", authenticator => sub {
        $auth->authenticate(@_):
    };
    $app;
};

The same way you can use lots of Authen::Simple backends with small changes.

With URLMap

URLMap allows you to compound multiple apps into one app, so combined with Auth middleware, you can run the same application in a auth vs. non-auth mode, using different paths:

use Plack::Builder;
my $app = sub {
    my $env = shift;
    if ($env->{REMOTE_USER}) { 
        # Authenticated
    } else {
        # Unauthenticated
    }
};

builder {
    mount "/private" => builder {
        enable "Auth::Basic", authenticator => ...;
        $app;
    };
    mount "/public" => $app;
};

This way you run the same $app in "/public" and "/private" paths, while "/private" requires a basic authentication and "/public" doesn't. (Inlining $env->{REMOTE_USER}, or whatever application logic in .psgi is not really recommended -- i just used it to explain it in an obvious way)