An Introduction To OpenResty - Part 3

Jan 04, 2016

In the previous two parts, we setup and introduced OpenResty. It's now time to build something with it.

In part 2, we covered some of the things you need to know, such as nginx's and OpenResty's scaling model (processes and coroutines), and the importance of various phases. That leaves Lua and the nginx lua API still to learn. Both are lightweight and you should be able to pick up the basics from even simple examples. I'll put links at the end of this post for further reading.

Lua Basics

I'm going to give a bullet-list introduction to Lua, which should help if you've used any other dynamic language:

Array indexes start at 1.
~= is used for not equal.
nil is used for nil/null/None.
-- for comments, and --[[ BLAH ]]-- for multi-line comments.
or and and instead of || and &&.
.. to concatenate strings.
Variables and functions are global by default. You'll end up putting local infront of everything.
Modules in Lua behave like modules in Node: what you return from a file is what get assigned when you require from another.
The built-in string library probably doesn't have the method you're looking for; everything is regular expression driven.
Arrays and dictionaries both use {}: {1,2,3,4} and {name = "leto"}. These are called tables.
You can make one table a metatable of another. You can do a lot with metatables, such as implementing inheritance. There are a number of special key names that, if the metatable contains, will cause the original table to behave a certain way. For example, the function assigned to the __index key of the metatable, allows you to build functionality similar to Ruby's method_missing.

Nginx Lua Basics

If you're already familiar with nginx, you'll find the the nginx lua API familiar. While we won't go over every method of the nginx lua API, here's an overview of some useful ones.

All of the variables which nginx makes available can be accessed via ngx.var.VARIABLE_NAME. For example:

# get the page value from the querystring
local page = ngx.var.arg_page

# get the authorization header
local authorization = ngx.var.http_authorization

# get the host of the request
local host = ngx.var.host

In an nginx configuration file, the above three variables would be accessed via $arg_page, $http_authorization and $host. In cases where you want to access many querystring or header variables, you can also get a table of them:

local args = ngx.req.get_uri_args()
local headers = ngx.req.get_headers()

It's worth pointing out that you can write to an ngx.var.VARIABLE value. These variables can be accessed in later parts of the code, including nginx configuration section which occur in later phases. For example, an upstream can be dynamically determined via:

location / {
  # the variable needs to exist before we write to it
  set $upstream '';
  access_by_lua_block {
    ngx.var.upstream = require('upstreams')()
  }
  proxy_pass http://$upstream;
}

One of the more powerful features of nginx lua is that you can issue one or many requests to other nginx locations:

location /v1/users {
  content_by_lua_block {
    res = ngx.location.capture("/legacy/users")
    -- we can do something with the response
  }
}

location /legacy {
  proxy_pass http://legacy;
}

Although this looks like it might be issuing another http request to the server's legacy location, it's actually just a function call. Similarly, you can use capture_multi to issue multiple requests in parallel (Lua supports multiple return values). Somewhat related is the ngx.exec method which does an internal redirect (again, no HTTP request), stopping the existing request (and clearing any state associated with it) and starting a new one at the specified location. ngx.redirect on the other hand returns a 302 (can be overridden to 301) to the client.

ngx.say and ngx.print are used write a message to the response. The difference between them is that ngx.say emits a trailing newline. Both these methods will first write out the response header if it hasn't already been written.

ngx.exit stop processing and return the response when given an argument greater than or equal to 200. Given 0, it only quits the current phase. Here's a pattern you'll often use:

ngx.status = 401
ngx.print('not authorized')
-- idiomatic to return from any function that ends the phase/request
-- makes it more obvious what's happening
return ngx.exit(401)

Wondering why you have to set the status AND specify the exit code? The first, ngx.status, sets the response's header (important that we do that before using print or say). ngx.exit on the other hand, is used to control the internal flow of the request (in this case, stopping further processing).

Verifying Signed Requests

We'll keep our example straightforward: authenticating a request based on a provided signed parameter. Specifically, we'll expect requests to provide an Authorization header in the following form:

-- $BODY is omitted for bodiless requests
Authorization: SHA256 hex(sha256($SECRET + $URL + $BODY))

The nginx configuration is similar to what we've already seen:

upstream app {
  server localhost:3001;
  keepalive 16;
}

location / {
  access_by_lua_block {
    require('authentication')()
  }
  proxy_pass http://app;
}

And, in full, src/authentication.lua looks like:

-- OpenResty ships with a number of useful built-in libraries
local str = require('resty.string')
local sha256 = require('resty.sha256')

local secret = "it's over 9000!"

-- a helper function, probably useful to have a 'response.lua' file
-- and do something like:
--   local response = requre('response')
-- (inlined here though for completeness)
local function notAuthorized()
  ngx.status = 401
  ngx.header.content_type = 'application/json'
  ngx.print('{"error":"not authorized"}')
  ngx.exit(401)
end

local function authenticate()
  local auth = ngx.var.http_authorization
  if auth == nil or auth:sub(0, 7) ~= 'SHA256 ' then
    return notAuthorized()
  end

  -- force nginx to read the body, without this, get_body_data() will return nil
  ngx.req.read_body()
  local msg =  secret .. ngx.var.request_uri
  local body = ngx.req.get_body_data()
  if body ~= nil then
     msg = msg .. body
  end

  local hasher = sha256:new()
  hasher:update(msg)
  if auth:sub(8) ~= str.to_hex(hasher:final()) then
    return notAuthorized()
  end
end

return authenticate

The module exports the authenticate function, since that's what it returns. The first thing it does is check for the presence of the Authorization header as well as checking that it's a supported type (only SHA256 in our case). Next, we force the body to be read into memory by calling ngx.read_body(). This must be called before we try to access the body. It's safe to call this multiple times (without incurring a performance penalty) and it won't block the worker from serving another request while the body is being read.

We then re-sign the request using the configured secret, request_uri variable and, optionally, the body. If the signature that we create doesn't equal to the provide signature, an error is returned and processing stops. Otherwise, processing passes to the content phases (in this case, the proxy_pass directive in our nginx configuration file).

The biggest problem with the above code is that signatures are valid forever and for all users. However, improving the code isn't much work: we simply need the client to include more information (a date, an ip address) as part of the request and the signature.

Admittedly, this functionality is already provided by the http secure link module, but that's only because we've stuck with basic functionality. Just the ability to use something other than md5 is a useful (we use this Lua code from Adobe for HMAC support via OpenSSL). But you could also load the secret from redis or postgres based on a provided client id, or implement finer access control. There's very little that you can't do.

An Introduction To OpenResty - Part 3

Lua Basics

Nginx Lua Basics

Verifying Signed Requests

Further Reading