Scripting Nginx with Lua

by Piotr Klibert

NGINX

Lightweight, very fast
async web server async framework

What does Nginx provide?

  • master/worker process management
  • event loop and event dispatch API
  • async socket APIs on top of that
  • modular architecture and many extension modules providing additional functionality

What modules are there?

  • some core modules implement various protocols: HTTP, SMTP, SSL, FastCGI, IMAP
  • the rest deal with configuration (e.g. `if` directive), content transformations (e.g. SSI, XSLT) and request parsing (e.g. Basic Auth support)
  • 3rd party modules are legion: https://www.nginx.com/resources/wiki/modules/

Thought Experiment!

Can we use nginx modules to develop web app?

  • it would be very fast
  • and massively scalable for free!

Yeah, but...

  • you'd have to write it in C, or using C-level APIs
  • you'd have to recompile Nginx after every change
  • (no support for dynamic loading of modules yet)
  • (and you'd have to write parts of it in Nginx config file syntax...)

Lua

Lua

  • Clean, expressive, minimal scripting language
  • Co-routines in the core language
  • Mostly procedural, with some OO and FP features
  • Simplified error recovery model
  • Easy for both Python and JS programmers to understand

Example?

-- split string on a given delimiter, return a table
-- with all the substrings
function split(str, sep)
   if not sep then sep = "&" end

   local parts = {}
   local start = 0

   for _it, n, char in _.enumerate(str) do
      if char == sep then
         table.insert(parts, strip(string.sub(str, start, n-1)))
         start = n+1
      end
   end

   table.insert(parts, strip(string.sub(str, start, -1)))

   return parts
end

Lua is Embeddable

  • very low memory footprint
  • good support for calling C code and being called from C
  • used successfuly as a scripting lang in many games

LuaJIT

  • all of the above plus incredible speed
  • 3x - 100x speedups when running pure Lua code
  • competes in performance with Java and C#
    rather than Python or Ruby
  • still embeddable and minimal

Lua rocks!

  • luarocks is a package manager for Lua
  • similar to NPM or PIP
  • allows installing both pure Lua and C modules
  • it's a young project, not that many libs yet

WAIT!

Aren't NGINX and LUA
a great fit?

Apparently, agentzh thought so and had a bit too much free time...

OpenRESTY

  • a distribution/bundle of Nginx (not a fork)
  • bundles Nginx core with LuaJIT
  • bundles additional Nginx modules and Lua libs
  • you can replace your nginx binary right now and everything will work

Nginx Scripting

Overview

  • OpenRESTY is the easiest way to start scripting Nginx
  • You can script almost any aspect of Nginx
  • Your code is asynchronous and non-blocking by default
  • You can communicate with any number of external services
  • You can embed arbitrarily complex logic in your scripts

Example: authorization

Lua:

local redis = require "resty.redis"
local red = redis:new()
red:set_timeout(1000)
local ok, err = red:connect("127.0.0.1", 6379)
if not ok then
    ngx.say("failed to connect: ", err)
    return
end
if red:smember("blocked_ips", ngx.var.remote_addr) then
    ngx.exit(ngx.HTTP_FORBIDDEN)
end

Nginx:

location / {
    access_by_lua_file "lua/access.lua";
    proxy_pass http://localhost:8080;
}

Example: redirection

Lua

local headers = ngx.req.get_headers()
if headers["Content-Length"] > MAGIC_NUMBER then
    return ngx.redirect("/upload_big_file")
else
    return ngx.redirect("/upload_small_file")
end

Nginx

location /upload {
    rewrite_by_lua_file "lua/rewrite.lua";
}
location /upload_big_file {
    # proxy_pass straight to s3 for example
}
location /upload_small_file {
    # uwsgi_pass to the app    
}

Example: content

Lua

ngx.header.content_type = "text/plain"
local headers = ngx.req.get_headers()
ngx.say("You've sent following headers:")
for k, v in pairs(headers) do
    ngx.say( k .. ": " .. v )
end

Nginx

location / {
    content_by_lua_file "lua/content.lua";
}

Other scriptable phases:

  • init_by_lua - before everything
  • set_by_lua - with other variable setting directives
  • log_by_lua - after other loggers ran
  • (header|body)_filter_by_lua - before headers/body is sent
  • ssl_certificate_by_lua - when accepting SSL connection

Gotchas?

Where to put Lua files?

  • know what your "server prefix" value is:
    nginx -V
  • you can override this value with -p option:
    nginx -p path
  • all *_by_lua_file directives use this value for resolving paths

Where to put additional Lua libraries?

  • wherever, as long as you register the path
  • use lua_package_path directive for this
  • luarocks can install packages to $HOME/.luarocks
    with --local option
  • libs containing C modules need to be added to
    lua_cpackage_path instead

The end