Better poly than sorry!

Python interoperability options

Recently I was writing a little set of scripts for downloading and analyzing manga meta-data, for use with my Machine Learning pet project. True to a polyglot style, the scripts were written in two languages: Python and Elixir. Elixir is a language that works on Erlang VM (see my previous blog post on it, Weekend with Elixir) and Python doesn't need any introductions I think. I used Elixir for network related stuff, and Python with pandas for analysis.

The biggest problem with such a setup (and with polyglot approach) is passing the data around between the languages. At first I just used a couple of temporary JSON files, but then I remembered a project I once saw, called ErlPort, which is a Python implementation of the Slang's external term protocol. In short, it enables seamless integration between Python and Erlang and - by extension - Elixir. ErlPort not only lets you serialize data, but lets you also call functions across language boundaries. It also supports Ruby besides Python.

In general, using ErlPort was a success in my case, but it's limited as it can only connect three languages. I'd normally say it's good enough and leave it, but a couple of days later, in "the best of Python lib of 2015" thread on HN, I discovered another project called python-bond, which provides similar interoperability features for Python and three more languages: PHP, Perl, JavaScript. The two libraries - ErlPort and python-bond - make it embarrassingly easy to integrate a couple of different languages in a Python project. Along with Cython, which lets you easily call C-level functions, it makes Python a very good base language for Polyglot projects.

The hardest part with polyglot style, and in particular architecture, is deciding which language should be a base one, the one which runs and integrates the rest of the languages. In my case, I ended up with most of the control flow inside Elixir, because of the very neat supervision tree feature it provides (thanks to Erlang/OTP). It remains to be seen if Elixir is capable of filling the role of a main implementation language in a polyglot project. I'll make sure to write a post about it in the future.