The State of LuaJit for ARM 64 - March 2016

Prepared by Charles Baylis, Adhemerval Zanella, & Ryan S. Arnold

Lua is "a powerful, fast, lightweight, embeddable scripting language". At the time of this writing the current version of the Lua language is 5.3.2.

LuaJIT is a Just-In-Time Compiler for the Lua language that enables execution of the language at near byte-code speeds rather than interpreted-language speeds.  Enterprise applications that desire to use an embedded scripting language will always choose JIT compiled Lua over interpreted Lua, as the performance impacts are significant.  Using an embedded scripting language will allow them to runtime optimize certain code-paths on-the-fly, after introspection.

For the executive summary of LuaJIT support for ARM64 please see the Collected Concerns header at the bottom of this analysis.

ARM64 Support

The latest official stable LuaJIT release is LuaJIT-2.0.4 but this version of LuaJIT does not have ARM64 support.

ARM64 enablement is only available in the most recent development snapshot of LuaJIT-2.1.0-beta2.

Enterprise Support

What version of LuaJIT do Enterprise Linux Distributions ship?

• RHEL7 - LuaJIT version 2.0.4
• OpenSUSE 13X - LuaJIT version 2.0.3
• SuSE SLE 11 SP2 - LuaJIT version 2.0.3
• CentOS - LuaJIT version 2.0.2
• Ubuntu LTS Trusty - LuaJIT version 2.0.2
• Ubuntu Wily - LuaJIT version 2.0.4
• Ubuntu Xenial - LuaJIT version 2.0.4
  

None of the current Enterprise Linux Distributions ship a version of LuaJIT with any ARM64 support.

Orphaned Project

The LuaJIT maintainer has retired from maintaining the project.

Per the mailing list post, a company called CloudFlare will act as stewards of the LuaJIT project, "Please note that stewardship does not mean ownership. The maintainers and contributors decide. A stewardship is mainly a commitment to help organize a group of developers. And to solve some issues that may be difficult for individual contributor."

The previous maintainer said "I'll move LuaJIT 2.1 (as-is) into beta and cut the 2.1.0 release."

LuaJIT 2.1 has moved into Beta, but the 2.1.0 release has not been made and the date is now beyond his prescribed retirement window. None-the-less, there is a possibility that he will make the release as he said.

Per this mailing list post, there is currently no LuaJIT maintainer at all.

LuaJIT Technical Issues

LuaJIT and Lua 5.1 Test-Suite

LuaJIT has standardized on the Lua 5.1 ABI/API.  The Lua language itself has moved on to version 5.3.

LuaJIT utilizes the Lua language test-suite.  Unfortunately, the lua5.1-tests test-suite doesn't work very well:

• Lua 5.1.5 passes 17/24 tests on x86_64 and ARM64 (both interpreted)
• LuaJIT 2.1.0 passes 11/24 tests on both x86_64 and ARM64 (both interpreted)
  

Lua-5.3.2 tests test-suite does better, but:

• Lua-5.3.2 passes all x86_64 tests successfully (interpreted).
• For LuaJIT, the Lua 5.3.2 test-suite does a Lua language version check which prevents LuaJIT from running the tests, because Luajit implements the Lua language version 5.1.

LuaJIT github official clone also has its own testcase suite, however as explicit stated in README "THIS IS NOT THE TEST SUITE FOR LUAJIT!". The document states the repo is still just a collection of a set of different tests, however it lacks some important points. The idea is to eventually integrate this repo to official luajit one.

64-bit Address Support in LuaJIT Backend

Presently there is no support in LuaJIT for using the JIT backend when needing to use 64-bit pointers, meaning only 4Gb of addressible memory is possible.

https://github.com/LuaJIT/LuaJIT/issues/25

This is definitely a blocker for ARM64 LuaJIT JIT support.

There is also no support for mix-mode apps (32-bit and 64-bit).  It requires shipping two sets of byte-code files

http://comments.gmane.org/gmane.comp.lang.lua.luajit/5981

48-bit Addressibility in LuaJIT Backend

Although Luajit still implements the hard upper limit of 47-bit of addressibility (128 TB), its usage on 48-bit kernel is no longer an issue. Luajit now implements a mix of linear probing and pseudo-random probing to limit the dynamic memory allocation through mmap to first 47-bit of memory address, thus allowing it to be used on its internal GC objects that pack the information on 8 bytes (more information at http://lua-users.org/lists/lua-l/2009-11/msg00089.html).

LuaJIT JIT support for ARM64

Presently the LuaJIT 2.1.0-beta only has interpreter support for ARM64.  JIT support for ARM64 remains to be implemented:

https://github.com/LuaJIT/LuaJIT/issues/26

As documented in TCWG-431, it will require considerable work to enable the full JIT support for ARM64 in LuaJIT.

LuaJIT implements a compiler infrastructure to convert LUA code to binary target instructions. Unfortunately, different from other modern JIT implementations which use an already existing backend (LLVM for instance), luajit implements its own.

This means that for full JIT support the LuaJIT backend will need to be extended to support ARM64.  This requires creating all the hooks and transformations required to convert to Lua intermediary representation (a representation language analogous to llvm IR or gcc gimple but simpler due limited opcode) to its internal SSA form and then to the ARM64 target representation.

 For other architecture implementations in LuaJIT there are 7 specific files required to implement a new port, plus hooks in common ones:

It is roughly 9830 LOC with a mixture of C and Lua code for each architecture port.  ARM64 already has the following files:

• src/lj_target_arm64.h (94 loc)
• src/vm_arm64.dasc (3700 loc)
• dynasm/dasm_arm64.h (518 loc)
• dynasm/dasm_arm64.lua (1166 loc)

So for full JIT support we need to implement:

• src/lj_asm_arm64.h
• src/lj_asm_arm64.h
• src/vm_arm64.dasc
• src/jit/dis_arm64.lua.

LuaJIT JIT optimization for ARM64

It will require additional development investment for ARM64 optimized instruction selection.  We haven't looked at the areas of optimization opportunity yet.  Presumably this will require profiling.

NGiNX

NGiNX Builds with LuaJIT, but for AArch64 this is obviously currently interpreted only (not JIT compiled).  Unfortunately runtime problems can only be debugged by running NGiNX.

Snabb Switch

Snabb Switch is an open source virtualized Ethernet networking stack that uses LuaJIT (and claims competitive performance to C-source in doing so).

Snabb Switch was developed on x86_64 and has some x86_64 specific parts, which result in build failures on ARM64.

• explicitly uses old memcpy symbol (memcpy@GLIBC_2.2.5)  -> link failure
  This doesn't work on ARM64 because there was no version of glibc for ARM64 in glibc version 2.2.5.
• custom profiling function uses REG_RIP in signal handler -> compile failure
• builds AVX2 and SSE2 source files with -mavx2 and -msse2 -> compile failure
• at startup snabb fails to find the arm64 ffi support (ffi = foreign function interface) because the makefile is hardcoded to x64.
• Some new parts of Snabb Switch use LuaJIT dynasm (inline assembler).
  • There is already some support for arm64 in the dynasm source in the LuaJIT tree (see above)
  • x86 assembly code in dynasm code would need porting to ARM64
  • Not every use case of snabb requires dynasm code, so it can be ignored for a while.

There will need to be optimizations for ARM64:

• Optimized checksum functions should use specialized NEON SIMD instructions (see src/arch/) or they'll default to slower C implementations.

Finally, any bugs in LuaJIT that come through Snabb Switch would require runtime debugging of Snabb Switch.

Collected Concerns

1. Who will move the LuaJIT 2.1.0-beta to LuaJIT 2.1.0 release, as there is no maintainer?
   Since LuaJIT 2.1.0-beta contains ARM64 interpreter support, ARM64 JIT compiler support will need to be built upon this beta version!
2. Who is the maintainer?
   Will an ARM64 port be accepted without a general maintainer?
3. Lua 5.1 test-suite is presently static, and Lua 5.3 test-suite will not execute against LuaJIT.
4. LuaJIT
   1. 64-bit address support is missing (on both ARM64 and x64)
   2. 48-bit virtual memory addressibility support is missing
   3. LuaJIT JIT support for ARM64 is completely missing and will require approximately 4.3 KLOC of code additions.
   4. LuaJIT JIT support for ARM64 will need optimizations to be competitive with x64.
5. NGiNX
   1. dependence on LuaJIT JIT support for ARM64
   2. NGiNX un ARM64 will underperform against x86_64 until LuaJIT supports JIT compilation on ARM64.
6. Snabb Switch
   1. As a primary consumer of LuaJIT in the enterprise and networking spaces do we need to port Snabb Switch to ARM64?
   2. As described above, this is a non-trivial endeavor requiring a physical network setup (maybe a virtual one), as well as open-source participation in Snabb Switch, and software porting.
   3. Snabb Switch on ARM64 will underperform against x86_64 until LuaJIT supports JIT compilation on ARM64.
7. Enterprise GNU/Linux Distribution Adoption
   Given Enterprise Linux Distribution LTS time-tables, what's the soonest that an Enterprise Distro can ship a version of LuaJIT with ARM64 JIT support?  Does this accelerate our ARM64 porting timetable?

Questions

If you have any questions or comments please email Ryan S. Arnold <ryan.arnold@linaro.org>