The State of LuaJit for ARM 64 - April 2016

Owned by Ryan Arnold
Last updated: Nov 04, 2016
2 min read

By Charles Baylis & Adhemerval Zanella

This page is outdated. Please see LuaJIT for ARM64 for the status of LuaJIT JIT support on ARM64.

ARM64 LuaJIT Interpreter Crash

Currently support only exists for LuaJIT interpreter support for ARM64.

It has been discovered that on 48-bit VA systems there is a bug that causes the interpreter to crash.  Here is the reproduction case:

  1. The source is here: http://luajit.org/download/LuaJIT-2.1.0-beta2.tar.gz
  2. Configure & Build
  3. Using a kernel configured with CONFIG_ARM64_VA_BITS_48=y CONFIG_ARM64_VA_BITS=48, run luajit in the source directory.

The previous maintainer, Mike Pall created a fix for this issue which restricts the memory addresses to the 47-bit range:

The memory block allocator rewrite patch from Mike Pall
fixes the 48-bit VMA issue we are observing (commit id
0c6fdc1039a3a4450d366fba7af4b29de73f0dc6).

 

This has been applied to the v2.1 branch of LuaJIT:

commit 0c6fdc1039a3a4450d366fba7af4b29de73f0dc6
Author: Mike Pall <mike>
Date: Mon Apr 18 10:57:49 2016 +0200

Rewrite memory block allocator.

Use a mix of linear probing and pseudo-random probing.
Workaround for 1GB MAP_32BIT limit on Linux/x64. Now 2GB with !LJ_GC64.
Enforce 128TB LJ_GC64 limit for > 47 bit memory layouts (ARM64).

After applying the patch and testing we verified that the bug that caused the interpreter to crash is now fixed on an AArch64 machine that is configured for 48-bits of virtual addressibility.

This issue has been resolved!

ARM64 LuaJIT JIT Implementation

Presently, the interpreter is enabled with LJ_GC64 support for ARM64, but not for x86_64.  This prevents the addressing of more than 4B of memory in an x86_64 application.

 

In the LuaJIT JIT backend code-base there is currently not an implementation of LJ_GC64 (which is required for addressing larger amounts of memory).

 

Since x86_64 doesn't implement it in the interpreter, and LuaJIT doesn't implement it in the JIT, there's not a problem on x86_64.  On ARM64, on-the-other-hand, we need congruence between the interpreter and the JIT (due to runtime fallback from JIT to interpreter) and therefore we need to make sure that LJ_GC64 is implemented in the JIT.

 

Therefore, on ARM64, not only must the ARM64 port be made, but LJ_GC64 must be enabled as well as it is a co-requisite.

Note: It's unknown whether the ARM64 interpreter can be reverted to not use LJ_GC64 support.

ARM64 JIT Enablement Approach

  1. Try enabling for x86_64 to see how far along it is – We won't dwell on this for too long as our concern is for ARM64.
  2. Use the ARM32 port as the basis for the ARM64 port.
  3. Do a minimum prototype to get 'hello_world' running.  
  4. Address LJ_GC64 issues as the arise.
  5. Do full port in later stages.
Note: The alternative is to do the LJ_GC64 enablement for x86_64 first but this isn't really in Linaro's immediate interest.  We're more interested in having an ARM64 port at the same time that LJ_GC64 is enabled.

Looking Forward - Supporting 52-bit (and greater) VA Sizes in LuaJIT

Three options immediately identified, ranked as the most desired to least desired.  All three approaches involve changes in the kernel to restrict the address range of memory returned from mmap:

  1. Per-process rules
  2. Per-cgroup rules
  3. Per-system (/proc/cpu)