Espruino Interpreter Internals

Please see the Performance section first for a rough overview, and for the practical implications of the implementation.

Note: There's also a forum thread with some more in-depth answers to questions about the interpreter.

Compilation

Espruino's Makefile calls a few different Python scripts that precompute various things:

• Parse information on the board and chip, and create:
  • A platform_config file - with information about the current system (how many of each peripheral, what pin buttons + LEDs are on, etc)
  • A pininfo source file - listing what peripherals are on what pins
  • Board Documentation (if required)
• Parse all the jswrap_ files (looking for JSON-formatted comments above each function), and create:
  • A jswrapper.c file containing a symbol table and calls to the functions that need to be executed
  • API Documentation (if required)

Much more detailed info is in the Espruino repository at https://github.com/espruino/Espruino/blob/master/README_BuildProcess.md

Parsing

There is no bytecode, so execution happens inside the parser in jsparse.c

The parser is a hand-written recursive-descent parser, and code to be executed is executed directly from variables (see below), not from a C-style string or flat buffer. The Performance page has more information on this.

Variable Storage

Variables are usually stored in fixed-size blocks defined by the JsVar data type. The size of them depends on how many variables need to be addressed. In small devices JsVar can get down to 12 bytes (10 bit addresses), but a device like Bangle.js 2 JsVar will have 14 byte blocks, and larger devices may use 16 bytes or more.

You can check the size of JsVar using process.memory().blocksize from within Espruino

Using fixed sized blocks has several implications:

• Free variables are stored in a linked list, so memory allocation and deallocation is O(1)
• Garbage collection passes are fast
• Memory fragmentation is far less of a problem as most structures do not need to be contiguous in memory
• Malloc would generally have a 4 byte allocation overhead for each memory block - we can do without this.

Each block has optional links to children and siblings, which allows a tree structure to be built. However in many cases these references aren't needed and can be used to store other data.

What follows are the basic variable fields and offsets for 16 byte variable:

| Offset | Size | Name | STRING | STR_EXT | NAME_STR | NAME_INT | INT | DOUBLE | OBJ/FUNC/ARRAY | ARRAYBUFFER | NATIVE_STR | FLAT_STR | | 16b | | | | | | | | | | | FLASH_STR | | |--------|------|---------|--------|----------|----------|----------|------|---------|----------------|-------------|------------|----------| | 0 - 3 | 4 | varData | data | data | data | data | data | data | nativePtr | size | ptr | charLen | | 4 - 5 | ? | next | data | data | next | next | - | data | argTypes | format | len | - | | 6 - 7 | ? | prev | data | data | prev | prev | - | data | argTypes | format | ..len | - | | 8 - 9 | ? | first | data | data | child | child | - | data? | first | stringPtr | ..len | - | | 10-11 | ? | refs | refs | data | refs | refs | refs | refs | refs | refs | refs | refs | | 12-13 | ? | last | nextPtr| nextPtr | nextPtr | - | - | - | last | - | - | - | | 14-15 | 2 | Flags | Flags | Flags | Flags | Flags | Flags| Flags | Flags | Flags | Flags | Flags |

On platforms where the variable size is under 16 bytes, some elements may not be byte-aligned, for example prev, first and last

Note: NAME_INT_INT, NAME_INT_BOOL, and NAME_STRING_INT follow the same pattern as NAME_STR/NAME_INT - they just use child to store a value rather than a reference.

Garbage Collection, Reference Counts and Locks

Espruino contains a mark/sweep garbage collector as well as reference counting.

Garbage Collection is only needed when objects contain circular references. For the vast majority of allocations/deallocations reference counting can be used.

In Espruino, each JsVar contains:

• A Lock Counter - this is a (usually 4 bit) counter that keeps track of Locks.
  • A Lock is needed when some code has a pointer to the variable
  • You increase locks with jsvLock and decrease with jsvUnLock
  • Generally a function that returns a JsVar* will return a locked variable. When JsVar* is passed as an argument it's usually the caller's responsibility to unlock the variable, unless the function's name is something like js....AndUnLock
  • When a variable is locked, it cannot be moved around in memory (because that would change the pointer)
  • All variables contain a lock counter
• A Reference Counter is a 4-8 bit counter that tracks the amount of times a variable is referenced
  • A reference is when another variable links to the current variable (eg. a variable is a child of an object)
  • You increase reverences with jsvRef and decrease with jsvUnRef but generally you never have to do this. If you're using jsvAddChild/etc then they handle references for you
  • When a variable is referenced but not locked, it can be moved around in memory by jsvDefrag to help avoid fragmentation
  • The only variable type that doesn't contain a reference cound is StringExt - this is a part of a string that gets added on the end with more characters, so it's always assumed that it's only referenced once (by the `String`` it is part of)

In most cases a variable can be freed without a GC pass. It will be allocated, used, and then when finally unlocked, if the reference count is zero then it and all its children will be freed.

When Garbage Collecting, Espruino will sweep over all variables in memory setting the JSV_GARBAGE_COLLECT bit unless they are locked. It will then traverse all children of non-garbage collected variables and clear the JSV_GARBAGE_COLLECT bit on them, and finally anything still with the JSV_GARBAGE_COLLECT bit set will be freed.

When writing C code to run inside Espruino the rules are reasonably straightforward:

• Use the built-in jsv* functions for adding/removing children and setting values and you won't have to worry about using jsvRef/jsvUnRef
• If you call a function and it returns a JsVar* in pretty much all cases you'll be responsible for calling jsvUnLock on it
• If you call another function with a JsVar*, unless it's called something like js....AndUnLock, you're still responsible for freeing the variable
• You should avoid having global pointers to JsVar structures, and if sensibe avoid even having JsVarRef. However if you do keep them you need to ensure they are locked/referenced as appropriate, and that you have a jswrap_..._kill handler that will unlock/reference them when the interpreter needs to be torn down.