Emacs Lisp multi return values
The missing feature
Did you ever wrote a function in Emacs Lisp which should return more than one result?
Emacs Lisp has no native support for multiple return values,
but provides cllib
that emulates it in a Common Lisp style.
In this article I will show that clvalues
is suboptimal and
can be replaced without any sacrifices to the convenience.
Naive solution
cllib
implements clvalues
in terms of list
.
This approach is naive because each time you return with that,
an allocation is involved. GC will trigger more frequently
and perfomance will degrade.
(let ((lexicalbinding t))
(benchmarkruncompiled 1000000
(clmultiplevaluebind (a b c) (clvalues 1 2 3)
(ignore a b c)))) ;; => (0.8493319750000001 59 0.7827748330000008)
;; ...more than 50 garbage collections,
We see the bottleneck now: proposed solutions should
avoid memory allocations.
In other words: no list
, cons
, vector
, …
No allocations with preallocations
We can still use lists and vectors if preallocation is done. Multiple return values are mostly consist of 24 elements => the set of required containers is fixed and known beforehand.
(defvar mv2 (makevector 2 nil))
(defvar mv3 (makevector 3 nil))
;; ... as many as we need.
When 2 value tuple must be returned, mv2
vector
is populated with corresponding values.
For 3 value tuple, mv3
is used.
Filled vector is returned to the caller.
Special macro can be used to extract vector elements
into specified bindings.
This brings us close to the cllib
, but without allocations.
Emacs Lisp has no real multithreading, so it is safe to store results inside private global variable.
List vs vector
The choice between vector
and list
is not easy,
especially if you know
Emacs bytecode.
First operation we care about is return efficiency. To make multi value return, preallocated list/vector must be filled with data.
f(x, y) = x, y+1
(defvar mv2 '(nil . nil)) (defvar mv2 [nil nil])
constants=[mv2] maxStack=5 constants=[mv2 0 1] maxStack=6

add1 <x y+1>  add1 <x y+1>
varref 0 <x y+1 mv2>  varref 0 <x y+1 mv2>
dup <x y+1 mv2 mv2>  dup <x y+1 mv2 mv2>
stackref 3 <x y+1 mv2 mv2 x> + constant 1 <x y+1 mv2 mv2 0>
setcar <x y+1 mv2> + stackref 4 <x y+1 mv2 mv2 0 x>
dup <x y+1 mv2 mv2> + aset <x y+1 mv2>
stackref 2 <x y+1 mv2 mv2 y+1> + dup <x y+1 mv2 mv2>
setcdr <x y+1 mv2> + constant 2 <x y+1 mv2 mv2 1>
ret + stackref 3 <x y+1 mv2 mv2 1 y+1>
+ aset <x y+1 mv2>
 ret
Left block shows list implementation. Right block is for vector.
As you may see, for N=2
case cons cell is better than vector in many ways:
 Bytecode is shorter
 Less stack space is used
 Smaller constant vector (no need for indexes)
Second operation is return value receive.
let x, y = f(...)
call ... <mvret2>  call ... <mvret2>
dup <mvret2 mvret2>  dup <mvret2 mvret2>
car <mvret2 x> + constant X <mvret2 mvret2 0>
stackref 1 <mvret2 x mvret2> + aget <mvret2 x>
cdr <mvret2 x y> + stackref 1 <mvret2 x mvret2>
+ constant Y <mvret2 x mvret2 1>
+ aget <mvret2 x y>
What about 3 or more return values? General algorithm for lists is:
 For
N
return values use dedicated preallocated list  First value bound with
setcar
 Last value bound with
setcdr
 Values in between set with
setcar
AND performcdr
Note that used list is not proper list. The last cdr
is not nil
.
At N=3
vector and list are nealy equal in efficiency, N=4
favors vectors.
List becomes less and less efficient as the N
grows.
In my experience 2value returns cover 90% of cases.
This means that list is a winner here.
Another thing worth considering is ability to discard some
of the return values. In Go you can do a, _, c := f()
which assigns 1st and 3rd returned values; 2nd value is ignored.
Generally, lists are slower here because you still need to
traverse ignored elements.
Next section describes another implementation option which is a good compromise between list and vector in terms of {return/assign/discard} operations perfomance.
Neither list, nor vector?
It is possible to avoid lists and vectors completely.
For each additional return value it is possible to use single global variable.
First value is returned as usual, while others
use varset
(setq) to bind additional data.
On the caller side, function result is bound to
the first variable; other variables read from
corresponding global variables.
;; Return "a", "b", "c":
(progn
(setq mv3 "c")
(setq mv2 "b")
"a")
;; Bind results:
(let ((x1 (f ...)
(x2 mv2)
(x3 mv3)))
...)
This gives us very compact bytecode. Perfomance depends on many factors, but it can match implementation based on preallocated lists.
Let’s use this idea to create mvlib
.
mvlib
The minimal mvlib
should consist of at least two macros:
mvret
 yield a multi valuemvlet
 bind multi value to local variables
Like with other solutions, predefined globals are required.
For simplicity, they have 0based suffixes.
That is, second return value is stored inside mv0
(not in mv2
).
(defconst mvmaxcount 10) ;; Arbitrary limit
(defun mvvar (index)
"Get return value variable symbol by INDEX"
(when (>= index mvmaxcount)
(error "Index %d is too high (%d is max)" index (1 mvmaxcount)))
(intern (format "mv%d" index)))
(dotimes (i mvmaxcount)
(eval `(defvar ,(mvvar i) nil)))
mvret
and mvlet
are convenience wrappers for code that is
presented in previous section.
;; Bind multiple values with some of them being ignored.
(mvlet (a _ b _) (mvret 1 2 3 4)
(+ a b)) ;; => 4
;; Compare with `cllib'.
(let ((lexicalbinding t))
(benchmarkruncompiled 1000000
(mvlet (a b c) (mvret 1 2 3)
(ignore a b c)))) ;; => (0.174552687 0 0.0)
;; 0 GC runs!
Multitple values return with zero allocations achieved
Each _
bind variable does not produce any code,
they truly ignore the result.
Important exception is the first binding. It can not
discard the bound expression because it has sideeffect
of setting rest return values.
Why I prefer goism
Macro can help a lot with many features, but what about packages or namespaces?
It is tedious and ugly to use prefixed identifiers for everything. Even C has better modularity and encapsulation with internal linkage and opaque pointers.
Everyone understand complications that arise with modules for Emacs. Luckily, there is another way. Some languages already have modules. With goism it is possible to write Go code that is translated into Emacs Lisp.
As a bonus, when goism will be complete, we could use Go libraries inside Emacs.