SafeTable/SafeFunction: Overcoming Lua's hideous nil construct

[Ian]

While I will argue that type safety is simply unnecessary in a dynamically typed language like Lua, the fact that Lua does not generate an error when accessing a variable that doesn't exist is horribly unsafe. This means that any typographical or logic error involving variables can cause confusing and hard to fix bugs. The error won't propagate until the nil value is used in a way that Lua defines as erroneous which doesn't tell you anything about how the variable became nil to begin with. To combat this I've tried to write some functions that do generate errors if you do something unintentional. :mad:

This first function wraps a table and prevents you from trying to read from a non-existent index. Lua would do this by default were it sane. To use it just wrap your table like so: SafeTable({1, 2, 3}) or alternatively SafeTable{1, 2, 3}. SafeTable has side-effects--that is it alters the table in place. I could change this behavior by adding a copy function, but I didn't think it worth it.


-- SafeTable raises an error if a non-existing index is accessed. Note that
-- indices can still be added and removed like usual. Also, while SafeTable
-- returns the table it was passed it also alters it in place.
function SafeTable(tbl)
return setmetatable(tbl, {
-- Define the __index metamethod to change the behavior of indexing.
__index = function (tbl, key)
-- If normal indexing is used then this metamethod will be called
-- causing infinite recursion; rawget ignores the metamethod.
result = rawget(tbl, key)
-- If the value for the given index/key is nil then generate an
-- error. Note that tostring is called because string
-- concatenation is arbitrarily NOT polymorphic and will generate
-- an error if the key is not a string or number.
assert(result ~= nil, "Invalid index: " .. tostring(key) .. ".")
return result
end
})
end

This next function wraps a function so that errors will be raised if it is called with too many or too few arguments, or if it is passed a nil value. To use it, wrap the function declaration with it or decorate it like so: f = SafeFunction(f). The optional second argument tells the function how many arguments to expect; there is sadly no way to glean this information via introspection.



-- SafeFunction raises an error if a nil argument is passed to it that precedes
-- a non-nil argument. Furthermore, if numberOfArguments is defined then it
-- will require that number of arguments be passed to it.
function SafeFunction(func, numberOfArguments)
-- Return the wrapper for func.
return function (...)
-- Create a table from the arguments passed to the wrapper.
local arguments = {...}
local totalArguments = numberOfArguments or #arguments
if #arguments ~= totalArguments then
-- The function was passed more arguments then it takes.
print(string.format("Function takes %d arguments; %d %s passed.",
numberOfArguments, #arguments,
#arguments == 1 and "was" or "were"))
end
for index = 1, totalArguments do
if arguments[index] == nil then
if index > #arguments then
-- The function was not passed enough arguments or the last
-- arguments were nil.
error(string.format(
"Function takes %d arguments; %d %s received.",
numberOfArguments, index - 1,
index - 1 == 1 and "was" or "were"))
else
-- An argument was passed that was nil. This makes the
-- assumption that a symbol was never declared somewhere.
error("A non-existent symbol was passed to parameter " ..
index .. ".")
end
end
end
func(...)
end
end


This last function tests to see if some local variables are nil or if upvalues are nil (these are used in closures.) Note that this is probably the least useful of the three. To use it you can stick it at the end of your function (before the return statement.) For some reason it doesn't work half the time (d'oh!) Alternatively (as a last resort!) you can hook it to be used on every function call using the hook functions provided. This may cause a lot of erroneous errors to be generated. I might add a way to selectively disable them if someone actually finds this useful. (I was too lazy to document this.)



function SafetyTest(event)
-- Test local variables.
local localIndex = 1
while true do
local name, value = debug.getlocal(2, localIndex)
if not name then
break
end
assert(value ~= nil or name == "arg" or name == "(*temporary)", name)
localIndex = localIndex + 1
end

-- Test upvalues.
local func = debug.getinfo(2).func
local upValueIndex = 1
while true do
local name, value = debug.getupvalue(func, upValueIndex)
if not name then
break
end
assert(value ~= nil, name)
upValueIndex = upValueIndex + 1
end
end

function EnableSafetyTest()
debug.sethook(SafeTest, "c")
end

function DisableSafetyTest()
debug.sethook()
end


Phew.

P.S.

Seth, it's not too late to switch to Python (maybe). :(

[Seth]

I totally agree, I really want giant errors when undefined variables are used, will check those out when I get a chance.

There is a script that lua come with called "strict.lua" to help with this for globals but it couldn't get it working a few weeks ago when I played with it. Sort of crashes everything. :confused:


--
-- strict.lua
-- checks uses of undeclared global variables
-- All global variables must be 'declared' through a regular assignment
-- (even assigning nil will do) in a main chunk before being used
-- anywhere or assigned to inside a function.
--

local mt = getmetatable(_G)
if mt == nil then
mt = {}
setmetatable(_G, mt)
end

mt.__declared = {}

mt.__newindex = function (t, n, v)
if not mt.__declared[n] then
local w = debug.getinfo(2, "S").what
if w ~= "main" and w ~= "C" then
error("assign to undeclared variable '"..n.."'", 2)
end
mt.__declared[n] = true
end
rawset(t, n, v)
end

mt.__index = function (t, n)
if not mt.__declared[n] and debug.getinfo(2, "S").what ~= "C" then
error("variable '"..n.."' is not declared", 2)
end
return rawget(t, n)
end


Seth, it's not too late to switch to Python

Indentation for block-delimiting.. nooooo!!!! :eek: ;)

[sphair]

Not that it is realistic, but if you would ever consider another scriptlanguage, I'd recommend SpiderMonkey (Mozillas Javascript implementation)

:)

[Ian]

There is a script that lua come with called "strict.lua" to help with this for globals but it couldn't get it working a few weeks ago when I played with it. Sort of crashes everything. :confused:

H'm, that's a lot more useful then what I came up with. But you're right that for some reason it exits Novashell without so much as an error message. However, this only appears to occur when you import it into system/startup.lua. It works perfectly when it's imported into a world-specific script. Are there multiple global frames somehow? :confused:


Indentation for block-delimiting.. nooooo!!!! :eek: ;)

Pft. If you don't already indent your code you're a goober. :p

[Ian]

I'd call it safety.lua instead. Anyway, the code contained herein should execute fine from system/startup.lua. I fixed several bugs from the original implementation and added automatic SafeTable wrapping for global tables. SafeTable also has several new features.

(It discovered errors in its own implementation, even. Now it is impossible to make the mistake of using "x = 1" instead of "local x = 1" inside of a function. Unless x is a global, of course.)


-- Requires that all global variables be assigned a value (even if nil) in the
-- global scope ("main chunk") before it may be read.

do
-- This magic constant is used by the error function. The other possible
-- values are useless, so you probably shouldn't change this.
local errorLevel = 2

-- This unique object replaces nil internally.
local safeNil = {}

-- Get the metatable for the global namespace.
local globalMetatable = getmetatable(_G)
-- Augment the metatable if it already exists; otherwise make a new one.
if globalMetatable == nil then
globalMetatable = {}
setmetatable(_G, globalMetatable)
end

-- A table that holds all global declarations.
globalMetatable.__declared = {}

-- Invoked whenever a global variable is read.
function globalMetatable:__index(variable)
if globalMetatable.__declared[variable] == nil and
-- C functions are allowed to do weird stuff.
debug.getinfo(2, "S").what ~= "C" then
error(string.format("Variable '%s' is not defined.", variable),
errorLevel)
end
return rawget(self, variable)
end

-- Invoked whenever a global variable is created.
function globalMetatable:__newindex(variable, value)
if globalMetatable.__declared[variable] == nil then
local what = debug.getinfo(2, "S").what
-- Assignment to undefined variables is allowed in the global
-- namespace ("main chunk") and C functions.
if what ~= "main" and what ~= "C" then
error(string.format("Assignment to undefined variable '%s'.",
variable), errorLevel)
end
globalMetatable.__declared[variable] = true
end
-- If the new variable is a table then replace it with a SafeTable.
if type(value) == "table" then
SafeTable(value)
end
rawset(self, variable, value)
end

-- Raises an error if a non-existing index is accessed. Elements can be
-- added to a SafeTable like normal, but setting an element to nil will
-- not delete it as a SafeTable can store nil values unlike regular
-- tables. To delete an element, use table.delete instead.
-- Implementation notes: returns the table it was passed but also alters it
-- in place as a side-effect. This function must be implicitly called to
-- change the behavior of local tables, but will automatically wrap global
-- tables.
function SafeTable(tbl)
if tbl.__unsafe then
tbl.__unsafe = nil
return tbl
end
local metatable = {}
-- Define the __index metamethod to change the behavior of indexing.
function metatable:__index(key)
-- If normal indexing is used then this metamethod will be called
-- causing infinite recursion; rawget ignores the metamethod.
local result = rawget(self, key)
-- If the value for the given index/key is nil then generate an
-- error. Note that tostring is called because string.format is
-- arbitrarily NOT polymorphic and will generate an error if the
-- key is not a string or number.
if result == nil then
error(string.format("Invalid key '%s'.", tostring(key)),
errorLevel)
elseif result == safeNil then
return nil
end
return result
end
function metatable:__newindex(key, value)
if value == nil then
value = safeNil
end
rawset(key, value)
end
return setmetatable(tbl, metatable)
end
end

-- If for some reason the default behavior a global table is desired it can be
-- wrapped with this function. This may be useful for backwards compatibility
-- or interoperability.
function UnsafeTable(tbl)
tbl.__unsafe = true
return tbl
end

-- Deletes a global variable.
function delete(variable)
rawset(_G, key, nil)
end

-- The following functions pretend to be part of the table package:

-- Deletes an element from a table.
function table.delete(tbl, key)
rawset(tbl, key, nil)
end

-- Tests whether a table contains a given key.
function table.haskey(tbl, key)
return rawget(tbl, key) ~= nil
end

[Seth]

Cool.

hmm, I RunScript this at the top of startup.lua but get:


---------------- Lua Error! Stack Dump ----------------

-1: system/sound.lua
-2: ...libstuff\novashell\bin/base/script/system/safety.lua:90: bad argument #1 to 'rawset' (table expected, got string)
Initting world
---------------- Lua Error! Stack Dump ----------------

-1: intro/intro_menu.lua
-2: ...libstuff\novashell\bin/base/script/system/safety.lua:90: bad argument #1 to 'rawset' (table expected, got string)
---------------- Lua Error! Stack Dump ----------------

-1: worlds/RT_TreeWorldTest/script/game_start.lua:18: Invalid key 'new'.


Any ideas?

[Ian]

For some reason all these errors appeared even though it seemed to work fine before I posted the code. :confused: Hurray nondeterminism. :mad:

I fixed it (sort of) but I had to change the implementation in the following ways:

1. The SafeTable thing had too many potential problems to make it the default behavior so I removed it.
2. For some damn reason the program bails if an error is discovered in the system scripts but works in world scripts. (Currently there is an error.) To "solve" this I had to add a global (g_safeTest) which enables error raising if set. In other words, you have to set it in game_start.lua. :(


-- Requires that all global variables be assigned a value (even if nil) in the
-- global scope ("main chunk") before it may be read.

-- Set this global to true in your world to enable errors to be raised.
g_safeTest = false

do
local function raise(err, ...)
local stackLevel = 3
if g_safeTest then
error(string.format(err, ...), stackLevel)
else
print(debug.traceback(string.format(err, ...), stackLevel))
end
end

-- Get the metatable for the global namespace.
local globalMetatable = getmetatable(_G)
-- Augment the metatable if it already exists; otherwise make a new one.
if globalMetatable == nil then
globalMetatable = {}
setmetatable(_G, globalMetatable)
end

-- A table that holds all global declarations.
globalMetatable.__declared = {}

-- Invoked whenever a global variable is read.
function globalMetatable:__index(variable)
if globalMetatable.__declared[variable] == nil and
-- C functions are allowed to do weird stuff.
(not g_safeTest or debug.getinfo(2, "S").what ~= "C") then
raise("Variable '%s' is not defined.", variable)
end
return rawget(self, variable)
end

-- Invoked whenever a global variable is created.
function globalMetatable:__newindex(variable, value)
if globalMetatable.__declared[variable] == nil then
local what = debug.getinfo(2, "S").what
-- Assignment to undefined variables is allowed in the global
-- namespace ("main chunk") and C functions.
if what ~= "main" and what ~= "C" then
raise("Assignment to undefined variable '%s'.", variable)
end
globalMetatable.__declared[variable] = true
end
rawset(self, variable, value)
end
end