Common pitfalls to avoid in Lua (1 Viewer)

This guide assumes you are already familiar with the basics of Lua, including tables, strings, the number type, and functions. Here are a few things you can do to avoid common and/or hard-to-debug mistakes in Lua.

• Do not compare tables with ==.
  Ex:
  

  --Do not do this!
  {a = 5} == {a = 5} --=> false

  This returns false! When comparing tables, lua uses what's called reference equality, which means that it will only return true if both sides are the same table, not just the same properties with the same values.
  

  local tableA = {a = 5}
  local tableB = tableA
  local tableC = {a = 5}
  tableA == tableB --=> true
  tableA == tableC --=> false
  tableA.a == tableC.a --=> true

  Try to avoid comparing tables, it's complicated no matter how you do it. If you really have to, then see How to check if two tables have the same value in lua.
• Do not use # on anything except an "array-like" table or a string*.
  

  local coolString = "abc123"
  local arrayLike = {10, 20, 30, 40}
  local alsoArrayLike = {
  a = 5,
     [1] = 10,
     [2] = 20,
     [3] = 30
  }
  local notArrayLike = {10, 20, nil, 40}
  
  #coolString --=> 6
  #arrayLike --=> 4
  #alsoArrayLike --=> 3
  #notArrayLike --=> undefined behavior! Avoid at all costs!

  
  "array-like" in this case means that there are no "holes" in the numbers; Either no number rows are defined or every number row up to the maximum has a value that isn't nil. This is also refered to as a "sequence" in the docs.
  
  If you invoke # on a table with holes in it, the result is undefined behavior! Undefined behavior means that the Lua interpreter can do whatever it wants. It can return 0, 100, nil, it can abort, explode, set the array to the lyrics of the bee movie, anything. Anytime you hear the words undefined behavior, run in the opposite direciton and avoid at all costs. Act as if every time you invoke undefined behavior, millions of poisonous spiders are going to crawl out of your keyboard and eat you alive.
  
  Note that while you can use # on a table with non-numeric keys as long as any numeric keys are array-like, I wouldn't reccomend it simply because the behavior (only counting the numeric keys) can be confusing.
  
  If you need to count how many rows a table has, you have to do it like this:
  
  

  local tableToCount = {
     a = 1,
     [3] = "abc",
     x = {1,2,3}
  }
  
  local count = 0
  for _,_ in pairs(tableToCount) do
     count = count + 1
  end
  count --=> 3

• Do not compare decimal numbers.
  

  0.3 - 0.2 --=> print will tell you "0.1", but it's actually 0.09999999999999998
  (0.3 - 0.2) == 0.1 --=> false

  Lua internally uses 64-bit floating-point numbers. This is usually what you want, but they are ever-so-slightly imprecise. Lua even rounds numbers when displaying them, so you don't see the imprecise result with print. This usually doesn't matter, as they're more than precise enough for most calculations, but if you try to compare them you may get unexpected results.
  
  If you really need to compare decimal numbers, you use what's called an epsilon, which is a very small number that you've decided, if two numbers are within epsilon of eachother, they're "close enough".
  

  local function compareDecimals(a, b, epsilon)
     return math.abs(a - b) < epsilon
  end
  local epsilon = 0.0001
  compareDecimals(0.3 - 0.2, 0.1, epsilon) --=> true

  
  If you're only comparing integer numbers, you don't need to worry about this unless...
• Make sure your integers don't go above 2^53 or below -2^53
  

  local function returnZero(a) --silly function, but it should always return 0, right?
     local b = a + 1
     local c = b - a
     return c - 1
  end
  returnZero(5) --=> 0
  returnZero(100) --=> 0
  returnZero(-43893) --=> 0
  --so far so good...
  returnZero(2^53) --=> -1
  --oh noes!

  All numbers are represented as a float, which is only precise up to a certain amount. This applies even to integers, when the integer has enough digits.
  
  Fortunately you won't need to worry about running out of precision for array indexes; An array with 2^53 elements would take up over 9 petabytes of RAM. That's 9 million gigabytes.

*or on a table where the metafunction __len is defined and you're using lua 5.2, see Lua reference manual 3.4.6

 

[damian]

It's a pet peeve of mine how programming languages don't perfectly compare decimals. But what is the reason for this? A flaw with the programming language itself, or how the computer running it responds to it?

 

[mid]

It's a pet peeve of mine how programming languages don't perfectly compare decimals. But what is the reason for this? A flaw with the programming language itself, or how the computer running it responds to it?

There is no good solution to that problem, which is why programming languages leave it up to you to decide. This is a fundamental issue with how floating point works, it's inaccurate by design (the operations, not the numbers themselves), but it has huge range.

There's ideas for other, different ways to represent non-integers in computers, all of them vary. Fixed-point is probably the next best thing. Some have even proposed representing them as two integers, forming a rational. The problem with that is that 50% of all possible combinations is a half (1 / 2, 2 / 4, 4 / 8, etc.), 25% of all combinations is a quarter, and so on.

There's also the very interesting Unum format, although it's variable-sized which already makes it impractical.

 

[damian]

There is no good solution to that problem, which is why programming languages leave it up to you to decide. This is a fundamental issue with how floating point works, it's inaccurate by design (the operations, not the numbers themselves), but it has huge range.

There's ideas for other, different ways to represent non-integers in computers, all of them vary. Fixed-point is probably the next best thing. Some have even proposed representing them as two integers, forming a rational. The problem with that is that 50% of all possible combinations is a half (1 / 2, 2 / 4, 4 / 8, etc.), 25% of all combinations is a quarter, and so on.

There's also the very interesting Unum format, although it's variable-sized which already makes it impractical.

I guess the tradeoff for larger over more accurate numbers is fair.

 

[stetre]

It's a pet peeve of mine how programming languages don't perfectly compare decimals. But what is the reason for this? A flaw with the programming language itself, or how the computer running it responds to it?

Most programming languages (including Lua) use the IEEE 754 standard for encoding real numbers.
You can google it up to find tons of explanations about the standard, but basically the reason for its quirks is that
it has many conflicting requirements to fulfill: we want it to represent a large range of numbers, from very small to
very large, to represent negative numbers, to represent irrational numbers as well (as accurately as possible), etc.
All this using a finite number of bits. This inevitably leads to some trade-offs because real numbers not only are
infinite, but they are also dense, so there are infinite of them even in a finite range (however small) and thus we do
not have the option to say "let's represent accurately all the numbers in a range" as we do with integers.

 

[Bonfire]

If you invoke # on a table with holes in it, the result is undefined behavior! Undefined behavior means that the Lua interpreter can do whatever it wants. It can return 0, 100, nil, it can abort, explode, set the array to the lyrics of the bee movie, anything. Anytime you hear the words undefined behavior, run in the opposite direciton and avoid at all costs. Act as if every time you invoke undefined behavior, millions of poisonous spiders are going to crawl out of your keyboard and eat you alive.

This is not true.
The Lua reference describes the behavior of the length operator:
Lua 5.3 Reference Manual

While the results of the length operator of a “table with holes“ are practically useless because they are implementation dependent and hard to predict it is by no way undefined. Especially it cannot have side effect like aborting the program or changing the contents of the table it is invoked on. The result can also be only less than the number of elements in the table, never more.

 

[Bonfire]

It's a pet peeve of mine how programming languages don't perfectly compare decimals. But what is the reason for this? A flaw with the programming language itself, or how the computer running it responds to it?

It has nothing to do with programming languages, it has to do with computer algebra and number representation.

A floating point number is an approximation of real numbers. As you may remember from school only a small fraction of real numbers can be represented with a finite number of digits. E.g. the number 1/4 can be represented exactly as 0.25.
The number 1/3 or irrational numbers like pi or sqrt(2) cannot.

Floating point numbers in computing further complicate things because they usually represent fractions as sum of negative powers of two ( 1/2, 1/4, 1/8, 1/16). They map not well do decimal fractions. A number which can be represented in decimal like 0.4 has no exact representation as binary fraction.
0,3125 in contrast has a exact binary representation (1/4 + 1/16).

So computers can perfectly compare numbers, so the problem is not comparison. But they cannot represent all numbers.

 

[shelvacu]

This is not true.
The Lua reference describes the behavior of the length operator:
Lua 5.3 Reference Manual

While the results of the length operator of a “table with holes“ are practically useless because they are implementation dependent and hard to predict it is by no way undefined. Especially it cannot have side effect like aborting the program or changing the contents of the table it is invoked on. The result can also be only less than the number of elements in the table, never more.

Yes, I glossed over that for dramatic effect. It depends on which version of Lua is in use; Lua 5.2 says:

Unless a __len metamethod is given, the length of a table t is only defined if the table is a sequence, ...

Thus, if a __len metamethod is not defined and the table is not a "sequence", then technically it's undefined behaviour. However I suspect that in the standard implementation of Lua it's the same behaviour as Lua 5.3, and that it's a binary search forward then backward.

 

[Bonfire]

The reference manual seem to be clarified with 5.3 and later.
Your warnings would be right with some undefined behavior in C, it has a lot of cases where undefined behavior can indeed have severe side effects. E.g while dereferencing NULL on most operating systems just lead to an OS exception, on microcontrollers without memory protection can be undetected and lead to memory corruption.

Lua as interpreted language is a lot more forgiving

 

[stetre]

I would say that the length of a table with holes is an undefined concept, rather than an undefined behavior
(for example, should the length of {10, 20, nil, 40} be 4, the highest index, or 3, the number of elements)?

 

[Paul Merrell]

Don't feed Lua or Luajit unicode strings unless you are using a unicode module in conjunction. A good pure Lua utf8 module is the pure Lua utf8 library developed by starwing available at starwing/luautf8 (.) It provides substitutes for the Lua string library's functions that are unicode-insensitive.

We have embedded the module along with Lua in NoteCase Pro for many years without a single report of an issue.

 

[dinsdale247]

Don't feed Lua or Luajit unicode strings unless you are using a unicode module in conjunction. A good pure Lua utf8 module is the pure Lua utf8 library developed by starwing available at starwing/luautf8 (.) It provides substitutes for the Lua string library's functions that are unicode-insensitive.

We have embedded the module along with Lua in NoteCase Pro for many years without a single report of an issue.

I'm not knowledgeable in unicode issues but Lua 5.4 seems to have at least partially addressed that?

From the release notes:
utf8 library accepts codepoints up to 2^31

 

[dinsdale247]

I would say that the length of a table with holes is an undefined concept, rather than an undefined behavior
(for example, should the length of {10, 20, nil, 40} be 4, the highest index, or 3, the number of elements)?

Except that it is defined. You will get as many items as available up until the hole. If you are using a non integer key, that means it uses a hash lookup and the hole will be indexed accordingly. Us meat-bags may think that the result is undefined, but it's not. It is the same behaviour on all platforms (Windows, Linux, Arm, Intel, AMD) and under all circumstances.

They had proposed a LUA_NILINTABLE compile option early in 5.4 working releases that would allow nils and defeat this problem. They scrapped that and there was a second (reverse) proposal but I don't see any compile flags in the newly released 5.4.2 sources.

 

[GavinW]

One of the troubles with floating point arithmetic is that addition is not associative; i.e. x+(y+z) may not be equal to (x+y)+z - and the same goes for multiplication. However, there are representations of real numbers for which the associativity laws (and the other laws of algebra) hold precisely. Look up the work of Abbas Edalat, and of his student John Potts. The problem is that though there is an infinity of possible ways of representing real numbers in a computer's memory, the only systems that most people have ever encountered are the floating point representations, and it does not occur to them that there are others. Which choice of datatype to take depends, as always, on what you want you want to do with their values.

As Bonfire pointed out, there are problems with using rational numbers (fractions) because the numerators and denominators can get very large if you insist on exact numbers. Recall that a real number is specified by which rational numbers it exceeds, and which it is less than; both being infinite sets. A general real number requires an infinite amount of information for its specification. Edalat's system supposes that you start your calculation by giving a measure of how precise you want an answer - say precision to a fixed number of binary points. Numbers are then given by a lazy stream of rational approximations. You need a lazy programming language, such as Haskell, to do this tidily.

Very few people who have not studied pure mathematics have a proper understanding of what real numbers are. Sadly, engineers and computer scientists are not usually among them. A friend of mine once offered a barrel of beer to any student in his class who could give an articulate explanation of how to define (not necessarily calculate) the number 2 to the power of the square-root of 2. The beer was never taken.

 

[stetre]

Except that it is defined. You will get as many items as available up until the hole. If you are using a non integer key, that means it uses a hash lookup and the hole will be indexed accordingly. Us meat-bags may think that the result is undefined, but it's not. It is the same behaviour on all platforms (Windows, Linux, Arm, Intel, AMD) and under all circumstances.

Still, it is implementation dependent and so you'd better not rely on this behavior and assume what the manual says: "When t is not a sequence, #t can return any of its borders."