More graphics library stuff…

Here is a quick update on Knights development.

Unfortunately I have still been struggling with graphics libraries for what feels like ages now.

Last time I posted that I was trying out SFML and have been doing quite well with it. Unfortunately I have found what seems to be a fatal problem – SFML is completely unable to detect keypresses of the “backquote” key (that is, the key above TAB). This is important for Knights because we use that key as a chat key. Unfortunately I haven’t found any way to work around this, so it looks like we won’t be able to use SFML for Knights right now.

I also looked into SDL 2. However, I found the rendering performance to be very slow. It seems that to get decent performance out of SDL 2, you have to be making direct OpenGL calls in your code (instead of relying on the so-called “SDL_Renderer” layer). Unfortunately this would be quite a lot of work (especially since I don’t really know any OpenGL at the moment…) so this is not really feasible for me right now.

Therefore I think the plan will have to be:

  1. I will keep Knights as it always was — i.e. using DirectX on modern windows machines, and using SDL (version 1 – which uses software rendering)ย on Linux or older versions of Windows.
  2. I have made a few changes to my DirectX code — in particular I have removed the dependencies on D3DX and D3DCOMPILER. This should mean that the code will run on any modern Windows machine without any “DirectX runtime library” downloads being required. (I have also put in some simple UTF-8 text handling, but that is another story…)
  3. At some point I will get in touch with the author of SFML to see if the keyboard handling can be improved. That would eventually mean that I could switch Knights over to using SFML on all platforms. Not only would this bring hardware accelerated graphics to Linux (finally!), it would also mean that I only have one lot of graphics code to maintain instead of two, which will make things easier. This is a longer-term goal though.

Knights ADF available

I got contacted a couple of months back by someone who said “I would love to play the original Knights on my old Amiga 500 but I can only transfer ADF Files to it“.

I had no idea people still played Knights on old Amigas (anyone remember setting up the piece of cardboard to divide the screen in two? ๐Ÿ™‚ ) but I was happy to help, so I made a bootable ADF from the Aminet version of Knights and sent it to him. The ADF file is now available for download. Enjoy ๐Ÿ™‚

Note that to use this you will need either a real Amiga or an emulator. Regular users will probably want to check out the PC version available from http://www.knightsgame.org.uk/.

Investigating SFML for graphics…

So, recently I bought a new laptop. And with it being a new year and all, I thought this might be a good opportunity to get back to some Knights development (something that, sadly, I haven’t had much time for these last few months). So, I installed the latest Visual Studio onto the laptop, set up the Knights code directory, and prepared to do some coding…

… and immediately hit a problem. The game would not build. Partly this was because the Microsoft compiler now supports C++11 (the latest version of the C++ language), which caused a few incompatibilities with my code (admittedly this was mostly my fault though!). And partly it was because Microsoft have changed the way DirectX works – in particular, “D3DX” is no longer a supported technology. Unfortunately for me, I had some code that was using D3DX… which now no longer compiles. Grr. (At least on Linux, once you get your code working, it tends to stay working… ๐Ÿ™ )

Well, I could just fix my DirectX code, but I thought it would be better in the long run to bite the bullet and set up a proper cross-platform solution for graphics. (The initial reason why I added DirectX support to Knights was just as a quick way of testing out hardware-accelerated graphics on Windows; it was never meant as a permanent solution.)

So, if one is looking for a modern 2D graphics library there seem to be two choices: SDL2 and SFML. I had a look at both of these, it seems to me that they both do roughly the same thing, but they have different kinds of API – SFML tries to be more modern and object-oriented and so on, while SDL is more “old school” with low-level C-style functions and structs and things. Personally I prefer the former, so I decided to try out SFML.

I have to say, I was not disappointed – SFML was very easy to set up and integrate into the game and seems to work very well. Performance was much faster than software rendering, although admittedly slightly slower than I got from my DirectX code (not sure why this is; presumably it’s just the overhead of having an extra library in between you and the graphics card; in any event, it’s not that big a deal and I think it’s a price worth paying given how easy to work with SFML is).

I also had some small keyboard handling problems with SFML (e.g. it seems the ` key – the “team chat” key in Knights – is not recognized, but I think I can work around that).

Overall, therefore, I hope to have the SFML integration work done in the next few days, which means the next version of Knights will support fast hardware-accelerated graphics in both Windows and Linux. (As a side note, there is also a problem on some Windows machines where the Knights DirectX graphics doesn’t work if you don’t have the right DirectX runtime libraries installed… that should also be fixed by the move to SFML.)

For more information about Knights please visit http://www.knightsgame.org.uk/.

 

Knights version 23 released!

Hi everyone,

I decided that it was about time to do another Knights release (has it really been a year since the last one?), so I have just released version 23.

This is just a small bug-fix update. The main bug that was fixed was that I finally tracked down why the server list page was sometimes showing 0 players online, even when people were online. This turned out to be an overflow problem (basically, after the game had been running a few days, a 32-bit timer would overflow and this would prevent any further updates being sent after that point).

There are also a couple of minor fixes/changes for the Linux build.

Most users won’t need to upgrade, although if you’re running your own server you should upgrade in order to get the server list fix. (I’ve already applied this to the official server.)

Hopefully next year I’ll get a bit more time to work on the game!

The new release is available from the download page.

About Knights

Knights is a multiplayer dungeon bashing game. Players must explore randomly generated dungeons and compete to solve various quests. For more information please visit http://www.knightsgame.org.uk/.

Knights version 22 released!

I am pleased to announce the release of a new version of Knights.

This release adds a number of new Lua scripting features, and it also fixes a couple of bugs. See the download page for full details.

Happy new year to everyone ๐Ÿ™‚

About Knights

Knights is a multiplayer dungeon bashing game. Players must explore randomly generated dungeons and compete to solve various quests. For more information please visit http://www.knightsgame.org.uk/.

Knights new version coming soon…

Just a quick Knights update here. I have been away from Knights for a while (sorry about that) but I do have a bit of spare time over Christmas and I intend to use it to make a new Knights release.

The plan is to include the following features (the numbers refer to Trac):

  • Implement the “Monster Proposal
  • #215 – Add GetRespawnFunction to Lua API
  • #203 – Error message or warning if Lua “prefix” forgotten
  • #206 – Update manual to refer to new Boost version
  • #164 – Ability to cycle each observer window independently

Hopefully I can get this done fairly quickly, as I know some people on the forum have been waiting for a new release for some time now…

After that I’d ideally like to get the map editor up and running again, as this would complement the Knights Lua scripting features nicely. This could really open the door for people to make new rooms or maybe even entire custom maps for Knights. That will be a task for next year though ๐Ÿ™‚

About Knights

Knights is a multiplayer dungeon bashing game. Players must explore randomly generated dungeons and compete to solve various quests. For more information please visit http://www.knightsgame.org.uk/.

Knights version 21 released!

A new version of Knights has been released!

This time there are two major new features:

  1. There is a brand new Tutorial level. This should make it a lot easier for newbies to learn how to play. Old timers might also like to play through the Tutorial, as there are a couple of interesting challenges in there ๐Ÿ™‚
  2. Lua support has been added. This allows players to make their own modifications (mods) for the game. Already some players have created mods, adding things like spiders, living armours, and a hilarious (imho) new wand type. To see what people are doing, check out the forums.

As usual, head to the download page for downloads and detailed release notes.

P.S. Due to a technical issue the official Knights game server is currently offline. I’ll fix this later today.

About Knights

Knights is a multiplayer dungeon bashing game. Players must explore randomly generated dungeons and compete to solve various quests. For more information please visit http://www.knightsgame.org.uk/.

Notes on Lua/C++ error handling

This is a technical post about how to handle Lua errors in C++ code.

Background

Recently I added support for Lua scripting in Knights. During this process I found out that there are several “gotchas” in terms of how to correctly handle Lua errors in your C++ code. I’m writing this blog post (1) as a way of documenting the issues I found, and (2) because it might be helpful to other programmers who run into similar issues.

I will present this as a series of problems that can occur, and possible solutions to each.

Problem 1: C++ destructors may not called when doing a “longjmp”

The scenario:

Your C++ code calls a Lua API function. Something goes wrong and a Lua error is generated. This causes Lua to do a “longjmp” through your code. Your C++ code contains at least one local variable with a non-trivial destructor.

The problem:

The problem in this scenario is that “longjmp” does not guarantee that destructors will be called. Clearly, this is bad news if you were relying on the destructor to clean up memory or other resources.

Example code:

int MyFunction(lua_State *lua)
{
   std::vector<int> v(10);
   // ...
   lua_call(lua, 0, 0);ย  // call some lua function
   // ...
   return 0;
}
void Foo(lua_State *lua)
{
   lua_pushcfunction(lua, &MyFunction);
   int result = lua_pcall(lua, 0, 0, 0);
   // ...
}

Discussion:

Imagine that the lua_call above (in MyFunction) results in a Lua error being raised.

At first sight this appears perfectly OK. Lua will handle the error by doing a longjmp. As per the specification of lua_pcall, the stack will be unwound back to the lua_pcall statement in Foo. An error code will be returned into “result”, and execution will continue, with an error message sitting on top of the Lua stack.

The problem is that, on some compilers at least, longjmp() does not call C++ destructors as it unwinds the stack. On such a compiler, the destructor for vector “v” would not be called, and we would have a memory leak.

Indeed, the C++ standard says this:

The function signature longjmp(jmp_buf jbuf, int val) has more restricted behavior in this International Standard. A setjmp/longjmp call pair has undefined behavior if replacing the setjmp and longjmp by catch and throw would invoke any non-trivial destructors for any automatic objects.

(Note: I have seen different wording for this in different places; the above comes from a Stack Overflow answer).

If you think about it, this means that most C++ programs that use longjmp would have undefined behaviour. Considering that Lua uses longjmp ubiquitously for error handling, this is not ideal ๐Ÿ™‚

The solution:

The easiest solution is to compile Lua as C++. (In Visual Studio, use option /TP; in GCC, compile the code with g++ instead of gcc.) When Lua is compiled as C++, it uses try/catch instead of setjmp/longjmp for error handling. Problem solved.

Of course, compiling Lua as C++ might not always be practical. For example, we might be working with an external Lua library that has already been compiled as C, or we might want to disable C++ exception handling for performance reasons. In these cases, the only real solution is “don’t do that”; in other words, don’t create stack objects in such a way that destructors would need to be called if there was a Lua error. Of course, this requires some careful programming; if you are going down this route, it would be worth reading this mailing list thread where the issue and some possible solutions are discussed.

What does Knights do?

In Knights I opted to compile Lua as C++. This is the simplest solution, and the overheads of exception handling have not proved to be a problem in practice (remember that Knights is only a simple 2D game with low CPU and RAM requirements in any case).

Problem 2: C++ exceptions should not propagate through Lua

The scenario:

We have written a C++ function and made it available to Lua (via lua_pushcfunction or lua_pushcclosure). Our C++ function can throw exceptions.

The problem:

If Lua calls our C++ function then there is the risk that an exception might be thrown and propagate up through the Lua source code. This is a problem because Lua is written in C, and therefore cannot be assumed to be exception safe in the C++ sense. Although we cannot know for sure that throwing an exception through Lua will be unsafe, we should err on the side of caution and assume that it is unsafe unless proven otherwise.

Example code:

int MyFunction(lua_State *lua)
{
   throw std::runtime_error("Boo!");
}
void Test(lua_State *lua)
{
   lua_pushcfunction(lua, &MyFunction);
   lua_pcall(lua, 0, 0, 0);
}

This code will cause a std::runtime_error to propagate up through the Lua implementation, which is a bad idea for the reasons mentioned above.

The solution:

There is only really one solution, which is to ensure that C++ functions exposed to Lua do not throw exceptions (i.e., they should have the nothrow guarantee).

The simplest way to ensure that is to enclose each function in a try/catch block, e.g.

int MyFunction(lua_State *lua)
{
   // This function is to be exposed to Lua
   try {
      // insert code here
   } catch (std::exception &e) {
      luaL_error(lua, "C++ exception thrown: %s", e.what());
   } catch (...) {
      luaL_error(lua, "C++ exception thrown");
   }
}

This code converts any C++ exception thrown into a Lua error. (If the exception is derived from std::exception, we construct a more useful Lua error message by calling std::exception::what(); otherwise, we just use a generic error message.)

What does Knights do?

One objection to the above is that it would be tedious to write those try/catch clauses in each and every function that we wish to expose to Lua. In Knights, I addressed this by defining a wrapper function “PushCClosure” that is a replacement for lua_pushcclosure. This automates the task of wrapping the enclosed C++ function in a try/catch block. Those interested in the implementation of PushCClosure may wish to refer to its source code.

Problem 3: Avoiding Lua “panics”

The scenario:

C++ calls a Lua API function, and the Lua API function raises a Lua error. Unfortunately, the C++ code did not call Lua in the so-called “protected mode” and therefore, Lua “panics”.

The default action after a panic is for Lua to abort the program. Clearly, this is undesirable in production code.

Example:

The following code illustrates the problem. The code is supposed to read the contents of the global variable “EXAMPLE” and return it as a std::string.

std::string Example_Bad(lua_State *lua)
{
   lua_getglobal(lua, "EXAMPLE");ย ย          // push value of EXAMPLE onto the stack
   const char *c = lua_tostring(lua, -1);   // read stack top (as char *)
   std::string result = c ? c : "";ย ย        // store to a std::string
   lua_pop(lua, 1);                      ย ย  // restore the stack
   return result;
}

Discussion:

The problem with the above code is that the calls to either lua_getglobal or lua_tostring might raise an error. For example, a maliciously-minded user might write the following:

local m = {}
function m.__index(tbl, key)
   error("Boo!")
end
setmetatable(_G, m)

Then, the call to lua_getglobal will execute the __index metamethod, which in turn raises an error. The C++ code does not handle the error, and the program aborts.

The solution:

Unfortunately, there is no way to directly check for errors from a Lua API function. Instead, you are supposed to call the Lua API in “protected mode” if you care about error handling. (This often traps beginners, who just write Lua API calls without thinking about what happens when there is an error.)

Calling Lua in “protected mode” is, unfortunately, somewhat tedious. You have to wrap your use of the Lua API inside a lua_pcall, as follows:

std::string Example2_Good(lua_State *lua)
{
   // Create a dummy struct which will be used for the pcall.
  ย struct Pcall {
      std::string result;ย  // Will hold the result.

      static int run(lua_State *lua) {
         // Read the pointer to the Pcall struct.
         Pcall *p = static_cast<Pcall*>(lua_touserdata(lua, 1));

         // Get the global "EXAMPLE" and store it in the Pcall struct.
         lua_getglobal(lua, "EXAMPLE");
         const char *c = lua_tostring(lua, -1);
         if (c) p->result = c;

         // Done.
         // (Note there is no need to pop the stack, 
         // Lua will do that for us when we leave the function.)
         return 0;
      }
   };

   Pcall p;
   lua_pushcfunction(lua, &Pcall::run);
   lua_pushlightuserdata(lua, &p);
   int result = lua_pcall(lua, 1, 0, 0);

  ย if (result != LUA_OK) {
      // Handle error here (e.g., throw C++ exception).
      // Don't forget to pop error message off the Lua stack.
   } 

   // Operation was successful. Return the result.
   return p.result;
}

As you can see, the general idea is to create a local struct, with a static “run” method, and then lua_pcall that method. A pointer to the struct is passed as the sole Lua argument; this allows you to read from and/or write to the struct from within the “run” method.

Importantly, neither lua_pushcfunction nor lua_pushlightuserdata can raise Lua errors, therefore they are safe to call outside of the pcall wrapper.

Note: Panic Functions

An alternative strategy would be to install a panic function, using lua_atpanic. However, it’s not really recommended to use panic functions for regular, everyday error handling. (For one thing, one is never quite sure what is left behind on the Lua stack after a Lua panic.) Instead it is better to use the lua_pcall method as described above.

What does Knights do?

In Knights I don’t currently wrap Lua API calls in a lua_pcall. (The exception is when I call Lua functions, where I do generally use lua_pcall instead of lua_call.) Therefore, if any top-level API calls trigger a Lua error, then a Lua panic occurs.

In Knights I have set up a panic function that throws a C++ exception. When this exception is caught, Knights assumes that the lua_State is unusable and closes it as soon as possible. The server then resets itself and execution continues.

While this method works, it is not particularly pleasant because any clients connected to the server do not receive any explanation of what happened. Their connections are just immediately closed when the Lua panic is handled.

Some time I would like to modify Knights to wrap all uses of the Lua API inside lua_pcall wrappers, as described above. This would ensure clean handling of all Lua errors. (The panic function would be kept, but it would only be there as a “backup” in case I ever forgot to put a lua_pcall wrapper in some place where it was needed.)

Further Reading

The Lua wiki has a page on Lua/C++ error handling:
http://lua-users.org/wiki/ErrorHandlingBetweenLuaAndCplusplus

Here is another blog post on this topic:
http://julipedia.meroh.net/2011/01/error-handling-in-lua.html

Knights Lua module system added

I’ve done quite a bit of work recently on making Knights Lua-scriptable.

  • Knights has been completely converted to use Lua for configuration. The old *.txt files in knights_data are gone.
  • “knights_data” has been split into separate “client” and “server” subdirectories.
  • A Lua module system has been implemented. Basically, a “module” is a self-contained directory that goes into knights_data/server.
  • Knights itself comes with a module called “classic” which implements the classic Knights gameplay that we all know and love.
  • Additional modules can be installed, by copying them into knights_data/server. These will modify the gameplay in various ways.
  • Modules only need to be installed on the server, not on the client machine. If the module contains graphics or sound files then these will be downloaded to the client machine as required.
  • As an example of what modules can do, I have converted ImpassIve_rus’s “multiwand” modification to be a Knights module. Installing this module on a server does two things:
    • The wand graphics are replaced so that each wand is colour coded.
    • Two new “Type of Wand” options are added: “Multiple Wands” (this adds the wand of securing and the wand of destruction at the same time); and “All Wands” (this adds all four wand types to the dungeon at the same time).
  • Users who know Lua programming can write their own modules. A module needs to contain at least one Lua code file, plus it can optionally contain new graphics and sound files.

There is a lot of documentation available on the wiki for those who want to make modules of their own.

Of course, this change is not without its disadvantages; the main one being that mod-making now requires a bit more knowledge, since you have to know Lua programming, whereas before it was just editing a txt file. However, in exchange for that, we get a lot more flexibility — you can do things in Lua code that simply aren’t possible in .txt files. I think the tradeoff is worth it.

If you do want to start Knights Lua programming and you find you don’t understand something, or run into other problems, then feel free to post on the Knights forum and I will try my best to help. I expect there to be a few “teething problems” initially, as the Lua API is new and no doubt has bugs and/or missing features. But with your help (in reporting issues), we can get those things fixed, over time ๐Ÿ™‚

Finally, just to be clear, note that the Lua version of Knights isn’t released yet. I aim to do that by the end of August.

Tutorial improvements

For my next trick I intend to add a better tutorial to Knights. Although the current tutorial is OK, it doesn’t really explain things in a very structured way and I think it probably blasts too much information at the player at once. The result is that people get confused, don’t really understand how to play Knights, and as a result, no doubt they think the game is “rubbish” and go do something else. Whereas if they had a better understanding, they would enjoy it a lot more.

For those reasons, I have decided to do some work on the tutorial. I posted some ideas in this forum thread for those who are interested. But anyway, that is what I am going to be working on for the next couple of weeks. After that the plan would be to do a release with the new tutorial and the Lua changes.