How to Choose a Game Engine
// October 6th, 2012 // Robofish
Good news: Jaret and I have begun work on Sparkrift’s next game! Bad news: it’s going to be a while before you can play it. Good news: the game is going to be awesome! We’ve been using the past couple of months to start laying the foundation for our next game. Jaret is doing a lot of concept art, and I’ve spent a lot of time looking into game engines and graphics frameworks, which brings us to to the topic of this post.
There are a lot of game engines and graphics frameworks to choose from, so how can you pick the right one? Maybe you shouldn’t even pick one; why not just write your own from scratch? That way you’ll have complete control and can tune it to match your needs exactly! Only you can decide what’s right for you, and I won’t go into writing an engine versus writing a game in this post. Here I’m going to walk you through our experience with Robofish.
Our goal for Robofish was pretty simple: prove that we work together and make a game that runs on the Xbox 360. Short of paying thousands of dollars for an official Xbox 360 developer kit that leaves you with XNA. That was fine with me because I’m a fan of C#, and since Robofish was our first game I didn’t think I’d be able to code it in OpenGL or Direct3D directly without it taking an extremely long time. After looking into XNA I realized that it’s really just a framework. That was good, but I knew if we used a game engine we’d probably finish the game even sooner.
Cue Torque X 2D. For a while XNA had a partnership with Torque where they would provide XNA developers with free access to the Torque X 2D game engine binaries. So I checked it out, read through the tutorials, coded up a little test to see if I could be productive, and it looked good! In only a few days I had gone from never creating a game to being able to move a little box around the screen and “shoot” smaller boxes. This was going to be cake!
I showed Jaret my fantastic demo and he was sold! After working on Robofish for a few months, I became a little frustrated with the documentation though. The tutorials were pretty decent, but there were only half a dozen or so, and they didn’t cover all the things I wanted to do in Robofish. Worse though was the fact that engine API documentation was practically nonexistent. I had no idea what features were available to me through the Torque engine. Posting on the Torque forums would lead to a handful of responses from a few dedicated Torque users if you were lucky.
Well I really needed to know how to use this engine, so we decided to buy the source for the engine, which was an option at the time. Once I had the source I was able to divine some of the meanings of the API calls. The source was decently documented, so that was how I proceeded. It was slow, and I was very unfamiliar with game engine frameworks and 2D graphics in general. Luckily I found a few things that seemed to work for the graphics and stuck with them while working on things like weapons and fish AI which didn’t need much from the engine. Things were going fairly smoothly until I realized XNA 4 was coming out soon.
Hurray, an upgrade to XNA! That should bring better performance and lots of cool new features and bug fixes, right? Well sure, it did some of that but there was a problem. Microsoft stated that you wouldn’t be able to publish XNA games on Xbox 360 unless they were using XNA 4 after a certain date. Robofish was never going to be done by that date. That meant we needed to upgrade to XNA 4. Torque X 2D was based on XNA 3, and I knew it wasn’t going to compile if I linked it with XNA 4. Well crap, to the Torque forums! Bad news led to worse news as I discovered that Torque was discontinuing support for the Torque X 2D engine. Holy crap, this game that we’ve been working on for over a year isn’t ever going to make it to Xbox 360 now!
You know that wasn’t the case though, so how did we do it? Not alone, that’s for sure! Torque X 2D was ported to XNA 4 by some very dedicated and generous Torque X 2D users. They named the port Torque X 2D CEV, which stands for “Community Enhanced Version”. Without their dedication I don’t know what would’ve happened. I doubt I would’ve been able to port the engine to XNA 4 by myself. Unfortunately the problems with Torque didn’t end there.
Performance was the largest problem we ran into with Torque. Maybe this isn’t an actual Torque problem because I don’t think they advertised blazing speeds with their engine. As an amateur I just assumed that if they were making this engine available to develop games for Xbox 360 that it would perform well. I did not benchmark it, which was my own fault. If I had benchmarked it at the start I would’ve discovered two things: the number of sprites that can be drawn to the screen is severely limited and games deployed to the Xbox 360 encounter massive stuttering periodically.
I was, and still am, extremely angry about the sprite drawing performance with Torque X 2D. It turned out that we were limited to somewhere around 200 to 300 sprites on the screen at once if we wanted to stay close to 60 FPS. That may be more than enough for a lot of games; it is not nearly enough for a SHMUPS. The number of sprites shown on the screen can add up quickly. Robofish had enemy sprites, weapon projectile sprites, effect sprites, and clam sprites on the screen all at once. The largest number of sprites comes from the weapon projectiles.
Let’s say you have a weapon that shoots 8 projectiles at once. Each time the weapon fires that’s 8 new sprites to draw to the screen. Depending on the weapon rate of fire and the movement speed of the projectiles there could be up to or more than 200 projectiles, and their corresponding sprites, on the screen at once. That’s pretty much all of available sprites right there. So we had to be really careful about limiting exactly how many weapon projectile sprites could be on the screen at once. Don’t forget that some of the enemies shoot projectiles as well. Get a few of those on the screen and that’s dozens more projectiles right there as well.
We also wanted to add particle effects to the weapons, which we did have in there at one point. The weapons looked awesome with particle effects, and it killed us to take them out, but we just couldn’t maintain a playable framerate with them in there. So why couldn’t Torque X 2D handle very many sprites on the screen at once? After investigating the engine source code I discovered that they weren’t using any kind of sprite batching. This means that each sprite gets its own GPU draw call, which is extremely inefficient. The difference in the number of sprites that can be drawn one at a time versus batching them together is a couple orders of magnitude; it’s that big of a deal. So if we were limited to 200 sprites before, if we had been using sprite batching we could’ve potentially been drawing 20,000 sprites at once with similar performance.
The second major performance problem in the Torque X 2D engine was the stuttering we encountered whenever we ran Robofish on Xbox 360. Anyone who has ever developed a game for the Xbox 360 using XNA knows what’s coming next: the garbage collector. Other people have written on the Xbox 360 .NET garbage collector before, so you can read about it from them if you like. The short version is you have two options when it comes to garbage collections: don’t create any garbage so the garbage collector doesn’t have to run or limit the number of objects on the heap so that the garbage collector can run quickly.
Running Robofish on top of the Torque X 2D engine meant the second option wasn’t really available to us. There were already tens of thousands of objects in the heap, so whenever the garbage collector ran it took a few milliseconds, sometimes up to 40 or 50 milliseconds, which is around 3 frames. Not drawing anything to the screen for 3 frames or more would definitely cause noticeable stutter for us. So if a lot of garbage was being generated by the game the garbage collector would run to free up memory and cause Robofish to stutter quite frequently. It wasn’t fun to play in that state, so I began to dig into the engine to find out where this garbage was coming from.
After digging into the Torque X 2D engine for weeks and months I basically came to the conclusion that whoever developed the engine hadn’t tested it very well on the Xbox 360. Maybe not at all. Maybe it was still a work in progress; I don’t know. What I do know is that I ended up making dozens of fixes to the engine source itself to keep it from creating garbage. One of the biggest improvements was the object pooling system.
Since Robofish’s weapons shoot so many projectiles, a good way to save memory is reuse the projectiles after they’re off screen. Instead of creating a new projectile when the weapon is fired and destroying the projectile when it hits an enemy or goes off screen you can turn the projectile into a new projectile. Recycling! Great idea in practice, but it wasn’t implemented very well in the Torque X 2D engine, and it would still leak garbage during the recycling process.
Even after all the time I spent trying to improve the engine I never truly got rid of the garbage allocation. I just got it into a state that was good enough to release. I’d like to note that both of the performance issues were not caused by Torque X 2D alone. There were many things I personally did wrong that caused performance issues. However, after I had fixed those issues I still had to go into the engine to fix what was left before the game was playable.
The point of this post is that game engines can be a blessing and a curse. I’m not sure if Torque X 2D saved us work or delayed our game release after everything is said and done. I suspect, even with all the issues we had, that we still came out ahead by using the engine… barely. That’s probably due more to me being a complete newbie to game programming at the start of the project, so it would’ve been really rough to create an engine for Robofish from scratch.
After thinking about it a lot, there are many things we could’ve considered to prevent the issues we ran into:
- Performance: We should’ve benchmarked the engine to find out if it would meet our needs. If we had done that at the beginning we would’ve caught most of the problems we ran into, and we could’ve avoided them.
- Community: The size of the user base and how active they are makes a big difference. If no one is using the game engine it’s probably for a reason. Having a large number of users also means quicker answers to questions you post on the forums.
- Documentation: If a game engine isn’t well documented it’s not a good sign. If you’re designing a game engine to be used easily you want to provide good docs to make it easy for the developer. I don’t want to use a game engine, or any library, that doesn’t have good docs; it’s just too much of a pain.
- Source: When the docs don’t provide enough information you can always fall back on the source… unless it’s closed. If you have the source you can also implement bug fixes yourself if the engine developer is too slow to provide fixes. Although if the developer is slow to provide fixes that’s a bad sign in itself.
- Maturity: If a game engine is brand new it’s probably still going to be changing, full of bugs, and not well documented. So just keep in mind that the newer something is, the less tested it will be and the more likely you’ll be doing the testing yourself.
- Features: The whole point of using a game engine is to save time. Verify that the game engine provides the features you will need for your game and test them to see that they work well.
Keep these factors in mind when choosing a game engine, and I think you’ll avoid most of the problems we ran into with Robofish. One final note on Torque X 2D, though: I tried to specifically call out the fact that we used Torque X 2D because there are lots of other Torque engines out there for other platforms. I have no comments on other Torque engines because I haven’t used them.
Stay tuned for my follow up post where I post some benchmarks on current game frameworks to show how we determined what framework to use for our next game!