Wednesday, 6 May 2015

Automated testing in game development

Automated testing in game development is practiced by some companies, but it is, to my knowledge, not common and not discussed much on the web. I am attempting to apply my four years of experience as a developer at LShift, which is not a games company, to my new life as an indie game developer.

First, is automated testing a good idea at all?
Automated testing is a frequently controversial topic in software development. I have had plenty of discussions with other developers outside the game world where testing has been met with skepticism.This is a big topic, covered elsewhere in depth. Just a few notes from me.

Writing tests does slow you down. It will take you longer - perhaps twice as long - to write tested code as untested code. Definitely in the short term you will make less progress. But in the mid to longer term you will start regaining the lost time, and eventually it will generally make things faster. This is because when your code is thoroughly covered by tests you can refactor and extend with ease.

On projects I have worked on I wouldn't hesitate to sit down in the middle of a large, complex system I hadn't touched in months and make changes to its core functionality. If the tests passed, I'd be happy to deploy it straight out to our client. Whereas without testing I would have had to spend more time reacquainting myself with the code, and double and triple checking my understanding and my changes to make sure I haven't introduced inadvertent side-effects. I have found testing invaluable, because it means I don't have to worry about breaking things - the tests will catch that for me.

In my game, I am not writing very many unit tests, except where individual classes or functions have unusually complex behaviour. Instead, I am largely writing higher-level tests of game logic - they might be called integration tests.

Here's what a moderately complex test looks like. I'm writing in C# using NUnit as my testing framework.
public void CanShootOtherHuman(PeerGraph graph)
    var harness = graph.MakeHarness();
    var shooter = makeHuman(new Position(0, 0));
    shooter.Weapon = new LaserWeapon(shooter, new Ammo(AmmoType.Laser, 35),
        new PhysicalAttack(1));
    var target = makeHuman(new Position(0, 100));

    shootTarget(harness, graph, shooter.Yield(), target);
    harness.RunUntil(new HasDecreased<double>(() => target.Health.Amount,
        "Target avatar has taken damage"));
As you can see, this test walks through one player shooting another. The test's assertion is contained within the line that begins harness.RunUntil. This checks that when one player shoots another their health decreases. If this doesn't become true within a certain time period the test fails.

The way these tests work is by setting up a harness that runs the full game engine, but with a very simple, pre-determined scenario - the only elements in this test are two player characters and, briefly, a laser beam. My engine is designed to be able to vary the simulation speed, so I run the tests at about ten times normal game speed. I also run the tests without any rendering, although I can turn rendering on for a test when I want to understand what's going on.

My game is multiplayer between computers, so I use NUnit theories and that PeerGraph parameter to verify the test works in different multiplayer setups. Each test is run with multiple different PeerGraphs - that is, different network topologies. In each one the shooter and the shootee will be on different machines (well, mocked out machines). In one the shooter is the server, and the other a client. In another it's the other way around. Still another will check that a client can shoot another client, and that the server (not actively participating) relays the state correctly. Network code is always complicated, despite my best efforts to keep it simple, so being able to check that the game's logic works the same way across different networking situations is very helpful.

Here's another helpful test:
public void CannotWalkThroughCreatures()
    var distance = SpaceGrid.DefaultSize * 2;
    var a = engine.Add(new SimpleCreature(new Aspect(new Position(-distance, distance)), 500));
    var b = engine.Add(new SimpleCreature(new Aspect(new Position(+50, 50)), 50));
    engine.FocusedElement = new MockPointOfView();

    var meetingPoint = new Position();
    harness.Do(new GameAction[]
        new GameAction(() => a.MoveTo(meetingPoint)),
        new GameAction(() => b.MoveToAndKeepGoing(meetingPoint), new IsFalse(() => a.IsMoving, 
            "Don't move until the other guy has stopped")),

    harness.AddInvariant(new IsTrue(() => a.Aspect.Position.X < b.Aspect.Position.X,
        "Creatures never move past each other")); 
    harness.RunUntil(new ValueHasBeenTrueFor(() => 
        a.Position.DistanceTo(b.Position) < a.Size.Width * 2,
        TimeSpan.FromSeconds(1), "Reach each other and don't separate"));
This test is one of a raft of tests that check the collision detection is working. You can see here that the harness helper I wrote allows you to add invariants as well as conditions that must be true before the test finishes. This one is the real meat of the test, as it checks that the two creatures can't walk through each other.

The harness clocks in at about 300 lines of code, plus the classes for the various invariant/end conditons, like "ValueHasBeenTrueFor". I won't say that these are easy tests to write, although I am getting quicker. It's not uncommon for me to spend an hour or two trying to figure out why one is not producing the results I'm expecting. But they give me confidence that once I have finished a subsystem, and written tests for it, it really is finished. I can change other parts of the game and add new features without worrying that a previous part will stop working.

Are you using automated testing in a game? Please tell me in the comments; I'd love to hear about it.

No comments:

Post a Comment