Next up in our NXT Dev Blog series, it's Mod Dunk telling us all about the changes needed to achieve seamless movement in the new RuneScape client.

---

RuneScape is a huge game filled to the brim with characters to meet, quests and regions to explore. 15 years of content weighs heavily on the current Java client, and NXT aims to bring you a modern engine that can efficiently handle RuneScape’s sheer size – now and in the future.

Loading is a problem in game development – particularly for open-world games like RuneScape. Since a player could go in any direction, and the area is too large to keep in memory, the game has to smoothly load assets without interrupting the gameplay.

Now, this is where you might think, 'Wait, I’m pretty sure RuneScape pauses the game to load' – and that’s because it does. In fact, every RuneScape player is probably familiar with the ‘Loading – please wait’ box that pauses the game as you walk around.

This little box is no longer needed in NXT – here’s the 'load-down' on how we did it!



Streaming
The world of RuneScape is divided up into map squares. As you move, the squares ahead of your character are loaded in, while the ones behind you are removed to save memory. The Java client will halt the whole game as it does this, hence the pausing.

One of the main objectives with NXT was to make this loading happen in the background, never interrupting the gameplay. The technique for doing this is called 'streaming' or 'asynchronous loading' because it is done in parallel to the main process or thread of the game.

To do this, we encapsulate each loading request into a 'job' object, and hand each job over to one of a set of worker threads. These worker threads then process these jobs on their own and simply return the results when they’re finished.

Unfortunately, this required all of our processing for 3D models, animations, textures, particle systems, scripted objects - everything in the game - to be rewritten. As you might imagine, this was a huge task, but definitely worth the effort because it allows us to offer smooth movement around the world that would be impossible in the Java client!

Packing
Being able to load many fragments of the world quickly is an essential requirement for supporting our extended draw distances in NXT.

The maximum draw distance in NXT is more than 4 times the Java draw distance, and this means that NXT requires roughly 16 times the number of models loaded at any one time. Loading in parallel is one thing, but the sheer number means that we need to load each individual bit faster too.

With every model that Java loads, it does a lot of processing to calculate the information necessary for its rendering. As with any game, every 3D model is made up of triangles that form its visible surface. The 3 corners (or vertices) of a triangle each contain information such as a surface normal, a colour, and texture coordinates.

In the Java client, the loading code runs algorithms to calculate these values, which can be very slow. For NXT, we now pre-compute them in an external packing stage, and simply read the values directly at runtime. This greatly speeds up loading times (but increases the size of the assets that must be downloaded), and is our preferred approach whenever possible.

Similarly, we have an external packing stage for compressing textures, and this means less data to process at runtime.



Above: Java view distance. Below: NXT view distance.

Preloading
In the Java client, reaching the login screen itself requires a loading screen. In NXT, this time has been cut down significantly. We now do a lot more of the necessary work while the player is in the lobby. At this stage, the computer is not doing anything other than rendering the menu anyway, so we have spare time to pre-load as much as we can.

The main work done here is in pre-loading the world. If you logged out in Lumbridge, for example, the client will pre-load that area in the background, as we know that is where you will be standing when you get back into the world.

Another thing done at this stage is shader preloading. Shaders are small programs that run on the graphics card and perform rendering calculations. In NXT, we maintain a list of commonly used shaders and make sure that they’re available before play begins.

Map Square Level of Detail
In NXT, we optimise the loading for distant map squares by only building and rendering the most important objects. For example, the walls of a building should always be visible at a distance, but the interior details of the building don’t need to be loaded until you get close enough to see inside it. This 'level of detail' system is very effective at speeding up loading and rendering when draw distances are so long.


Large objects such as Armadyl's tower are still loaded from far away, but small detail such as grass or flowers around it would not be

Shader permutations
RuneScape has many objects in the world, made from various materials. Each material can use a combination of visual effects, such as environment mapping, texturing, specular lighting, alpha transparency, emissive lighting and more.

In NXT, we have introduced further features, including hardware animation, improved fog, and shadow mapping. Allowing every feature to be toggled independently caused many hundreds, or even thousands of shader permutations (or variants) to be generated. Thankfully, we have whittled this number down to around 100, and further code to reduce this is in development.

We achieve this in a few ways:
1) We identify the shader features that are producing the most permutations, using in-house shader debugging output.

2) Highlighted features that are cheap to use are simply made available all the time.

3 )Highlighted features that are expensive to use can be changed to switch themselves on or off inside the shader program itself.

A big issue with generic game engines such as Unreal or Unity is that a user can easily create different materials that require many thousands of shader permutations, increasing the time it takes to load them all. Since NXT only has to deal with a very specific (though large) set of data, we can optimise our rendering and shader code to perform the job more efficiently by generating fewer permutations.

All of this work has been towards a worthy goal – to make sure that the NXT client can show much more of the RuneScape world, and still beat the Java client on loading times.

Mod Dunk
Senior Engine Developer