This weeks tech blog is courtesy of our friends over at Butterscotch Shenanigans, and they'll be talking about how to set up and manage data transfer between GameMaker Studio 2 games and a server.
Hey, Game Makers! I’m Seth Coster, the game programmer from Butterscotch Shenanigans. And I’m here to talk about... THE INTERNET. More specifically, how to use GameMaker’s HTTP functions in a manner that’s reusable, powerful, and clean... like a Tesla Roadster.
First, a little background. We recently launched a game called Levelhead into Steam Early Access. Levelhead is platformer-maker style game where you teach a robot how to deliver packages.
The game is made up of three core pillars:
- A single-player campaign
- A powerful level editor
- Web features to allow players to publish levels for others to play
When we set out to build Levelhead, we knew we were going to need some seriously robust solutions to handle web requests. The game is constantly talking with our server for leaderboards, stats, downloads, uploads, tags, searching, caching, etc. You name it, Levelhead is doing it.
Handling this many web requests in your game might sound like a lot to deal with, but it’s actually pretty straightforward if you have a good system to manage it all, which is what I’m here to talk about! So... LET’S GET IN THERE.
NOTE: For those who are more of the “learn by doing” types, I have included a link to download a .yyz file containing all the code from this article, so you can use it as a foundation for building your own custom HTTP controller. It might be helpful to download the .yyz file first, poke around in it, and then refer back to this article for explanations! Check the end of the article for the download link.
THE BASICS OF HTTP REQUESTS
What can make HTTP requests so dang difficult to deal with, is that they’re asynchronous (meaning you have to wait for them), and they require some external server to do its job.
Because of that, much like your old college roommate, you just can’t count on them all the time. Sometimes you’ll say, “Hey, Steve, if you’re running to the store, can you grab me some chips?” And Steve will say, “Yeah. I’ll get you those chips.” Then Steve comes back, and... Where are the chips, Steve? I don’t see any chips. In other words, we got a 404 error. Chips not found.
Or maybe Steve is gone for four hours. And sure, Steve comes back with the chips, but guess what? Steve took so long, you already ate a bag of candy and fell asleep, and you don’t even want the chips anymore.
Or perhaps Steve never comes back. Now what? How are you going to get those chips? And perhaps less importantly, where is Steve? Will Steve ever come back? Whatever ate Steve, we just call that a server timeout.
Just as there are a million things that could happen to Steve as he quests for your chips, there are a million ways an HTTP request could go wrong. And if you’re sending a lot of HTTP requests, you could go crazy trying to keep track of it all, unless you have a good way to do it. But before we talk about how to handle all that, let’s quickly cover the basics of GameMaker HTTP requests.
REQUEST IDS AND HTTP_REQUEST()
In GameMaker, you simply use the function http_request() to send a message to your server. What server? What message? Well, that all depends on how your server is set up! But this post is about the GameMaker side of all this, so I’ll just assume that you have a functioning server somewhere that you can send messages to.
When you call http_request(), GameMaker Studio 2 will return an id number, which we’ll call the Request ID. So your code may look something like this:
new_request_id = http_request(url, method, header_map, body);
We store the Request ID in a variable, so you can check in on it later. It’s kind of like calling Steve on his cell phone, so you can see if he’s gotten home with your tasty chips. To better keep track of it, we might name it something like:
chips_request_id = http_request(url, method, header_map, body);
THE ASYNCHRONOUS EVENT
After you’ve sent off an HTTP request, at some point, an Asynchronous HTTP event will trigger. This means your HTTP request has completed, and there are goodies (or errors) waiting for you to unpack.
Inside this Async - HTTP event, there will be a special DS map called async_load that contains all the information you will ever need about your HTTP request -- including the Request ID. This allows you to easily identify what this HTTP request is even for since you now know which Request ID it’s associated with. In our Steve example, we would ask whether the ID of this particular Async event matches with chips_request_id. If it does, we know to check for chips! Then, we just dunk our hand into the async_load map and pull the chips out. (We’ll talk about how to do this a little later.)
So, to recap the basic structure of performing an HTTP request to a server:
- Send off an HTTP request using
http_request() - Store the Request ID number in some sensibly-named variable
- When an Async - HTTP event occurs, you can check the Request ID of the server response to figure out what to do with the information you got back.
- Once you know what to do with the information, you can get stuff out of
async_load, such as the actual message the server sent back to you.
This is all well and good if you’re sending off one or two requests. No problem. But what should you do if you’re sending... a thousand? What if you have tons and tons of different HTTP request for a huge variety of things, all being fired off at different times, in different contexts? Do you need to rewrite the code to do the above four steps every single time? What about error checking? What about timeouts? What about verifying that you’re getting the correct data? It sounds like a lot to deal with!
I’ve found that a helpful analogy to understand this is the “Air Traffic Controller”...
AIR TRAFFIC CONTROL
When you have a lot of planes flying around an airport, the last thing you want is to have each pilot deciding on their own when to land. If each pilot is deciding where to go, when to go there, and why, then you’re going to end up with a very fiery airport.
Instead, Airports solve this problem by centralising all decision-making about air traffic to one place: the Air Traffic Control tower. Each pilot just talks to Air Traffic Control, and no decision about takeoff or landing is made outside of this context. This way, we always know when each plane is leaving and where it’s going. And when a new plane comes into land, we know where it came from, and we can make sure it lands safely and that it has a gate to connect to.
So if we want to streamline our handling of HTTP functions in our game, we should create something similar -- an HTTP Traffic Controller object! We will call it o_http_controller.
SETTING UP AN HTTP CONTROLLER
Coming back to our Steve metaphor, it’s the job of our HTTP controller object is to do the following:
- Know that Steve has been sent to get some chips.
- If Steve comes back, make sure he brought the chips (a successful response), and not something weird like Funyuns (an error).
- If Steve did bring back the chips, get the chips from him and put them in a safe place, so we can eat them when we want to.
- Know when to bail on Steve if he doesn’t come back fast enough.
Basically, our HTTP controller is a Steve babysitter. So, let’s break it down!
PART 1: KNOWING WHAT WE SENT
The first thing our HTTP controller needs is a list of all of the HTTP ids we have sent out. This will allow us to later check on those requests and perform operations on them if needed. So, once we create our o_http_controller object (and made it persistent so it lives forever), let’s set up a list in the HTTP controller’s create event, where we’ll store all those HTTP ids:
Next, instead of directly calling http_request() when we want to send an HTTP request, we will make a script that does it for us. When we make a request through this script, it will also document our HTTP ID and register it with our HTTP Controller. We can call this script http_request_c(), which is shorthand for “HTTP Request through the HTTP Controller Object.”
For now, we will just have this script send HTTP requests, add those request IDs to our HTTP Controller’s list, and give us the request ID back. Like this!
As a bonus, the way we’ve set this up allows us to use the http_request_c() script in the exact same way we would use the original GameMaker Studio 2 http_request() function since it takes the same arguments and returns the same value.
With this script, we can now send requests, and we have a list of all active requests we have sent!
PART 2: CHECKING WHAT WE GET BACK
As soon as we start sending messages to our server with http_request_c(), we’re going to start getting stuff back. So we need somewhere to put all that stuff! To store the server’s responses, we will set up a DS map in our o_http_controller’s create event, which we will call request_responses.
Next, in the o_http_controller object, we will create an Async - HTTP event, and this event will watch for all incoming HTTP traffic. Remember, this is the event that triggers whenever we get a response back from the server, and it contains that special async_load DS map. Once we have the event created, we can pull those response codes out of the async_load map and store them!
NOTE: The “?” symbol in the square brackets is a shorthand way to retrieve a value from a DS map.
As you can see above, we are retrieving the “id” and “status” of this response. The “id” is our request ID number, and the “status” will be equal to 0 if the request was successful (which just means that we heard back from the server).
But there’s a bit more going on in this code snippet. Let’s break it down!
STORING INFORMATION FOR LATER
If we have a successful request, we will create a new DS map, pull the “http_status” information out of async_load, and then copy it to our new map. This map is then stored in our request_responses map.
At first glance, this might seem weird. After all, we already have all the information in async_load. Why are we just copying that information over to another map? Doesn’t this come out the same? A map is a map, right?
Well... not exactly! For starters, async_load is transient. It only exists inside the Async - HTTP event. So if we want to get that information from somewhere else in our game, like, say, a different object, that gets pretty tricky. By moving the information to a more permanent map (our request_responses map), we can now do whatever we need to with the data we got back from the server. We’ve got it on lockdown now!
And second, maybe you don’t need exactly what the server sends you, but instead, some processed version of it that is more useful for your purposes. With the above approach, you can have your HTTP controller convert any information from the server into a more usable format, and then store that in your request_responses map, instead of just the raw data.
HTTP STATUS CODES
So, in the previous section, you may notice that we specifically grabbed “http_status” from the server’s response and stored it in our request_responses map. What’s this “http_status” thing, and why are we storing it?
When you are performing HTTP requests, you will typically get back some kind of status code that means something specific. Here are some examples:
- 200: Everything is great, and it went as expected!
- 403 You don’t have sufficient permissions to do whatever you were trying to do.
- 404: Whatever you were looking for, the server doesn’t know where it is.
- 500: Something really bad happened on the server. And it’s so bad, the server doesn’t even know what it was!
GameMaker drops these status codes into the “http_status” key of async_load. By storing these status codes, we can query them later to figure out whether something went wrong (or right) with one of our HTTP requests.
So we can now send a message to a server, and record the HTTP Status Code of our request once we hear back.
But wait! We’re not done! The HTTP Status Code is basically just Steve saying, “Hey. I’m back, and I have the chips.” But we don’t actually have the chips in our hand yet -- we just know that Steve has them. So let’s get those chips!
THE CONTENT
Within async_load, there’s another key called “result”. Under normal circumstances, this key will map to a value that is a string, which will contain the information that the server sent back to you. Just like we did with “http_status”, we will once again record this result in our request_responses map.
And now we have the content locked down! To recap up to this point, we can now:
- Send an HTTP request, using our HTTP controller object.
- Store the HTTP Status code and the content we got back.
So we’ve got the content stored... Now how do we use it?
PART 3: USING THE INFORMATION
Since we’re going to be using HTTP functions all over the place in our game, we want a standardised way of retrieving information out of the HTTP controller object, so we don’t have to reinvent the wheel every time we want to talk to a server. To that end, we can make a set of short scripts that reach into the HTTP controller and pull stuff out. Here are some examples:
A script to check whether an HTTP request has succeeded:
A script to check the status code of an HTTP request:
A script to pull the result out of an HTTP request if we have it:
NOTE: This isn’t the cleanest way to do this, but I made each script a standalone script so you can see what it’s doing in its entirety.
Once we have these scripts, we can use them in an object to send an HTTP request and get the results back with only a few lines of code!
To test this, we’ll make an object for doing a test HTTP request. We’ll call this something creative, like o_http_test_object. In the create event of this object, we will send out an HTTP request using our new script:
And in the step event, this object will wait until we’ve got a successful request back, and output the result to the debug console.
Boom! Just like that, we can now send an HTTP request and read out the results in just a few lines of code with any object.
PART 4: CLEANUP
As an astute observer, you’ll notice that we’re creating a lot of DS maps here. With every HTTP request we do, we have a new DS map! So, how do we take care of all this garbage and make sure we don’t have memory leaks all over the place?
To make cleanup easy, we’ll make a new script that purges any data associated with a particular request ID from our o_http_controller object. We’ll call this script http_request_purge(). It looks like this!
Note: ds_list_delete_element() is a custom script as well. It’s included in the .yyz at the end!
And last, since we’re lazy programmers, we don’t want to have to manually purge all of this information all the time. That’s a lot of work, and it’s subject to human error! Instead, We can create expiration rules, like this:
- The HTTP controller will only hold on to data received from a server for one second at most. After 1 second, the HTTP controller will destroy the data.
- Any object that needs to store or use the data for longer than one second must make its own copy of that data before the data gets purged.
To do that, we’ll increment a timer in the o_http_controller object for each request response, and once that timer surpasses one second, we’ll purge the response information.
First, we’ll initialise a DS map called response_timers in the create event of our o_http_controller:
Then, in the step event of the HTTP controller, we can iterate through our request_ids list, and run timers on any that have gotten a response. Any requests that have had a response lying around for more than a second, we purge.
Last, we’ll want to update our Async - HTTP event to make sure we are purging any information about this request ID if our request threw some kind of an error. We can find this out by checking if the “status” is less than zero.
And there we go! We now have automatic cleanup of any data received from the server, in all cases -- success or failure.
PART 5: TIMEOUTS
The last bit of advice I have with handling HTTP functions is to always be wary of server timeouts. This simply means that you have sent out a request, and you haven’t heard back in a really long time. Generally, if you have a reliable server and you haven’t heard back for more than a few seconds, something has probably gone wrong. To prevent having your player sitting and waiting 30 full seconds for a timeout, it’s a good practice to just head this off at the pass, and instead make your own timeout system.
You can use a system similar to to the “response_timers” system above, where you track how long each request has gone out. If the request sits for more than 10 seconds or so, it’s probably fine to just kill it. I’ve included the code for this in the attached .yyz file, for those interested!
BUILDING FORWARD
ALL RIGHT, I ADMIT IT. This is a lot of stuff to learn. And if you’re like me, you have a hard time just reading about something and then immediately putting it to practice. Personally, I love to just start tinkering with things to figure out how they work (and how they don’t). So, for those of you who learn best by tinkering, here is a link to a .yyz of all the code content from this article (plus a few bonus snippets, like handling timeouts). This code is free to use for any means. Put it in your project, redistribute it, modify it... Do whatever you want!
Now you have the basics of an HTTP controller, and you can adapt it to start handling your own web requests! Bear in mind that there is a lot of extra stuff going on with web requests, and you will probably need to heavily adapt your HTTP controller object to do all the things you need it to do. The way I have set it up here is for illustrative purposes, but it’s not meant to be a full stand-in for an HTTP controller in a fully functioning game.
To give you a sense of all the kinds of things you can adapt this HTTP controller to do, here are some examples of what we’ve done in Levelhead with our HTTP controller.
- It logs all HTTP requests to the debug console, so we can easily track what the game is doing.
- It parses out cookies and then re-attaches cookies to future requests to the same URL.
- It automatically creates buffers to download non-string content into and automatically cleans up those buffers.
- It auto-converts certain web formats to more GML-friendly ones, like JSON into ds_maps.
- It handles login credentials and security tokens automatically.
Once you have all of your HTTP action happening in one place, it’s pretty straightforward to add in all these fantastic functions!
Thank you all for reading, and I hope this was helpful! Happy developing!
