Chris Smith in Gaming

Hosting Factorio on Google Compute Engine

Factorio is a video game that’s been in development for several years now, and I’ve been hooked since the first time I tried it. If you are unaware, I would highly recommend you check it out on Steam Early Access.

While I would love to wax poetic about how amazing the game is – especially the latest update, v0.15 – this blog post details how I play Factorio with my friends: using a dedicated server hosted on Google’s Cloud. Formally “Google Cloud Platform” or GCP.

Rather than needing to coordinate schedules when people can be online and/or sending a save file back and forth, by running a dedicated Factorio instance in the cloud people can just sign on whenever they want. Everything “just works”. Even if you log on from a different machine. Your game state is stored in the cloud.

Anyways, this post is about how to set up your own Factorio game server on GCP.

Project Overview

Perhaps I’ll later host a cluster of Factorio games using Google Container Engine / Kubernetes. Or use network schenanigans to have multiple games running on the same VM. But for now, I’m just doing the simplest thing that works. I’m hosting a single game instance on a single Google Compute Engine (GCE) VM.

To do this, there are two main components:

• Game Container and Settings - How will we package/distribute the Factorio game binary and provide the map generation and server settings for the game.
• Hosting on Google Compute Engine - The specific virtual machine, and how we automate startup and shutdown of our instance.

You’ll see the terms “factorio-aas” and “deathworld” come up in the scripts. “factorio-aas” is the name of the GCP project I’m using (covered next). I am using “deathworld” to describe the game instance of factorio. (To differentiate this VM from other Factorio games I am hosting, which corresponds to the game settings used as well.)

Creating a GCP Project

First thing you need to do is to create a Google Cloud Platform (GCP) project. All GCP resources are tied to a project, which includes GCP VMs and Storage buckets.

To create a project on Google Cloud Platform, head here: https://cloud.google.com/console.

In order to be able to create GCE VMs, you’ll first need to enable billing. Unfortuantely running VMs in the cloud isn’t free. Sorry. But you can host a Factorio megabase on an n1-standard-1 instance, which will cost less than $25 a month. You might even be able to reduce the CPU or memory required for Factorio. And/or shut it down at night to save on costs.

And of course there is a Free Trial of GCP too. You can get $300 in free credits on GCP, which is more than enough to host a few Factorio games all year.

Once you have the project created and billing set up, you’ll then need to “enable” Google Compute Engine and Google Cloud Storage for your project. Just go to these links and click “OK”.

• https://console.cloud.google.com/compute
• https://console.cloud.google.com/storage

Game Container and Settings

Obviously, the most important part of hosting Factorio in the cloud is the Factorio game itself.

You can download the latest “headless” instance of Factorio from the web. But downloading and running binaries can quickly become a pain as you upgrade Factorio versions, upgrade runtime or OS dependencies, etc. The current state of the art for this sort of thing is to distribute software as a container. If you are unfamiliar with Docker Moby, just think of it as an easy way to package and distribute software. Here is a decent introduction that doesn’t have too much marketing-speak.

Anyways, all the hard work of creating a Moby container for Factorio has already been done. Actually, there are several Factorio containers on DockerHub. But it appears that the best maintained one is by David Anderson:

• GitHub - github.com/dtandersen/docker_factorio_server
• DockerHub - dtandersen/factorio/

Thanks David for sharing your work! With him taking care of the Factorio packaging and deployment step, all we need to do is configure the game.

Map generation settings

The Factorio instance will create a new map on first startup. But you might want to alter the settings to adjust ore richness, resource patch frequency, etc. And when you start the game, you might want to adjust biter AI and/or the impact of pollution.

If you go to whever you have Factorio installed, you’ll find a data folder with sample configuration files map-gen-settings.example.json and map-settings.exmaple.json.

Make a copy of these files, and customize as desired. If you are feeling adventerous, I uploaded the settings used by Factorio’s new “Death World” mode to a GitHub Gist. If you are Factorio pro looking for a new challenge, Death World is highly recommended!

To get the map settings files onto the VM, I simply stored them on Google Cloud Storage. To do this yourself, create a new Cloud Storage Bucket. You can do this in the Cloud Console UI. Or just use the Google Cloud SDK command-line tool gcloud.

I’ll opt for the CLI as it is easier for those of you following along. To create a bucket for your Factorio data, use the gsutil’s make bucket (mb) command. Note that GCS bucket names are in a gobal namespace. So you’ll get an error if you try to create a bucket named factorio-aas-files as that is already taken.

# Create the bucket "factorio-aas-files".
gsutil mb gs://factorio-aas-files

# Copy the files into that bucket.
gsutil cp \
    ./map-gen-settings.json \
    gs://factorio-aas-files/gameconfig/deathworld/map-gen-settings.json
gsutil cp \
    ./map-settings.json \
    gs://factorio-aas-files/gameconfig/deathworld/map-settings.json

Server settings

The game’s server is also configured through a server-settings.json file, and just like before you can find an example version with your local Factorio install.

However, there are a few very important values you need to change for your VM.

Authentication Key

In order to host a public game, you need to provide our Factorio login credentials. while you could enter your username and password if you purchased the game directly from Factorio.com, the much safer route is to includes an opaque authentication token.

You can find this in a player-data.json file. To be honest I don’t know when exactly it gets created. But you can find it on Windows in the AppData\Roaming\Factorio folder. The thing you are looking for is the “service-token” value.

Enter that “service-token” value into server-settings.json where it asks for a “token”.

Admins

Admin users on a multiplayer server have the ability to use console commands. You’ll probably want to enter your user name there too.

Like the map settings files, we’ll store this on GCS as well.

gsutil cp \
    server-settings.json \
    gs://factorio-aas-files/gameconfig/deathworld/server-settings.json

With the Factorio game-side of the equation taken care of. Now all we need is to up and run it on the Cloud.

Hosting on Google Compute Engine

Now let’s move onto the GCP-side of things to configure the VM that will host our game.

Network

Once you have your project set up, the first thing you’ll want to do is create a custom network for your Factorio VMs. Factorio communicaites via UDP on port number 34197. You could simply enable that on the “default network” for your project, but I would recommend putting your Factorio VMs on their own network. (To isolate any custom firewall rules just to the machines which need them.)

To create the network you could use the Google Cloud Console UI, or simply use the Google Cloud SDK command-line too gcloud. I’ll opt for the CLI as it is easier for those of you following along.

# Switch the current Cloud SDK project to whatever your GCP project is.
# I'm using factorio-aas, but your project name will obviously be different.
gcloud config set project factorio-aas

# Create the network and unblock key firewall rules.
gcloud compute networks create  factorio-vms
gcloud compute firewall-rules create allow-ssh --network factorio-vms --allow tcp:22
gcloud compute firewall-rules create allow-game-traffic --network factorio-vms --allow udp:34197

Startup script

With the Docker Moby container and custom network, we have everything we need to run Factorio. But – just like the spirit of game itself – it is best to automate things.

We’ll next create startup and shutdown scripts to install/upgrade Factorio and gracefully shutting down the VM.

On startup we want to do several things:

• Install Docker Moby if it is not already available on the system.
• Install/upgrade the Factorio container to the latest version.
• Bring down the latest game configuration files from GCS, in case you tweaked map or server settings.

To do all of these things, we’ll use the following Bash script.

#!/bin/bash
set -x

# Install Moby if needed. Full instructions:
# https://docs.docker.com/engine/installation/linux/debian/#install-using-the-repository
echo "Checking Moby installation status."
if docker -v ; then
  echo "Moby installed."
else
  echo "Moby not detected. Installing..."

  sudo apt-get -qq update
  sudo apt-get -qq -y install \
     apt-transport-https \
     ca-certificates \
     curl \
     gnupg2 \
     software-properties-common

  curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -

  sudo add-apt-repository \
    "deb [arch=amd64] https://download.docker.com/linux/debian \
    $(lsb_release -cs) \
    stable"

  sudo apt-get -qq update
  sudo apt-get -qq -y install docker-ce

  echo "Moby CE installed."
  docker -v
fi

# Create the various directories the container reads/writes to.
mkdir -p /factorio-instance/saves
mkdir -p /factorio-instance/mods
mkdir -p /factorio-instance/config

# Bring down the latest config files.
# TODO(chrsmith): This is the only part of the script that is dependent on
# the game instance name. Perhaps get that from the GCE metadata server.
rm /factorio-instance/config/*
gsutil cp gs://factorio-aas-files/gameconfig/deathworld/* /factorio-instance/config/

# Delete the container if it already exists. (Reboot-to-upgrade scenario.)
docker stop factorio
docker rm factorio
docker pull dtandersen/factorio
# Get the latest instance. But you can peg it to a specific version with:
# "stable", "0.15.13", etc.
docker run -d \
   --name factorio \
   --publish 34197:34197/udp \
   -e FACTORIO_SAVE_NAME=gcp-cloud-save \
   -v /factorio-instance/saves:/factorio/saves \
   -v /factorio-instance/mods:/factorio/mods \
   -v /factorio-instance/config:/factorio/config \
   dtandersen/factorio:latest

To make this script available, I’m uploading it to Cloud Storage:

gsutil cp startup-script.sh gs://factorio-aas-files/deathworld/startup-script.sh

Shutdown script

When you shut down the VM, you can have it run a custom shutdown script too. We’ll tell Moby to stop the container, which will allow Factorio to shutdown gracefull. (Or at least as gracefully as possible in the X-seconds before the VM powers down.)

For convenience, we also write the Factorio instance’s output (via docker logs) to disk. I’m going to assume there is a much better way to do that…

#!/bin/bash
set -x

mkdir -p /factorio-shutdown-logs/
LOG_FILE=/factorio-shutdown-logs/factorio-log-`date +%Y%m%d_%H%M%S`.txt
docker logs factorio > $LOG_FILE
docker stop factorio

gsutil cp shutdown-script.sh gs://factorio-aas-files/dearthworld/shutdown-script.sh

Setting up the VM

Setting up the VM requires checking a few boxes, but don’t be too intimidated. This is the set of things we need to do:

• Use the n1-standard-1 machine type. That’s enough horsepower to host a game. (Even more than is needed for smaller bases.)
• Specify a startup and shutdown script, pointing to our files hosted on GCS.
• Give the machine read-only access to GCS.
• Connect it to our customized GCE network.

Doing this all in the Google Cloud Console UI is probably the easiest way to set things up. But if you want to do it on the CLI, this is the command:

GCSFILES="gs://factorio-aas-files/deathworld"

gcloud compute instances \
    create "factorio-deathworld-vm" \
    --machine-type=n1-standard-1 \
    --metadata=startup-script-url=$GCSFILES/startup-script.sh,shutdown-script-url=$GCSFILES/shutdown-script.sh \
    --scopes=https://www.googleapis.com/auth/devstorage.read_only \
    --network=factorio-vms \
    --image-family=debian-8 --image-project=debian-cloud \
    --zone=us-west1-a

Pro Tip: If you want to see the VM’s output as it boots up and runs the starup script, edit the VM in the Google Cloud Console and click “View Serial Port” at the very bottom. It’s an invaluable tool in debugging VM startup issues.

Alternatively, the output of the startup script is written to /var/log/daemon.log. So you can SSH into the VM and look at that file as well.

Maintenance

Of course your job is not done. You are now on official Server ADministrator, and so put your SRE hat on. Here are some things

SSHing into the VM

If you want to connect to the VM, e.g. poke it for whatever reason, this is the command. It requires that we unblocked port 22 from earlier.

gcloud --project "factorio-aas" \
    compute ssh --zone "us-west1-a" "factorio-deathworld-vm"

Looking at Factorio logs

To look at the output from the Factorio instance, e.g. status messages with version information and notifications when players join/exit, run:

sudo docker logs factorio

Updating version of Factorio

How to update versions of Factorio. Or what about a new version of Factorio comes out?

Just restart the VM. It will gracefully shut down the Factorio game. It should “just work”. Though to be hoenst I haven’t tested this too much.

This is because it grabs the “latest” docker image…

Deleting your VM

If you are done with your Factorio game, you will eventually want to delete the VM. However, doing so will delete your hard disk – losing any save games you have. So be careful.

gcloud --project "factorio-aas" \
    compute instances delete "factorio-deathworld-vm" \
    --zone "us-west1-a"

Future work

Would like to eventually add the ability to dynamically spin up instances. Perhaps in a Kubernets cluster, etc. In theory that should be possible with [network port forwarding etc]. And by reusing multiple instances per machine, get better utilizing of CPU resources.

Random features list

Here is full list of random ideas I have for better managing Factorio instances. Essentially it is blogging fodder as I find the time time:

• Storing data (e.g. game configs) in Google Cloud Source Repositories.
• Using the GCE metadata server. If you are hosting mutliple games, you can reuse the same image template. Such as within a GCE Instance Group, and use game-specific settings via VM metadata. e.g. tag a VM with the game name “deathworld” or “with-bobs-mods”.
• Integration with Cloud Logging. (Stackdriver agent?) Make it easier to inspect what is going on with the game instance without SSHing into the VM.
• Push notifications. Use Cloud Pub/Sub to send notifications when somebody joines or leaves the server.
• Backing up save files to Google Cloud Storage.
• Web frontend to configure and manage mods associated with the game.
• Web frontend showing game and server stats, etc.
• Google Maps integration! Via the Google Maps mod.