Setting up an EC2-instance for load testing with k6

November 24, 2020

Running Grafana k6 at meaningful load from a fresh EC2 box hits two snags before any test runs: the kernel and shell defaults aren't tuned for tens of thousands of connections, and the install instructions floating around still point at Bintray, which Grafana sunset in May 2021. Here's the current path.

Note on Amazon Linux versions. AL2 is in maintenance support; AL2023 is the current generation. AL2 uses yum, AL2023 uses dnf. The commands below show dnf for AL2023 — swap in yum if you're still on AL2.

Initial configuration

Update the base packages:

sudo dnf update -y

The k6 docs on running large tests recommend tuning the kernel for higher network throughput:

sudo sysctl -w net.ipv4.ip_local_port_range="1024 65535"
sudo sysctl -w net.ipv4.tcp_tw_reuse=1
sudo sysctl -w net.ipv4.tcp_timestamps=1

The docs also tell you to raise file-descriptor limits with ulimit. That alone won't stick: ulimit is a shell built-in, so any change dies with the session.

The fix is to edit /etc/security/limits.conf and add (tab-separated):

*	soft	nofile	250000
*	hard	nofile	250000

Log out and back in for the new limits to apply.

Installing k6

Grafana publishes an RPM repo at dl.k6.io. On AL2023:

sudo dnf install https://dl.k6.io/rpm/repo.rpm
sudo dnf install k6

On AL2, substitute yum for dnf. Verify:

k6 version

Getting tests onto the instance

scp is the shortest path for a handful of test files and JSON payloads:

scp -i "~/<path_to_cert_for_ec2>.pem" ./<path_to_local_file>.js \
  [email protected]:<path_to_remote_file>

For anything bigger, pull from git on the instance.

Run inside a screen session

A long-running k6 test should not die with your SSH session. Wrap it in screen (or tmux):

# Create a labeled session:
screen -S k6

# Detach with ctrl+a, d
# List sessions:
screen -ls

# Resume:
screen -r <id>

Avoid nesting screen sessions — it's easy to leave a test running in a forgotten inner session. The official k6 Docker images are an alternative if you'd rather not deal with that.

Run the test

k6 run --out csv=test_result.csv test.js

k6 also supports loading dummy data from files and orchestrating multiple serial or parallel scenarios with staged load levels. One caveat worth knowing up front: any file you open in a test is opened by every single virtual user, so size your dataset accordingly.

For sustained load tests beyond a single box, k6's hosted option (now Grafana Cloud k6) is the easy out — pricey, but it spares you the cluster operations.