Making Cybersecurity Fun with a Capture the Flag Event, Part 2
Check out the technology behind our competition - and attempt the hacking challenges yourself.
In Part 1 of our Capture the Flag (“CTF”) writeup, we explored the social and business value of running a hacking competition. Now, let’s take a behind-the-scenes look at the technologies we used to run the event (and if you’re feeling brave, try to hack your own way to victory).
Test your hacking skills now
This Github repo - and accompanying CTF primer - is your ticket to attempting our CTF 2022 challenges, with a few notable alterations:
- 21 total challenges - We limited our contest to 10.
- No time limits - We had 10 minutes for easy challenges, and 20 minutes for medium ones.
- No hints - You’re on your own.
Have fun, and happy hacking!
Sudo: a fickle beast
Our resident Site Reliability Engineer, Bob Micheletto, has a penchant for finding flaws in poorly written
sudo rules. He initially wrote 21
sudo-based challenges, but to keep things simple, we pared them down to 10 for our contest. (Note: you can try all 21 here.)
We decided that participants in the event would:
- SSH to a preconfigured server,
- Examine the
- Attempt to gain root access by exploiting those
If and when they got root, hackers would run a special
/root/ctf.sh command, which would send a secret hash and automatically update the leaderboard.
Come work with us!
We’re 100% remote and loving it. Check the jobs page for opportunities to join our diverse team.
Technology to run the event
We considered a few hosting options:
- Docker containers with screen recording & manually submitting solutions, which we ruled out because they’d be too difficult to support. (Note: if you want to attempt the challenges yourself, you will be using Docker.) ❌
- Shared AWS ec2 instances, which were nixed because it’d be too easy to accidentally disable the system for all - plus folks might leave clues to challenges laying around. ❌
- Single AWS t2.nano instances for all, to be refreshed every day. It cost less than $5 for a server month, and we’d only keep them up for a couple weeks anyway. Perfect. ✅
TL;DR - folks would ssh to their own t2.nano instance each day, and figure out the challenge from there.
Spawning the servers
Each challenge server was provisioned using AWS CloudFormation and configured using our agentless Puppet configuration. Our cloud-init script fetched the Puppet configuration from code-commit and applied the appropriate challenge tailored for a specific user based on the EC2 instance tags.
User accounts & authentication
We used a Puppet module to create users on our servers, and user assignment is done through Hiera, so we generated a bunch of configuration files for the challenge servers that put only User X, on Server X. “We didn’t want anyone to be tempted to log into another user’s challenge server,” Bob explains.
We added our users public keys using the same module that adds our users. The snag was, we were going to be deleting and recreating these servers 10 times a piece. To avoid users running into SSH warnings about changing host keys, we decided to preserve the keys and provision them through Puppet.
To enforce the time limits for challenges - with automatic logouts after a specified time - we used a
shutdown trick from Unixdaemon.net. “We needed a way to keep the challenges short and sweet for our contestants,” Bob says. “Adding time limits - 10 minutes for easy challenges, and 20 minutes for medium ones - proved ideal.”
Here’s the Puppet template we used to create the timer events on login:
if [ “$PAM_USER” = ‘<%= @user %>’ ] ; then # Start the events echo /usr/local/bin/shutdown_timer.sh | /usr/bin/at now echo /usr/local/bin/hint.sh | /usr/bin/at now fi
Note: we still wanted the ability to login to these servers with administrative user, so we check the
PAM_USER environment variable - and only kickoff the events if the challenge user logs in.
Here’s the Puppet template file we used to generate the shutdown:
# Shutdown Delay sleep <%= @shutdown_delay %>
cat /usr/local/shutdown_message.txt | /usr/bin/wall
/usr/sbin/shutdown -h now
Note: the duration of
shutdown_delay was different depending on the difficulty of the challenge (10 minutes for easy ones, 20 minutes for medium).
For automated scoring - with a leaderboard contestants could check at any time - we wrote a script that:
- Sent out a
- Sent a verification email (using AWS SNS) with a secret string for verification, unique per user, per challenge. Without this, anyone could “hack” the leaderboard by spoofing
statsdmessages - and that’s not the type of hacking we’re looking for here.
We then built a leaderboard using AWS managed Grafana, and allowed our users to login to the leaderboard using their usual SSO logins. Here’s a glimpse of the final standings:
“Automating the scoring process took a bit of wrangling, but saved us a lot of time and manual work in the end,” Bob concludes.
Automated hints appeared for contestants halfway through each challenge using
wall. The hint timer was set at the same time as the shutdown timer (on user login). “Frustration would ruin the fun, but we also didn’t want to give away too much,” Bob says. “Our hints provided just enough info to point folks in the right direction.”
Here’s the Puppet template we used to send the hint:
sleep <%= @hint_delay %>
cat /usr/local/hint.txt | /usr/bin/wall
hint.txt file was also configured via Puppet, and was specific to each challenge.
Network Ninja branding
A special flourish for contestants was this custom Network Ninja ANSI MOTD that appeared on login.
Support, documentation, & fun provided via Slack
Each morning at 9am ET, we’d announce the daily challenge in our company Slack at #ctf2022.
Contestants had 24 hours to complete each challenge. For educational purposes, we also provided reference solutions for the prior day’s challenge in the channel.
“I accidentally trashed the sudoers file”
Here are a smattering of contest highlights from Slack. Note to the reader: don’t do the following.
Hacker creativity reigned supreme.
After several lockout notices, this warning attempted to keep people from harming their own shells…
…But not everyone listened.
Seriously, it’s really easy to break things with sudo/sudoers!
When stuck, extra help was available upon request.
In contrast, others gloated about their hacking prowess, in song form.
And now, a final song to send us off. We can’t wait for #ctf2023!
Throughout the competition, we had fun, collaborated together across product teams, and learned some things about security along the way. Be sure to check out Part 1 of our CTF event writeup if you haven’t already, and if you want to participate in the competition next time, come work with us.
Previous articleMaking Cybersecurity Fun with a Capture the Flag Event, Part 1
Are you a developer? We’re hiring! Join our team of thoughtful, talented people.