Announcement 🔥March Improvement Project🔥 - A Dev Diary

Greetings Citizens of Heckfire!

Our development team has been hard at work to significantly reduce lag during swarms over the past few months and as many of you may have noticed in our Free For All events, these improvements are available now!

We are incredibly grateful for everyone's patience and support while we worked through this - as fellow citizens of Heckfire we know how frustrating dealing with lag is, and we are so pleased to have been able to complete this work for the benefit of all.

We want to give a special shout out and huge thanks to all those that participated in the tests we ran through this process and the thorough feedback we received in help tickets and in the #march-testing channel on discord!

Now that we've tackled this, we can start working on all the exciting updates we've been wanting to do for years! Anyone looking to burn castles in neighboring realms sometime soon? Scroll to the bottom for a sneak peek! 😏

For those interested in a deep dive, keep reading to hear about the work we’ve been doing and how we did it!


The Long Version

Kingdoms of Heckfire has been around for almost 7 years! As the game grew up, new elements were layered in to create a breadth of mechanics, events and systems for players to enjoy (new Competitions, Idols, Booty and more!)

Such delightful chaos!
​

As Heckfire grew, the march system got really pushed to its limits! With the well coordinated clans of up to 50 players launching hundreds of attacks on a single player, all within seconds of each other, came dreaded latency issues 😰 (or lag). This lag was frustrating for players and developers alike as it could impede gameplay, causing frustration and panic, and it limited our ability to release cool new mechanics. The lag eventually led to rescheduling of various competitions to limit its impact, as well as the sunsetting of the Realm versus Realm events. All this to say, it got pretty bad.

We felt the pain this was causing our players and put together a dedicated team to face this issue and reduce the impacts of March Swarm Lag once and for all!

Specifically, our goals were to reduce the time it takes for marches to process at a node before they return home. We identified this was an issue in both Player vs Enemy (PvE) and Player vs Player (PvP) attacks, and set our sights on a solution that would speed up the processing of marches queued up to hit a node (player or monster on the realm map).

After breaking the work into chunks, we kicked things off in September 2023. The team investigated, scoped out a solution we had high confidence in and started tackling low hanging fruit to get the project underway. We even built new ways to measure the impact of lag on players to ensure we captured the cleanest data in testing phases to come!

October brought excitement as work began in full! Our early prototype improvements were ready to go and we began tests behind the scenes! Test, iterate, test, iterate, test, iterate . . . needless to say, the work took some time. Naturally we uncovered new issues, as is always expected when you try to blow something up like this, but we were able to catch and develop solutions for them along the way.

Through this period we developed improvements that we felt confident to begin sharing with our patient players and the Elf in Shelter Events were born! These PvE events encouraged users to swarm a single monster node in the same way they would swarm another player in PvP. For those that need reminding, here’s the post about how the Elf in the Shelter events worked:
https://forums.kingdomsofheckfire.com/threads/elf-in-a-shel-ter-test-event.2836/

These cute little gingerbread houses were filled with angry elves!​

With a good amount of scrambling just before the holidays, much like elves in Santa's workshop, we jammed all the important folks onto group calls to coordinate these events LIVE! We used an A/B format that allowed us to adjust different factors in our tests to collect as much data as we could. We were very excited to see what players had to say this was our first chance to see the new tech in action, and we hope folks participating had as much fun as we did!

After the events wrapped, we posted an update about our progress in January to share where we were at and keep y’all in the loop. You can check it out here:
https://forums.kingdomsofheckfire.com/threads/march-improvements-are-on-their-way.2843/

Check out the impressive swarms that came together to smite those pesky Elves!

Christmas is cancelled!


DIE ELVES!​

We then set our sights more directly on PvP, conducting a ton of research on the most popular strategies to ensure we replicated scenarios that could feasibly happen the way players like to play. Shoutout to some of our top combatants for giving us great ideas on how to build the perfect accounts for PvPing purposes! We used this info to make a ton of developer accounts to battle it out in test servers and replicate swarming behaviours. These accounts would be the seeds of a growing Developer army . . .

Why do I hear the Imperial March every time I see this screenshot? ​

After testing PvP ourselves, we couldn’t release it into the wild on just any FFA day as we needed to make sure no issues occurred when players finally got their hands on the improvements! We do our best to plan for every potential situation, but folks always find the most interesting ways to do things you never thought possible 🧐. So we figured the best way to do this was to give Heckfire players the chance to beat us up!

Dusting off the old Realm versus Realm Competitions, we made even more juicy Developer accounts ripe for pillaging, stacked up a system of rewards to ensure no players would be punished for setting our castles aflame, and the Dev Invasion was ready to commence!

You can check out the full details of the Developer Invasion here: https://forums.kingdomsofheckfire.com/threads/invasive-species-dev-invasion.2849/

These events ran as A/B tests over the course of several days, allowing us to collect final feedback and data, while really stress testing these improvements! This also gave us great feedback on how the Realm Versus Realm tech held up and let us test it in a live environment once again!

🥳 And it worked!!! 🥳


Y’all really made ATA Will regret coming into work that day!​

As the event wrapped up, we turned the improvements back on quietly to attempt to get unbiased feedback. Between what we observed in metrics and the folks writing in to tell us they’d had the smoothest FFA in years, we felt confident it was a success!

Here is a graph that may better explain!

​

This graph is from one of the Dev Invasion test days where we did not have March Improvements turned on during the 3:00pm PST (15:00) Dev Invasion, but we did have them turned on during the 4:00pm PST (16:00) Dev Invasion.

The Y Axis here shows a performance metric that describes the average amount of time it takes for combat marches to process. The X axis is the time in PST.

As you can see the time it takes for Marches to process drops off steeply after the 3:00pm Invasion ends and it never even spikes during the 4:00pm Invasion!

On March 14th we brought back an all Realm Free For All Day for the first time in years and we are working on plans to bring back even more great PvP events for y’all to really enjoy!

​

Here's a little sneak peak of what has us dancing for joy like a pants-less Cedric!

• 🌏⚔️🌏 The RETURN of Realm versus Realm events!
  
  
• 🤺 Only All Realm Free For Alls going forward! (this will allow further flexibility in the competition schedule)
  
  
• 📆 Free For All’s on Weekends!
  
  
• 🌏 Ultra Free For All’s (where all realms compete on the same leaderboards!)
  
  
• 🌋 More frequent Shrine and Altar events!

We know there is so much else we want to bring to the dedicated Citizens of Heckfire and so we say once again a big thanks to everyone for their patience and help getting here!

- The Heckfire Dev Team