Changing Frame of Mind

Long time no see! When spring rolled around I got caught up in wedding season (so many BEAUTIFUL grooms and brides finally tied the knot this year <3 <3 <3 !!!!) and now that everyone has escaped the danger zone I’ve been relaxing this week with our old arch nemesis: COVID. It has been the best case scenario for an unfortunate situation; vectors were immediately identified and notified, I knew I had it and was isolated days before my tests detected it, my fridge was stocked, and my symptoms have been more mild than flus of previous years. So I’m feeling very lucky and doing not too bad!

There’s no full essay from me this time, but instead one from my colleague Hunter Bridges! We’ve been sparring for years over whether our game’s time based variables should be in units of seconds or frames, and this tiny clash has challenged and remolded our game design philosophies. I do not know if I have settled into a concrete final stance, but my current liquidity is a testament to how persuasive Hunter’s argument is. I am glad that he has published this perspective, which I know he has been refining carefully for quite a while. This lesson has made me a better game developer, and I am sure it will make others better game developers as well!

Make sure to give it a read before going any further — my own short response will make no sense otherwise!

So my counter-point to using frames as the base metric is two-fold:

Firstly, we experience all other aspects in life using seconds, so using frames makes me have to multiply and divide things by 60 constantly. I am not good at multiplying and dividing things by 60, but a computer is extremely good at that. In fact, it was born to do it! So is the computer working for me, or am I working for the computer?

This is the line of thinking I first came into our discussions with, flabbergasted that any other approach even existed. But it was a deeply immature understanding of the value of frames. Learning to speak the language of the computer, not just in terms of code and logic, but phenomenologically reframing my own senses to those of the computer can unlock a far greater potentiality of nuance in game design. Hunter nailed the crux of it. When you are designing the physics of a universe (as every game must do) you must do so at an infinitesimal scale. Action and reaction connect and spread through literal lightning. Humans are not rocks, they do not experience life at geologic scale; nor are they computers, experiencing life in electric cycles. But as many an animator could tell you, with a little bit of training and focus you can see and count in “1s” and “2s”. It is crucial for an animator to break down an action into the standard metrics of animation so the action can be translated into one drawing every 1/24th of a second. Game designers must be able to do this as well, though with more than twice as small a gradation. The job of a game designer is to bridge the computer experience and the human experience, and so yes that means doing the multiplying and dividing by 60 frames, until you can see a game on the screen and accurately guess how many frames an animation lasts for. 

My second argument is more complicated, and not one I surrender so easily. When a game measures time as the distance between frames rather than the distance between seconds, it locks the game to a specific frame-rate. This demands a consistent technical quality, which can be a good thing, but it might lose out on future-proofing. If we make games for the machine, what happens if the machine changes? A second will always be a second, will a frame always be 1/60th of a second?

My friend Alec has a 120Hz monitor which he uses to play Stepmania. He is very good at Stepmania. However, it turns out he is twice as good at Stepmania as his scores were suggesting, because the copy of Stepmania he was using counts accuracy by frames, and consequently his window for “Flawless” hits was half the size as the vast majority of other players. Truly Flawless! If an experienced developer can see in 60 frames per second, then Alec has pushed to the next level and can now see in 120. 

This is a technical bug, and one Alec quickly discovered was fixed in a later build of the game, but it still represents the type of issue that can easily slip by when building for machine time. There are old, defunct browser games that I have downloaded but have become unplayable because they assumed the refresh rate of world wide web would never improve beyond what was possible in 2002. This happens with all old games, though the work emulators do hides that this could even be an issue. Developers in the 1980’s weren’t thinking about how their game would run on system architecture four decades later. Perhaps neither should we! But as games become exponentially more complex pieces of technology, they will become much harder to build emulation sandboxes for. Building fundamental gameplay code on top of system defined frame rate seems really sandy to me. Less than 1% of people are playing games with 120Hz displays, but will that always be the case?

There is a theory that our human senses are influenced in some degree by the technology around us. A professor of aural studies once told me that as soon as there were ways to record sound, musicians and singers were being caught in lip-sync scandals. What was interesting about the records, though, is that the low quality scratchiness of phonographs and gramophones which are so obvious to us now didn’t give away the deception. The majority of people simply didn’t hear it as low quality until higher fidelity technology incrementally transformed standards, and that prolonged mediated exposure changed our brain’s digestion of our senses.

I bring up that tangent because it relates to the concession in my first argument, that we can augment our senses to a higher fidelity and improving our own senses is one of the duties of a game developer, but it also breeds a warning for this second argument. In fifteen years, computer monitor companies will have capitalized on all the profit to be had when they persuade us that our many 60Hz monitors look unbearably worse than the 120Hz ones, and could our games of today be experienced not just under a technical difference but an aesthetically different one? Will the eyes of our audience see our games running meaningfully choppier on their screens, compared to the notification animation playing on their second monitor at twice the frame rate?

I’ll admit, that is perhaps too heady of a line of thinking. But that is what I love about this discussion — working in frames isn’t about methodology, it is about philosophy. Evening Star‘s policy is to use frames as the basic metric, because that is how the best games have been made. The games developed for the consoles of our childhoods were counting on the dependable specifications of their fixed platforms, and sometimes couldn’t even technically spare the calculations needed to compare the distance in time between frames. Ain’t no way an SNES game is querying a clock whenever it’s calculating a movement speed — pretty sure that thing didn’t even have a clock! And if Harvest Moon can count down to its sunsets one frame at a time, why can’t people today?

The biggest difference with games today is that they have to exist agnostic of their platforms. Independent games must be easily portable to as many systems as possible, or they will never find a big enough audience to survive. Even the largest platform owners like Sony are going multi-platform and having to port their console selling hits to PC. The direction of games today, like all media, is towards abstracted “content” whose utility is measured by how much it can be repackaged. So for all my headiness and worries about future-proofing, games are hitting portability issues around frame rates today. Dark Souls 2 had weapons degrade twice as fast on the 60FPS PC port, because the game was developed around the PS3’s hardware, and assumed a 30FPS lock. (Thanks again to Alec for telling me about this as I was writing!) 

As with Stepmania, this was eventually patched, though it took a shockingly long time. I’m sure the delay was a result of bureaucratic rather than technical reasons, but while frame rate dependency issues are usually easy to fix, they are also very easy to miss. Among thousands of lines of code on a perfectly functioning object, it is easy to glance over durability -= 2.0; and not realize it needs * deltaTime appended for it to scale properly with time. This has to be caught regardless of whether frames or seconds are being used as the scale, and it is the job of a programmer to catch this, but it is so much easier to slip up when the equation is missing a “multiply by 0.01666” (the fraction of a second between frames aka deltaSeconds) rather than missing a “multiply by 1” (the deviation from intended frame rate aka deltaFrames). Materially, that is what the difference between centering a game’s code around seconds vs frames looks like. Assuming a fixed frame rate makes gameplay math much cleaner for programmers, because 1 is a much nicer number than 0.01666! It also means unnoticed mistakes can easily slip by in games of all sizes.

That said, the industry’s unmooring from fixed frame rates has had some clear negative consequences. Because the games of today can run at any which frame rate, they often do. But rather than run better, faster, this has instead manifested in more games with frame rate dips all over the place. A game can drop to half or a third of the intended frame rate, and it will still technically “work.” The player will still move at a consistent speed, the camera will rotate without slowing down, and the player experience can continue. The game will look worse, but it doesn’t stop the show. This is good, but as a developer I can’t help but feel like standards have gotten sloppier. Older games ran at consistent clocked speeds because they had to — if the frame rate dropped in F-ZEROthen all the cars would move slower too. In many ways, it is a blessing to not be tied to frame rate dependency like that. But my friends, we have abused our power. When the gods of videogames come down to look at us and ask to see what we have done with our power to query a clock, whenever we desire, for the time between frames, will we be proud? I am sure they will high five Animal Crossing, but maybe the rest of us will be forced to share Promethus’ fate.

So what do you think? Are you team deltaFrames or team deltaSeconds! Please leave a comment on Hunter’s blog or send either of us a message — let’s continue the discussion! I’m sure many a developer has lived through the shift from console “locked frames” assumption to the PC native “frame independent” assumption. What were the code standards like? What are the sneakiest frame-rate dependent bugs that slipped by? I want to hear stories!

Wow can’t believe I said I only had a short response instead of an essay, now we’re 1,900 words later. Sorry Hunter, you worked way harder on editing your piece to perfection while I have vomited a massive first draft, this is why your code runs at 60FPS B^) EVERYONE GO SHARE HUNTER’S COOL BLOG AND MAKE HIM WRITE MORE! Also everybody else write more, so that I have more to write about.

Alright enough about all that. Gotta end my monthly wrap up. It’s June now, which is my birthday month, so once I recover from COVID somebody better come see my favorite movie Macross Frontier: The Wings of Farewell making its international debut on the big screen on my birthdayyyyy!!!!!

Games I’ve played this month (and the last three months oops):

  • Stranger of Paradise (PS4)
    • I wrote half an essay about how bad my taste is and how no one should ever ask me for recommendations because I told my coworker The Batman is good but I like Batman & Robin more, and I un-ironically adore Star Wars Episode III, Pirates of the Caribbean 3, Spider-Man 3, and all gratuitous threequels. It was about learning that to other people, “good movie” meant “good to take people on a date to” and that means it is less about engaging with a film than it is about representing oneself through a film. This is why Marvel films excel with cool powers, glib humor, and little moments of vulnerability (that have been focus tested to not be too vulnerable). The characters in those films are very dateable, and that makes them good date movies! Anyways Strangers of Paradise is not cool nor dateable but I love it. I sincerely have had a good time playing it, but it is condescending flavor of love. That made me think about how there is other popular media that is also built on the audience being condescending. I was watching Succession at the same time, and that is a show that is not full of cool dateable characters but ones that the audience is supposed to condescendingly root for their downfalls. So why does that work as mass media when Strangers of Paradise doesn’t? Well it seems to be all about intent! Succession makes it clear that it is in on the joke, and it is asking you to laugh with it. Strangers of Paradise I don’t think anyone alive can tell you where the line is between the things that are intentionally funny and accidentally funny. That lack of clarity is a big problem for dates! When sharing media with another person, you want to laugh at the same things, and if anyone is ever thinking “Do they think that I think…” then that’s a showstopper for enjoying anything! So when making stuff for a big audience make sure your intentions are clear and your characters are really hot.
  • Franken (PC)
    • ​this game is good and all the characters are really hot.
  • I don’t remember any other games I played in the last three months
  • i wrote down “DK King of Swing (GBA) is better than DKC” so I guess I played that sometime and that is my opinion and I guess i;ll fight you to the death over it!!!!!!!!

    love y’all wishing you good health!
    – esteban

If you have enjoyed reading this, consider signing up for the Saving Game newsletter! I will always post these on my website, but with the newsletter all future essays can go straight to your inbox.