There's a topic that's been weighing heavily on our minds lately and it's Game Review Culture. We've made some adjustments to how we produce content and review games because of it, and those changes are detailed below, but first, our thoughts on why we're doing so...
There's a Right Way and a Wrong Way...
With every new game release you have 20-30 networks all scrambling to have the best, shiniest, and most opinionated review. Some choose a more honest route, sticking to being critical and not awarding high scores to anything that isn't truly ground-breaking, while others it's clear as day there's a toxic pattern or bias behind their reviews and structure. We're not going to be so bold as to say there's a "right" way to review games or anything in general, but there is definitely a "wrong" way.
What's the wrong way? Well, that's all opinion based. To us, it's any review that doesn't provide any real depth to it. Those who stick to surface-level compliments or complaints without breaking it down or going into detail about why they feel a specific way. This leaves entire gaps in something that's supposed to inform the consumer as to why they should or shouldn't purchase the product. In fact, most reviews we've read lately have had nothing but statements such as, "It would've been better if they'd made it like this game", where the complaints are so biased we can't bother reading to the end. Or, in the more positive ones, they throw fancy statements with tier 4 words that have your average person googling to understand. There have been both instances where we play a title that averages a 6-7 among large name reviewers, that we end up adoring more than most titles, and others that score 9-10s that leave us with little ????'s hovering over our head.
This is why we choose to never bring scores into our Reviews. Quantifying a game and one's enjoyment level, and whether they'd recommend it based upon something as simple as a 1-5, 1-10, and even a 1-100 scale, simply doesn't do a game justice. How can you give a game with 7 hours of gameplay a 9, then turn around and award a 100-hour game an 8? Scoring systems are immensely flawed and yes, you're welcome to have the opposite opinion, we're not going to argue it with anyone, but there's no way to convince us that this is a good system.
Fresh experiences, judged accordingly...
Now, we absolutely understand that there are certain "factors" that get calculated into review scores. A few of these include genre, game length, and comparing it to other games similar to it, and all of this just feels wrong. If you start throwing in stipulations and factors you only make the scoring system that much more flawed. Just because someone who loves Super Mario will rate every Super Mario game a 10, doesn't mean they're going to give other Platformers fair scores. They likely have so much bias towards their favorite game, that they're just going to spend the entire time comparing it and faulting it for not being the same (which, we see so much it hurts our brain), then actually treating the game like a fresh entry to the genre, and allowing it to be a fresh experience and judge it accordingly. It's fine to draw comparisons so that the community can better understand how a game looks, plays, and feels, but it's another to fault them for not being your favorite game. If you want to review your favorite game again, go do that.
This leads us to what we've named "Hype" and "Hate" culture. Put simply? Reviewers that spend so much time hyping up a game, or hating on it, that their reviews should be tossed to the side entirely. Anyone who sounds like they're blindly praising a game without solid reasons as to why, or, the reverse, should absolutely be put on the chopping block for being an untrustworthy source of game reviews. We understand that some people get a kick out of the extreme reactions of individuals, and we definitely want to boost up the games we enjoy and caution people about the ones we don't, but there's a vast difference between honest positive/negative reviews and over-hyped/over-hateful reviews.
Day 1 is the worst state of a game...
So... where is all this going and why has it led us to want to change our structure? Well, with all the talk about hype and hate culture, we've evaluated where a lot of it stems from and it's Day 1 Reviews. Why? Well, if you consider the fact that the day a game launches is the worst state that a game is usually going to be in, it also raises the question of how reliable those reviews are past Week 1 of the game.
We as reviewers and creators spend about 20 hours per game we review between playing, story-boarding/writing, editing, post-production and social media sharing, and that's usually at a minimum. Sure, some smaller games might be closer to 10 hours, but for games offering 20-40+ hours of content, that number starts to double to 40 and sometimes triple past 60. We don't want to be reviewing a game we don't feel fit to review and we'll put in as much playtime as we find necessary to put out a good one. This creates another problem in that if we put out a Day 1 review after spending 40 hours in Early Access, and by Week 2 it has a huge patch fixing half the problems we experienced, we just feel bad about having an inaccurate or less positive review than seems fair for the game. That's bad marketing. It doesn't hurt us or our content in any way, and that's the honest truth, but it does hurt the games.
Reviews that can't be undone...
Imagine if you click on a review. It's a bad one. You move on to the next one. That one's also bad. So you choose not to buy the game not realizing that the only reason they're bad is because there were a few game-breaking bugs that got fixed weeks ago. However, those creators didn't care to update their review because... well it's too damned time-consuming and that's the truth. Even we don't want to be updating reviews. It sucks going through hours of voice recording, editing, thumbnail creation, posting schedules, it's just exhausting, and we don't want to do it again just because a game drastically improved over the first month.
SO! What does that mean for us here at The Pillow Fort? It means that we're putting Day 1 Review culture in the grave and leaving it there. We're sticking to our guns and refusing scores like the plague they are. We know that you as a developer want a shiny new number to throw up on your page of accolades, but more often than not those scores aren't as shiny as you were hoping, and did far more to impact your sales negatively than you're going to admit publicly. Why? Because hundreds of games are being released a month and one bad review can have so much impact it hurts. Imagine those reviewers saying, "Don't play this game, go play this one that we enjoyed way more instead". Now, instead of going home to the team and sharing a heartfelt victory, you're left in tears, all over things that could've been fixed if you'd had the time, knowledge, and opportunity to do so. Developers get blamed for various bugs and performance issues when they genuinely had no idea they existed until they presented their game to the public to be slaughtered mercilessly.
One platform does a fantastic job of not penalizing or over-awarding a game for its recent reviews and has implemented a scale that compares recent reviews to all reviews over time, and that, our dear friends, is Steam. Yes, Steam puts the review bombing culture behind them by sharing both the Recent Review Score (Positive, Mixed, Negative), with the overall review score for the game (Mixed, Positive, Mostly Positive, Overwhelmingly Positive). This gives us a better understanding of the current state of a game and more often times than not, we'll see games with a "Mostly Positive" overall rating, yet rocking a "Mixed" state for their recent reviews due to a patch that did more harm than good.
It's important to know that both circumstances can happen to a game. We've seen plenty of titles that do incredibly well during their early life, only to turn around and take a review score plummet on Steam due to implementing something, or worse, removing something, from the game that just makes it feel ... well... bad. To name an example, Destiny 2 obliterated its community by removing huge chunks of their lore and story prior to releasing the new expansion. Even though Destiny 2 itself had / has a loyal fanbase, plenty of people were rightly upset when they decided to just flush large chunks of their game down the toilet. The speed at which people changed their Reviews from a positive one to a negative one was sad, but warranted.
On the opposite spectrum, you have titles that release like Redfall that have an absolute flop of a launch. Now, we personally enjoyed a lot of aspects of the game, but our party of 4 was greatly hindered by the fact that 3 of us were having some sort of issue, 90% of the time. From game-breaking bugs to annoying ones that made the game unnecessarily hard, to an empty world that felt truly barren of the interaction you'd look for in an open-world RPG title. All of which could've easily been fixed with a simple 2nd-week patch, or heck, 4th for that matter, but the lack of immediate response from the development team and the sheer womping price of the game made it hard to forgive. To this date, many have a bad taste in their mouths. That's detrimental for a title because even if they fix it, they've lost the trust of their community and future games will be scrutinized. Now, we're not saying that's the right way to approach it, but rather, being transparent about the way the gaming community tends to deal with negative reviews.
Let's focus on the facts...
This lack of updated reviews in this industry, and the culture around being negative when a game has a less-than-optimal launch, is why we're seeking to fill the gap. We're no longer going to be giving games an "official" rating on Day 1 of release, we'll instead be waiting until 1-3 months in. Once we finally feel like the game is in a steady state with no big updates in the near future, we'll formulate an official review article and video.
Our community has established that they're happy to play Early Access titles and deal with less than successful launches, so long as the developers care to make their games better. Even if games lack certain features we may want, at the end of the day, that's forgivable. What isn't? Bugs or performance issues that make the game downright unplayable. That is where we come in. We've developed what we're calling a "5-Step Process" to playing and reviewing games.
Here's our new Content Structure breakdown: Step 1: Regardless of whether we get a game prior to its launch, or on the same day, we're ONLY going to be posting a 2-4 minute "First Impressions" video in which we highlight the features of the game with absolutely no opinions. These videos will seek to showcase what the game has to offer and only document any bugs we experience. We will make no comparisons to other titles, or insert thoughts such as whether we enjoyed the game or not. We want people to have the facts first and foremost, and our official "review" and opinions later. We will then submit all feedback, thoughts for improvement, and bug reports to the developer and put the game down for a week or two.
Step 2: After the first week (possibly two) for smaller titles, we'll post a 2-4 hour video that features raw gameplay aspects with fantastic chaptering so that the community can jump around and check out the different aspects of the game. For larger or longer titles, we'll host a stream that focuses on being a Q&A for the community to get a look at the game and ask any questions they might have.
Step 3: After 2-4 weeks post-launch (depending on hotfixes / patches), we'll pick the game up and re-experience what we thought were the best and worst parts of the game, even if that's 10 hours in. This is where we'll also formulate our Review Article and post any Guide or Informative style videos. This article will also seek to be more guide-like than opinionated. We know how much our community has enjoyed our deep-dive reviews in the past, so we're revamping these and focusing on being entertaining and educational.
Step 4: Once we're sure there aren't going to be any major updates or patches that will drastically improve a game's playability and feel, or add additional features, we'll formulate an official review for YouTube that goes into detail about all its features. These will be longer 10-20 minute videos that are aimed at providing the most thorough and honest reviews we as a team can provide for the games we play. They'll educate the consumer on everything we feel they should know going into it while leaving out heavy spoilers or features that we feel are important to keep a surprise. All of that type of content will be contained in specialized videos.
Step 5: For games that offer roadmaps or have big content plans for the future, as we can and are able, we'll formulate "Patch Review" videos, similar to how we'll operate for Early Access games. The Pillow Fort will always aim to provide continuous coverage for as many titles as we can, so if you enjoy one of our reviews and want us to keep updates about that game in particular coming then please feel free to tell us, we weigh requests pretty heavily.
NOTE: Early Access titles automatically disqualify from this 4-5 Step Plan. Instead, we'll ONLY post a First Impressions review and gameplay footage, and aim to post updated videos for your larger roadmap improvements. Games are only eligible for an official rating once they've entered the full release stage.
Even though these changes won't eliminate the problem of our reviews becoming outdated as time goes on, we can at least ensure that they're not outdated right after release. We want to provide our first impressions of a game on Day 1 when possible, and within the first week when not. We of course want to help aid in getting games into the hands of people we know are going to love it, but we want to keep from dissuading people from buying due to small issues, or slice-of-life improvements that come within a few days of the feedback.
We know and understand that some developers will always reject our content due to the fact that we won't provide a score, and to top it off, we're on the smaller side despite our growth and reach rates. This will exclude us from covering a lot of titles that our community requests as our personal budget for buying games is extremely limited being parents of 3. Beyond that, we can't pretend to have the sort of time that warrants buying a game that doesn't allow us to cover it. We have too many requests from those who do want us to cover their game that, even if we don't want to, we'll have to move on from those that don't. Believe us when we say that's not a brag, just an acknowledgment that sticking to our values means not prioritizing a specific game just because we have a personal desire to play it, and not penalizing games that aren't our favorite, just because they (lol) aren't our top upcoming games. We do try to avoid being hypocrites after all. So, consider that us stating our commitment to providing quality feedback for a game, even if we don't personally enjoy it, and presenting it properly to the community that will enjoy it.
How is a Dev to know until Launch Day?
The idea is that our content approach will allow developers the chance to listen to the community and implement proper changes to the game prior to the fear of an official review. After all, how is a dev to know what the community wants until the community plays it? On top of that, we can ensure that at least one review channel is looking out for the consumer and putting all the console, genre, and review biases behind them.
With that all out of the way, you can enjoy some changes to the website that have made our entire content process plain as day, our creator stats incredibly visible, and our coverage request application transparent and thorough.
Thanks for your continued support!
We want to state that we are getting more requests than ever and it's been mind-boggling at the love our channel has received. We've only been at this for 7 & 1/2 months and it never ceases to amaze us the sheer amount of growth, visibility, and reach our channels receive. We couldn't do this without our community and it's why we're so driven to provide accurate and thorough information in every aspect of content we put out. If you want to keep up-to-date with all the happenings here at The Pillow Fort, consider joining our Discord as it's the best way to interact with us, submit requests, and catch all the news in one easy place, rather than scattered about our different social platforms. Thanks as always, and please feel free to share your thoughts on the changes below, or suggest any ideas you might have for us to improve!
Comments