The internet as we know it was not made for games, certainly not real time games like Riot's League of Legends. Problems with 'ping' and 'packet loss' aren't just issues for individual players sitting at home. They're serious business concerns for developers and publishers. And, at the scale at which a company like Riot operates, small improvements in customer experience can result in big changes to the bottom line.
From Guinsoo's twitter: 'If you're having trouble downloading the new LoL patch (speed-wise) set your default browser to IE, close launcher,. Client install taking forever?? And have had no problems with other games that use a digital download (WoW andSc2). League of Legends and PvP.net are.
To smash more people into their game, to get more players to spend time and treasure on their champions, Riot needed more bandwidth. The big guys like AT&T and Comcast were unable (or unwilling) to play ball, so Riot built their own private internet, literally cobbling together their own high-speed network from unlit fiber optic cables sitting in the ground. They claim to have created 'one of the fastest networks on the continent' just so people have a better experience playing their game.
That, coupled with the hard work of moving their game servers across the country to a new datacenter in Chicago, Riot says increased the number of players with 80 millisecond pings 'from 50 percent to 80 percent overnight.'
Riot claims to have made 'one of the fastest networks on the continent'
If their self-reported figures are to be believed, it's an incredible technical achievement. And the story behind it — told in twoparts on the Riot Games Engineering blog — is a great read ... if you're into that kind of thing.
I realize that you may not be into that kind of thing, so let me try and break it down for you in layman's terms.
This time last year when you connected to a game of LoL you were connecting through your home router, then through your rented internet gateway and then across a bewildering collection of routers to dozens of different datacenters all the way to Riot's game servers in San Jose, California.
Imagine flying from your house to San Jose on a flight that has 13 connections and four layovers. It would take a while to get where you're going, and you might not be in the best shape once you got there.
So, what Riot did was cut out as many legs of that trip as it could. The collection of routers in dozens of different datacenters? The 13 connections and four layovers? Gone.
They did it by lighting up so-called 'dark fiber' that's been buried in the ground, unused for years and supplementing that new, private network with a set of leased connections to fill in the gaps and provide redundancy.
So now, when you want to fly out to San Jose for some LoL, you've got a direct flight.
But then Riot went a step further and actually moved San Jose closer to your house.
League Of Legends Taking Forever To Download Pc
Well, not really. They moved their game servers to a new facility in Chicago, right in the sweet spot for the new network they'd just designed. As you can see from their chart below, the day they made the cut-over to the new facility the percentage of their users with the kind of low-latency connections needed for a quality multiplayer online experience jumped dramatically.
In a previous life I used to do a bit of the salesmanship for this kind of dark fiber and colocation... thing. I have an incredible level of respect for the amount of technical acumen — and the sheer amount of labor — involved in making something like this happen. For a single game company to go to such lengths for a single game product is remarkable, and a real testament to how much Riot values their community.
Despite being a very fun game, there are many, many things in League of Legends that desperately need to be fixed, improved upon, or added in the first place. A key figure at League’s development studio Riot wants the game’s tens of millions of fans to know that delays for crucial improvements aren’t the result of misguided priorities.
Jeffrey Lin, Riot’s social system designer who’s better known inside the League of Legends community by his handle “Riot Lyte” (or just Lyte) gave an explanation for Riot’s workflow last night on his ask.fm page in response to a fan question that many players have been wondering about: why hasn’t the game gotten a new (and hopefully better) replacement for its outdated client yet? Developers at Riot have spoken about releasing a revamped client for more than a year now, but players haven’t seen any hints of the thing show up in the game yet.
The concern from fans is both a popular and real one. It’s easy to look at things like League’s archaic in-game client, or its awesome-sounding-but-never-fully-functioning “team builder” matchmaking mode, or Riot’s many attempts at a streamlined system to combat player toxicity, and see little more than a bunch of exciting but unfulfilled promises. And the situation only looks worse for fans when Riot keeps making bigger and crazier promises before actually seeing through the ones it’s already made, like the company did at the beginning of 2015 by announcing the crazy lag-killing network its working on. That, and the fact that Riot excuses delays for things like a new client by saying they take time...while simultaneously pumping out stuff like priced character skins and new champions with the speed of a charged up gatling gun.
Add all that up, and you can understand why many reasonable League of Legends players like myself sometimes feel that Riot has got its priorities wrong: taking its sweet time with much-needed fixes that they can’t really charge people money for, while continuing to put out premium content that will deliver them short-term returns. In his response, Lin counters this line of thinking by pointing out that producing stuff like characters or skins doesn’t mean Riot is not prioritizing important things like the new client. It just looks like that’s the case because, unsurprisingly, releasing an individual character skin is a hell of a lot easier than making a whole new client for 67 million-odd people (emphasis mine):
It [a new client] is a priority. However, to explain a little bit about development, we have many different teams with different expertise. For example, a Champion Team has team members who specialize in game engineering, art, game mechanics design, and although they can make excellent champions, they aren’t well-suited to make new skins or systems like Leagues or Tribunal or a client. So, different teams have different priorities and every project is a different size and timeline. So, Feature A might be a 3 month project, while Feature B might be a 6 month project and Feature C might be a 9 month project. But, Feature C might be the highest priority at the studio, even though it would ship “last” in this example. You really can’t determine a studio’s priority based on what “launches,” because all of these features launch on very different cadences because of their scope, not because of the size of their teams necessarily or their priority in the studio.
As a studio, you tend to have a few “big bets” at a time which are big projects that take quite a long time, but we believe are worthwhile. The client and Team Builder Draft are considered mid- to larger sized projects so it takes longer to launch these than say, one new skin. So, when you see a bunch of smaller features and projects “finish” before the big projects, it does NOT mean we only prioritize smaller features and projects—it just means smaller projects tend to have shorter timelines and therefore ship earlier and more often.
Advertisement
These sorts of “why is it taking so damn long?” questions have been on Lin’s mind a lot this month. Earlier in July, he sounded aggravated responding to a similar question about why Riot “takes so long to implement stuff.” He suggested that concerns like this are “naive and ignorant” about the realities of game development:
League Of Legends Taking Forever To Download Movie
I’m going to be honest, every time someone asks about why Company A or B is moving so slow developing new software whether it’s a game or app, developers are crushed inside. It’s such an unfair question, and people need to stop. Many years ago when I was just a gamer, I used to think the same things. I thought, “Why does it take so long to implement this feature, it seems so easy.” I was an idiot.
I started visiting studios whether it was some at EA, Ubisoft, Blizzard, Valve and Riot, and companies like Google, Facebook and Amazon. We can all agree that these are respectable companies, and guess what? All of them have their pros and cons and you can’t even imagine the challenges that face these companies everyday. How long do you think it took Google to develop Gmail? How many projects lasted years at these companies and had to be canceled before you ever heard about them? How long did it take Facebook to re-design their Thumbs Up icon? Developers do what they do because they are passionate about building stuff. They put all of their heart, sweat and tears into these games and products. There are nightmares in the games industry about “crunch” where teams work excessive hours every single week just to get a game out the door a few months early and guess what? These developers burn out and quit and never make another game. How many amazing games will never be made because of the non-stop harassment and stress that gamers put on them? Developers are never going to be fast enough in some eyes because the outside person doesn’t know how difficult it is to build industry-changing, high quality software that can scale to the global masses.
“Could we be faster at Riot? Of course,” Lin concluded. “Every company could be faster, and there are always things to improve. But, when you’re making stuff for over 67 million players around the globe, even the simplest designs become quite complicated.”
Advertisement
Lin makes a lot of good points that are supported by League of Legends’ recent history. When the developer finally got around to putting out an often-asked-for update to its in-game HUD recently, the game’s forums and subreddit exploded with hyperbolic exclamations about how it’s the worst thing ever. And when they redesigned a woman champion recently to make her slightly less of a “sexy MOBA lady,” fans recoiled in disgust in an almost identical way. Having an audience the size of League’s can make the developer seem risk averse and conservative by any number of people’s individual standards.
At the same time, Lin’s points are ones that game developers always seem to rush to whenever they’re faced with criticism from fans and players. The “you don’t understand how hard it is!” line can be legitimate, but it can also be an easy way to disengage from a fight you know you’re not going to win, to put it in a very MOBA way. Lin can’t really account for the ways that League of Legends’ neverending parade of new premium content makes Riot Games tons of money that a) its publicly-traded parent company Tencent Holdings undoubtedly demands that it makes and b) pressures them to keep making new pricey stuff rather than diverting more resources to something essential that they couldn’t charge people for, like a new client. If League of Legends existed in an economic environment that didn’t require it to churn out content at a nonstop clip (much like Gawker Media, the company that owns and publishes Kotaku), then its publishers could hire more developers to focus on improving and maintaining the content that already exists.
But calling out gamers for harassment is a much simpler solution than taking a hard look at the business models that’ve been applied to League of Legends as a free-to-play game and Riot Games as its publisher.
Advertisement
To contact the author of this post, write to yannick.lejacq@kotaku.com or find him on Twitter at @YannickLeJacq.