Question

[ryancole]

Hello,

I'm trying to understand your blowfish decryption of the chunk and keyframe data. I had a few questions. The payload header contains an encryption key, but in decrypt.rb LN31-33 it looks like you're pulling the blowfish decryption key from the JSON metadata portion of the file.

To decrypt the keyframe and chunk data, we do not use the encryption key from the payload header, but instead we use a key derived from the JSON metadata - is this correct?

This is confusing me, because in the JSON metadata that I'm extracting from my own replay files, I do not see the gameKey.gameId, or key keys that you are using to extract the decryption key. I do see a gameId, which I assume is the same as the first, but I do not see a JSON metadata property named key. Have these changed, or am I not understanding your code? I'm not a Ruby programmer and your code is pretty straightforward, but I may be misinterpreting something.

Thanks!

[lukegb]

Here's some code I wrote with robertabcd's help - its a Python script which should successfully parse and decrypt LoL (official) replays.

To clarify, the JSON file referred to in the decrypt script is a custom one, not the Riot metadata.

https://gist.github.com/d2997a5fc7970ce6e1e1

[lukegb]

I'd love to be updated on anything you're doing - trying to figure out the key frame/chunk format myself ;-)

[ryancole]

@lukegb Ah. Ok that makes some sense then. I saw a disconnect between the Ruby script that reads the file, and the other script that decrypts the chunk data. So, I figured it might be something like that. I'll check out your code. I've basically just converted his Ruby code to C#, and took more a stream-based approach to reading it. My code is here: https://github.com/ryancole/LeagueReplayReader

Also, thanks for your code. I will check it out and see if it clarifies anything!

Edit: Could you possibly give a simple explanation of the answer to my question, also? I'm looking over your code, but it has been a long time since I've used Python. So I just want to be sure I understand it properly as I look over it. What key is used for decrypting the chunk and keyframe data?

[robertabcd]

First to answer your question. When decrypting ROFL files, the gameId and key in the header should be used, and there should be no key in the metadata json.

It comes with a historical reason. The decrypt.rb is originally designed to work with the output of download.pl, and it is written before the ROFL format is reversed.

download.pl produces meta.json by combining json returned from getGameMetaData, and a user-supplied key attribute added in. (The key only can be obtained from the retrieveInProgressSpectatorGameInfo RPC call through PVP.net client, or featured games json. Not exposed in the "replay cloud.")

After ROFL format is reversed, I found that keyframes and chunks are identical to those downloaded from the "replay cloud", so I tried to reuse the script. Thus, the json metadata in ROFL files contains different information.

Keeping this open unless clarified somewhere in the source tree.

[lukegb]

Clarified file format at https://github.com/robertabcd/lol-ob/wiki/ROFL-Container-Notes

Might want to edit that - I just braindumped into it.

[themasch]

Do you know if the same format applies to the chunks coming over the Spectator REST API? I tried to decode such a chunk using the described mechanism but just getting random data. zlib (Gunzip) fails with "incorrect header check"
[EDIT] whoop nvm, working! Just confused "key" with "password" when creating a decipher.

[Tastefull]

Hi there,

I have a little "project" that i would like some help with if posible. Its not to hijack the thread, but its relevant to the decrypt of the chunk part your talking about.

Outline of what I want to do:
We are doing a lot of live streaming of our LoL tournaments and league, but we cant stream them all. So i want to make a "LoL text event echo'er". I have done this many years ago for Counter-Strike and is a simple way for people to follow a match to get the main actions. I would integrate it into to a IRC Bot and do the "live echo" in a channel there.

The output im looking for is something similar to this:
**** Player 1 (2/1/4) killed Player 2 (1/3/2)
**** Teamscore: Blue 11 (2 turrents) - 2 Red (2 turrents)
**** Player 3 killed the dragon
**** Player 2 destryed a outer turrent
etc
etc

Is this in anyway posible when decrypting the stream from the spectator stream?

Kind regards
\Thomas Hansen

[themasch]

Thats pretty much what my goal for this project was. I'm hoping that we get all this information when we are able to decrypt the stream.
I'd really love to get some people together and collect some know how to be able to get this done, currently I don't see a way to make any progress.
Downloading spectator data is possible, decrypting and decompression seems to work but I've no clue how to read the data. Enconding is still unclear.

[Zero3]

@themasch Count me in. I got a couple of ideas too, but a little lack of time at the moment. I have previously spent some time inspecting the data though, and the good news is that it definitely is structured in some way. The bad news is that I don't yet know in which format. If you start something up, I'm interested in hanging around. I can't promise when I will have time to properly dig into this though.

[themasch]

@Zero3 that sounds good. I know these time issues very well, suffering from the same.
Might drop some code soon but I can't promise anything currently. I'll let you know if something becomes availiable.

[Tastefull]

Can you try to send me som samples of the chunks that are decrypted and decompressed, then i'll try to take a look at them :)

\T

[robertabcd]

This project may help: https://code.google.com/p/packet-lol/
Last time I tried, it matches some part of data in chunk files, but not matched perfectly.

[ryancole]

You're not going to be able to make sense of the chunks without being able to look at it inside of a debugger. There are some plain text strings throughout it, but the majority is just bytes of data.

[jaagupkymmel]

I would love to help, but I dont't have any experiance with decrypting packets etc.

[themasch]

@ryancole i'm afraid you are right.

I just collected any information I could find into one document (with aweful english). If someone know something thats lacking, feel free to add it: https://gist.github.com/themasch/8375971

[Divi]

@themasch https://gist.github.com/themasch/8375971#endofgamestatsregion-gameid--amf- endOfGameStats is actually an AMF base64 encoded file.
Here an example after converted AMF to an array : http://pastebin.com/KB4TUPhs

[themasch]

@Divi wow, nice. Thanks. Will update the document soonish.

[robertabcd]

@themasch Just noticed. Consider changing "region" to "platformId", which conforms to the JSON returned from "../featured"?

[Divi]

@robertabcd In the RTMP API, this value is called "originalPlatformId" when retrieving current game data.

[robertabcd]

@Divi I see... Any ideas on differences between original and not-original?
Edit: mentioned wrong person.

[themasch]

I think platfromId does the job until we know if theres original and a not-original version.
I just updated the gist. Should I make a normal repository out of this so we can have issues and commits and stuff?

[Divi]

@themasch I guess repository is better.
@robertabcd The RTMP API has pretty old data/keynames, I guess platformId is correct.
Btw, if you want to see the full current game API response : http://pastebin.com/rZx4tjEE

EDIT:
About the packet sniffing, maybe we should check some of hacking/cheat forums on LoL, they are often the first to decode packets.

This one has many tools on the forum : http://botoflegends.com/forum but it seems to be down a lot of time (DDoS or host issue, don't know).

[Divi]

@robertabcd @themasch I made a lot of researchs, and spectator packets are not the same as live game packets. Chunk/keyframe files are clearly a list of packets, but I don't know the separator between each packet. I don't have the skill for decoding packet, maybe @Zero3 with his idea can do that.

[Zero3]

@Divi If a spectator "frame" actually contains several "packets", there might not even be a separator (it might be implicit given the context). It really all depends on the encoding used, which we don't know yet. I think decoding the packets will be a fun challenge, and I really want to help out with this once I get some time on my hands (which unfortunately is not in the very near future because of studies). Hint: There are a lot of plaintext strings in the decrypted packets which should make this job significantly easier.

[Zero3]

By the way: If someone goes ahead with this, they should definitely get in contact with the guy behind http://www.leaguereplays.com/. By the looks of the decompiled .NET code, this guy knows a lot about the internal data structures used by LoL. Chances are that a lot of things are reused in the spectator packet format. There is of course also the possibility of asking Riot about the format. They probably won't make this stuff public (given their efforts with encrypting the packets in the first place), but perhaps they are willing to cooperate under an NDA or some other legal contract about the purpose of using the data. Who knows? :).

[Divi]

@Zero3 I don't think "LoL Replay" knowns about data structure, because it just downloads and put chunks and keyframes in one unique file. And when a player want to watch a game, it creates a local REST server readable by LoL Client. Btw, good luck for yours studies :)

[themasch]

@Zero3 not sure about how much these guys know. Besides that, I'm no fan of stuff like NDAs at all ;)
@Divi @robertabcd @ryancole and all the others:
https://github.com/themasch/leaguespec
I just created a repository for collecting informations. I'd really love to see pull request and issues pile up ;)
I don't feel like giving random ppl commit permissions yet but I'll give them later to ppl who participate in the project and behave nicely. I hope you understand that, I'm just trying to prevent a mess. Is that fine for everyone?

[ryancole]

As most people know, the chunks and keyframes within a replay file do indeed contain packet data. This has been confirmed to me by two friends of mine who both work at Riot. But even without that, I think it's pretty obvious to most people who have seen the chunks and watched that data flow through League in a debugger.

Anyway, the packets are not delimited by anything, based on my research. Some packets have static sizes, and so the size of the packet within the chunk is known / hard coded. Other packets, with variable sized payloads, have length identifiers, which tell you how large the packet is going to be on a per packet basis. This is common among many different game protocols. namely Blizzard products.

Now, I spent about a month looking at the replay file's chunks going through the League client, in a debugger. The logic is pretty simple and works as you'd expect. The game uses the chunk data to increment the game state in real time, as you'd expect. I never did manage to fully map out any significant packet format. I have an extremely well documented IDA project file, from about 7-8 months ago, in which I have the replay system's main loop fully documented. I have bookmarks set on many of the different packet handler functions.

The reason I stopped working on this though is because i encountered some code that completely confused me and I just couldn't wrap my head around it any more. It threw me out of the zone mentally, if you will. Basically, while debugging one the packet handlers, I watched as the League client was doing it's normal parsing of the replay file, grabbing a length, reading in the data, etc, and then all of a sudden the function decided to go read some data from way down in the bottom of the chunk file. This is confusing, because you'd expect it to just read from top to bottom (which it does do for the most part). This lead me to believe that the packets, in the chunk file, are able to reference sort of a shared memory or something.

Basically, my final concluding thought on the replay files is that a single chunk contains more than just packets. It contains packets, as well as some sort of shared state buffer or memory. Packets can reference this chunk of memory, it looks like. The packet came first in the chunk file and the shared memory appeared to be at the bottom of the chunk. That's about all I determined before I decided to take a break on it.

[ryancole]

Does anybody in here happen to have a .rofl file that I could test with? My PBE account appears to have been marked inactive.

[Divi]

@themasch thanks for the repo :)
@ryancole what kind of soft/debugger do you use to inspect the client ? I haven't .rofl file unfortunately. I'm pretty sure that if you ask for a file on Reddit, someone will give it to you in PM.

[jaagupkymmel]

I worked on @tyscorp gist that contained some data about keyframe format, you can find it at https://gist.github.com/jaagupkymmel/a36e07f4abb4b012c66e

[themasch]

Thats nice work. I've found some additions, too. Maybe we could put this doc into loldevs/leaguespec? @tyscorp ?

[tyscorp]

https://github.com/loldevs/leaguespec/wiki/Keyframe-Specification

[themasch]

wow you are fast. Great work!

[Valandur]

Hey guys, thought I'd join in on the discussion because this sounds rather interesting.
@themasch You asked in an earlier comment if the REST API was using the same kind of encryption method as the .rofl files. Then you edited saying that you confused the key with the password for the blowfish decipherer. I have tried doing the same, by taking a game from the /featured list, which already contains the encryption keys, but I then keep getting an incorret header check after decyphering when unzipping. Can you elaborate on how you went about doing this?

[tyscorp]

Are you decrypting the "encryption key" with the gameId to get the real key?

On Tuesday, 4 February 2014, Valandur notifications@github.com wrote:

Hey guys, thought I'd join in on the discussion because this sounds rather
interesting.
@themasch https://github.com/themasch You asked in an earlier comment
if the REST API was using the same kind of encryption method as the .rofl
files. Then you edited saying that you confused the key with the password
for the blowfish decipherer. I have tried doing the same, by taking a game
from the /featured list, which already contains the encryption keys, but I
then keep getting an incorret header check after decyphering when
unzipping. Can you elaborate on how you went about doing this?

Reply to this email directly or view it on GitHubhttps://github.com//issues/1#issuecomment-33978457
.

[Valandur]

@tyscorp Yes. If I understood the documentation correctly you use the gameId to decrpyt the base64-decoded "encryption key", then use this key to decrypt the chunks and keyframes.

[tyscorp]

@Valandur are you using unzip or gunzip? The data is gzipped before being encrypted.

[Valandur]

@tyscorp I'm using gunzip. First I decrypt the data as decribed above (Blowfish in ECB mode, with Pkcs5/Pkcs7 padding), then I try to unzip it.
According to the gzip file specifications the magic number (first two bytes) should be 0x1f followed by 0x8b, which I never actually get.

Maybe an example would help - taken from one of the featured games.
The game has the id

751095183

The encryption key is

vzRVWdZRzBKby4BiBvVOaaMHVNe6TITq

or converted from base64 to a byte array in hex notation

BF-34-55-59-D6-51-CC-12-9B-CB-80-62-06-F5-4E-69-A3-07-54-D7-BA-4C-84-EA

For me this yields a real encryption key of

CA-7A-8F-3E-20-DB-BC-CD-6F-0C-0E-0C-E3-E5-73-B1

what would your decrypted real key be?

[tyscorp]

I get

C2-8A-5E-48-C3-AC-C3-9F-C3-80-C3-A5-1D-2E-49-19-C3-BB-C3-B8-09-47-C3-A6

[Valandur]

Ok so it looks like I'm doing something wrong when decrypting the key. Maybe I'm using some weird Blowfish algorithm, or I'm not using the class how it's meant to be used. I'll look into it, thanks for your help so far :)

[tyscorp]

I had problems where I was using the wrong input format. Converting the observerEncryptionKey from base64 to binary didn't seem to work. I ended up using the "base64" flag in my encryption lib instead of converting it myself.

[Valandur]

Hmm, I think I'm just like, being stupid right now :/
Are you interpreting the game id as a 64bit integer? so the game id mentioned above would become

8F-CD-C4-2C-00-00-00-00

[tyscorp]

It has to be a string.

[Divi]

@Valandur You have your encryption key (as string, like @tyscorp said), base64 decode on it, and Blowfish ECB with padding#5 with game id as the key. Do not unzip the result.

After, do a blowfish ECB padding#5 on chunk/keyframe with the decrypted result string above, then GZ decode it.

[Valandur]

@tyscorp @Divi What I meant with my previous comment is that the blowfish decryption library I'm using requires your key to be a byte array. So, I take the game id as a 64bit integer, and convert it to a byte array. Then I pass that array as the key of the blowfish algorithm to my decrypter, which decrypts the base64-decoded observerEncryptionKey. And this is gives me the "real" decryption key, which seems to differ from what it should actually be.

[tyscorp]

gameId has to be a string.

[robertabcd]

As @tyscorp said, you have to convert gameId into string (eg. "751095183"). Blowfish supports variable key length, most implementation just repeats the bytes until it fits in.

[themasch]

Thats the code I'm using in node.js:

function decrypt(pass, data) {
    if(!Buffer.isBuffer(pass)) {
        pass  = new Buffer(pass.toString());
    }
    var decipher  = crypto.createDecipheriv('bf-ecb', pass, "")
    var inBuf = new Buffer(data, 'base64');
    decipher.write(inBuf);
    return decipher.read();
}

Related doc entry: http://nodejs.org/api/crypto.html#crypto_crypto_createcipher_algorithm_password

[Valandur]

Thanks a lot for the help guys, was able to fix the problem. It was a combination of not removing the padding from the encryptionKey after decrypting it, and not using the gameId as a string.

[snowl]

How did your remove the padding from the encryptionKey?

[Divi]

You must not remove the encryption key padding.

[jaagupkymmel]

May I suggest making the wiki private, atleast for the time being. I'm fairly certain none of us want this information to get in hands of people who might use it to create malicious content and I don't think Riot would be too happy to see that someone can read all their spectator data, especially if they share it publicly.

We should still give everyone who wishes to contribute access to it.

[Zero3]

@jaagupkymmel I think this is an important ethical decision we ought to make ASAP. I would personally support either way.

[themasch]

I'm really against close-sourceing this. This started as a public project, done by and done for the community. There may be ways to misuse this data but currently I don't see any evil things to do with what we got yet.

We should still give everyone who wishes to contribute access to it.
How should this protect the information from getting misused?

But this isn't my project and I might be the least relevant contributor so I'd really like to hear some more opinions ( @robertabcd, @tyscorp, @trebonius2, @Divi ... )

If the project members decide to hide this, I won't stop them ;)

[lukegb]

I disagree with close-sourcing this. It's useful information - it's nothing especially private; after all, League Replays does this sort of thing (but much more coarsely, I admit) and they haven't been explicitly killed.

Doesn't mean I won't advise that we keep individual copies of the wiki git repository though, and mirror that offsite...

[themasch]

Why do we discuss this on another projects issues anyways?
loldevs/leaguespec#3