New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Database structure #3

Open
XePeleato opened this Issue Mar 12, 2014 · 67 comments

Comments

Projects
None yet
6 participants
@XePeleato
Copy link
Collaborator

XePeleato commented Mar 12, 2014

Sorry, I was busy and i have created the Db as i said in #1
I think that for a basic system at the moment, it's fine.
We need to work with the Xenia author because he will tell us what functions are and are not going to be implemented.
I have created the database like this:

Table 1:GamerData:
ID - AN ID it's always useful
Gamertag - The gamertag, i don't know if it's prohibited to use this name
Password - For the gamertag
Gamescore - "The "G" Points
Email - The email of the user
Rep - 1 to 5 stars
Banned - 1 or 0
Reports - Number of times reported
Admin - I think this can be useful with the panel

Second Table: LiveSystem // At the moment it's for statistics more than for configuration.
Online - 1 or 0 if it is under maintenance
Registered - Number of registered gamertags

Please feel free to tell if you think that something needs to be added or deleted.
I have uploaded the .sql to my server for downloading it. http://82.165.170.28/Aixen.sql

@Jan4V

This comment has been minimized.

Copy link
Collaborator

Jan4V commented Mar 12, 2014

Password will be hashed with SHA-512.
Gamertag will be just ok.
Server MySQL ok?

@XePeleato

This comment has been minimized.

Copy link
Collaborator

XePeleato commented Mar 12, 2014

Yes, server OK

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Mar 12, 2014

Actually, I'd prefer if we were to use the 256 bit version of BLAKE2 for
64-bit processors.

On Wednesday, March 12, 2014, XePeleato notifications@github.com wrote:

Yes, server OK

Reply to this email directly or view it on GitHubhttps://github.com//issues/3#issuecomment-37444928
.

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the specific
individual(s) to whom it is addressed. It may contain information that is
privileged and confidential under state and federal law. This information
may be used or disclosed only in accordance with law, and you may be
subject to penalties under law for improper use or further disclosure of
the information in this e-mail and its attachments. If you have received
this e-mail in error, please immediately notify the person named above by
reply e-mail, and then delete the original e-mail. Thank you.

@Jan4V

This comment has been minimized.

Copy link
Collaborator

Jan4V commented Mar 12, 2014

But the password will need to be hashed also on client.

BTW: @haneefmubarak Did you see the servers i sent you by email?

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Mar 12, 2014

Also, @XePeleato, we should implement it however we like, seeing as that we
should be able to add new fields dynamically as we wish. I'm leaning
heavily towards using a NoSQL solution, I just need to see which one has
the best client driver for C.

I think it would be great I we could make the service generic so it can be
extended to support other games and or platforms in the future as clients.

On Wednesday, March 12, 2014, Haneef Mubarak
<haneef503@gmail.comjavascript:_e(%7B%7D,'cvml','haneef503@gmail.com');>
wrote:

Actually, I'd prefer if we were to use the 256 bit version of BLAKE2 for
64-bit processors.

On Wednesday, March 12, 2014, XePeleato notifications@github.com wrote:

Yes, server OK

Reply to this email directly or view it on GitHubhttps://github.com//issues/3#issuecomment-37444928
.

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the specific
individual(s) to whom it is addressed. It may contain information that is
privileged and confidential under state and federal law. This information
may be used or disclosed only in accordance with law, and you may be
subject to penalties under law for improper use or further disclosure of
the information in this e-mail and its attachments. If you have received
this e-mail in error, please immediately notify the person named above by
reply e-mail, and then delete the original e-mail. Thank you.

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the specific
individual(s) to whom it is addressed. It may contain information that is
privileged and confidential under state and federal law. This information
may be used or disclosed only in accordance with law, and you may be
subject to penalties under law for improper use or further disclosure of
the information in this e-mail and its attachments. If you have received
this e-mail in error, please immediately notify the person named above by
reply e-mail, and then delete the original e-mail. Thank you.

@Jan4V

This comment has been minimized.

Copy link
Collaborator

Jan4V commented Mar 12, 2014

Ok so use nosql?

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Mar 12, 2014

BLAKE2 libs are available fairly universally, so this shouldn't be all that
much of an issue. I can implement this pretty quickly.

Jano, we shouldn't settle on anything particular since it's so early in the
project. Yes, I received your email on the keys, thx for that.

On Wednesday, March 12, 2014, Jano Varga
<notifications@github.comjavascript:_e(%7B%7D,'cvml','notifications@github.com');>
wrote:

But the password will need to be hashed also on client.

BTW: @haneefmubarak https://github.com/haneefmubarak Did you see the
servers i sent you by email?

Reply to this email directly or view it on GitHubhttps://github.com//issues/3#issuecomment-37445913
.

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the specific
individual(s) to whom it is addressed. It may contain information that is
privileged and confidential under state and federal law. This information
may be used or disclosed only in accordance with law, and you may be
subject to penalties under law for improper use or further disclosure of
the information in this e-mail and its attachments. If you have received
this e-mail in error, please immediately notify the person named above by
reply e-mail, and then delete the original e-mail. Thank you.

@Jan4V

This comment has been minimized.

Copy link
Collaborator

Jan4V commented Mar 12, 2014

ok

@Jan4V

This comment has been minimized.

Copy link
Collaborator

Jan4V commented Mar 12, 2014

So just to tell everyone,
We have one master server: bigger in EU
and two slaves in US and ASIA
And xepeleato's server.

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Mar 12, 2014

You require multiple masters, actually, two minimum, recommended three.

I think that any VPS or other cloud hosted servers ought to be set as
masters (since the masters run in peer to peer mode), slaves should be home
servers and the like, I will connect a home server myself as a slave once
we have enough basic code for the entire framework. After that, I encourage
others to connect their home servers as slaves too.

Perhaps writing is not the best way of explaining this, I'll draw up a
picture when I get home today.

On Wednesday, March 12, 2014, Jano Varga notifications@github.com wrote:

So just to tell everyone,
We have one master server: bigger in EU
and two slaves in US and ASIA
And xepeleato's server.

Reply to this email directly or view it on GitHubhttps://github.com//issues/3#issuecomment-37446991
.

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the specific
individual(s) to whom it is addressed. It may contain information that is
privileged and confidential under state and federal law. This information
may be used or disclosed only in accordance with law, and you may be
subject to penalties under law for improper use or further disclosure of
the information in this e-mail and its attachments. If you have received
this e-mail in error, please immediately notify the person named above by
reply e-mail, and then delete the original e-mail. Thank you.

@Jan4V

This comment has been minimized.

Copy link
Collaborator

Jan4V commented Mar 12, 2014

I also created a picture but about counting to which server connect and loadbalancing.
I dont have home server nor public ip...

@XePeleato

This comment has been minimized.

Copy link
Collaborator

XePeleato commented Mar 12, 2014

@haneefmubarak It seems that MongoDB works well with C

@perezpaya

This comment has been minimized.

Copy link

perezpaya commented Mar 12, 2014

What about achivements? You should take in to account the achivements of the player to calculate the G points.

Imho MongoDB is not a choice for production, it makes so weird things, high memory usage usually. I prefer using cassandra or mariadb/posgress. Also we should take into account Redis if we want fast response times and to cache some data about sessions, users, queues, distributed events for all machines (pub/sub).

@albertofem

This comment has been minimized.

Copy link

albertofem commented Mar 12, 2014

Hi, new here. Project looks interesting. One thing I noticed after looking at the database schema, is that there is no salt field! By all means we should hash passwords using salting, as hashing alone is proved to be insecure. Also I noticed that you are using the MyISAM engine, which does not support foreign keys and contraints. Nevertheless, maybe we should discuss about schemas on theory for now without bothering about choosing a backend. At the end, once you have a solid database design, choosing it is a minor issue.

@perezpaya

This comment has been minimized.

Copy link

perezpaya commented Mar 12, 2014

Lol, I didn't had time to read code, but seriusly we are not using Salts? That will be a big failure of security if someone steals data from databases, he could use rainbow tables and get lots of passwords.

@XePeleato

This comment has been minimized.

Copy link
Collaborator

XePeleato commented Mar 12, 2014

@alexperezpaya The achievements are stored into the profile.

EDIT: The Gamescore shouldn't be on the server, sorry.

@perezpaya

This comment has been minimized.

Copy link

perezpaya commented Mar 12, 2014

@XePeleato Do you know that there are some limitations at NoSQL databases e.g MongoDB where there are some limitations for each documents, each document can storage up to 16mb and that means that if a user has lots of data the document will be full and could start throwing exceptions.
I had bad experiences with documents that exceed the limit and broked all the db, a friend debugging one app managed to write into a document the full Don Quixote and corrupted all the database.

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Mar 12, 2014

KV stores might be a possibility too. As for the passwords thing, obviously
we would use a salted hash. Worst case scenario, you could simply use the
persons last time of login combined with their user as a salt, and
recalculate the stored hash upon every login.

On Wednesday, March 12, 2014, Alejandro Perezpayá notifications@github.com
wrote:

@XePeleato https://github.com/XePeleato Do you know that there are some
limitations at NoSQL databases e.g MongoDB where there are some limitations
for each documents, each document can storage up to 16mb and that means
that if a user has lots of data the document will be full and could start
throwing exceptions.
I had bad experiences with documents that exceed the limit and broked all
the db, a friend debugging one app managed to write into a document the
full Don Quixote and corrupted all the database.

Reply to this email directly or view it on GitHubhttps://github.com//issues/3#issuecomment-37472921
.

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the specific
individual(s) to whom it is addressed. It may contain information that is
privileged and confidential under state and federal law. This information
may be used or disclosed only in accordance with law, and you may be
subject to penalties under law for improper use or further disclosure of
the information in this e-mail and its attachments. If you have received
this e-mail in error, please immediately notify the person named above by
reply e-mail, and then delete the original e-mail. Thank you.

@XePeleato

This comment has been minimized.

Copy link
Collaborator

XePeleato commented Mar 12, 2014

@alexperezpaya Ok, but we need to get the DB working with C Code, so a NoSQL DB seems to be the best option. ¿Any idea?

@perezpaya

This comment has been minimized.

Copy link

perezpaya commented Mar 12, 2014

@haneefmubarak that seems weird but so secure :)

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Mar 12, 2014

I specialize in WTF solutions :)

On Wednesday, March 12, 2014, Alejandro Perezpayá notifications@github.com
wrote:

@haneefmubarak https://github.com/haneefmubarak that seems weird but so
secure :)

Reply to this email directly or view it on GitHubhttps://github.com//issues/3#issuecomment-37473408
.

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the specific
individual(s) to whom it is addressed. It may contain information that is
privileged and confidential under state and federal law. This information
may be used or disclosed only in accordance with law, and you may be
subject to penalties under law for improper use or further disclosure of
the information in this e-mail and its attachments. If you have received
this e-mail in error, please immediately notify the person named above by
reply e-mail, and then delete the original e-mail. Thank you.

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Mar 13, 2014

we're starting to clog up the issue - so for aixen architectural concerns see #5

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Mar 16, 2014

@XePeleato how's progress coming along?

@XePeleato

This comment has been minimized.

Copy link
Collaborator

XePeleato commented Mar 16, 2014

I am testing some NoSQL DB's , for Key-Value DB's i have tried Cassandra, MariaDB and Bigtable.
Cassandra: seems to be a good option but i couldn't find any good C Driver.
MariaDB: The structure is similar to MySQL (same founder), it has a C Driver, it seems that is a very good option but i don't know if is a trouble the similarities with MySQL.
Bigtable: Cassandra and MariaDB seem to be better than this.

@Jan4V

This comment has been minimized.

Copy link
Collaborator

Jan4V commented Mar 16, 2014

I can work with MySQL/MariaDB so for me no problem.

@Jan4V

This comment has been minimized.

Copy link
Collaborator

Jan4V commented Mar 16, 2014

@haneefmubarak @XePeleato BTW why dont you want to use MySQL/MariaDB

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Mar 16, 2014

I'm familiar with the known limitations (latest: https://foundationdb.com/documentation/known-limitations.html), but what specifically do you object to? Most of the limitations are common to most DBs (others just won't tell you), and the remaining ones seem rather reasonable.

So just list off what you dislike with the research from the rest of the above.

But before you do that, please read the guide on markdown @ https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet (you can use markdown when writing in issues)

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Mar 22, 2014

@XePeleato how's progress on the research coming along?

@XePeleato

This comment has been minimized.

Copy link
Collaborator

XePeleato commented Mar 27, 2014

Sorry @haneefmubarak and all the Team I was very busy.
I am currently researching about this and i wil edit the comment with the info

Aerospike 3

  • They have an "Enterprise License", we need to contact their sale department for more info.
  • It has ACID functionality, for high scale applications, it process 200K transaction per second.
  • The queries are asynchronus.

CouchBase

  • It features the Scalability, Performance and Availability, and their C SDK.
  • Is written in Erlang and C/C++
  • Yes it's Multithreaded
  • Non-Blocking

Riak

  • Riak is written in Erlang with a bit of code in C.
  • It features the K/V storage, and the Scalability, it's pretty similar to Couchbase.
  • I need to research a little bit more, i think queries are non-blocking.

FoundationDB
For me, FoundationDB is fine, now i have researched more data and in my oppinion it is valid.

If you or any member of the team have any Question please ask.
Have a nice day.

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Mar 30, 2014

@XePeleato

This comment has been minimized.

Copy link
Collaborator

XePeleato commented Apr 3, 2014

@haneefmubarak
Updated.

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Apr 3, 2014

Excellent work, @XePeleato.

When you have time, could you please read the guide on markdown @ https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet (you can use markdown when writing in issues) - it'll help a lot in improving clarity if you used formatting (for example, in the above, bullet points would have been nice 😺)? Thanks!

@XePeleato

This comment has been minimized.

Copy link
Collaborator

XePeleato commented Apr 3, 2014

Ok, thanks @haneefmubarak i will take a look.

EDIT: Edited above

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Apr 3, 2014

@XePeleato - if it's not too much trouble, could you also edit your earlier posts in this thread - whenever you get the time of course?

Thanks!

@XePeleato

This comment has been minimized.

Copy link
Collaborator

XePeleato commented Apr 4, 2014

Edited. @haneefmubarak it's big but clear, in my oppinion

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Apr 4, 2014

Cool @XePeleato, thanks again. I'd like your opinion here: if it was entirely up to you, which DB would _you_ pick out of the four we have narrowed it down to and why would you pick that particular DB?

EDIT: added to question

@XePeleato

This comment has been minimized.

Copy link
Collaborator

XePeleato commented Apr 4, 2014

@haneefmubarak I would pick Aerospike 3. ¿Why? Well, it has an official C SDK, ACID Support, a trial of the Enterprise edition and all that i said above.

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Apr 4, 2014

Okay.

@alexperezpaya @MayeulC @Jan4V

If the decision wan entirely up to one of you, which DB would _you_ pick out of the four we have narrowed it down to and why would you pick that particular DB? Please only state what you actually think - don't try to intentionally agree or disagree with each other; it will make this process of selection less effective.

@Jan4V

This comment has been minimized.

Copy link
Collaborator

Jan4V commented Apr 4, 2014

https://www.youtube.com/watch?v=D-fOlAaVlsA

2014-04-04 3:32 GMT+02:00 Haneef Mubarak notifications@github.com:

Okay.

@alexperezpaya https://github.com/alexperezpaya @MayeulChttps://github.com/MayeulC
@Jan4V https://github.com/Jan4V

If the decision wan entirely up to one of you, which DB would you pick
out of the four we have narrowed it down to and why would you pick that
particular DB? Please only state what you actually think - don't try to
intentionally agree or disagree with each other; it will make this process
of selection less effective.

Reply to this email directly or view it on GitHubhttps://github.com//issues/3#issuecomment-39524037
.

@XePeleato

This comment has been minimized.

Copy link
Collaborator

XePeleato commented Apr 6, 2014

And which DB would you choose @haneefmubarak ?

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Apr 6, 2014

Haha. Well, seeing as I'll be ultimately making the choice, I'd like to leave others opinions unbiased for now, but I've narrowed my choice down to:

  • Aerospike3
  • Couchbase
  • FoundationDB

and at the moment, I'm leaning towards Couchbase and FoundationDB because the community edition of Aerospike3 is limited to two servers.

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Apr 13, 2014

Cool. I'll call that a wrap for our DB selection. At this point, I will start doing a series of tests upon each of the following DBs:

  • _Aerospike3_
  • _Couchbase_
  • _FoundationDB_

_Riak_ has been removed from the list because it has unclear documentation and other issues which will make it difficult to develop for, along with their exorbitant commercial pricing model (for the future).

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Apr 13, 2014

I will be testing two scenarios:

  • 4x
    • 4 servers, each has:
      • 2 cores
      • 6GB RAM
    • DBs:
      • AS3 - will _not_ be tested due to server limitation
      • CB - will be tested
      • FDB - will be tested
  • 2x
    • 2 servers, each has:
      • 4 cores
      • 12GB RAM
    • DBs:
      • AS3 - will be tested
      • CB - will be tested
      • FDB - will be tested
@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Apr 20, 2014

DB Chosen: Couchbase

Turns out I didn't even have to benchmark them. While reading through and playing with the code, I noticed that Couchbase was extremely easy to code for, in addition to being free for any number of DB servers. Also, when I played with the DB itself, Couchbase heals in a smart manner, and caches data intelligently in RAM.

This means that we can eliminate heartbeat and master communications, as we can just use the database as a message queue. This also increases the durability of the distribution of master servers, since there will be close to zero data that will be stored on the master servers themselves.

I also noticed that the DB was extremely fast, really only being limited by how fast data could be sent.

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Apr 20, 2014

I believe that we should use the database how it is at its core, namely as a Key-Document Store. This means storing JSON documents associated with keys, and using the keys for lookup along with a JSON parser to look into the documents.

Based on the fields that have been provided throughout the thread, I believe that we ought to have a format that goes something like this:


  • A set of keys, listing all users, indexed by prefix (sorted by date within document):
Key: userList_(prefix)

{
    count: (number of usernames in this document),
    (user n): (Aixen join timestamp ("YYYYMMDD:HHMMSS" in UTC)),
    ...
}

example (key L):

Key: userList_L

{
    "count": 5,
    "Liam": "20140420:000000",
    "Lauren": "20140420:001520",
    "Lea": "20140513:124531",
    "Lox": "20140612:172147",
    "Lund": "20130627:213456"
}

  • A set for users:
Key: user_(username)

{
    ID: (a unique ID positive integer),
    name: {
                given: (first name),
                middle: (null = none, string = middle name / initial),
                sur: (last name)
    auth: {
                pass: (the salted, hashed password - base16),
                salt: (a random unique salt for the password - base16),
                question: (a password recovery question),
                answer: (answer to the above)
    },
    points: (the equivalent of a gamerscore),
    email: (the formatted email address of the user),
    rep: (as a float between -10 and 10, default to 0),
    knownIPs: (an array of the IP addresses that this account has been accessed from),
    ban: {
                current: (0 = no ban, 1 = ban),
                end: (timestamp to end ban, -1 = forever),
                count: (number of previous bans)
    }
    clients: (an object of the clients user has used),
    games: (an object of the games user has used),
    friends: (an array of the users' friends),
    status: (true = online, false = away, string = logoff timestamp)
}

example (user Lund):

Key: user_Lund

{
    "ID": 71,
    "name": {
                "given": "Lund",
                "middle": "M.",
                "sur": "Bobbington"
    "auth": {
                "pass": "2c5711f07de5803bdb48f46609c5a396e00db890db8b933540b8ef600b09cbb5",
                "salt": "cd372fb85148700fa88095e3492d3f9f5beb43e555e5ff26d95f5a6adc36f8e6",
                "question": "best gaming network eva?",
                "answer": "Aixen"
    },
    "points": 85,
    "email": "lund@example.com",
    "rep": 4,
    "knownIPs": ["192.168.1.4", "10.4.1.3"],
    "ban": {
                "current": 1,
                "end": "20140420:000000",
                "count": 0
    },
    "clients": {
                "list": ["xenia", "rpcs3", "native"],
                "xenia": ["win7"],
                "rpcs3": ["osx", "ubuntu"],
                "native": ["osx", "debian"]
    },
    "games": {
                "xenia": ["halo3", "forza2"],
                "rpcs3": ["codmw2", "gta4"],
                "osx": ["client"],
                "debian": ["test"]
    },
    "friends": ["Lauren", "Liam"],
    "status": "20140416:000445"
}

A userIndex document:

Key: userIndex

{
    "userCount": 1000,
    "prefixCount": 3,
    "prefix": ["al", "ar", "L"]
}

A networkStatus document:

Key: networkStatus

{
    "masters": {
                "count": 3,
                "regions": ["US", "EU"],
                "list": ["US0", "US1", "EU0"]
    },
    "users": {
                "global": 21,
                "US": 13,
                "EU": 8
    }
}

Are there any other important document schemas that we need, @XePeleato or @MayeulC?

@XePeleato

This comment has been minimized.

Copy link
Collaborator

XePeleato commented Apr 20, 2014

I understand, it's fine for me.

@MayeulC

This comment has been minimized.

Copy link
Collaborator

MayeulC commented Apr 20, 2014

Fine for me too, I just have a basic remark :
Wouldn't it be better to store user ID instead of usernames? It would be better if the user wants to change its nickname.

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Apr 20, 2014

@MayeulC, updated two schemas:


  • A set of keys, listing all users, indexed by prefix - (sorted by date within document):
Key: userList_(prefix)

{
    count: (number of usernames in this document),
    (username): (userID),
    ...
}

example (key L):

Key: userList_L

{
    "count": 5,
    "Liam": 0,
    "Lauren": 1,
    "Lea": 2,
    "Lox": 3,
    "Lund": 4
}

  • A set for users:
Key: userID_(their full ID number)

{
    username: (a username),
    join: (Aixen join timestamp ("YYYYMMDD:HHMMSS" in UTC)),
    name: {
                given: (first name),
                middle: (null = none, string = middle name / initial),
                sur: (last name)
    auth: {
                pass: (the salted, hashed password - base16),
                salt: (a random unique salt for the password - base16),
                question: (a password recovery question),
                answer: (answer to the above)
    },
    points: (the equivalent of a gamerscore),
    email: (the formatted email address of the user),
    rep: (as a float between -10 and 10, default to 0),
    knownIPs: (an array of the IP addresses that this account has been accessed from),
    ban: {
                current: (0 = no ban, 1 = ban),
                end: (timestamp to end ban, -1 = forever),
                count: (number of previous bans)
    }
    clients: (an object of the clients user has used),
    games: (an object of the games user has used),
    friends: (an array of the users' friends),
    status: (true = online, false = away, string = logoff timestamp)
}

example (user Lund):

Key: userID_4

{
    "username": "Lund",
    "join": "20130627:213456",
    "name": {
                "given": "Lund",
                "middle": "M.",
                "sur": "Bobbington"
    "auth": {
                "pass": "2c5711f07de5803bdb48f46609c5a396e00db890db8b933540b8ef600b09cbb5",
                "salt": "cd372fb85148700fa88095e3492d3f9f5beb43e555e5ff26d95f5a6adc36f8e6",
                "question": "best gaming network eva?",
                "answer": "Aixen"
    },
    "points": 85,
    "email": "lund@example.com",
    "rep": 4,
    "knownIPs": ["192.168.1.4", "10.4.1.3"],
    "ban": {
                "current": 1,
                "end": "20140420:000000",
                "count": 0
    },
    "clients": {
                "list": ["xenia", "rpcs3", "native"],
                "xenia": ["win7"],
                "rpcs3": ["osx", "ubuntu"],
                "native": ["osx", "debian"]
    },
    "games": {
                "xenia": ["halo3", "forza2"],
                "rpcs3": ["codmw2", "gta4"],
                "osx": ["client"],
                "debian": ["test"]
    },
    "friends": [1, 0],
    "status": "20140416:000445"
}

Is there anything else we ought to alter?

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Apr 20, 2014

Well, I thought it would be nice to know which master server is handling a client, so I amended the userID schema:


  • A set for users:
Key: userID_(their full ID number)

{
    username: (a username),
    join: (Aixen join timestamp ("YYYYMMDD:HHMMSS" in UTC)),
    name: {
                given: (first name),
                middle: (null = none, string = middle name / initial),
                sur: (last name)
    auth: {
                pass: (the salted, hashed password - base16),
                salt: (a random unique salt for the password - base16),
                question: (a password recovery question),
                answer: (answer to the above)
    },
    points: (the equivalent of a gamerscore),
    email: (the formatted email address of the user),
    rep: (as a float between -10 and 10, default to 0),
    knownIPs: (an array of the IP addresses that this account has been accessed from),
    ban: {
                current: (0 = no ban, 1 = ban),
                end: (timestamp to end ban, -1 = forever),
                count: (number of previous bans)
    }
    clients: (an object of the clients user has used),
    games: (an object of the games user has used),
    friends: (an array of the users' friends),
    status: {
                current: (true = online, false = away, string = logoff timestamp),
                server: (master server name or last connected server)
    }
}

example (user Lund):

Key: userID_4

{
    "username": "Lund",
    "join": "20130627:213456",
    "name": {
                "given": "Lund",
                "middle": "M.",
                "sur": "Bobbington"
    "auth": {
                "pass": "2c5711f07de5803bdb48f46609c5a396e00db890db8b933540b8ef600b09cbb5",
                "salt": "cd372fb85148700fa88095e3492d3f9f5beb43e555e5ff26d95f5a6adc36f8e6",
                "question": "best gaming network eva?",
                "answer": "Aixen"
    },
    "points": 85,
    "email": "lund@example.com",
    "rep": 4,
    "knownIPs": ["192.168.1.4", "10.4.1.3"],
    "ban": {
                "current": 1,
                "end": "20140420:000000",
                "count": 0
    },
    "clients": {
                "list": ["xenia", "rpcs3", "native"],
                "xenia": ["win7"],
                "rpcs3": ["osx", "ubuntu"],
                "native": ["osx", "debian"]
    },
    "games": {
                "xenia": ["halo3", "forza2"],
                "rpcs3": ["codmw2", "gta4"],
                "osx": ["client"],
                "debian": ["test"]
    },
    "friends": [1, 0],
    "status": {
                "current": "20140416:000445",
                "server": "US0"
}

This was referenced Apr 20, 2014

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Jun 1, 2014

Some additional areas of possible research:

  • additional schemas (ie: games, logons from an IP address, etc.)
  • review of existing schemas (what needs to be added, what needs to be removed, etc.)

@Hykem, are there any changes to existing schemas (ie: more fields) or new schemas that would be useful in implementing an emulation of PSN?

@ghost

This comment has been minimized.

Copy link

ghost commented Jun 9, 2014

There are a few fields specific to the PSN that would be important for emulation (e.g.: trophy related and DRM fields).
I'm doing some research, mainly on the DRM, to understand what is truly necessary, but the generic fields presented seem to be quite satisfactory for both online platforms.

@haneefmubarak

This comment has been minimized.

Copy link
Owner

haneefmubarak commented Jun 9, 2014

@Hykem are PSN trophies similar to XBL achievements?

@ghost

This comment has been minimized.

Copy link

ghost commented Jun 20, 2014

@haneefmubarak Yes, it's a similar mechanism.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment