Redesign Of StorageDevice

[tomspilman]

As part of working to support Microsoft's .NET Native i'm looking at incompatibilities that we need to resolve.

Some of these issues are with the following APIs:

• StorageDevice.BeginOpenContainer
• StorageDevice.EndOpenContainer
• StorageDevice.BeginShowSelector
• StorageDevice.EndShowSelector

The issue with them is the use of AsyncCallback and IAsyncResult which is unsupported in .NET Native.

We need to decide how to modify, remove, and/or replace these with modern equivalents.

[tomspilman]

My initial thoughts are that none of these APIs make sense anymore.

Selecting the storage device doesn't happen on modern devices... mobile, console, or PCs... it simply always defaults to the user data area which is mirrored to the cloud in some cases.

The question then would be what does the new API here need to look like? Should it be more like this:

public StorageContainer OpenContainer(string gameTitle, PlayerIndex player)

My thinking here is that local multiplayer games with multiple users logged in means multiple areas for saving game data.

Thoughts?

[totallyeviljake]

I thought StorageContainer was supposed to be tied to your login profile. So you could login with JoeX and have a different storage area than SuzieY, rather than player 1, 2, 3, etc.

[tomspilman]

I thought StorageContainer was supposed to
be tied to your login profile.

It is... PlayerIndex is a generic way to specify that. Internally the API looks up the profile tied to it.

So you could login with JoeX

Of course StorageDevice wouldn't manage logins... logins are done somewhere else. All the StorageDevice API needs is a way to say which player you need storage for. Internally how that is resolved to a user profile is implementation specific.

[tomspilman]

Here is a pretty good overview of the existing XNA storage APIs:

http://msdn.microsoft.com/en-us/library/bb200105.aspx

[totallyeviljake]

Good link Tom! Looks like your API fits well into the current XNA storage management. As for the game title, is there any way to get that from the game object? Otherwise, can I write a game saver that saves like "OpenContainer("ARMED!", 1)" and kill someone's game save from another game?

[tomspilman]

Otherwise, can I write a game saver that saves
like "OpenContainer("ARMED!", 1)" and kill someone's
game save from another game?

I assume you mean on a PC... that still won't be secure or anything. Any game you install on a PC can delete or overwrite files in any area your user account has access to. This would be no different really.

I was simply following basically XNA already does

User storage is in the My Documents folder of the user who is currently logged in, in the SavedGames folder. A subfolder is created for each game according to the titleName passed to the BeginOpenContainer method.

So we would be no more or less secure than XNA in this respect on PCs.

On console and mobile this will be pretty secure as the OS only lets you write to the user storage area for your particular game.

[totallyeviljake]

I figured as much, but now at least those unaware have been given "awareness."

Looks good Tom!

[tomspilman]

I'm still hoping to get some more feedback. Anyone else have any experience in this area to help define what this API should be?

[tomspilman]

@theZMan - Any experience to share here?

[theZMan]

How is the async stuff not supported - isn't aysnc a fundamental part of W8/WP8/WinFX?

Aside from that the reason for the async IMO is the slow Xbox hard drive. To meet cert requirements you have to render at least 10fps all the time so this was the sensible way to do it.

On the other hand W8 also has similar requirements for repsonsivness which is why they encourage async stuff too.

So I would suggest looking at whatever WinRT does for file IO - buried deep in here I would guess http://msdn.microsoft.com/en-us/library/windows/apps/windows.storage.aspx - if they stick with async then so should we.

[tomspilman]

How is the async stuff not supported - isn't aysnc a
fundamental part of W8/WP8/WinFX?

The new async/await and Tasks totally work... the old async stuff used in XNA is not supported yet.

Aside from that the reason for the async IMO is the slow Xbox hard drive.

I guess a redesigned StorageDevice.OpenContainer() would still possibly do some sort of file operation when it is called, so not making this a blocking call might be a good thing. Although it might just be unnecessary complication as it will be pretty instant and probably a one time per-game call.

The real slow stuff will be calls to StorageContainer APIs themselves which don't have any sort of async support at all in XNA or MonoGame.

So I would suggest looking at whatever WinRT does for file IO

That isn't a bad idea.

I although I say that game developers tend to do async at a higher level. For example an async ContentManager isn't important to us as we typically put all content loading into a background task/thread anyway.

[tomspilman]

Going to tackle this soon. Any other feedback is appreciated before I begin.

[shmuelie]

I'd suggest taking a leaf from the Async/Await advice Microsoft gives and provide both synchronous asynchronous methods so the developer can chose if they want to block or not. Do remember that it's not threading that Async/Await is meant to solve as much as calling (mostly) IO operations without blocking. So even if a developer is loading on a background thread non-blocking code will use less resources.

[harry-cpp]

My idea for a new design for this would be:

public class StorageDevice
{
    public string[] GetStreamList();

    public bool StreamExists(string name);

    public Stream GetStream(string name);

    public void DeleteStream(string name);
}

For the most part just provide a stream which can be used for saving, and that is that.

[KonajuGames]

My idea for a new design for this would be:

It needs to work with multiple users logged in and have a separate area for each user, as happens on consoles. And also be a separate area for different games.

I'm not a fan of the repeated use of the word Stream in the APIs you suggest.

I believe new Storage APIs should use async/await. Don't rely on the developer to do the threading.

[harry-cpp]

It needs to work with multiple users logged in and have a separate area for each user, as happens on consoles.

I totally didn't think about that.

And also be a separate area for different games.

Did thought about that, it just needs to create a container for the game from the assembly information, tho leaving the user a choice to set it up manually might be good.

I'm not a fan of the repeated use of the word Stream in the APIs you suggest.

I didn't mean it as API, I meant it more as "let's try to describe what's currently in my head right now" kind of thing, something to discuss on. The basic concept of my idea is there.

I believe new Storage APIs should use async/await. Don't rely on the developer to do the threading.

I disagree on that, I would rather leave it to the developer to do async/await.

[shmuelie]

I disagree on that, I would rather leave it to the developer to do async/await.

Most developers are not threading experts and/or don't understand the difference between threading and IO async. And yes game developers need to be slightly more aware of it, they should be focused on the game itself, not how storage APIs should work. That is "our" job.

[tomspilman]

Since I wrote this issue I've ported games to all the current consoles... nothing suggested here so far will work on them all. Consoles all have curious ways to access the save data, read data, write data, and even specialized error cases.

I'm currently of the opinion that writing one API that wraps all of it might be a fools errand. It could result in a Frankenstein API that people on PCs will laugh at the complexity and absurdity of it all... not understanding it is that way because some consoles need it that way.

Unless you know the consoles and the technical requirements around their systems for storing user data... you really cannot design this API. There is so much more to consider and I just can't give you the details not being licensed developers for those consoles.

[KonajuGames]

I can agree with that, having used some of those console APIs. The XNA Storage APIs were designed to closely match the Xbox 360 API. Most platforms support IsolatedStorage in the .NET Framework. WinRT does not, but I have an IsolatedStorage wrapper available for it that works. For consoles, it is best to use the platform APIs. So we leave the Storage APIs removed and close this issue?

[tomspilman]

So we leave the Storage APIs removed

For sure.

and close this issue?

Maybe?

Unless myself or someone else with access to all the console SDKs gets involved we really can't even come to an API design. People proposing stuff to just be shot down is not a productive way to get to a design.