End-user/consumer dashboard/profile

[brylie]

User story

As an API consumer,
I would like a dashboard/profile where I can manage my details
So that I can view my API usage and recover my key.

Planning

There is an API Consumer Dashboard planning document on the API Umbrella wiki.

Support this issue

Financial supporters:

[darylrobbins]

Is this something currently in the works or a longer term roadmap item? This is functionality that we require as well at the moment. Would you see this as part of api-umbrella-web or as a new web app?

[brylie]

I set up a Bountysource page to support this issue financially.

[darylrobbins]

I was more thinking of contributing rather than funding it. I just wanted to make sure that it wasn't already in progress and I also want to make sure however we implement it fits in with your vision of the overall architecture. Thanks.

[brylie]

Cool. I am not the main developer.

We were just talking in our organization about contributing to the project financially, so had already set up the Bountysource page. Your post reminded me to submit the link to this issue is all :-)

[darylrobbins]

Got it. Based on your bounties, it sounds like we have similar ideas for how to use the platform.

[ccsr]

I have backed this with Bounty on https://www.bountysource.com/issues/7554569-end-user-consumer-dashboard-profile

[darylrobbins]

@GUI We are going to need to develop something within the next couple months to provide at least a basic end user portal. I would prefer if we were able to contribute back what we developed to the project, so I was hoping to get your view on how you saw this fitting in. I am seeing this as an extension to the existing web project.

The scope we have in mind is:

• registration process, which may or may not require approval for access to the API (dependent on config)
• logins for registered users (likely same login mechanism as admins but consumer role)
• one or more API keys tied to a registered user, with the ability to fully see the keys owned
• optional closed-access documentation site
• usage information for owned API keys (future)

p.s. We're preparing some feedback on your proposed new architecture. I only stumbled upon the issue tonight. Maybe a link from the project readme could garner more attention to the proposal? Thanks!

[GUI]

@darylrobbins Sorry for the delay in following up. We'd be happy to accept contributions regarding user registration or profile functionality. I have some loose ideas for now I've personally thought about implementing this feature, but everything you outlined sounds good and in a similar direction to what I've been thinking. There are probably some technical considerations with some of these items (like how a user with multiple keys is handled by the gatekeeper component, or how closed documentation works with our current model of treating documentation as an external static website repo), but this all sounds like the right direction.

I'd say our only real requirement on our end is that we do have to maintain our current signup approach for api.data.gov (which is what currently happens in API Umbrella, where we don't verify e-mail addresses and we don't have much of a concept of an account). But I think we are potentially interested in having more of a real account option either as an alternative or in tandem to our current approach. So I think we somehow need to support both signup models with API Umbrella, which I realize adds unfortunate complexity. However, if you'd prefer to fork and focus on the new functionality right now, I could try to lend a hand in making sure we can support both signup models at a later date.

So let me know how I can help or be of assistance. If you're comfortable with the code-base, you're certainly more than welcome to dive in and get started. But I also realize our documentation isn't great in areas, so I can certainly try to answer any questions you might have here. Alternatively, if it would be helpful, we could hop on a call sometime to chat through some of this in more detail.

Thanks!

[darylrobbins]

@GUI Thanks for all your support, Nick.

In the end solution, I'm seeing three basic access models:

1. Fully open - sign up with an email/password and get an API key for full access. Log back in with your profile to get your API key again. (hopefully meeting your requirements)
2. Semi open - sign up with an email/password and purpose of request but you can't do or see anything until the admin approves your application
   2a) More semi open - unapproved accounts have basic sandbox access to the environment. Hopefully, we can model this as a special case of 2.
3. Fully closed - accounts can only be created by admins; once they are created, login information is communicated to the registered email

Multiple API keys per account is definitely where things start to get interesting. Can a user split their entitlements into multiple keys or does an admin have to do it? Do they all have the same account id? What does each key represent? Different environments or different web properties/sources? At present, API umbrella doesn't really seem to support using common infrastructure for multiple environments (i.e. production, test).

Preferably, we would like to offer some degree of customizability of the admin/developer portal without the need for building your own package.

Closed documentation definitely adds some challenges to the process.

As you hint at, in an ideal world, we could deliver something as flexible as I just described. In reality, we may need to start somewhere and fork accordingly, iterating to bring the two together.

My team and I are definitely willing to invest some effort to build something that can be folded back into the project, but there are always going to be conflicting practical limitations.

At present, in our independent project, we have closed documentation generated from a RAML file and a fancy display of a single API key per account.

Let's look at having a call late next week or early the following week to go over architecture and high-level design of how we integrate a developer portal into the current code base. I have a development environment partially working, so I may ping you on a couple issues relating to getting that up and going. Vagrant is starting the chef provisioning process but it's failing part way through right now. In terms of scheduling, I'm guessing you are also located in the Eastern Timezone?

[GUI]

@darylrobbins: Sounds good. If you run into development setup issues, feel free to file issues or reach out. And if you want to have a call to discuss any of this in more detail, you're welcome to get in touch. In terms of scheduling, I'm actually in the Mountain timezone (if you want to schedule stuff, you can contact me directly at nick.muerdter@nrel.gov)

Oh, and one little side tidbit:

At present, API umbrella doesn't really seem to support using common infrastructure for multiple environments (i.e. production, test).

I might be misunderstanding you, but I think we may support this. If you want to proxy your development/staging/production APIs through a single API Umbrella instance, that is possible. You can do so, by having API Umbrella listen on multiple hosts or different paths to distinguish and route your staging vs production API traffic differently. However, if you mean something else, let me know.

[darylrobbins]

Yes, that's exactly what I had in mind. So, I take it if I setup different front-end hosts for each environment, I can route the traffic basic on domain name? That makes sense but it wasn't immediately obvious from the UI. With the emphasis around rewriting the front-end hostname, I wouldn't have jumped to the conclusion that it was also a constraint on which requests get sent there.

It's looking like we'll be in a position to start doing some work on the user dashboard late this week or early next week.

[brylie]

Hey all, just threading together a few issues that may be related to dashboard development

• RAML Support (RAML Support? #47)
• API Docs Section (API Docs section #55)
• Available APIs listing (Available APIs listing #53)
• Request API Key (Request API key #83)
• Dashboard for API Consumer analytics (Dashboard for API Consumer analytics #58, Public API key usage page #39)
• "My Profile" page ("My Profile" page #52)

[darylrobbins]

Unfortunately, it's looking like we aren't going to be able to work on the end-user dashboard as soon as we had hoped. Our web developer has accepted a position at another company, so I no longer have the resources available to develop this functionality until we are able to backfill and ramp up the new developer.

[brylie]

Hey all, what are some ideas you would like to see as components on the end-user (consumer) API Dashboard? We are planning to mock-up the API Dashboard, so your thoughts are really important! :-)

Also, lets try to keep the topic focused to Dashboard features, and add notes to other features on corresponding issues.

[ccsr]

We need to also keep in mind that there are some paid APIs and the details of the payment also need to be displayed on the dashboard. For example my paid is been accessed 100 times today so I earn $X.

[brylie]

I added an API Consumer Dashboard planning document to the API Umbrella wiki. @GUI and @darylrobbins will you please help describe the desired functionality?

[55]

Here is my first attempt, these are just wireframes of a consumer dashboard page.

[1] Sortable/resizable boxes with different type/view of statistics on selected API. By default it shows statistics of first API on user's list.
[2] By clicking on button a panel slides down with available analytical tools, [3] which by drag&drop user can add to the page.
[4] By clicking on button modal window opens with the user’s API keys list. [5] User can copy selected key or [6] see a statistics on a selected API.
[7] Shows brief info of a user (e.g., Settings, Log out, user’s name, avatar, edit profile) [8] by clicking on a Settings/Edit Profile opens a modal with all settings/options/fields user can modify.

[brylie]

@GUI can you weigh in on this design? @darylrobbins do you have any design documents that we could use as reference for this task? We would like to contribute to the development efforts for this functionality.

[brylie]

Please help us out by contributing to the API dashboard design document

[GUI]

Apologies for the delay in responding. From a design perspective, I unfortunately don't have a ton to contribute, but I think these wireframes look good.

From my own perspective, the main thing I'd like to see is that the dashboard be API driven so that it's somewhat easy to build other interfaces or embed the dashboard on multiple sites. However, this is admittedly just because I'm thinking about our api.data.gov use-case where we allow individual agencies to embed and tailor the tools around their own existing developer portal sites. But that's just me and our rather specific use-case, so I wouldn't feel like you have to worry about implementing it in that way for your own implementation.

And if you are planning to dive into the development, definitely feel free to give me a shout if you'd ever like to chat about this in more detail or if I can be of any assistance in getting oriented in the code-base.

Thanks!

[brylie]

We are currently working on a spec, with the expectation to develop this feature. That said, we definitely need guidance writing the spec, so that it aligns with API Umbrella trajectory. To this end, I created the design document to help capture the specification for the dashboard.

[darylrobbins]

Sorry for the delay. Unfortunately, we weren't able to get to a design document before our web developer moved to a new gig. This has left us quite understaffed, so I have been quite busy on other part of our architecture.

Here are some of the main factors that we identified from memory:

Signup Process

1. Open - anyone can signup and start using the API
2. Closed - only admins can create users
3. Approval - anyone can signup but they can't do anything until they are approved by an admin

Closed Documentation

Some installs may want to offer closed documentation, only to registered users (i.e. us).

API Tokens

Break the link between API User and API Token. It would make sense for a single user to have multiple API Tokens and for multiple API Users to share a single token (i.e. an entire team).

A user may want different tokens for each of their websites and/or for different environments (production/staging).

The question then becomes, how does a user get a new API key? Do we keep things simple and they get one when they registered (or are approved) and then an admin can assign more?

At least the administrator should be able to regenerate a key (and possibly a user).

Users should be able to see their API keys as well in the console.

Statistics

It would be nice if the user could have access to the same statistics as the admin, but constrained to their API keys.

Dashboard

It would be nice if there was a dashboard or an help page or something as the home page when one logs into the dashboard -- instead of the current blank page.

That's all I remember at the moment, but I will be sure to add anything else if I think of it. Thanks!

[brylie]

Hey all, we have been working for several months on our 'api-umbrella-dashboard' project, and have finally made a beta release. If you will, please help us test the beta project, and feel free to add feature requests, bug reports, etc. for anything needed.