-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
directory API -All argument not returning all #22
Comments
Focusing on the first paragraph first - to be clear, you have a primary and secondary domain set up, right - not just two domains under your control? If you go to the API Reference page and use the Try It! feature at the bottom, using "my_customer" as the customer ID, do you get the appropriate results there? What happens if you supply the actual customer ID for one of the domains? What happens if you use the customer ID for the domain in GShell, or use "my_customer" explicitly? You can find a customer ID like so: I can't reproduce the issue, I don't have multiple domains set up like that to test with; as such I'm not terribly familiar with how it works. I assume that each domain has a separate customer ID (what you see when you get a user from each domain), but you should be able to just call it with the customer ID for the primary domain, correct? From the list users documentation we have two parameters, only one of which can be filled out at a time - Customer and Domain. From the description of Customer:
And, as per the description of the Domain parameter:
In GShell, you can see that we set the Customer param to "my_customer" which is supposed to represent your user's customer ID without having to retrieve it. I just ran Regarding your second paragraph, it sounds like you have only authorized access to the first domain (the primary) so when you explicitly try to connect to the second domain, it's trying to authenticate against that domain separately, rather than using the authentication for the primary domain. When you call I do have a fault in my code, however, since I'm apparently just forcing everyone to use "my_customer". I'll need to set it up so that if you supply a domain it will use that instead of "my_customer", which apparently only includes the primary domain at this point. I'll make that change first. In the meantime, can you try the Try It! on the Users List API Reference page referenced above, using my_customer as well as the customer IDs for your domains to see what the behavior is? Also, try using the Get-GAUser or Group cmdlets with explicitly providing the Customer param as either "my_customer" or the customer ID and see if the behavior matches what you're seeing on the Try It! web thingy. |
Yes, that is correct. I have about 10 domains total.
Thanks for the extra explanation on customer/domain with user list, I must have missed that.
So I ended up testing and gshell returns all of the domains when I Specify -customer with my actual id as well as with "my_customer". I do see Where you mention using my_customer as default, but when I run
I believe you are correct with this, however gshell isn't working when calling secondary domains, and it is working on the Test directory API. This may have to do with the issue you just fixed with not writing to config whenever the config has already been written, however I don't think that's the case. Actually I just tried something else and will stucture it below:
When running any command that is using any of the extra domains, gshell will request authentication(which obviously does not write to the oauth2 file), then successfully completes(only when I specify customer on every command I try, otherwise the command completes for the primary domain. If it doesn't write to the oauth2 file, it would make sense that it requests for permission every time, although it requests for the same permissions that have been given to the primary domain. My theory is that I don't need to re authenticate at all for any other secondary domain(this is correct, I just tested with the reference test site), as there is no way to give specific access to only certain secondary domains via the api, it simply isn't possible. When I authenticate with a secondary domain I get this message Unlike gshell, directory API test is only created to work with 1 user, 1 authentication, so when I type a new domain that doesn't exist, it simply returns not found. Using the theoretical situation where a gshell user has 3 different google apps PRIMARY domains and accounts authenticated to the directory API, each with 10 secondary domains, gshell would not handle this very well. With my(extremely limited) understanding, the best way to fix the issue might possibly be: Assume Every domain used in any command is an addition to the primary domain. This would work fine for unlimited amounts of secondary domains, but would start breaking if there were multiple primary domains authed with the application. It seems you are using the Domain flag to check authentication(and authenticate once again if it isn't found in your oauth file), and any command that isn't ran with that is assumed to be the default user/domain of gshell. I think a user can only have a single Google Apps account(with up to 600 domains on this account, past this would require a new google apps account, thus needing to re authenticate a new user), so maybe gshell can track authentications on a per user/customer basis instead of per domain. Sorry if anything above was hard to read, I got a bit overwhelmed with it all. Another thing to note is that it seems you added the Domain parameter to every command, not just ones that have it by default, as a way to track authentications correctly. This doesn't work as intended, and may be better to use either a custom gshell ID for each authentication, using the customer ID, or a new -primarydomain flag. Finally for gmail api(and maybe some others(Drive for example) domain administrators do not have access to other gmail inboxes, the best way to get around this is to add an option to use a service account instead of the user. You can see how this application handles it - https://github.com/jay0lee/got-your-back/wiki#--service-account . This would obviously be a lot more work and require a rewrite of some things. I will do some more research and see if I can contribute. Since gshell is designed to work with multiple users it complicates things compared to others that would simply have a new instance/application each time it is ran. Thanks again for all the help, you're the best once again! |
Sorry again for the wait!
It's supposed to say 'no domain provided' if you run a first GShell cmdlet without having a default domain stored, and you didn't supply a domain via a Domain parameter. If you supplied a Domain param and you're getting that, let's keep that in mind and circle back to that after all else is resolved.
The thing is, GShell is directly based on those same APIs that are only meant to work with 1 user and 1 authentication. I just wrapped it in a bunch of piping that helps handle that. The end result is close to what you were saying:
With the Google APIs, your authentication tends to be based on the user, for that one single API. GShell, being a wrapper for a collection of APIs, already has to juggle an authentication for each one of those APIs. Based on what you're saying, consider this - assume you have one primary domain with one sub-domain to start with. You connect to the primary domain with admin1@primary.com and GShell saves the authentication info for that connection. This is the default connection for GShell, so any further calls where you don't specify the domain connects to this with the auth credentials for the admin1 account. This also seems to mean that you have access to the sub-domains as well, because your admin1 account has access to both its domain and the sub-domain, but in order to call information for all of your domains including the sub-domain, you have to specify my_customer as you saw. If you wanted to connect just to the sub-domain, you would have to create a new connection specifying that secondary domain, and using an admin account from that domain as well. This would be treated as a separate connection (I assume, unless being a sub-domain somehow prevents this in a way I'm not aware of). Now that you have two domains authenticated, each domain has it's own default user (the one you authenticated with), so you don't need to specify that. But, you DO need to specify which domain's authentication and default user you want to leverage. That's why every Cmdlet has a -Domain param. However, it seems that this Domain param, which gets populated for every GShell Cmdlet (in order to retrieve the appropriate authentication information for given domain), is setting the Domain parameter that is intended for the API call before sending it over to Google. Due to a recent update I made any directory Cmdlet with a -Customer parameter sets that Param to 'my_customer' by default, to remove the need for the user to specify either a Domain or a Customer. Buuuut by doing that the idiot developer, whoever that moron is, didn't realize that it prevented us from being able to specify a domain. Oops. I'll need to include a separate parameter for these types of cmlets where the GShell -Domain param would otherwise conflict. Rather than totally screwing up backward compatability for the sake of keeping the name, what if I called it -CustomerDomain, would that be clear? So you'd have to choose between Customer and CustomerDomain? If neither or both are specified, it'd default to Customer. I won't be splitting it out in to two parameter sets for other reasons I'd be happy to discuss another time, but if you have another suggestions on that naming let me know? Currently, I'm still not sure why it works when you manually specify the 'my_customer' though, and not when it's assumed - as far as I can tell in my testing, those should functionally be the same as of 0.9.6.0 - is that what you're using? I'll be updating the release to 0.9.7.0 to include the fix for the OAuth writing tonight after I finish this, so if nothing else grab that and give it a try. Worst case scenario, temporarily close PowerShell, rename the saved OAuth file (you can always go back and rename it again later) and start from scratch. Once we talk a bit more about the -CustomerDomain param, we can slate that for the 0.9.8.0 bugfix. My goal is to work all these kinks out before I launch the first stable version of 1.x, so anything until then is WIP.
Got you covered! Check it out, that was one of the major 0.9.x features. That said, you can check the discussion group for an example of how it can be used in action to delete emails from all users in your domain (at least, one domain haha, who knows what would happen in your case!)
Are you kidding me? It's rare that I don't feel so bad about being as verbose as I tend to be, it's nice to find a bird of a feather! I think at this point all I need is for your input on the bit I put in bold on the new parameter, and I'll start working on that. Would you be willing to test it out before I post it as a new bugfix? I'd rather be sure it's working before having others install it; I could temporarily just host it in google drive and post a link here, killing it off once it works out? I'm open to other ideas, too, but wouldn't want it hanging around indefinitely. |
This is the issue though, It's not possible to connect to the secondary domain and it's not possible to give someone administrator rights to the secondary domain. All extra domains are simply housed under the same umbrella, and cannot be managed differently via the API. The Directory Api -domain is simply a result refiner, nothing more.
Not wanting to screw up backwards compatibility is 100% understandable, however It doesn't make a ton of sense to keep using domain as the switch. As we found out above, secondary domains are not controllable by the API, and a single account can only authenticate with 1 domain, so on the grand scheme of things it makes more sense to me to organize on a per user basis. Imagine the current oauth scheme:
Under domain.com, the default domain will always be the same, the domain will always be the same, after the first authentication the only thing that would change are the token and scopes. If Someone authenticates another domain, they are really just authenticating another user, the API doesn't care what domain they have permissions of, or what domain the account is tied to(or any secondary domains that are also under control). So with the current system if I wanted to authenticate a second user, being user.name@secondarydomain.com, gshell would recreate everything in that file, and now call that user with -domain secondarydomain.com, even though it has the exact same permissions of the primary domain, since the api does not work with secondary domains. It seems as though the more and more complicated the user setup gets, the easier it would be to manage authentication on a per user basis. Then a gshell user could have 5 users from the same GSuite domain, then also having 3 from another GSuite domain, And finally 2 more users from a third GSuite domain. Let's say all 10 users are using different secondary domains, which would cause gshell to be used with 10 different -Domain options, when there is really only functionally 3. If we went even deeper and had some users on the primary domain and some on the secondary of each Primary GSuite domain, it makes managing on a per domain basis even more difficult. Oauth file with user authentication:
If you still feel that using the -Domain switch is the best way to go for the future, I'm not going to complain too much, as I would understand.
If you stick with -Domain for organizing authentications and selecting authentications, then this would work fine. I like the defaulting of customer being my_customer if nothing is supplied, and defaulting to Customer only if both CustomerDomain and customer are supplied.
I am on 0.9.5.0 currently. I will upgrade to 0.9.7.0(or 0.9.6.0) when I get the chance later today or tomorrow to test if this is still happening.
😃 I see the release is out, thanks!
Awesome! I must have missed it in documentation. 😃
Happy to test anything that you can throw my way! |
I think the main takeaway for me here is this:
That is, clearly, not how I had understood it when I had originally researched it. That's my bad, sorry! I really wish I had environments where I could test these things cough.HeyGoogleHelpAGuyOut.cough, but half the time I'm flying blind because we simply don't have things set up that way, or we don't use a feature - which is why I'm grateful for the help I get from you! Rather than all the quoting I'll say that I understand what you're saying. The JSON structure was more about a top-down approach to storing the information. As you said it may as well be a different user for each API, as far as the APIs care - it's GShell that is grouping them together. I get what you're saying about having multiple users and how that would affect the -Domain param. Under current circumstances, it would essentially require changing the default user any time you wanted to change users within the same domain. What I'm not sure of is this: is this a reasonable scenario to plan for? From what you were saying, if I'm understanding it correctly, we can't use the APIs or GShell with a user account from a secondary domain. Or perhaps, if we use some admin from that domain, it acts like it's from the primary domain? Either way, the ultimate result seems to be that for an umbrella scenario, your best bet is to use an admin account from the primary domain, and then filter results (where applicable) to those from the secondary domain. If so, why would I - using an admin account from the primary domain - want to use different user accounts at all, if they can all be managed by one single account from the primary domain? If I'm working in the primary domain, I'd use that primary admin account. If I'm working in the secondary domain, it sounds like I'd still need to use that primary admin account. If I'm working in both or more, same primary account. Only when switching to a completely separate primary domain would it matter that I have a second account, right? I feel like I'm missing why someone would be juggling multiple users aside from needing to use a different admin account to manage the admin account you normally use (like, in CPanel you can't make changes to your own admin account in some respects). What am I not seeing? |
Honestly explaining the way it works seems a bit strange, I guess google just decided that domain would be searched often enough so they threw it into the main parameters, instead of having to use query.
I can imagine a lot of niche situations that this would be wanted.
Obviously all four are pretty far fetched. I am actually doing number 3, so it's a real use case for me. I could obviously get around any of those situations rather easily, but my main point was designing gshell to be compatible with every single possible use case, no matter how far fetched(or seemingly illogical) it may be. Again, I could see wanting to keep backwards compatibility. I'm not sure what would be the best way to implement the new system if you are going to. My best idea would be to check if the oauth file is organized in the domain format or not. If it is, enable the legacy system, if it isn't, use whatever new organization you find best. Throw in a warning saying -domain will be depreciated by a certain version, and have a method to switch over. The amount of people using the -domain switch are probably rather small anyway, most likely 95% of people will be using gshell with just one account. I would personally just get rid of the older method, as it should be quite simple for anyone to adapt scripts to the new way. Finally is how service accounts(and gmail api) mix into everything. Currently you have service accounts tied to domains, with it being triggered by -TargetUserEmail. The thing I find strange about this(assuming you have it working the way I expect), is that after supplying -TargetUserEmail, I don't see anything that -UserID is being used for, My assumption is that it's simply thrown out and uses TargetUserEmail as the UserID. This is strange because if you have account delegation you would change the -UserID parameter to who you are targeting, which is fine, just not how I would expect it to work. Maybe a better option is to use a Boolean to enable or disable usage of the service account. This changes the code from: Service Account Would switch to(using non domain authentication): Service Account This would make it both consistent with any Delegation/self/service account commands, it would also make it consistent with the Gmail API itself. This isn't really that important, just something I noticed. If you are annoyed by my recommendation for change but lack of contribution just let me know and I can dive into the code a bit more. If you aren't annoyed yet then I am impressed! 😛 |
Good news! 0.9.7 is giving all users without supplying -customer! I also was able to update my oauth file and have it saved! It looks like Get-GAGroup is missing the customer parameter so it may need to be added to be able to get all groups. Also I finally get it. Domain is both being used as a switch for authentication, and as a way to allow supplying the user without domain name by combining the two(and using the default domain). I have only ever supplied the full email as that's what I was used to, and didn't realize you had set it up to work that way. That does make a ton of sense and is actually quite a smart way to do things if google apps domains worked the way you had thought. The reason I never ran into the issue before was because I didn't supply domain name until it was required(Get-GAUser -list). Damn, this makes the whole problem a lot more complicated. Nope, just went straight back to more confused. The second parameter is supposed to be customfieldmask? What is it actually being supplied as(I originally thought it was -domain)? I'm just going to throw as many examples as I can below and hopefully you can help. Here is a list of things that do and don't work.
I don't really understand why this is happening. From all that testing I assumed the second parameter used as |
Last comment for the day - promise! 😄
on 0.9.7 it is still showing no domain provided when I do have a default domain and I did specify a new domain. |
Trying to keep this on track, I've split your last point about the 'no domain provided' in to a separate ticket here - thanks for catching that. Regarding your comments on the service account and the -Domain switch, I'd like to keep that conversation going, but I'm not sure that here is the place. Would you be willing to start a new thread in the Discussion or Dev group (but honestly nobody uses the dev group yet, so if you used the former I'd not complain too loudly) and just reference your post here as where we left off? About the missing parameters for Get-GAGroup, I've created a new ticket for that here. For the Domain parameter question, I've split that out in to a ticket here. Again, if you'd prefer to use the dev group for that I wouldn't mind at all, but so far I've only run in to you here on github so I figured I'd stick with that until you make a move to change it. :) Back on track on the my_customer issue...
...from what you're saying, it sounds like that specifically has been resolved, can you verify? If so I'll close this out and we'll move the remaining discussion to the appropriate spots. |
Also, regarding this:
I'd love to hear more about that, if you'd be willing. I don't often get to hear who is using GShell and how, and this sounds pretty cool! Feel free to reach out to me separately if so. Have you considered forking GShell and pulling out some of the .Net methods it has and using those separately, as opposed to using an extra layer of PowerShell? It's been a while since I've tried it, but if you're interested I'd be happy to try and help. |
Correct!(Once The group -all works as well, but that is in another issue so no worries).
Absolutely! As you said we can go ahead and close this ticket and move it elsewhere. I'll keep this in mind |
Thanks! |
This is happening for
Get-GAUser -All
andGet-GAGroup -All
. Basically when using -All it only returns users or groups ending in @primarydomain.com. From the directory api documentation it seems like it should return all groups of an account if no domain or account are specified, but It is not returning anything from @secondarydomain.com. This makes it so I cannot fully script things as I can't find a way to actually get all users or groups.I can
Get-GAGroup test@secondarydomain.com
successfully so I know I have permissions. If I runGet-GAGroup -Domain secondarydomain.com -All
It invokes the scope manager for directory API even though I already have full permissions in my oauth file, It then opens in my browser, Received verification code. You may now close this window. It then gives me the exact same result if I didn't specify -domain, by giving only results of primarydomain.com, and nothing else. In our environment we have many domains being used currently, so getting only primarydomain.com is troublesome.Let me know if you are able to replicate the issue or if you need any more information.
Thanks!
The text was updated successfully, but these errors were encountered: