Skip to content

Commit

Permalink
Updates to Cloud formation scripts for administrators (#88)
Browse files Browse the repository at this point in the history 
* Fix wrong location in comment

* Merge dev branch into master for 0.2 Beta and Holistic Testing (#24)

* Updated publishing samples to reflect changes made to source code

* Updated Publihing SDs, shp, csv samples to reflect latest API changes

* Publisher sample updates - added samples for web maps and web scene (#3)

* Updated notebook title to match filename

* Added a new sample to showcase publishing web maps and web scenes

* Fixed credentials

* Updated sample titled "Using and updating GIS content" (#4)

* Updated notebook title to match filename

* Added a new sample to showcase publishing web maps and web scenes

* Fixed credentials

* Updated the sample with additional examples

* Fixed credentials

* Added an example for line featuers. Added some text explaining what smart mapping is

* Additional widgets to minimize scrolling. Fixed geocoder usage following API update

* Updated geocoder usage to match API updates

* Updated the sample to match gp service update. Added a bit more text explaining the usage of gp tools (#12)

* Minor - updated sample to search for desired item to add instead of a fixed index (#13)

* GIS analysts - sample illustrating spatial analysis using ArcGIS Python API (#14)

* Initial commit

* Furnished the sample with spatial analysis examples

* Fixes for API changes. Added summary text and elaborated a few steps (#15)

* new sample to showcase big data anlaysis using NYC taxi example (#16)

* Updated hurricane tracks sample to match API changes (#17)

* Updated hurricane tracks sample to match API changes and added descriptive text

* updated credentials

* Updated sample to match API changes, added explanatory text (#19)

* clean up sample (#18)

* clean up sample (#20)

* Remove usage statistics notebook as the API isnt available

This requires work on the portal backend as well as it does not yet
support ASM.

* bug fixes and added descriptive text (#22)

* synced raster product sample in 05 Content Publishers folder to match that in 04 GIS Analysts and Data Scientists (#21)

Thanks, @AtmaMani

* Updated Overview page - updated sample TOC (#23)

* doc updates

* apidoc update for 0.2 beta release

* updated install instructions

* Updated samples download link to point to v0.2-beta release (#25)

* fixed wrong import

* fixed execution count

* apache 2.0 license specified

* scripts will parse credentials as cmd line args

* ignore IDE files

* replacing \ with /. Added batch file to download, install miniconda, arcgis pkg and then call the script

* split cloud formation bat into two, other bug fixes

* piping all prints to log file, prints 0 on success else 1

* bug fixes

* silent install bug fixeS

* bug fixes

* uncomment executable lines. updated readme with screen print messages

* after publishing, share the webmaps to groups the user is a member of. when reassigning the webmap, store it in a new folder of naming convention: Lastname_webmaps

* scripts will parse credentials as cmd line args

* ignore IDE files

* replacing \ with /. Added batch file to download, install miniconda, arcgis pkg and then call the script

* split cloud formation bat into two, other bug fixes

* piping all prints to log file, prints 0 on success else 1

* bug fixes

* silent install bug fixeS

* bug fixes

* uncomment executable lines. updated readme with screen print messages

* after publishing, share the webmaps to groups the user is a member of. when reassigning the webmap, store it in a new folder of naming convention: Lastname_webmaps
  • Loading branch information
@AtmaMani @rohitgeo
AtmaMani authored and rohitgeo committed Mar 23, 2017
1 parent 96cb516 commit 6b4500e
Show file tree
Hide file tree
Showing 13 changed files with 922 additions and 209 deletions.
2 changes: 1 addition & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
guide/gis module/.ipynb_checkpoints/class gis.ContentManager-checkpoint.ipynb
.ipynb_checkpoints
.idea
.idea/
162 changes: 162 additions & 0 deletions samples/03_org_administrators/AdminCreatePortal/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# Introduction
This folder contains Python and batch scripts that will help populate a fresh portal
with users, groups and content for each of the user accounts. You can find the
list of users and groups in the corresponding `users.csv` and `groups.csv`
files.

## Steps
1. Run `cloud_formation_a.bat` which will download and install miniconda,
then install the `arcgis` package in a environment called `cloud_formation`
1. Run `cloud_formation_b.bat` which execute the Python scripts in sequence
and creates users, groups and content.
2. Optionally run `clean_up.bat` to erase all users, groups and content

# Sample outputs below

## Running the cloud formation batch files
Call the batch files one after another. These files return 0 upon success. The `cloud_formation_b.bat`
accepts 4 command line arguments. Pass the URL to the portal, admin username and password and the log file
to output the messages.

```
>> cloud_formation_a.bat
0
```

```
>> cloud_formation_b.bat https://ESRIwebgis.webgistesting.net/portal -u admin -p xxxxxx -l python_log.log
Creating new environment
Package plan for package removal in environment C:\Users\atma6951\miniconda3\envs\cloud_formation:
The following packages will be REMOVED:
arcgis: 1.0.1-py36_1 esri
bleach: 1.5.0-py36_0
colorama: 0.3.7-py36_0
...
Fetching package metadata ...........
Solving package specifications: .
Package plan for installation in environment C:\Users\atma6951\miniconda3\envs\cloud_formation:
The following NEW packages will be INSTALLED:
pip: 9.0.1-py36_1
python: 3.6.0-0
Installing arcgis package
Fetching package metadata .............
Solving package specifications: .
Package plan for installation in environment C:\Users\atma6951\miniconda3\envs\cloud_formation:
The following NEW packages will be INSTALLED:
arcgis: 1.0.1-py36_1 esri
bleach: 1.5.0-py36_0
...
Uninstalling prior versions of arcgis widget
- Validating: ok
0
0
```
# Running the Python scripts individually
If you choose to run the scripts individually instead of calling them using the batch files, you can do so
as shown below. All files accept command line parameters, to view the options, call them with `--help` flag.

## Running clean_up.py
`clean_up.py` accepts the credentials as command line args. Inquiring help
on the file returns the following:

```
E:\code>python cleanup.py --help
usage: cleanup.py [-h] [-u USER] [-p PASSWORD] url
positional arguments:
url Portal url of the form:
https://portalname.domain.com/webadaptor
optional arguments:
-h, --help show this help message and exit
-u USER, --user USER Administrator username
-p PASSWORD, --password PASSWORD
Administrator password
-l LOG_FILE --log Log file to record all process outputs
```

Running it with credentials returns the following
```
E:\code>python cleanup.py https://ESRIwebgis.webgistesting.net/portal -u admin -p xxxxxx -l python_log.log
=====================================================================
RUNNING CLEANUP
Deleting groups
---------------
Deleting Basemaps ## success
Deleting Central Services ## success
...
Deleting user content
---------------------
User : adams.powell #
Deleting : Adams Powell response locations # True |
Deleting : SC # True |
Deleting : SC # True | empty
User : admin # skipped
...
Deleting users
--------------
Deleting adams.powell ## success
Deleting allen.price ## success
Deleting anderson.bailey ## success
...
All clean
```

## Running create_groups.py
Similar to `clean_up.py`, `create_groups.py` also accepts command line args.
Running it with valid credentials prints the following output:

```
E:\code>python create_groups.py https://ESRIwebgis.webgistesting.net/portal -u admin -p xxxxx
CREATING GROUPS
Creating group: Basemaps ## success
Creating group: Central Services ## success
Creating group: Compliance ## success
Creating group: Customer Service, Finance, Billing and Accounting ## success
Creating group: Demographic Content ## success
```

### Running create_users.py
`create_users.py` will create users on the portal and add them to appropriate
groups specified in the `users.csv` file. Running it with valid credentials
reports below, truncated for brevity:

```
E:\code>python create_users.py https://ESRIwebgis.webgistesting.net/portal -u admin -p xxxxxxx
CREATING USER ACCOUNTS
Creating user: smith.collins ## success ##
Adding to groups: # Basemaps # Central Services #
Creating user: johnson.stewart ## success ##
Adding to groups: # Basemaps # Central Services #
Creating user: williams.sanchez ## success ##
Adding to groups: # Basemaps # Central Services #
...
```

### Running publish_content.py
This script will publish feature layers, web map with those layers and
assign it to each of the user. Running it will print the following output,
truncated for brevity:

```
E:\code>python publish_content.py https://ESRIwebgis.webgistesting.net/portal -u admin -p xxxxx
Publishing .\user_content\KS.csv # webmaps ## success. Assigning to: # smith.collins
Publishing .\user_content\NV.csv # webmaps ## success. Assigning to: # johnson.stewart
Publishing .\user_content\IN.csv # webmaps ## success. Assigning to: # williams.sanchez
Publishing .\user_content\NC.csv # webmaps ## success. Assigning to: # jones.morris
....
```
120 changes: 69 additions & 51 deletions samples/03_org_administrators/AdminCreatePortal/cleanup.py
View file
Original file line number Diff line number Diff line change
@@ -1,61 +1,79 @@
# Script to clean up the users, content and groups created for demo

from arcgis.gis import *
print("RUNNING CLEANUP")
print("---------------")
import argparse

gis = GIS("https://dev003327.esri.com/portal", "admin", "esri.agp")
try:
#region read cmd line args
parser = argparse.ArgumentParser()
parser.add_argument('url', help='Portal url of the form: https://portalname.domain.com/webadaptor')
parser.add_argument('-u','--user', help='Administrator username', default='admin')
parser.add_argument('-p','--password', help='Administrator password', default='x]984<ngb3!')
parser.add_argument('-l', '--log', help='Path to log file', default='python_process.log')

args = parser.parse_args()
#endregion

# Read the log file in append mode
log_file = open(args.log, 'a')

log_file.write("\n")
log_file.write("=====================================================================\n")
log_file.write("RUNNING CLEANUP\n")

gis = GIS(args.url, args.user, args.password)

# region remove groups
group_list = gis.groups.search("owner:" + args.user)
log_file.write("Deleting groups\n")
log_file.write("---------------\n")

# region remove groups
group_list = gis.groups.search("owner:admin")
print("")
print("Deleting groups")
print("---------------")
for group in group_list:
try:
log_file.write("\nDeleting " + group.title + " ## ")
group.delete()
log_file.write("success")
except Exception as group_del_ex:
log_file.write("Error deleting : " + str(group_del_ex))
# endregion

for group in group_list:
#region remove content for each user
log_file.write("\n\nDeleting user content\n")
log_file.write("---------------------\n")
user_list = gis.users.search("")
try:
print("Deleting ", group.title, end= " ## ")
group.delete()
print("success")
except Exception as group_del_ex:
print("Error deleting : " , str(group_del_ex))
# endregion

#region remove content for each user
print("")
print("Deleting user content")
print("---------------------")
user_list = gis.users.search("")
try:
for user in user_list:
log_file.write('\nUser : ' + user.username + " # ")
if user.fullName in ['Administrator', 'Esri', 'Esri Navigation']:
log_file.write('skipped')
continue

user_content = gis.content.search('owner:{0}'.format(user.username))
for item in user_content:
log_file.write('\nDeleting : '+ item.title + " # ")
delete_status = item.delete()
log_file.write(str(delete_status)+ " | ")
log_file.write('empty')

except Exception as content_del_ex:
log_file.write(str('content_del_ex'))
#endregion

# region remove users
user_list = gis.users.search()
log_file.write("\n\nDeleting users\n")
log_file.write("--------------\n")

for user in user_list:
print('User : ', user.username, end=" # ")
if user.fullName in ['Administrator', 'Esri', 'Esri Navigation']:
print('skipped')
if user.username == "admin" or user.username.startswith("esri_") or user.username.startswith("AVWORLD"):
continue
else:
log_file.write("\nDeleting " + user.username + " ## ")
user.delete()
log_file.write("success")
# endregion
log_file.write("\n All clean")

user_content = gis.content.search('owner:{0}'.format(user.username))
for item in user_content:
print('Deleting : ', item.title, end = " # ")
delete_status = item.delete()
print(str(delete_status), end = " | ")
print('empty')

except Exception as content_del_ex:
print(str('content_del_ex'))
#endregion

# region remove users
user_list = gis.users.search()
print("")
print("Deleting users")
print("--------------")

for user in user_list:
if user.username == "admin" or user.username.startswith("esri_") or user.username.startswith("AVWORLD"):
continue
else:
print("Deleting ", user.username, end = " ## ")
user.delete()
print("success")
# endregion
print("\n All clean")
print("0")
except:
print("1")

0 comments on commit 6b4500e

Please sign in to comment.