Update Okta Module to use Models #1243

csanders-git · 2023-09-19T19:34:10Z

No description provided.

achantavy

Thanks for taking on the data model refactor and using the new Okta async APIs (and getting rid of a long-standing "miss last page" bug).

I gave this a first pass and my main blockers are:

Getting rid of awssaml - we need to keep this.
Okta group role transform efficiency: the list of group roles can be huge so there is a lot of wasted work here. Let's come up with a faster and cleaner way of doing this.

Since this is a big PR, we might need a few rounds. In future for faster merging it'd be better to make refactors true refactors with no new added functionality.

achantavy · 2023-10-19T17:54:41Z

setup.py

@@ -1,31 +1,31 @@
 from setuptools import find_packages
 from setuptools import setup

-__version__ = '0.81.0'
+__version__ = "0.81.0"


Let's not include these changes since it's not directly related to this PR.

achantavy · 2023-10-19T17:56:39Z

cartography/models/okta/user.py

+    id: PropertyRef = PropertyRef("id")
+    lastupdated: PropertyRef = PropertyRef("lastupdated", set_in_kwargs=True)
+    login: PropertyRef = PropertyRef("login")
+    email: PropertyRef = PropertyRef("email")


Suggested change

email: PropertyRef = PropertyRef("email")

email: PropertyRef = PropertyRef("email", extra_index=True)

achantavy · 2023-10-19T17:58:14Z

cartography/models/okta/user.py

@@ -0,0 +1,212 @@
+from dataclasses import dataclass


Let's look at

cartography/cartography/data/indexes.cypher

Lines 217 to 232 in 2fa2cb0

CREATE INDEX IF NOT EXISTS FOR (n:OktaOrganization) ON (n.id);

CREATE INDEX IF NOT EXISTS FOR (n:OktaOrganization) ON (n.lastupdated);

CREATE INDEX IF NOT EXISTS FOR (n:OktaUser) ON (n.id);

CREATE INDEX IF NOT EXISTS FOR (n:OktaUser) ON (n.email);

CREATE INDEX IF NOT EXISTS FOR (n:OktaUser) ON (n.lastupdated);

CREATE INDEX IF NOT EXISTS FOR (n:OktaGroup) ON (n.id);

CREATE INDEX IF NOT EXISTS FOR (n:OktaGroup) ON (n.name);

CREATE INDEX IF NOT EXISTS FOR (n:OktaGroup) ON (n.lastupdated);

CREATE INDEX IF NOT EXISTS FOR (n:OktaApplication) ON (n.id);

CREATE INDEX IF NOT EXISTS FOR (n:OktaApplication) ON (n.lastupdated);

CREATE INDEX IF NOT EXISTS FOR (n:OktaUserFactor) ON (n.id);

CREATE INDEX IF NOT EXISTS FOR (n:OktaUserFactor) ON (n.lastupdated);

CREATE INDEX IF NOT EXISTS FOR (n:OktaTrustedOrigin) ON (n.id);

CREATE INDEX IF NOT EXISTS FOR (n:OktaTrustedOrigin) ON (n.lastupdated);

CREATE INDEX IF NOT EXISTS FOR (n:OktaAdministrationRole) ON (n.id);

CREATE INDEX IF NOT EXISTS FOR (n:OktaAdministrationRole) ON (n.lastupdated);

and add extra_index=True to each PropertyRef where needed, and then we can remove the line from indexes.cypher.

achantavy · 2023-10-19T18:01:53Z

cartography/models/okta/application.py

+class OktaApplicationNodeProperties(CartographyNodeProperties):
+    id: PropertyRef = PropertyRef("id")
+    lastupdated: PropertyRef = PropertyRef("lastupdated", set_in_kwargs=True)
+    # TODO: Find out when this is used


Nit: Can we remove the TODO here and replace it with a GitHub issue instead?

achantavy · 2023-10-19T18:02:54Z

cartography/models/okta/application.py

+    last_updated: PropertyRef = PropertyRef("last_updated")
+    licensing_seat_count: PropertyRef = PropertyRef("licensing_seat_count")
+    name: PropertyRef = PropertyRef("name")
+    # TODO: Figure out profile


Nit: TODOs to GH issues.

achantavy · 2023-10-19T18:20:57Z

cartography/intel/okta/groups.py

+) -> List[Dict[str, Any]]:
+    """
+    Convert a list of Okta group roles into a format for Neo4j
+    :param okta_group_roles: List of Okta group roles


The datatype is OktaGroup. Double checking but is this comment correct?

cartography/intel/okta/groups.py

achantavy · 2023-10-19T18:39:34Z

cartography/intel/okta/groups.py

+            match_user = {**group_props, "user_id": group_member.id}
+            transformed_groups.append(match_user)
+        # Check to see if this group has any matching group roles
+        # TODO: Converting this into a hashmap would make perform better


This feels like a lot of wasted work since the group-role list can be very big. Let's take some time to think about restructuring.

achantavy · 2023-10-19T18:39:55Z

cartography/intel/okta/authenticators.py

+    )
+
+
+# TODO: Authenticator's enrolled per user


Let's clean this up

achantavy · 2023-10-19T18:40:58Z

cartography/intel/okta/__init__.py

+        okta_client, neo4j_session, common_job_parameters
+    )
+
+    # TODO: Deprecate this while we determine a method of making it more generic


Blocker: This is still needed.

Update Okta Module to use Models

8aa8b1c

csanders-git requested a review from achantavy September 19, 2023 19:34

achantavy requested changes Oct 19, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Okta Module to use Models #1243

Update Okta Module to use Models #1243

csanders-git commented Sep 19, 2023

achantavy left a comment

achantavy Oct 19, 2023

achantavy Oct 19, 2023

achantavy Oct 19, 2023

achantavy Oct 19, 2023

achantavy Oct 19, 2023

achantavy Oct 19, 2023

achantavy Oct 19, 2023

achantavy Oct 19, 2023

achantavy Oct 19, 2023

	email: PropertyRef = PropertyRef("email")
	email: PropertyRef = PropertyRef("email", extra_index=True)

	CREATE INDEX IF NOT EXISTS FOR (n:OktaOrganization) ON (n.id);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaOrganization) ON (n.lastupdated);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaUser) ON (n.id);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaUser) ON (n.email);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaUser) ON (n.lastupdated);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaGroup) ON (n.id);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaGroup) ON (n.name);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaGroup) ON (n.lastupdated);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaApplication) ON (n.id);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaApplication) ON (n.lastupdated);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaUserFactor) ON (n.id);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaUserFactor) ON (n.lastupdated);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaTrustedOrigin) ON (n.id);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaTrustedOrigin) ON (n.lastupdated);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaAdministrationRole) ON (n.id);
	CREATE INDEX IF NOT EXISTS FOR (n:OktaAdministrationRole) ON (n.lastupdated);

Update Okta Module to use Models #1243

Are you sure you want to change the base?

Update Okta Module to use Models #1243

Conversation

csanders-git commented Sep 19, 2023

achantavy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment