Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automagic dashboards #7048

Merged
merged 245 commits into from
Mar 15, 2018
Merged

Automagic dashboards #7048

merged 245 commits into from
Mar 15, 2018

Conversation

sbelak
Copy link
Contributor

@sbelak sbelak commented Mar 1, 2018

This supersedes #6455, #6840, #6853, #6887, #6786, and #6978.

Automagic filters are no longer exposed via api, but only used internally. Added the option of explicitly defining which filters to have in dashboard heuristics (dashboard_filter field).

Things that need to be fixed before merging

  • Wire up Metabot section of the setup flow
  • Fix occasional empty dashboard titles in the right hand pane
  • Fix Metric dashboard generation
  • Segment dashboard generation
  • Comment out or remove "See something else"
  • Field dashboard generation
  • Question dashboard genreation
  • Ad Hoc question dashboard generation

["dimension" "Foo"]
["DIMENSION" "Foo"]
42
[:baz :bar]]))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you just make these all separate tests? Why combine them all into one test?

(expect
[true
false]
(map ga-dimension? ["ga:foo" "foo"]))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not really sure what's going on in this namespace -- what's the reasoning behind trying to combine a bunch of tests that anywhere else in the codebase would be separate into one single test? I just don't understand why

[{:foo 2}
{:foo {:bar 3}}]
[(update-in-when {:foo 2} [:foo :bar] inc)
(update-in-when {:foo {:bar 2}} [:foo :bar] inc)])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

multiple test combined into one :(

(s/defn ^:private save-field-updates!
(def ^:private FieldOrTableInstance (s/either i/FieldInstance i/TableInstance))

(s/defn ^:private save-model-updates!
"Save the updates in UPDATED-FIELD."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're going to change the function make sure you change the docstring

(log/debug (format "Based on classification, updating these values of %s: %s"
(sync-util/name-for-logging original-field)
(sync-util/name-for-logging original-model)
values-to-set))
;; Check that we're not trying to set anything that we're not allowed to
(doseq [k (keys values-to-set)]
(when-not (contains? values-that-can-be-set k)
(throw (Exception. (format "Classifiers are not allowed to set the value of %s." k)))))
;; cool, now we should be ok to update the Field
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sure you change the comments too

[#"vendor" :entity/CompanyTable]])

(s/defn ^:always-validate infer-entity-type :- i/TableInstance
[table :- i/TableInstance]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/defn generates basic docstrings which is why the linter didn't complain but it would still be nice to have a brief blurb explaining what exactly is going on here

@camsaul
Copy link
Member

camsaul commented Mar 14, 2018

This looks good to me except for:

  • The docstring and comment that need to be updated I mentioned in the comments
  • I don't really understand the logic behind trying to combine 5 separate tests into one test all over the place

Copy link
Contributor

@senior senior left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just made a quick pass over it and found some things. I think they are things we can fixup up after merge but before releasing.

@@ -44,6 +44,16 @@ const FieldSidebar = ({
icon="document"
name={t`Details`}
/>

{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason to keep this around?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left it commented out while we wait on the endpoint being done.

:visualization_settings {}
:id (gensym)}))))))

(def ^:private ^Long title-height 2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^:const would be good here

nil))

(defmethod automagic-analysis (type Field)
[field]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A comment around why this is empty would be good, maybe a nil too if that's what is intended

([context bindings filters metrics dimensions limit order_by]
(walk/postwalk
(fn [subform]
(if (rules/dimension-form? subform)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where are you expecting to find dimension-forms? Seems like it will be traversing a lot of stuff that couldn't possibly be a dimension form (i.e. :type and :query and [:type :query])

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anywhere in :query so it's just 2 keys, :type and :database which doesn't fell like that much.

true
false
false]
(map (comp boolean #'rules/dimension-form?) [[:dimension "Foo"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is no longer private

:fk_target_field_id [:not= nil]
:table_id from-table)
(filter (comp #{(:table_id to-field)} :table_id Field :fk_target_field_id))
(map :id)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

u/get-id

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm intentionally using :id as it will always be an object. Unless we have a convention that we always prefer u/get-id

(let [filter-id (-> candidate hash str)
dashcards (:ordered_cards dashboard)
dashcards-new (map #(add-filter % filter-id candidate) dashcards)]
(cond-> dashboard
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dashboard -*> dashcards -*> series -*> fk fields and each fk field will cause a separate Fields query to fire. Any idea how many we're talking here? We've been battling n+1 issues lately

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It sucks. I'll rework it so it's one query per filter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

(defn- filter-type
[{:keys [base_type special_type]}]
(cond
(isa? base_type :type/DateTime) "date/all-options"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar question as above, no unix timestamps?

(def ^Long default-card-width
"Default card width."
6)
(def ^Long default-card-height
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^:const on these three


(defn- create-collection!
[title color description]
(when api/*is-superuser?*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the intended behavior of non superusers here? Does everything have to deal with possible nil values from this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Permissions aren't really thought out yet. But in this case it works ok, as :collection_id = nil is the default anyway.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I'd characterize users spewing cards into the everything questions space as working ok ;)

@salsakran salsakran changed the title Automagic dashboards [WIP] Automagic dashboards Mar 14, 2018
@salsakran salsakran merged commit 2a185a0 into master Mar 15, 2018
@salsakran salsakran deleted the automagic-dashboards-stage1 branch March 15, 2018 00:41
@salsakran salsakran added this to the 0.29 milestone Mar 15, 2018
@sbelak sbelak mentioned this pull request Mar 15, 2018
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants