New Enterprise Search Kibana plugin #66922
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cee-chen
force-pushed
the
feature/enterprise-search-plugin
branch
2 times, most recently
from
e248aa7
to
a26773d
Compare
cee-chen
force-pushed
the
feature/enterprise-search-plugin
branch
from
a26773d
to
99f3912
Compare
cee-chen
force-pushed
the
feature/enterprise-search-plugin
branch
from
409b36a
to
d0279b0
Compare
cee-chen
force-pushed
the
feature/enterprise-search-plugin
branch
from
d0279b0
to
d18c40c
Compare
cee-chen
force-pushed
the
feature/enterprise-search-plugin
branch
from
4c927cd
to
1c4c990
Compare
|
@gammon @kobelb I believe we recently decided that we should not longer merge plugins that are disabled by default and/or have not passed a security review. Rationale being that we don't want users to be able to hurt themselves (eg. by exposing a security flaw) by turning this plugin on. Am I remembering correctly? And if so, should @constancecchen simply move this to a feature branch on our root repo until it's beta ready? |
|
@kovyrin is currently doing some work to make Enterprise Search run against 8.x, so it's likely we can remove the default disabled line shortly here in any case. The disabled line does seem to be making some functional test pass that "shouldn't" (were previously failing, I think a nav link issue that we're continuing to meet w/ @kobelb to discuss), so definitely makes sense not to merge in with it. I'll aim to remove the line here in a day or two once I'm confident I have ES and Kibana successfully and consistently both running against 8.x |
cee-chen
force-pushed
the
feature/enterprise-search-plugin
branch
3 times, most recently
from
da328d2
to
4bac2a5
Compare
|
Hey @constancecchen. I briefly ran through some of the code here on the design side. The big things you'll want to address before this goes up for review:
Yell over slack when you're ready for a review, otherwise we'll assume this is a WIP PR. |
cee-chen
force-pushed
the
feature/enterprise-search-plugin
branch
4 times, most recently
from
9f26662
to
15e8f50
Compare
cee-chen
force-pushed
the
feature/enterprise-search-plugin
branch
2 times, most recently
from
1af7a33
to
c0ccf22
Compare
- remove set - remove unnecessary capabilities registration - telemetry namespace agnostic
see elastic#66922 (comment) - add if statement - pass log dependency around (this is kinda medium, should maybe refactor) - update tests - move test file comment to the right file (was meant for telemetry route file)
cee-chen
force-pushed
the
feature/enterprise-search-plugin
branch
from
8d495f3
to
15a8b15
Compare
pgayvallet
reviewed
Jul 8, 2020
x-pack/plugins/enterprise_search/public/applications/shared/licensing/license_checks.ts
Show resolved
Hide resolved
Comment on lines
120
to
123
| registerTelemetryRoute({ | ||
| ...dependencies, | ||
| getSavedObjectsService: () => savedObjectsStarted, | ||
| }); |
There was a problem hiding this comment.
This is effectively registering the telemetry route after the plugin's start. This is currently working because we are waiting for the legacy plugins to initialize before starting the server (which are initialized after KP plugin's start), however this may break after legacy removal if we decide to change this workflow.
I see that you can't just rely on the route handler context because you are using the contract to create a repository to use incrementCounter, however as a workaround, you could do:
let soStart: SavedObjectsServiceStart;
getStartServices().then(([coreStart]) => {
soStart = coreStart.savedObjects
});
registerTelemetryRoute({
...dependencies,
getSavedObjectsService: () => soStart,
});@joshdover do we have any plan to allow access to incrementCounter from the SO client or from the SO context handler by any other mean?
There was a problem hiding this comment.
Thanks so much for the great catches & cleanup Pierre! 😁 This comment should be addressed in 9c6acdd
There was a problem hiding this comment.
@joshdover do we have any plan to allow access to incrementCounter from the SO client or from the SO context handler by any other mean?
No plans that I'm aware of.
I'm not sure why this was originally omitted from the Client interface (this all predates me at Elastic), but I suspect there may be security / authz concerns due to the scripted update that incrementCounter uses. In this specific case, authz doesn't really matter since we're accessing a telemetry object that (I assume?) is space agnostic and only accessible to the kibana_system user.
I think we don't have any internal repositories available on the RequestHandlerContext in order to prevent accidental usage when a scoped client is probably what you want. Another option would be to register your own context provider to supply this on the context object:
coreSetup.http.registerRouteHandlerContext('enterpriseSearch', () => {
const [coreStart] = await coreSetup.getStartServices();
return {
internalSavedObjectsRepository: coreStart.savedObjects.createInternalRepository()
}
});This would make the value accessible on context.enterpriseSearch.internalSavedObjectsRepository inside your route handlers.
- Remove unnecessary ServerConfigType - Remove unnecessary uiCapabilities - Move registerTelemetryRoute / SavedObjectsServiceStart workaround - Remove unnecessary license optional chaining
JasonStoltz
reviewed
Jul 8, 2020
...gins/enterprise_search/public/applications/app_search/components/setup_guide/setup_guide.tsx
Show resolved
Hide resolved
JasonStoltz
reviewed
Jul 8, 2020
...enterprise_search/public/applications/shared/kibana_breadcrumbs/generate_breadcrumbs.test.ts
Outdated
Show resolved
Hide resolved
Address type/typos
cee-chen
force-pushed
the
feature/enterprise-search-plugin
branch
from
054f659
to
3b715b5
Compare
- I can't even?? I swear charset=utf-8 fixed the same error a few weeks ago
- Fix incorrect navigateToApp slug (hadn't realized this was a URL, not an ID) - Update without_host_configured tests to run without API key - Update README
cee-chen
force-pushed
the
feature/enterprise-search-plugin
branch
from
5136952
to
f305f11
Compare
|
@elasticmachine merge upstream |
|
@legrego Our functional tests for Enterprise Search without a Is there anything else you'd like to add or review? |
pgayvallet
approved these changes
Jul 9, 2020
| }, | ||
| async (context, request, response) => { | ||
| try { | ||
| const enterpriseSearchUrl = config.host as string; |
There was a problem hiding this comment.
NIT: config.host is defined as optional in the config schema. currently, the endpoint would perform a unnecessary request that will fail:
const url = `${encodeURI(enterpriseSearchUrl)}/as/engines/collection?${params}`;
const enginesResponse = await fetch(url, {
headers: { Authorization: request.headers.authorization as string },
});would be better to 'fail fast' in the case of !config.host
There was a problem hiding this comment.
In theory the front-end should be taking care of this / not allowing this endpoint to be reached (the user/admin will get redirected to the Setup Guide which doesn't make a http call if config.host isn't set).
That being said, that's still a really good point and it's better to harden all pieces of an app! I'll definitely write this down as follow-up tech debt - I'd like to tackle it as a way to DRY out multiple endpoints at once vs per-endpoint (maybe http interceptors?)
Comment on lines
+37
to
+39
| const enginesResponse = await fetch(url, { | ||
| headers: { Authorization: request.headers.authorization as string }, | ||
| }); |
There was a problem hiding this comment.
cc @restrry: FYI, another usage relying on accessing the authorization header on the request.
There was a problem hiding this comment.
Does it perform request to a dedicated AppSearch server, but uses ES credentials? Would it cover this use-case if the platform provides request proxy functionality instead?
x-pack/plugins/enterprise_search/server/routes/app_search/engines.ts
Outdated
Show resolved
Hide resolved
Comment on lines
+33
to
+37
| export class MockRouter { | ||
| public router!: jest.Mocked<IRouter>; | ||
| public method: methodType; | ||
| public payload?: payloadType; | ||
| public response = httpServerMock.createResponseFactory(); |
There was a problem hiding this comment.
cc @joshdover @restrry I think I really like this testing utility. By improving it a little (like performing validation on all three body + query + params), that could be a good solution to provide route testing tool for plugin unit tests. WDYT?
There was a problem hiding this comment.
I'll definitely admit I didn't quite grok how to handle body/query/params and over-simplified it into the payload var (also probably not the best name 😅) - super cool to hear that others think it's useful though!!
x-pack/plugins/enterprise_search/server/routes/enterprise_search/public_url.test.ts
Outdated
Show resolved
Hide resolved
Comment on lines
+58
to
+59
| expect(mockLogger.error).toHaveBeenCalled(); | ||
| expect(mockRouter.response.internalError).toHaveBeenCalled(); |
There was a problem hiding this comment.
Optional: Assert the logged error content
expect(loggingSystemMock.collect(mockLogger).error).toMatchInlineSnapshot()There was a problem hiding this comment.
My only concern about the inline snapshot is that a stack trace is generated:
Array [
Array [
"App Search UI telemetry error: Error: Could not find Saved Objects service
at handler (/Users/constance/Documents/kibana/x-pack/plugins/enterprise_search/server/routes/app_search/telemetry.ts:35:44)
at MockRouter.callRoute (/Users/constance/Documents/kibana/x-pack/plugins/enterprise_search/server/routes/__mocks__/router.mock.ts:53:11)
at Object.it (/Users/constance/Documents/kibana/x-pack/plugins/enterprise_search/server/routes/app_search/telemetry.test.ts:69:24)
at Promise (/Users/constance/Documents/kibana/node_modules/jest-circus/build/utils.js:198:28)
at new Promise (<anonymous>)
at callAsyncCircusFn (/Users/constance/Documents/kibana/node_modules/jest-circus/build/utils.js:162:10)
at _callCircusTest (/Users/constance/Documents/kibana/node_modules/jest-circus/build/run.js:205:40)
at process._tickCallback (internal/process/next_tick.js:68:7)",
],
]
which of course is going to change per-machine and thus fail the snapshot 😬
There was a problem hiding this comment.
I changed this to
expect(loggingSystemMock.collect(mockLogger).error[0][0]).toEqual(
expect.stringContaining(
'App Search UI telemetry error: Error: Could not find Saved Objects service'
)
);
for now - would def love to hear if there's a better approach however!
There was a problem hiding this comment.
As a funny aside, when I was googling this to try and find better approaches, one of the top hits I got was Josh Dover's excellent blog post 😂 https://joshdover.com/when-not-to-use-jest-snapshots/
Comment on lines
+9
to
+13
| export default function ({ loadTestFile }: FtrProviderContext) { | ||
| describe('Enterprise Search', function () { | ||
| loadTestFile(require.resolve('./app_search/engines')); | ||
| }); | ||
| } |
There was a problem hiding this comment.
So this test suite is only for local testing, or is there a follow-up plan to be able to run the suite against a running enterprise search instance on the CI?
There was a problem hiding this comment.
Correct, per the README with_host_configured only runs locally right now. Larry let us know that Spencer added the option to run a docker image as a sidecar service on CI, but currently we're leaning as a team more towards running separate Cypress E2E tests (since we already have a full suite of Cypress tests on ent-search). It's definitely on our list to follow up on!
Comment on lines
+33
to
+34
| // Push the route to the history. | ||
| history.push(to); |
There was a problem hiding this comment.
Optional: (I know this means quite a lot of changes as you would need to add a specific context provider instead of using useHistory) It would be better to use applicationSetup.navigateToUrl here, to handle cases where you'll navigate to an internal link;
kibana/src/core/public/application/types.ts
Lines 742 to 769 in 159369b
This can be done as a follow-up.
There was a problem hiding this comment.
This is great, will definitely look into it as a follow-up PR!
legrego
approved these changes
Jul 9, 2020
JasonStoltz
reviewed
Jul 9, 2020
JasonStoltz
approved these changes
Jul 9, 2020
- remove unnecessary authz? - remove unnecessary content-type json headers - add loggingSystemMock.collect(mockLogger).error assertion - reconstrcut new MockRouter on beforeEach for better sandboxing - fix incorrect describe()s -should be it() - pull out reusable mockDependencies helper (renamed/extended from mockConfig) for tests that don't particularly use config/log but still want to pass type definitions - Fix comment copy
💛 Build succeeded, but was flaky
Test FailuresKibana Pipeline / kibana-xpack-agent / Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/lens/smokescreen·ts.lens app lens smokescreen tests should allow creation of lens visualizationsStandard OutStack TraceBuild metrics@kbn/optimizer bundle module count
History
To update your PR or re-run it, just comment with: |
|
|
cee-chen
pushed a commit
to cee-chen/kibana
that referenced
this pull request
Jul 9, 2020
* Initial App Search in Kibana plugin work - Initializes a new platform plugin that ships out of the box w/ x-pack - Contains a very basic front-end that shows AS engines, error states, or a Setup Guide - Contains a very basic server that remotely calls the AS internal engines API and returns results * Update URL casing to match Kibana best practices - URL casing appears to be snake_casing, but kibana.json casing appears to be camelCase * Register App Search plugin in Home Feature Catalogue * Add custom App Search in Kibana logo - I haven't had much success in surfacing a SVG file via a server-side endpoint/URL, but then I realized EuiIcon supports passing in a ReactElement directly. Woo! * Fix appSearch.host config setting to be optional - instead of crashing folks on load * Rename plugin to Enterprise Search - per product decision, URL should be enterprise_search/app_search and Workplace Search should also eventually live here - reorganize folder structure in anticipation for another workplace_search plugin/codebase living alongside app_search - rename app.tsx/main.tsx to a standard top-level index.tsx (which will contain top-level routes/state) - rename AS->ES files/vars where applicable - TODO: React Router * Set up React Router URL structure * Convert showSetupGuide action/flag to a React Router link - remove showSetupGuide flag - add a new shared helper component for combining EuiButton/EuiLink with React Router behavior (https://github.com/elastic/eui/blob/master/wiki/react-router.md#react-router-51) * Implement Kibana Chrome breadcrumbs - create shared helper (WS will presumably also want this) for generating EUI breadcrumb objects with React Router links+click behavior - create React component that calls chrome.setBreadcrumbs on page mount - clean up type definitions - move app-wide props to IAppSearchProps and update most pages/views to simply import it instead of calling their own definitions * Added server unit tests (#2) * Added unit test for server * PR Feedback * Refactor top-level Kibana props to a global context state - rather them passing them around verbosely as props, the components that need them should be able to call the useContext hook + Remove IAppSearchProps in favor of IKibanaContext + Also rename `appSearchUrl` to `enterpriseSearchUrl`, since this context will contained shared/Kibana-wide values/actions useful to both AS and WS * Added unit tests for public (#4) * application.test.ts * Added Unit Test for EngineOverviewHeader * Added Unit Test for generate_breadcrumbs * Added Unit Test for set_breadcrumb.tsx * Added a unit test for link_events - Also changed link_events.tsx to link_events.ts since it's just TS, no React - Modified letBrowserHandleEvent so it will still return a false boolean when target is blank * Betterize these tests Co-Authored-By: Constance <constancecchen@users.noreply.github.com> Co-authored-by: Constance <constancecchen@users.noreply.github.com> * Add UI telemetry tracking to AS in Kibana (#5) * Set up Telemetry usageCollection, savedObjects, route, & shared helper - The Kibana UsageCollection plugin handles collecting our telemetry UI data (views, clicks, errors, etc.) and pushing it to elastic's telemetry servers - That data is stored in incremented in Kibana's savedObjects lib/plugin (as well as mapped) - When an end-user hits a certain view or action, the shared helper will ping the app search telemetry route which increments the savedObject store * Update client-side views/links to new shared telemetry helper * Write tests for new telemetry files * Implement remaining unit tests (#7) * Write tests for React Router+EUI helper components * Update generate_breadcrumbs test - add test suite for generateBreadcrumb() itself (in order to cover a missing branch) - minor lint fixes - remove unnecessary import from set_breadcrumbs test * Write test for get_username util + update test to return a more consistent falsey value (null) * Add test for SetupGuide * [Refactor] Pull out various Kibana context mocks into separate files - I'm creating a reusable useContext mock for shallow()ed enzyme components + add more documentation comments + examples * Write tests for empty state components + test new usecontext shallow mock * Empty state components: Add extra getUserName branch test * Write test for app search index/routes * Write tests for engine overview table + fix bonus bug * Write Engine Overview tests + Update EngineOverview logic to account for issues found during tests :) - Move http to async/await syntax instead of promise syntax (works better with existing HttpServiceMock jest.fn()s) - hasValidData wasn't strict enough in type checking/object nest checking and was causing the app itself to crash (no bueno) * Refactor EngineOverviewHeader test to use shallow + to full coverage - missed adding this test during telemetry work - switching to shallow and beforeAll reduces the test time from 5s to 4s! * [Refactor] Pull out React Router history mocks into a test util helper + minor refactors/updates * Add small tests to increase branch coverage - mostly testing fallbacks or removing fallbacks in favor of strict type interface - these are slightly obsessive so I'd also be fine ditching them if they aren't terribly valuable * Address larger tech debt/TODOs (#8) * Fix optional chaining TODO - turns out my local Prettier wasn't up to date, completely my bad * Fix constants TODO - adds a common folder/architecture for others to use in the future * Remove TODO for eslint-disable-line and specify lint rule being skipped - hopefully that's OK for review, I can't think of any other way to sanely do this without re-architecting the entire file or DDoSing our API * Add server-side logging to route dependencies + add basic example of error catching/logging to Telemetry route + [extra] refactor mockResponseFactory name to something slightly easier to read * Move more Engines Overview API logic/logging to server-side - handle data validation in the server-side - wrap server-side API in a try/catch to account for fetch issues - more correctly return 2xx/4xx statuses and more correctly deal with those responses in the front-end - Add server info/error/debug logs (addresses TODO) - Update tests + minor refactors/cleanup - remove expectResponseToBe200With helper (since we're now returning multiple response types) and instead make mockResponse var name more readable - one-line header auth - update tests with example error logs - update schema validation for `type` to be an enum of `indexed`/`meta` (more accurately reflecting API) * Per telemetry team feedback, rename usageCollection telemetry mapping name to simpler 'app_search' - since their mapping already nests under 'kibana.plugins' - note: I left the savedObjects name with the '_telemetry' suffix, as there very well may be a use case for top-level generic 'app_search' saved objects * Update Setup Guide installation instructions (#9) Co-authored-by: Chris Cressman <chris@chriscressman.com> * [Refactor] DRY out route test helper * [Refactor] Rename public/test_utils to public/__mocks__ - to better follow/use jest setups and for .mock.ts suffixes * Add platinum licensing check to Meta Engines table/call (#11) * Licensing plugin setup * Add LicensingContext setup * Update EngineOverview to not hit meta engines API on platinum license * Add Jest test helpers for future shallow/context use * Update plugin to use new Kibana nav + URL update (#12) * Update new nav categories to add Enterprise Search + update plugin to use new category - per @johnbarrierwilson and Matt Riley, Enterprise Search should be under Kibana and above Observability - Run `node scripts/check_published_api_changes.js --accept` since this new category affects public API * [URL UPDATE] Change '/app/enterprise_search/app_search' to '/app/app_search' - This needs to be done because App Search and Workplace search *have* to be registered as separate plugins to have 2 distinct nav links - Currently Kibana doesn't support nested app names (see: elastic#59190) but potentially will in the future - To support this change, we need to update applications/index.tsx to NOT handle '/app/enterprise_search' level routing, but instead accept an async imported app component (e.g. AppSearch, WorkplaceSearch). - AppSearch should now treat its router as root '/' instead of '/app_search' - (Addl) Per Josh Dover's recommendation, switch to `<Router history={params.history}>` from `<BrowserRouter basename={params.appBasePath}>` since they're deprecating appBasePath * Update breadcrumbs helper to account for new URLs - Remove path for Enterprise Search breadcrumb, since '/app/enterprise_search' will not link anywhere meaningful for the foreseeable future, so the Enterprise Search root should not go anywhere - Update App Search helper to go to root path, per new React Router setup Test changes: - Mock custom basepath for App Search tests - Swap enterpriseSearchBreadcrumbs and appSearchBreadcrumbs test order (since the latter overrides the default mock) * Add create_first_engine_button telemetry tracking to EmptyState * Switch plugin URLs back to /app/enterprise_search/app_search Now that elastic#66455 has been merged in 🎉 * Add i18n formatted messages / translations (#13) * Add i18n provider and formatted/i18n translated messages * Update tests to account for new I18nProvider context + FormattedMessage components - Add new mountWithContext helper that provides all contexts+providers used in top-level app - Add new shallowWithIntl helper for shallow() components that dive into FormattedMessage * Format i18n dates and numbers + update some mock tests to not throw react-intl invalid date messages * Update EngineOverviewHeader to disable button on prop * Address review feedback (#14) * Fix Prettier linting issues * Escape App Search API endpoint URLs - per PR feedback - querystring should automatically encodeURIComponent / escape query param strings * Update server plugin.ts to use getStartServices() rather than storing local references from start() - Per feedback: https://github.com/elastic/kibana/blob/master/src/core/CONVENTIONS.md#applications - Note: savedObjects.registerType needs to be outside of getStartServices, or an error is thrown - Side update to registerTelemetryUsageCollector to simplify args - Update/fix tests to account for changes * E2E testing (#6) * Wired up basics for E2E testing * Added version with App Search * Updated naming * Switched configuration around * Added concept of 'fixtures' * Figured out how to log in as the enterprise_search user * Refactored to use an App Search service * Added some real tests * Added a README * Cleanup * More cleanup * Error handling + README updatre * Removed unnecessary files * Apply suggestions from code review Co-authored-by: Constance <constancecchen@users.noreply.github.com> * Update x-pack/plugins/enterprise_search/public/applications/app_search/components/engine_overview/engine_table.tsx Co-authored-by: Constance <constancecchen@users.noreply.github.com> * PR feedback - updated README * Additional lint fixes Co-authored-by: Constance <constancecchen@users.noreply.github.com> * Add README and CODEOWNERS (#15) * Add plugin README and CODEOWNERS * Fix Typescript errors (#16) * Fix public mocks * Fix empty states types * Fix engine table component errors * Fix engine overview component errors * Fix setup guide component errors - SetBreadcrumbs will be fixed in a separate commit * Fix App Search index errors * Fix engine overview header component errors * Fix applications context index errors * Fix kibana breadcrumb helper errors * Fix license helper errors * ❗ Refactor React Router EUI link/button helpers - in order to fix typescript errors - this changes the component logic significantly to a react render prop, so that the Link and Button components can have different types - however, end behavior should still remain the same * Fix telemetry helper errors * Minor unused var cleanup in plugin files * Fix telemetry collector/savedobjects errors * Fix MockRouter type errors and add IRouteDependencies export - routes will use IRouteDependencies in the next few commits * Fix engines route errors * Fix telemetry route errors * Remove any type from source code - thanks to Scotty for the inspiration * Add eslint rules for Enterprise Search plugin - Add checks for type any, but only on non-test files - Disable react-hooks/exhaustive-deps, since we're already disabling it in a few files and other plugins also have it turned off * Cover uncovered lines in engines_table and telemetry tests * Fixed TS warnings in E2E tests (#17) * Feedback: Convert static CSS values to EUI variables where possible * Feedback: Flatten nested CSS where possible - Prefer setting CSS class overrides on individual EUI components, not on a top-level page + Change CSS class casing from kebab-case to camelCase to better match EUI/Kibana + Remove unnecessary .euiPageContentHeader margin-bottom override by changing the panelPaddingSize of euiPageContent + Decrease engine overview table padding on mobile * Refactor out components shared with Workplace Search (#18) * Move getUserName helper to shared - in preparation for Workplace Search plugin also using this helper * Move Setup Guide layout to a shared component * Setup Guide: add extra props for standard/native auth links Note: It's possible this commit may be unnecessary if we can publish shared Enterprise Search security mode docs * Update copy per feedback from copy team * Address various telemetry issues - saved objects: removing indexing per elastic#43673 - add schema and generate json per elastic#64942 - move definitions over to collectors since saved objects is mostly empty at this point, and schema throws an error when it imports an obj instead of being defined inline - istanbul ignore saved_objects file since it doesn't have anything meaningful to test but was affecting code coverage * Disable plugin access if a normal user does not have access to App Search (#19) * Set up new server security dependency and configs * Set up access capabilities * Set up checkAccess helper/caller * Remove NoUserState component from the public UI - Since this is now being handled by checkAccess / normal users should never see the plugin at all if they don't have an account/access, the component is no longer needed * Update server routes to account for new changes - Remove login redirect catch from routes, since the access helper should now handle that for most users by disabling the plugin (superusers will see a generic cannot connect/error screen) - Refactor out new config values to a shared mock * Refactor Enterprise Search http call to hit/return new internal API endpoint + pull out the http call to a separate library for upcoming public URL work (so that other files can call it directly as well) * [Discussion] Increase timeout but add another warning timeout for slow servers - per recommendation/convo with Brandon * Register feature control * Remove no_as_account from UI telemetry - since we're no longer tracking that in the UI * Address PR feedback - isSuperUser check * Public URL support for Elastic Cloud (#21) * Add server-side public URL route - Per feedback from Kibana platform team, it's not possible to pass info from server/ to public/ without a HTTP call :[ * Update MockRouter for routes without any payload/params * Add client-side helper for calling the new public URL API + API seems to return a URL a trailing slash, which we need to omit * Update public/plugin.ts to check and set a public URL - relies on this.hasCheckedPublicUrl to only make the call once per page load instead of on every page nav * Fix failing feature control tests - Split up scenario cases as needed - Add plugin as an exception alongside ML & Monitoring * Address PR feedback - version: kibana - copy edits - Sass vars - code cleanup * Casing feedback: change all plugin registration IDs from snake_case to camelCase - note: current remainng snake_case exceptions are telemetry keys - file names and api endpoints are snake_case per conventions * Misc security feedback - remove set - remove unnecessary capabilities registration - telemetry namespace agnostic * Security feedback: add warn logging to telemetry collector see elastic#66922 (comment) - add if statement - pass log dependency around (this is kinda medium, should maybe refactor) - update tests - move test file comment to the right file (was meant for telemetry route file) * Address feedback from Pierre - Remove unnecessary ServerConfigType - Remove unnecessary uiCapabilities - Move registerTelemetryRoute / SavedObjectsServiceStart workaround - Remove unnecessary license optional chaining * PR feedback Address type/typos * Fix telemetry API call returning 415 on Chrome - I can't even?? I swear charset=utf-8 fixed the same error a few weeks ago * Fix failing tests * Update Enterprise Search functional tests (without host) to run on CI - Fix incorrect navigateToApp slug (hadn't realized this was a URL, not an ID) - Update without_host_configured tests to run without API key - Update README * Address PR feedback from Pierre - remove unnecessary authz? - remove unnecessary content-type json headers - add loggingSystemMock.collect(mockLogger).error assertion - reconstrcut new MockRouter on beforeEach for better sandboxing - fix incorrect describe()s -should be it() - pull out reusable mockDependencies helper (renamed/extended from mockConfig) for tests that don't particularly use config/log but still want to pass type definitions - Fix comment copy Co-authored-by: Jason Stoltzfus <jastoltz24@gmail.com> Co-authored-by: Chris Cressman <chris@chriscressman.com> Co-authored-by: scottybollinger <scotty.bollinger@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> # Conflicts: # .github/CODEOWNERS # x-pack/scripts/functional_tests.js
cee-chen
pushed a commit
that referenced
this pull request
Jul 9, 2020
* Initial App Search in Kibana plugin work - Initializes a new platform plugin that ships out of the box w/ x-pack - Contains a very basic front-end that shows AS engines, error states, or a Setup Guide - Contains a very basic server that remotely calls the AS internal engines API and returns results * Update URL casing to match Kibana best practices - URL casing appears to be snake_casing, but kibana.json casing appears to be camelCase * Register App Search plugin in Home Feature Catalogue * Add custom App Search in Kibana logo - I haven't had much success in surfacing a SVG file via a server-side endpoint/URL, but then I realized EuiIcon supports passing in a ReactElement directly. Woo! * Fix appSearch.host config setting to be optional - instead of crashing folks on load * Rename plugin to Enterprise Search - per product decision, URL should be enterprise_search/app_search and Workplace Search should also eventually live here - reorganize folder structure in anticipation for another workplace_search plugin/codebase living alongside app_search - rename app.tsx/main.tsx to a standard top-level index.tsx (which will contain top-level routes/state) - rename AS->ES files/vars where applicable - TODO: React Router * Set up React Router URL structure * Convert showSetupGuide action/flag to a React Router link - remove showSetupGuide flag - add a new shared helper component for combining EuiButton/EuiLink with React Router behavior (https://github.com/elastic/eui/blob/master/wiki/react-router.md#react-router-51) * Implement Kibana Chrome breadcrumbs - create shared helper (WS will presumably also want this) for generating EUI breadcrumb objects with React Router links+click behavior - create React component that calls chrome.setBreadcrumbs on page mount - clean up type definitions - move app-wide props to IAppSearchProps and update most pages/views to simply import it instead of calling their own definitions * Added server unit tests (#2) * Added unit test for server * PR Feedback * Refactor top-level Kibana props to a global context state - rather them passing them around verbosely as props, the components that need them should be able to call the useContext hook + Remove IAppSearchProps in favor of IKibanaContext + Also rename `appSearchUrl` to `enterpriseSearchUrl`, since this context will contained shared/Kibana-wide values/actions useful to both AS and WS * Added unit tests for public (#4) * application.test.ts * Added Unit Test for EngineOverviewHeader * Added Unit Test for generate_breadcrumbs * Added Unit Test for set_breadcrumb.tsx * Added a unit test for link_events - Also changed link_events.tsx to link_events.ts since it's just TS, no React - Modified letBrowserHandleEvent so it will still return a false boolean when target is blank * Betterize these tests Co-Authored-By: Constance <constancecchen@users.noreply.github.com> Co-authored-by: Constance <constancecchen@users.noreply.github.com> * Add UI telemetry tracking to AS in Kibana (#5) * Set up Telemetry usageCollection, savedObjects, route, & shared helper - The Kibana UsageCollection plugin handles collecting our telemetry UI data (views, clicks, errors, etc.) and pushing it to elastic's telemetry servers - That data is stored in incremented in Kibana's savedObjects lib/plugin (as well as mapped) - When an end-user hits a certain view or action, the shared helper will ping the app search telemetry route which increments the savedObject store * Update client-side views/links to new shared telemetry helper * Write tests for new telemetry files * Implement remaining unit tests (#7) * Write tests for React Router+EUI helper components * Update generate_breadcrumbs test - add test suite for generateBreadcrumb() itself (in order to cover a missing branch) - minor lint fixes - remove unnecessary import from set_breadcrumbs test * Write test for get_username util + update test to return a more consistent falsey value (null) * Add test for SetupGuide * [Refactor] Pull out various Kibana context mocks into separate files - I'm creating a reusable useContext mock for shallow()ed enzyme components + add more documentation comments + examples * Write tests for empty state components + test new usecontext shallow mock * Empty state components: Add extra getUserName branch test * Write test for app search index/routes * Write tests for engine overview table + fix bonus bug * Write Engine Overview tests + Update EngineOverview logic to account for issues found during tests :) - Move http to async/await syntax instead of promise syntax (works better with existing HttpServiceMock jest.fn()s) - hasValidData wasn't strict enough in type checking/object nest checking and was causing the app itself to crash (no bueno) * Refactor EngineOverviewHeader test to use shallow + to full coverage - missed adding this test during telemetry work - switching to shallow and beforeAll reduces the test time from 5s to 4s! * [Refactor] Pull out React Router history mocks into a test util helper + minor refactors/updates * Add small tests to increase branch coverage - mostly testing fallbacks or removing fallbacks in favor of strict type interface - these are slightly obsessive so I'd also be fine ditching them if they aren't terribly valuable * Address larger tech debt/TODOs (#8) * Fix optional chaining TODO - turns out my local Prettier wasn't up to date, completely my bad * Fix constants TODO - adds a common folder/architecture for others to use in the future * Remove TODO for eslint-disable-line and specify lint rule being skipped - hopefully that's OK for review, I can't think of any other way to sanely do this without re-architecting the entire file or DDoSing our API * Add server-side logging to route dependencies + add basic example of error catching/logging to Telemetry route + [extra] refactor mockResponseFactory name to something slightly easier to read * Move more Engines Overview API logic/logging to server-side - handle data validation in the server-side - wrap server-side API in a try/catch to account for fetch issues - more correctly return 2xx/4xx statuses and more correctly deal with those responses in the front-end - Add server info/error/debug logs (addresses TODO) - Update tests + minor refactors/cleanup - remove expectResponseToBe200With helper (since we're now returning multiple response types) and instead make mockResponse var name more readable - one-line header auth - update tests with example error logs - update schema validation for `type` to be an enum of `indexed`/`meta` (more accurately reflecting API) * Per telemetry team feedback, rename usageCollection telemetry mapping name to simpler 'app_search' - since their mapping already nests under 'kibana.plugins' - note: I left the savedObjects name with the '_telemetry' suffix, as there very well may be a use case for top-level generic 'app_search' saved objects * Update Setup Guide installation instructions (#9) Co-authored-by: Chris Cressman <chris@chriscressman.com> * [Refactor] DRY out route test helper * [Refactor] Rename public/test_utils to public/__mocks__ - to better follow/use jest setups and for .mock.ts suffixes * Add platinum licensing check to Meta Engines table/call (#11) * Licensing plugin setup * Add LicensingContext setup * Update EngineOverview to not hit meta engines API on platinum license * Add Jest test helpers for future shallow/context use * Update plugin to use new Kibana nav + URL update (#12) * Update new nav categories to add Enterprise Search + update plugin to use new category - per @johnbarrierwilson and Matt Riley, Enterprise Search should be under Kibana and above Observability - Run `node scripts/check_published_api_changes.js --accept` since this new category affects public API * [URL UPDATE] Change '/app/enterprise_search/app_search' to '/app/app_search' - This needs to be done because App Search and Workplace search *have* to be registered as separate plugins to have 2 distinct nav links - Currently Kibana doesn't support nested app names (see: #59190) but potentially will in the future - To support this change, we need to update applications/index.tsx to NOT handle '/app/enterprise_search' level routing, but instead accept an async imported app component (e.g. AppSearch, WorkplaceSearch). - AppSearch should now treat its router as root '/' instead of '/app_search' - (Addl) Per Josh Dover's recommendation, switch to `<Router history={params.history}>` from `<BrowserRouter basename={params.appBasePath}>` since they're deprecating appBasePath * Update breadcrumbs helper to account for new URLs - Remove path for Enterprise Search breadcrumb, since '/app/enterprise_search' will not link anywhere meaningful for the foreseeable future, so the Enterprise Search root should not go anywhere - Update App Search helper to go to root path, per new React Router setup Test changes: - Mock custom basepath for App Search tests - Swap enterpriseSearchBreadcrumbs and appSearchBreadcrumbs test order (since the latter overrides the default mock) * Add create_first_engine_button telemetry tracking to EmptyState * Switch plugin URLs back to /app/enterprise_search/app_search Now that #66455 has been merged in 🎉 * Add i18n formatted messages / translations (#13) * Add i18n provider and formatted/i18n translated messages * Update tests to account for new I18nProvider context + FormattedMessage components - Add new mountWithContext helper that provides all contexts+providers used in top-level app - Add new shallowWithIntl helper for shallow() components that dive into FormattedMessage * Format i18n dates and numbers + update some mock tests to not throw react-intl invalid date messages * Update EngineOverviewHeader to disable button on prop * Address review feedback (#14) * Fix Prettier linting issues * Escape App Search API endpoint URLs - per PR feedback - querystring should automatically encodeURIComponent / escape query param strings * Update server plugin.ts to use getStartServices() rather than storing local references from start() - Per feedback: https://github.com/elastic/kibana/blob/master/src/core/CONVENTIONS.md#applications - Note: savedObjects.registerType needs to be outside of getStartServices, or an error is thrown - Side update to registerTelemetryUsageCollector to simplify args - Update/fix tests to account for changes * E2E testing (#6) * Wired up basics for E2E testing * Added version with App Search * Updated naming * Switched configuration around * Added concept of 'fixtures' * Figured out how to log in as the enterprise_search user * Refactored to use an App Search service * Added some real tests * Added a README * Cleanup * More cleanup * Error handling + README updatre * Removed unnecessary files * Apply suggestions from code review Co-authored-by: Constance <constancecchen@users.noreply.github.com> * Update x-pack/plugins/enterprise_search/public/applications/app_search/components/engine_overview/engine_table.tsx Co-authored-by: Constance <constancecchen@users.noreply.github.com> * PR feedback - updated README * Additional lint fixes Co-authored-by: Constance <constancecchen@users.noreply.github.com> * Add README and CODEOWNERS (#15) * Add plugin README and CODEOWNERS * Fix Typescript errors (#16) * Fix public mocks * Fix empty states types * Fix engine table component errors * Fix engine overview component errors * Fix setup guide component errors - SetBreadcrumbs will be fixed in a separate commit * Fix App Search index errors * Fix engine overview header component errors * Fix applications context index errors * Fix kibana breadcrumb helper errors * Fix license helper errors * ❗ Refactor React Router EUI link/button helpers - in order to fix typescript errors - this changes the component logic significantly to a react render prop, so that the Link and Button components can have different types - however, end behavior should still remain the same * Fix telemetry helper errors * Minor unused var cleanup in plugin files * Fix telemetry collector/savedobjects errors * Fix MockRouter type errors and add IRouteDependencies export - routes will use IRouteDependencies in the next few commits * Fix engines route errors * Fix telemetry route errors * Remove any type from source code - thanks to Scotty for the inspiration * Add eslint rules for Enterprise Search plugin - Add checks for type any, but only on non-test files - Disable react-hooks/exhaustive-deps, since we're already disabling it in a few files and other plugins also have it turned off * Cover uncovered lines in engines_table and telemetry tests * Fixed TS warnings in E2E tests (#17) * Feedback: Convert static CSS values to EUI variables where possible * Feedback: Flatten nested CSS where possible - Prefer setting CSS class overrides on individual EUI components, not on a top-level page + Change CSS class casing from kebab-case to camelCase to better match EUI/Kibana + Remove unnecessary .euiPageContentHeader margin-bottom override by changing the panelPaddingSize of euiPageContent + Decrease engine overview table padding on mobile * Refactor out components shared with Workplace Search (#18) * Move getUserName helper to shared - in preparation for Workplace Search plugin also using this helper * Move Setup Guide layout to a shared component * Setup Guide: add extra props for standard/native auth links Note: It's possible this commit may be unnecessary if we can publish shared Enterprise Search security mode docs * Update copy per feedback from copy team * Address various telemetry issues - saved objects: removing indexing per #43673 - add schema and generate json per #64942 - move definitions over to collectors since saved objects is mostly empty at this point, and schema throws an error when it imports an obj instead of being defined inline - istanbul ignore saved_objects file since it doesn't have anything meaningful to test but was affecting code coverage * Disable plugin access if a normal user does not have access to App Search (#19) * Set up new server security dependency and configs * Set up access capabilities * Set up checkAccess helper/caller * Remove NoUserState component from the public UI - Since this is now being handled by checkAccess / normal users should never see the plugin at all if they don't have an account/access, the component is no longer needed * Update server routes to account for new changes - Remove login redirect catch from routes, since the access helper should now handle that for most users by disabling the plugin (superusers will see a generic cannot connect/error screen) - Refactor out new config values to a shared mock * Refactor Enterprise Search http call to hit/return new internal API endpoint + pull out the http call to a separate library for upcoming public URL work (so that other files can call it directly as well) * [Discussion] Increase timeout but add another warning timeout for slow servers - per recommendation/convo with Brandon * Register feature control * Remove no_as_account from UI telemetry - since we're no longer tracking that in the UI * Address PR feedback - isSuperUser check * Public URL support for Elastic Cloud (#21) * Add server-side public URL route - Per feedback from Kibana platform team, it's not possible to pass info from server/ to public/ without a HTTP call :[ * Update MockRouter for routes without any payload/params * Add client-side helper for calling the new public URL API + API seems to return a URL a trailing slash, which we need to omit * Update public/plugin.ts to check and set a public URL - relies on this.hasCheckedPublicUrl to only make the call once per page load instead of on every page nav * Fix failing feature control tests - Split up scenario cases as needed - Add plugin as an exception alongside ML & Monitoring * Address PR feedback - version: kibana - copy edits - Sass vars - code cleanup * Casing feedback: change all plugin registration IDs from snake_case to camelCase - note: current remainng snake_case exceptions are telemetry keys - file names and api endpoints are snake_case per conventions * Misc security feedback - remove set - remove unnecessary capabilities registration - telemetry namespace agnostic * Security feedback: add warn logging to telemetry collector see #66922 (comment) - add if statement - pass log dependency around (this is kinda medium, should maybe refactor) - update tests - move test file comment to the right file (was meant for telemetry route file) * Address feedback from Pierre - Remove unnecessary ServerConfigType - Remove unnecessary uiCapabilities - Move registerTelemetryRoute / SavedObjectsServiceStart workaround - Remove unnecessary license optional chaining * PR feedback Address type/typos * Fix telemetry API call returning 415 on Chrome - I can't even?? I swear charset=utf-8 fixed the same error a few weeks ago * Fix failing tests * Update Enterprise Search functional tests (without host) to run on CI - Fix incorrect navigateToApp slug (hadn't realized this was a URL, not an ID) - Update without_host_configured tests to run without API key - Update README * Address PR feedback from Pierre - remove unnecessary authz? - remove unnecessary content-type json headers - add loggingSystemMock.collect(mockLogger).error assertion - reconstrcut new MockRouter on beforeEach for better sandboxing - fix incorrect describe()s -should be it() - pull out reusable mockDependencies helper (renamed/extended from mockConfig) for tests that don't particularly use config/log but still want to pass type definitions - Fix comment copy Co-authored-by: Jason Stoltzfus <jastoltz24@gmail.com> Co-authored-by: Chris Cressman <chris@chriscressman.com> Co-authored-by: scottybollinger <scotty.bollinger@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> # Conflicts: # .github/CODEOWNERS # x-pack/scripts/functional_tests.js
9 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a new Enterprise Search plugin to Kibana x-pack. It's very MVP right now and eventually will be fleshed out to match the UI/UX of App Search currently (e.g. creating/deleting engines, analytics, etc.). Additionally, Workplace Search will also eventually be added.
Since there's some extra/additional lift required to run this plugin (requires App Search running locally/on same Elasticsearch cluster), here's a few additional screenshots/screencaps to help visualize the current plugin MVP.
Animated screencap
Static screenshots
Various empty/error states
Checklist
Delete any items that are not applicable to this PR.
For maintainers