LPS-123084 POC obtain user segments only when it is needed #505

cgoncas · 2020-11-04T13:07:14Z

Motivation

The method getSegmentsEntryIds is called in every request that receives the Liferay server. It decreases the performance of the server.

Proposed solution

Move the code from the SegmentsEntryPreAcction to where it is used.

Create a utility class SegmentEntryRetriever (maybe we need to rename it), that returns the SegmentsEntryIds of a given group, user and context (LPS-123084 Create utility class SegmentsEntryRetriever (that encapsulates common code for obtaining segments) #640).
Create a new method getSegmentsExperienceIds that doesn't contain the segmentEnryIds (LPS-123084 Add a new method getSegmentsExperienceIds, without the parameter segmentsEntryIds. #659).
Call SegmentEntryRetriever directly where SegmentsWebKeys.SEGMENTS_ENTRY_IDS is used (LPS-123084 Obtain user segments only when it is needed #665).
Call the new method getSegmentsExperienceIds so we don't need to get the segmentEntryIds when the page has NOT experience (LPS-123084 Calculate the segments only when there are experiences associated with the layout. #686).

Performance Results

With this fix, the results are almost as good as not use segmentation.

You can check the full results in this document.

Portal Version	Version	Session Count	Mean Time of Login	Portal CPU Usage (%)	DB CPU Usage (%)	Session count changes(%)
BaseLine	`2b20ab5`	38428	269	96,96,96	0,0,0	-
Test 21	Segment(100 segments)	38203	275	96,96,96	0,0,0	-0.5855105652

liferay-continuous-integration · 2020-11-04T13:07:18Z

To conserve resources, the PR Tester does not automatically run for every pull.

If your code changes were already tested in another pull, reference that pull in this pull so the test results can be analyzed.

If your pull was never tested, comment "ci:test" to run the PR Tester for this pull.

…ociated with the layout.

manoelcyreno · 2020-12-16T08:55:19Z

ci:test:tango

manoelcyreno · 2020-12-16T08:58:07Z

ci:test:echo

liferay-continuous-integration · 2020-12-16T10:19:30Z

✔️ ci:test:tango - 6 out of 6 jobs passed in 1 hour 24 minutes

Click here for more details.

Base Branch:

Branch Name: master
Branch GIT ID: 012adf4b2836179a5478bf04b5337955917fccb8

Copied in Private Modules Branch:

Branch Name: master-private
Branch GIT ID: 0ff04d9f73a4643abdd2b335581feef26b985b8a

ci:test:tango - 6 out of 6 jobs PASSED

6 Successful Jobs:

For more details click here.

liferay-continuous-integration · 2020-12-16T10:24:06Z

Jenkins Build:test-portal-acceptance-pullrequest(master)#3965
Jenkins Report:jenkins-report.html
Jenkins Suite: tango
Pull Request:liferay-tango#505
Spira Release:/Liferay DXP 7.4/Pull Request/ci:test:tango
Spira Release Build:master - cgoncas > liferay-tango - PR#505 - 2020-12-16[00:55:33]
Spira Jenkins Build:publish-spira-report#16451

liferay-continuous-integration · 2020-12-16T11:21:05Z

❌ ci:test:echo - 6 out of 20 jobs passed in 2 hours 23 minutes

Click here for more details.

Base Branch:

Branch Name: master
Branch GIT ID: 012adf4b2836179a5478bf04b5337955917fccb8

Copied in Private Modules Branch:

Branch Name: master-private
Branch GIT ID: 0ff04d9f73a4643abdd2b335581feef26b985b8a

ci:test:echo - 6 out of 20 jobs PASSED

14 Failed Jobs:

6 Successful Jobs:

For more details click here.

Failures unique to this pull:

test-portal-acceptance-pullrequest-batch(master)/functional-tomcat90-mysql57-jdk8/1
Job Results:
54 Tests Passed.
1 Test Failed.
1. AXIS_VARIABLE=20,label_exp=!master #429560
  1. LocalFile.Sites#EditSiteFriendlyURLInvalid - Poshi Report - Poshi Summary - Console Output
test-portal-acceptance-pullrequest-batch(master)/functional-tomcat90-mysql57-jdk8/10
Job Results:
97 Tests Passed.
2 Tests Failed.
1. AXIS_VARIABLE=5,label_exp=!master #459194
  1. LocalFile.WebContentWithCustomStructures#AddWCWithFirstComplexStructure - Poshi Report - Poshi Summary - Console Output
  2. LocalFile.WebContentWithCustomStructures#AddWCWithSecondComplexStructure - Poshi Report - Poshi Summary - Console Output
test-portal-acceptance-pullrequest-batch(master)/functional-tomcat90-mysql57-jdk8/11
Job Results:
64 Tests Passed.
4 Tests Failed.
Click here for more failures.
test-portal-acceptance-pullrequest-batch(master)/functional-tomcat90-mysql57-jdk8/2
Job Results:
61 Tests Passed.
5 Tests Failed.
Click here for more failures.
test-portal-acceptance-pullrequest-batch(master)/functional-tomcat90-mysql57-jdk8/3
Job Results:
57 Tests Passed.
2 Tests Failed.
1. AXIS_VARIABLE=13,label_exp=!master #457242
  1. LocalFile.WebContentDisplay#AddWithStructure - Poshi Report - Poshi Summary - Console Output
2. AXIS_VARIABLE=2,label_exp=!master #457242
  1. LocalFile.RSS#RemoveRSSPortlet - Poshi Report - Poshi Summary - Console Output
...

liferay-continuous-integration · 2020-12-16T11:43:28Z

Jenkins Build:test-portal-acceptance-pullrequest(master)#7410
Jenkins Report:jenkins-report.html
Jenkins Suite: echo
Pull Request:liferay-tango#505
Spira Release:/Liferay DXP 7.4/Pull Request/ci:test:echo
Spira Release Build:master - cgoncas > liferay-tango - PR#505 - 2020-12-16[00:58:19]
Spira Jenkins Build:publish-spira-report#11386

manoelcyreno · 2020-12-16T12:27:45Z

@david-gutierrez-mesa can you take a look, if the Echo failures have something related to the PR?

Tim-Cao · 2020-12-17T20:46:47Z

ci:test:echo

liferay-continuous-integration · 2020-12-17T23:40:18Z

❌ ci:test:echo - 7 out of 20 jobs passed in 2 hours 52 minutes

Click here for more details.

Base Branch:

Branch Name: master
Branch GIT ID: 6db1295ed2ab7bbf5d05a27108a80c399369c249

Copied in Private Modules Branch:

Branch Name: master-private
Branch GIT ID: 0ff04d9f73a4643abdd2b335581feef26b985b8a

ci:test:echo - 7 out of 20 jobs PASSED

13 Failed Jobs:

7 Successful Jobs:

For more details click here.

Failures unique to this pull:

test-portal-acceptance-pullrequest-batch(master)/functional-tomcat90-mysql57-jdk8/10
Job Results:
23 Tests Passed.
0 Tests Failed.
1. AXIS_VARIABLE=0,label_exp=!master #447581
  Build was aborted
2. AXIS_VARIABLE=6,label_exp=!master #447581
  1. LocalFile.WebContentWithCustomStructures#AddWCWithFirstComplexStructure - Poshi Report - Poshi Summary - Console Output
  2. LocalFile.WebContentWithCustomStructures#AddWCWithSecondComplexStructure - Poshi Report - Poshi Summary - Console Output
test-portal-acceptance-pullrequest-batch(master)/functional-tomcat90-mysql57-jdk8/11
Job Results:
68 Tests Passed.
3 Tests Failed.
test-portal-acceptance-pullrequest-batch(master)/functional-tomcat90-mysql57-jdk8/2
Job Results:
63 Tests Passed.
3 Tests Failed.
test-portal-acceptance-pullrequest-batch(master)/functional-tomcat90-mysql57-jdk8/3
Job Results:
57 Tests Passed.
2 Tests Failed.
1. AXIS_VARIABLE=12,label_exp=!master #449519
  1. LocalFile.WebContentDisplay#AddWithStructure - Poshi Report - Poshi Summary - Console Output
2. AXIS_VARIABLE=14,label_exp=!master #449519
  1. LocalFile.LocalizationWithSitePages#AddPageAndChildPageTranslations - Poshi Report - Poshi Summary - Console Output
test-portal-acceptance-pullrequest-batch(master)/functional-tomcat90-mysql57-jdk8/4
Job Results:
69 Tests Passed.
3 Tests Failed.
...

liferay-continuous-integration · 2020-12-18T00:22:56Z

Jenkins Build:test-portal-acceptance-pullrequest(master)#7295
Jenkins Report:jenkins-report.html
Jenkins Suite: echo
Pull Request:liferay-tango#505
Spira Release:/Liferay DXP 7.4/Pull Request/ci:test:echo
Spira Release Build:master - cgoncas > liferay-tango - PR#505 - 2020-12-17[12:46:58]
Spira Jenkins Build:publish-spira-report#16592

Tim-Cao · 2020-12-18T08:28:14Z

Hi @cgoncas and @david-gutierrez-mesa,
According to analysis results of ci:test:echo, the failure cases are not related to this PR. Those are known failure in our team.
ci:test:echo(Run in PR)

cgoncas · 2020-12-29T08:18:45Z

Closing this PR, since this solution is already merged into master

cgoncas added the draft 📝 Draft pull request label Nov 4, 2020

cgoncas changed the title ~~UTIL~~ LPS-123084 POC obtain user segments only when it is needed Nov 4, 2020

cgoncas force-pushed the MOVE_CODE_WHERE_IT_IS_USED branch from d4d1bb2 to fc4f873 Compare November 4, 2020 13:14

dgarciasarai marked this pull request as draft November 5, 2020 11:19

liferay-continuous-integration added ci:test:relevant - pending ci:test:stable - pending ci:test:relevant - failure and removed ci:test:relevant - failure ci:test:stable - failure ci:test:relevant - pending labels Nov 13, 2020

cgoncas force-pushed the MOVE_CODE_WHERE_IT_IS_USED branch from fc4f873 to d40b526 Compare December 1, 2020 17:40

liferay-continuous-integration added ci:test:sf - pending and removed ci:test:sf - failure ci:test:relevant - failure labels Dec 1, 2020

brianchandotcom and others added 2 commits December 15, 2020 22:02

LPS-124751 Makes more sense like this

012adf4

LPS-123084 Calculate the segments only when there are experiences ass…

ef7634a

…ociated with the layout.

cgoncas force-pushed the MOVE_CODE_WHERE_IT_IS_USED branch from ba5b50e to ef7634a Compare December 16, 2020 08:22

cgoncas mentioned this pull request Dec 16, 2020

LPS-123084 Calculate the segments only when there are experiences associated with the layout. #686

Closed

liferay-continuous-integration added ci:test:tango - pending and removed ci:test:tango - pending labels Dec 16, 2020

liferay-continuous-integration added ci:test:echo - pending ci:test:tango - success and removed ci:test:echo - failure ci:test:tango - pending labels Dec 16, 2020

liferay-continuous-integration added ci:test:echo - failure and removed ci:test:echo - pending labels Dec 16, 2020

liferay-continuous-integration added ci:test:echo - pending ci:test:echo - failure and removed ci:test:echo - failure ci:test:echo - pending labels Dec 17, 2020

cgoncas closed this Dec 29, 2020

cgoncas deleted the MOVE_CODE_WHERE_IT_IS_USED branch January 28, 2021 15:42

LPS-123084 POC obtain user segments only when it is needed #505

LPS-123084 POC obtain user segments only when it is needed #505

Conversation

cgoncas commented Nov 4, 2020 • edited

liferay-continuous-integration commented Nov 4, 2020

manoelcyreno commented Dec 16, 2020

manoelcyreno commented Dec 16, 2020

liferay-continuous-integration commented Dec 16, 2020

✔️ ci:test:tango - 6 out of 6 jobs passed in 1 hour 24 minutes

Base Branch:

Copied in Private Modules Branch:

For more details click here.

liferay-continuous-integration commented Dec 16, 2020

liferay-continuous-integration commented Dec 16, 2020

❌ ci:test:echo - 6 out of 20 jobs passed in 2 hours 23 minutes

Base Branch:

Copied in Private Modules Branch:

14 Failed Jobs:

For more details click here.

Failures unique to this pull:

Job Results:

Job Results:

Job Results:

Job Results:

Job Results:

liferay-continuous-integration commented Dec 16, 2020

manoelcyreno commented Dec 16, 2020

Tim-Cao commented Dec 17, 2020

liferay-continuous-integration commented Dec 17, 2020

❌ ci:test:echo - 7 out of 20 jobs passed in 2 hours 52 minutes

Base Branch:

Copied in Private Modules Branch:

13 Failed Jobs:

For more details click here.

Failures unique to this pull:

Job Results:

Job Results:

Job Results:

Job Results:

Job Results:

liferay-continuous-integration commented Dec 18, 2020

Tim-Cao commented Dec 18, 2020 • edited

cgoncas commented Dec 29, 2020

cgoncas commented Nov 4, 2020 •

edited

Tim-Cao commented Dec 18, 2020 •

edited