Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PERF-3888 Add low-priority background operation to sinusoidal workload #841

Closed
wants to merge 5 commits into from
Closed

PERF-3888 Add low-priority background operation to sinusoidal workload #841

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c7b4052
PERF-3888 Add low-priority background operation to execution control …
pvselvan Mar 13, 2023
f80633d
add updates and indexes to workload
pvselvan Mar 21, 2023
d46197c
reduce threads in workload
pvselvan Mar 21, 2023
77182d2
only update one document per thread
pvselvan Mar 23, 2023
73125de
introduce more iteractions of read and update actors
pvselvan Mar 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 43 additions & 12 deletions src/workloads/execution/SinusoidalReadWrites.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,26 +9,36 @@ Description: |
to effectively stop work on one actor while the other reaches its peak. We then
alternate for a couple repetitions.

Simultaneously, we will run the TTL monitor in the background to test that it
is deprioritized when the system is under load.

In this workload, we try to ensure that reads and writes do not overlap much.
Keywords:
- sinusoidal
- execution control
- insertMany
- find
- CrudActor
- ttl

dbname: &dbname mixed_sinusoidal
DocumentCount: &NumDocs 500
CollectionCount: &NumColls 1

NumPhases: &NumPhases 2
NumPhases: &NumPhases 3

Document: &doc
a: {^RandomInt: {min: 0, max: *NumDocs}}
b: {^RandomString: {length: 16}}

ExpiredDocument: &expiredDoc
expireDate: {^Date: "1970-01-01"} # Instantly expire the document
x: 999

Clients:
Default:
QueryOptions:
maxPoolSize: 1001
maxPoolSize: 1002

ActorTemplates:
- TemplateName: WritesActorWithSleepTemplate
Expand All @@ -47,10 +57,12 @@ ActorTemplates:
Repeat: 6
CollectionCount: *NumColls
Operations:
- OperationName: insertOne
- OperationName: insertMany
OperationMetricsName: WriteDocs
OperationCommand:
Document: *doc
Documents: # insert regular and expired docs
- *doc
- *expiredDoc
ThrowOnFailure: false
- TemplateName: ReadsActorWithSleepTemplate
Config:
Expand All @@ -75,37 +87,56 @@ ActorTemplates:
ThrowOnFailure: false
Actors:
- Name: Setup
Type: Loader
Type: AdminCommand
Threads: 1
Phases:
OnlyActiveInPhases:
Active: [0]
NopInPhasesUpTo: *NumPhases
PhaseConfig:
# Have 500 documents ready at start
Repeat: 1
BatchSize: 100
Database: admin
Operations:
- OperationName: AdminCommand
OperationCommand:
setParameter: 1
ttlMonitorSleepSecs: 1
gregorynoma marked this conversation as resolved.
Show resolved Hide resolved
SleepAfter: 60 seconds # Wait 60 seconds so that the new ttl sleep config is active.
gregorynoma marked this conversation as resolved.
Show resolved Hide resolved


- Name: InsertData
Type: Loader
Threads: 1
Phases:
OnlyActiveInPhases:
Active: [1]
NopInPhasesUpTo: *NumPhases
PhaseConfig:
# Add a large set of expired documents at start to make sure TLL monitor doesn't catch up in future phases.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we confirm in the FTDC data that we were indeed always behind on TTL deletes?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the FTDC data, the TTL monitor is deleting fewer documents than it was deleting after the load phase, but @haleyConnelly is there another metric I should be using besides deletedDocuments to measure this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, in general there should be a queueLength of non-zero to show low priority operations are at least queueing. Compared to the load phase, aren't there more threads the TTL is competing with in phase 2?

Repeat: 1
BatchSize: 10_000
Threads: 1
DocumentCount: *NumDocs
DocumentCount: 200_000
Database: *dbname
CollectionCount: *NumColls
Document: *doc
Document: *expiredDoc
Indexes:
- keys: {a: 1}
- keys: {expireDate: 1}
options: {expireAfterSeconds: 0}

- ActorFromTemplate:
TemplateName: WritesActorWithSleepTemplate
TemplateParameters:
Name: {^Parameter: {Name: "WritingName", Default: WritingActor_50_50}}
Threads: {^Parameter: {Name: "WritingThreads", Default: 500}}
ActivePhases: [1]
ActivePhases: [2]

- ActorFromTemplate:
TemplateName: ReadsActorWithSleepTemplate
TemplateParameters:
Name: {^Parameter: {Name: "ReadingName", Default: ReadsActor_50_50}}
Threads: {^Parameter: {Name: "ReadingThreads", Default: 500}}
ActivePhases: [1]
ActivePhases: [2]

AutoRun:
- When:
Expand Down