Skip to content
This repository has been archived by the owner on Apr 2, 2023. It is now read-only.

Improve HttpOfferSet performance #214

Merged
merged 11 commits into from
May 20, 2021
Merged

Improve HttpOfferSet performance #214

merged 11 commits into from
May 20, 2021

Conversation

lenhattan86
Copy link
Collaborator

Description:

  • Improve json encoding performance by replacing gson with jackson.
  • Use ClosableHttpClient for multiple requests.
  • Monitor the differences between the original offerset and the received ones.

Testing Done:

  • performance test on 2000 offers with ~3ms improvement per scheduling cycle.
  • verify the correctness on 2-agent cluster setup.

ridv
ridv reviewed May 19, 2021
View reviewed changes
Copy link
Contributor

@ridv ridv left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments

src/main/java/io/github/aurora/scheduler/offers/HttpOfferSetImpl.java Outdated Show resolved Hide resolved
src/main/java/io/github/aurora/scheduler/offers/HttpOfferSetImpl.java Show resolved Hide resolved
src/main/java/io/github/aurora/scheduler/offers/HttpOfferSetImpl.java Outdated Show resolved Hide resolved
src/main/java/io/github/aurora/scheduler/offers/StatCalculator.java Show resolved Hide resolved
src/main/java/io/github/aurora/scheduler/offers/HttpOfferSetImpl.java Outdated Show resolved Hide resolved
src/main/java/io/github/aurora/scheduler/offers/HttpOfferSetImpl.java Outdated Show resolved Hide resolved
src/main/java/io/github/aurora/scheduler/offers/HttpOfferSetImpl.java Outdated Show resolved Hide resolved
@lenhattan86
Copy link
Collaborator Author

I saw "CI build error" will push another fix.

ridv
ridv approved these changes May 20, 2021
View reviewed changes
Copy link
Contributor

@ridv ridv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor fix and it's ready from my end

src/main/java/io/github/aurora/scheduler/offers/HttpOfferSetImpl.java Outdated Show resolved Hide resolved
@lenhattan86
Copy link
Collaborator Author

@zorro786 If you have time, please take another look.

zorro786
zorro786 reviewed May 20, 2021
View reviewed changes
Copy link
Collaborator

@zorro786 zorro786 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some more comments

src/main/java/io/github/aurora/scheduler/offers/HttpOfferSetImpl.java Outdated Show resolved Hide resolved
src/main/java/io/github/aurora/scheduler/offers/HttpOfferSetImpl.java Outdated Show resolved Hide resolved
src/main/java/io/github/aurora/scheduler/offers/HttpOfferSetImpl.java Outdated Show resolved Hide resolved
@lenhattan86
Copy link
Collaborator Author

@zorro786 another push. Thanks for the quick response.

zorro786
zorro786 approved these changes May 20, 2021
View reviewed changes
Copy link
Collaborator

@zorro786 zorro786 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last review comment above and LGTM

@lenhattan86
Copy link
Collaborator Author

I resolved the last comment from @zorro786 and replaced "this." in the plugin.

@zorro786 zorro786 merged commit 667a5f1 into aurora-scheduler:master May 20, 2021
ridv pushed a commit that referenced this pull request May 20, 2021
@lenhattan86 @ridv
Improve HttpOfferSet performance (#214)
5177ee3 
### Description:
- Improve json encoding performance by replacing gson with jackson.
- Use ClosableHttpClient for multiple requests.
- Monitor the differences between the original offerset and the received ones.

### Testing Done:
- performance test on 2000 offers with ~3ms improvement per scheduling cycle.
- verify the correctness on 2-agent cluster setup.
@lenhattan86 lenhattan86 added java Pull requests that update Java code enhancement New feature or request labels Jun 17, 2021
@lenhattan86 lenhattan86 added this to the 0.24.2 milestone Jun 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@ridv ridv ridv approved these changes

@zorro786 zorro786 zorro786 approved these changes

Assignees
No one assigned
Labels
enhancement New feature or request java Pull requests that update Java code
Projects
None yet
Milestone
0.24.2
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@lenhattan86 @ridv @zorro786