Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes / Updates / Dates for Round 19 #4966

Open
2 tasks done
nbrady-techempower opened this issue Jul 31, 2019 · 24 comments
Open
2 tasks done

Changes / Updates / Dates for Round 19 #4966

nbrady-techempower opened this issue Jul 31, 2019 · 24 comments

Comments

@nbrady-techempower
Copy link
Member

@nbrady-techempower nbrady-techempower commented Jul 31, 2019

I'll be editing this pinned issue to link to other issues or pull requests that may affect tests for this round.

Round 19 testing will begin on or around May 1st. Last call for changes to be reviewed and merged is April 29th.

  • #4984 - Updated major versions of MySQL and MongoDB
  • #5015 - The requirements are now just on GitHub, and the results web site links there instead of having a duplicate copy.
  • #5167 - Single and Multi Query tests should be reading the entire row from the database. This was stated in the rules but have added an extra sentence to make it clearer that selecting just the randomNumber is incorrect.
  • #5199 - Postgres to version 12
  • Citrine environment from CentOS to Ubuntu 18.04
  • #5285 - Wrk improvements and MongoDB update
  • Azure environment from Ubuntu 16.04 to 18.04 and changing the terraform script from using latest image to a specific version for better reproduction of test results. Apr 23, 2020

Blockers for Round 19

  • Investigating issue related to #4092 and other start/stop issues
  • Add a message to tfb-status with some advance notice before PR submissions are closed for the round

Round 19 results have been posted: https://www.techempower.com/benchmarks/#section=data-r19

@nbrady-techempower nbrady-techempower pinned this issue Jul 31, 2019
@bhauer
Copy link
Contributor

@bhauer bhauer commented Aug 22, 2019

  • #5015 The requirements are not just on GitHub, and the results web site links there instead of having a duplicate copy.

@zloster
Copy link
Contributor

@zloster zloster commented Nov 5, 2019

Hi all. I want to bring attention to the following issues:

  1. There is no response for quite long time on these two: #4092 (comment) and #4885 (comment) I'm not able to pinpoint the problem that is causing the described behaviour.
  2. #4083 (comment) I would like to hear how to proceed with this. It concerns the frameworks using Java and servlet container. The data suggests there is a difference.

@volyrique
Copy link
Contributor

@volyrique volyrique commented Nov 9, 2019

Are there any plans for OS upgrades? Newer kernel versions might be affected differently by the security countermeasures that led to a lot of performance regressions before. CentOS 8 got released more than a month ago (CentOS 7 is used in the physical hardware environment), and I believe that we have more recent options for the Azure VMs (which use Ubuntu 16.04 if I am not mistaken).

@bhauer
Copy link
Contributor

@bhauer bhauer commented Nov 10, 2019

@volyrique Indeed, we are presently considering switching over to Ubuntu 18.04.

@hushino
Copy link

@hushino hushino commented Nov 14, 2019

Update package.json of Hapi from "hapi": "16.1.1" to "@hapi/hapi": "^18.4.0"
package.json of hapi tests

@volyrique
Copy link
Contributor

@volyrique volyrique commented Nov 21, 2019

How about any plans to visualize the cached queries test type?

@bhauer
Copy link
Contributor

@bhauer bhauer commented Nov 21, 2019

@volyrique Good question. I'll see if I can get that enabled for the next round.

@OneCricketeer
Copy link

@OneCricketeer OneCricketeer commented Dec 31, 2019

Please upgrade Dropwizard to at least 1.3.17 or to 2.0

https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Java/dropwizard/pom.xml#L17

@OneCricketeer
Copy link

@OneCricketeer OneCricketeer commented Dec 31, 2019

@Liroo
Copy link

@Liroo Liroo commented Feb 13, 2020

Adding uWebSocket @alexhultman

@nbrady-techempower
Copy link
Member Author

@nbrady-techempower nbrady-techempower commented Mar 24, 2020

Round 19 will run in April. Pull requests not ready by 4/3 won't be guaranteed to be part of Round 19.

@zloster
Copy link
Contributor

@zloster zloster commented Mar 24, 2020

@nbrady-techempower @bhauer
#4092 (comment)
The problem with the TCP/IP connections is still there and there is no comment from your side. I've found a possible reproduction steps. It's just a configuration change.
Please, take a closer look.

@nbrady-techempower
Copy link
Member Author

@nbrady-techempower nbrady-techempower commented Apr 8, 2020

We're taking Citrine down for maintenance and working on some pressing issues that may have a negative impact on the next round. We'll be postponing round 19 for a couple of weeks, so please feel free to keep submitting pull requests. We'll have messaging on Citrine and in this thread when things are back up.

@sebastienros
Copy link
Contributor

@sebastienros sebastienros commented Apr 8, 2020

pressing issues that may have a negative impact on the next round

Can you share the details if we want to repro the environment?

@msmith-techempower
Copy link
Member

@msmith-techempower msmith-techempower commented Apr 8, 2020

Can you share the details if we want to repro the environment?

We will share any changes we make.

@nbrady-techempower
Copy link
Member Author

@nbrady-techempower nbrady-techempower commented Apr 8, 2020

@sebastienros We're working more closely on this issue right now #4092 and anything else we do will be completely transparent.

@OneCricketeer
Copy link

@OneCricketeer OneCricketeer commented Apr 8, 2020

I would contribute a PR, but tied up with other priorities. Can someone take care of Dropwizard, please?

@joanhey
Copy link
Contributor

@joanhey joanhey commented Apr 9, 2020

Before close the PRs for the next Round.

Please add a message in advance in https://tfb-status.techempower.com/
Like the one that you place when you're doing maintenance.
DeepinScreenshot_Seleccionar área_20200410003104

@nbrady-techempower
Copy link
Member Author

@nbrady-techempower nbrady-techempower commented Apr 9, 2020

Thank you @joanhey for making that request here. I've added it to the top as a reminder to myself.

@nbrady-techempower
Copy link
Member Author

@nbrady-techempower nbrady-techempower commented Apr 17, 2020

We're moving forward with Round 19:

Round 19 testing will begin on or around May 1st. Last call for changes to be reviewed and merged is April 29th.

@joanhey
Copy link
Contributor

@joanhey joanhey commented Apr 23, 2020

Update Azure to Ubuntu 18.04 like in Citrine.

@Phara0h
Copy link

@Phara0h Phara0h commented Apr 29, 2020

Are you hard set on the deadline for today? I was planning on having mine by the end of the weekend. Pretty sure I have the fastest web framework on the web right now, minium fastest with node.js.

@nbrady-techempower
Copy link
Member Author

@nbrady-techempower nbrady-techempower commented Apr 29, 2020

@Phara0h A continuous run started today on Citrine. The next run will be reviewed for Round 19, so technically you do have a few days, but it would not give you any time to review data and fix problems.

@nbrady-techempower
Copy link
Member Author

@nbrady-techempower nbrady-techempower commented May 28, 2020

Round 19 results have been posted: Round 19 results have been posted: https://www.techempower.com/benchmarks/#section=data-r19

Unpinning this issue but will leave open for a bit. Pinning a new issue to start tracking changes for Round 20

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests