Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Python] Flask: delete repeated tests #8183

Merged
merged 1 commit into from
May 2, 2023

Conversation

remittor
Copy link
Contributor

Issue #8178

@joanhey
Copy link
Contributor

joanhey commented Apr 27, 2023

@nbrady the variants is limited, but per framework. And will be the contrary.

I think that will be better by language, all variants.
If each fw test with ~12 platforms it's a big problem, for the time to run the bench.
And the result and difference will be the same.

It's better to limit frameworks to 2-4 platforms.

@remittor
Copy link
Contributor Author

If each fw test with ~12 platforms it's a big problem, for the time to run the bench.
It's better to limit frameworks to 2-4 platforms.

There is an option to consider the time limit for all tests that relate to a particular framework.
Here is a list of the longest tests (frameworks):

3:32:47 (0:16:55 + 3:12:59)  http4k
2:52:40 (0:09:15 + 2:40:25)  officefloor
2:26:19 (0:39:43 + 1:42:57)  ubiquity
2:18:05 (0:55:08 + 1:17:46)  cutelyst
2:17:09 (0:24:17 + 1:49:05)  php
2:13:02 (0:09:16 + 2:01:09)  rack-sequel
2:11:21 (0:24:27 + 1:44:36)  falcon
2:02:47 (0:14:38 + 1:46:31)  hexagon
1:53:41 (0:11:17 + 1:36:59)  play2
1:51:43 (0:08:45 + 1:40:14)  sinatra-sequel
1:49:15 (0:24:53 + 1:15:56)  micronaut
1:48:16 (0:09:01 + 1:33:21)  pippo
1:47:35 (0:14:04 + 1:31:34)  ktor
1:47:08 (0:07:52 + 1:36:34)  fastapi
1:43:51 (0:17:08 + 1:24:15)  imi
1:42:47 (0:07:49 + 1:31:18)  nestjs
1:38:16 (0:58:47 + 0:37:35)  spliffy
1:38:02 (0:35:07 + 1:00:59)  treefrog
1:29:30 (0:02:07 + 1:25:02)  chi
1:28:29 (0:21:39 + 1:05:01)  laravel
1:27:15 (0:07:24 + 1:17:58)  roda-sequel
1:26:32 (0:46:42 + 0:38:57)  warp
1:23:44 (0:38:40 + 0:42:07)  axum
1:20:41 (0:34:36 + 0:43:50)  actix
1:20:13 (0:08:10 + 1:10:07)  sinatra
1:18:43 (0:09:42 + 1:05:01)  spring
1:16:43 (0:23:15 + 0:52:01)  lumen
1:13:53 (0:02:33 + 1:09:07)  ffead-cpp
1:09:14 (0:01:22 + 1:04:12)  nodejs
1:07:30 (0:08:54 + 0:56:07)  flask

It can even be automated when running tests. It will just be necessary to report which tests did not fit in the time quota.

@NateBrady23 NateBrady23 merged commit 7b354da into TechEmpower:master May 2, 2023
@NateBrady23
Copy link
Member

Wow 3.5 hours for http4k.. thanks for this list. it would be nice if we could at least cut back on some of the permutations in these worst offenders to start.

franz1981 pushed a commit to franz1981/FrameworkBenchmarks that referenced this pull request Jun 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants