Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SqlServer on Linux DB? #3786

Open
benaadams opened this issue May 25, 2018 · 25 comments
Open

Add SqlServer on Linux DB? #3786

benaadams opened this issue May 25, 2018 · 25 comments
Assignees

Comments

@benaadams
Copy link
Contributor

Could a SqlServer DB be added?

Based on docker FROM microsoft/mssql-server-linux:latest

@michaelhixson michaelhixson self-assigned this May 31, 2018
michaelhixson added a commit to michaelhixson/FrameworkBenchmarks that referenced this issue May 31, 2018
I added [ci skip] because this requires a new techempower/tfb image, so
the travis build would fail I think.

I don't know whether this will work out because of licensing.  See the
places where I put "ACCEPT_EULA=Y" in the dockerfiles.  I don't know what
EULA that refers to and it's possible we won't be able to accept it.
I put this together anyway just to see if it's technically possible (it
is) and to get review feedback.

I added a variant of gemini that uses SQL Server and passes all tests.

I did not measure performance at all or do any performance tuning on the
database side.

Fixes TechEmpower#3786
@bhauer
Copy link
Contributor

bhauer commented May 31, 2018

Just out of curiosity, what is licensing like for SQL Server on Linux?

The free Express edition would not be the right fit since Express is limited to 4 CPU cores. I suspect we'd want to use the Developer edition.

@benaadams
Copy link
Contributor Author

Same as Windows I believe; SQL Server 2017 brought parity between the editions in base db features/programmability (whereas previously whether you could use memory optimised, column store or data-compression was based on the edition - so Express was more a SQL Server Lite)

At SQL Server 2017, Express is now fully featured from a base db perspective; with editions upping core, memory, db size limits as well as adding "bigger database" features: scaling (failovers, clustering etc), analytics, auditing etc - so using Developer edition (which is Enterprise), should still be a valid representation of Express.

Going from the pricing page; Developer and Express are both free; with the caveat that Developer can only be used for build, test and demonstration - but not production.

So I think it would be correct to use Developer edition for the benchmark tests.

@bhauer
Copy link
Contributor

bhauer commented Jun 14, 2018

@benaadams We were just discussing this internally. We believe that Citrine qualifies as a "test" environment rather than a "production" environment for the purposes of licensing. Is it possible to get anyone at Microsoft who knows the SQL Server license model well to confirm this for us? We just don't want any bad surprises.

@bhauer
Copy link
Contributor

bhauer commented Jun 14, 2018

We will also need to contend with clause 6 from the license terms:

  1. BENCHMARK TESTING. You must obtain Microsoft's prior written approval to disclose to a third party the results of any benchmark test of the software.

@benaadams
Copy link
Contributor Author

benaadams commented Jun 15, 2018

We will also need to contend with clause 6 from the license terms:

BENCHMARK TESTING. You must obtain Microsoft's prior written approval to disclose to a third party the results of any benchmark test of the software.

Oh wow, that's definitely an issue... @DamianEdwards any idea how to proceed?

Also earlier one on licencing:

We believe that Citrine qualifies as a "test" environment rather than a "production" environment for the purposes of licensing. Is it possible to get anyone at Microsoft who knows the SQL Server license model well to confirm this for us? We just don't want any bad surprises.

@bhauer
Copy link
Contributor

bhauer commented Aug 2, 2018

Does anyone watching this conversation know who we could contact on the SQL Server team to get the necessary waivers or permission to proceed?

@jamiere-msft
Copy link

Hi. I can handle getting the data reviewed and provide written permission post-review. I do this for all of our TPC benchmark results as well as our Data Warehouse Fast Track program.

@michaelhixson
Copy link
Contributor

@jamiere-msft We'd like to publish results continuously without anyone needing to review each individual set of results. Currently, we publish results automatically at https://tfb-status.techempower.com/ and manually at https://www.techempower.com/benchmarks/. The amount of SQL Server-related data in our results would increase over time, because any contributor to this project would be able to write a test that pairs SQL Server with some web framework of their choosing.

Is this model of benchmarking compatible with SQL Server's licensing?

@jamiere-msft
Copy link

Thanks Michael. I need to run this by a few people here to see if we could do a global permission for data coming from this workload.

@mi-hol
Copy link

mi-hol commented Sep 2, 2018

@jamiere-msft did you have any luck with this question yet?
Corporate decisions are often influenced by creating potential bad headlines as a test case ;-)

@jamiere-msft
Copy link

@mi-hol Yes and no. I reached out to @benaadams to see if he could share with us some preliminary data points to help our argument. Once we get a data point or two we can continue the internal discussions.

@mi-hol
Copy link

mi-hol commented Oct 8, 2018

@jamiere-msft @benaadams I wonder if there might be news to share?

@Wraith2
Copy link

Wraith2 commented Apr 17, 2019

These benches would be handy for perf comparisons on work i've done.
Would they have to use linux builds? the linux perf of SqlClient lags behind native

@salarzobir
Copy link

salarzobir commented May 1, 2019

@benaadams any news about SqlServer?

@msmith-techempower
Copy link
Member

@Wraith2 Currently, because of a bug in Docker4Win, the toolset is limited to Linux-only. This will, presumably, be corrected at some point.

@bruno-garcia
Copy link

I'd be very curious to see how it performs with MS SQL Server.
Anything the community can do to help here?

@Wraith2
Copy link

Wraith2 commented Aug 30, 2019

no, waiting on Microsoft.

@Wraith2
Copy link

Wraith2 commented Feb 4, 2020

Still waiting on the MS side of things, an update would be useful given the performance work that i've put into Microsoft.Data.SqlClient

@Wraith2
Copy link

Wraith2 commented May 21, 2020

We're coming up to the 2 years anniversary of this thread and we're still waiting on movement from the Microsoft side on the licensing question. Given that the TE benchmarks are being used as optimization targets for much of aspnet core it'd be really good to get some benchmarks for the DB in there as well.

@Wraith2
Copy link

Wraith2 commented Nov 17, 2020

Another 6 months. Given how often techempower was mentioned a .netconf it's still a big omission that sql server isn't present in the benches. It gives the impression of a lack of confidence in the product.

@Wraith2
Copy link

Wraith2 commented Feb 8, 2021

Another round missed, https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=fortune
Will it make 3 years without anything happening? Looks likely.

@Wraith2
Copy link

Wraith2 commented May 26, 2021

3 Years old today.

@Wraith2
Copy link

Wraith2 commented May 26, 2022

4 Years old today! 🎉

Still waiting on Microsoft.

@Wraith2
Copy link

Wraith2 commented May 26, 2023

5 Years old today!

Still waiting on Microsoft. It is possible that the request has been lost internally. Does anyone know who to contact who is currently in a position to do something about this?

@Wraith2
Copy link

Wraith2 commented May 26, 2024

6 Years old today.

Still waiting on Microsoft.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants