Lets type the world
- Type providers offer you a way of working with data in highly information rich, strong typed manner with a few lines of code
- ORM at its best ;-)
How is the general way to do that?
- Write code or use an existing lib to access the data source (e.g. a database (sql or nosql), file, web site, REST service, ...)
- Create data structures / a domain model, that will represent your world in your code
- Read the data from your source
- Transform the data into your typed data structures
- Work with your data structures
FSharp Type Provider
- A Design-time component that provides a computed space of types and methods...
- An adaptor between the data / services and .NET languages...
- Is no static code generator, but targets in the same area
- Uses the F# Compiler to generate types, properties and methods based on meta information of the data sources
- Lift type inference to a usage maximum => inferes types from the data sources
- A type provider can also ensure that groups of types are only expanded on demand. This allows on-demand integration of large-scale information spaces in a strongly typed way.
CSV Type Provider
See Sample by F# Data
Date,Open,High,Low,Close,Volume,Adj Close 2012-01-27,29.45,29.53,29.17,29.23,44187700,29.23 2012-01-26,29.61,29.70,29.40,29.50,49102800,29.50 2012-01-25,29.07,29.65,29.07,29.56,59231700,29.56 2012-01-24,29.47,29.57,29.18,29.34,51703300,29.34
CSV Type Provider
type Stocks = CsvProvider<"../data/MSFT.csv"> // Download the stock prices let msft = Stocks.Load("http://ichart.finance.yahoo.com/table.csv?s=MSFT") // Look at the most recent row. Note the 'Date' property // is of type 'DateTime' and 'Open' has a type 'decimal' let firstRow = msft.Rows |> Seq.head let lastDate = firstRow.Date let lastOpen = firstRow.Open // Print the prices in the HLOC format for row in msft.Rows do printfn "HLOC: (%A, %A, %A, %A)" row.High row.Low row.Open row.Close
Excel & SQL Type Provider
type EngagementBookingFile = ExcelFile<"EngagementBookingFile.xlsx", "A6:N1000"> [<Literal>] let connectionString = @"Data Source=...\...;Initial Catalog=...;Integrated Security=True" type EngagementTrackingDb = SqlProgrammabilityProvider<connectionString> // Booking DAL type Booking = EngagementTrackingDb.dbo.``User-Defined Table Types``.BookingTableType type UpdateBookings = EngagementTrackingDb.dbo.stUpdateBookings let readBookingExcel month (engagementCode: int) filePath = // Let the type provider do it's work let file = new EngagementBookingFile(filePath) let filteredData = file.Data |> Seq.filter(fun el -> el.``Entry date``.Month = month) let stUpdateBookings = new UpdateBookings() stUpdateBookings.Execute( [ for row in filteredData -> new Booking(EmployeeGUI = row.``Employee GUI``, EngagementCode = engagementCode, ActivityId = (if row.Activity = "ERW1" then 17 else 1), BookDate = row.``Entry date``.Date, PersonHours = Convert.ToDecimal(row.``Hours Charged``))]) |> ignore
Cool? Yes! But, it goes further
- Take other programming languages as a data source and use it integrated and typed in F# with the power of FSharp Type Providers
R Type Provider
The R Type Provider makes it possible to use all of R capabilities, from the F# interactive environment. It enables on-the-fly charting and data analysis using R packages, with the added benefit of IntelliSense over R, and compile-time type-checking that the R functions you are using exist
R Type Provider Sample (using ggplot)
See sample by Evelina Gabasova
open RProvider open RProvider.ggplot2 let x = [0.0 .. 0.1 .. 10.0] let y = x |> List.map (fun value -> sin(value)) // create a data frame let dataframe = namedParams ["X", x; "Value", y] |> R.data_frame G.ggplot(dataframe, G.aes(x="X", y="Value")) ++ R.geom__line()
Extract of further Type Providers
- HTML Type Provider
- JSON Type Provider
- Azure Storage Type Provider
- WSDL Type Provider
- WorldBank Type Provider
Program your own type providers
- You have the possibility to create your own type provider.
- Remember, before you start to create your own type provider:
- Type providers are best suited to situations where the schema is stable at runtime and during the lifetime of compiled code.
- When you create your own type provider, basically you need to do the following steps:
- Create a namespace, where the provided type could live
- Create the types, with members and methods inferred by the schema and add it to the namespace and assembly
See sample by Microsoft
// See https://msdn.microsoft.com/de-de/library/hh361034.aspx // This type defines the type provider. When compiled to a DLL, it can be referenced [<TypeProvider>] type SampleTypeProvider(config: TypeProviderConfig) as this = // Inheriting from this type provides implementations of ITypeProvider inherit TypeProviderForNamespaces() let namespaceName = "Samples.HelloWorldTypeProvider" let thisAssembly = Assembly.GetExecutingAssembly() // Make one provided type, called TypeN. let makeOneProvidedType (n:int) = // Implementation ... // Now generate 100 types let types = [ for i in 1 .. 100 -> makeOneProvidedType i ] // And add them to the namespace do this.AddNamespace(namespaceName, types) [<assembly:TypeProviderAssembly>] do()
Generative VS Erasing Type Providers
Explanation by Thomas Petricek in a stackoverflow post
The reason why standard type providers (for OData, LINQ to SQL and WSDL) work with C# is that they generate real .NET types behind the cover. This is called generative type provider. In fact, they simply call the code generation tool that would be called if you were using these technologies from C# in a standard way. So, these type providers are just wrappers over some standard .NET tools. Most of the providers that are newly written are written as erasing type providers. This means that they only generate "fake" types that tell the F# compiler what members can be called (etc.) but when the compiler compiles them, the "fake" types are replaced with some other code. This is the reason why you cannot see any types when you're using the library from C# - none of the types actually exist in the compiled code. Unless you're wrapping existing code-generator, it is easier to write erased type provider and so most of the examples are written in this way. Erasing type providers have other beneftis - i.e. they can generate huge number of "fake" types without generating excessively big assemblies. Anyway, there is a brief note "Providing Generated Types" in the MSDN tutorial, which has some hints on writing generative providers. However, I'd expect most of the new F# type providers to be written as erasing. It notes that you must have a real .NET assembly (with the generated types) and getting that is not simplified by the F# helpers for building type providers - so you'll need to emit the IL for the assembly or generate C#/F# code and compile that (i.e. using CodeDOM or Roslyn).
- The great advantage over other codegen tools is that this "service" is provided at compiler level!
- This means, that we have compile time safety in the binding process to the data source
- If some meta / structure information is changed in your data source, you will get a compiler error, rather than a runtime error
- The type provider feature itself is limited to F#, but the generated types are accessible, if we have a generative type provider
- Very, very powerful feature in the data science area, where you have to deal with a lot of crapy ;-), semistructured data