U-SQL Avro Example
This example demonstrates how you can use U-SQL to analyze data stored in Avro files.
- Open Microsoft.Analytics.Samples.sln in Visual Studio 2017
- Build the Microsoft.Analytics.Samples solution
- Copy the following files from your build directory to a directory in Azure Data Lake Store (e.g. \Assemblies\Avro):
- Create a database (e.g. run 1-CreateDB.usql.cs), switch to the new database
- Check file paths in 2-RegisterAssemblies.usql and update them if necessary
- register the assemblies which have previously been uploaded to ADLS by submitting 2-RegisterAssemblies.usql
Upload sample data
- Get an Avro sample file which contains twitter data from here.
- Use the Azure Data Lake Explorer (in Visual Studio, or the Azure Portal) or any other ADLS client to upload twitter.avro to a directory in Azure Data Lake Store (e.g. /TwitterStream/2016/12/twitter.avro)
- Check file paths in 3-SimpleAvro.usql and update them if necessary
Run the sample job
- Submit 3-SimpleAvro.usql and wait for the U-SQL to finish.