Azure Function App V2 that converts CSV data to JSON
I created this Azure Function as part of a larger solution to process incomming SFTP files. One possible soltion is to convert the CSV data to JSON and then perform Azure SQL Bulk OPENROWSETS Bulk Inserts.
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.
The JSON the function accepts consists of the following fields.
rowsToSkip - This indicates the number of rows to skip in the conversion process.
fileName - This is the source file where the data came from. This is passed in for downstream processes that may need to know which file was processed.
csv - This is the raw data from the source file that you want to be converted to JSON. This content may contain \r\n or \n end of line markers. The function will detect them an process the csv data accordingly.
{
"rowsToSkip": 1,
"fileName": "MyTestCSVFile.csv",
"csv":"ID,Name,Score
1,Aaron,99
2,Dave,55
3,Susy,77
"
}
fileName - This is the source file where the data came from. This is passed in for downstream processes that may need to know which file was processed.
rows - list of the CSV data in JSON format, with the field name from the header row.
{
"fileName":"MyTestCSVFile.csv",
"rows":[
{
"ID":"1",
"Name":"Aaron",
"Score":"99"
},
{
"ID":"2",
"Name":"Dave",
"Score":"55"
},
{
"ID":"3",
"Name":"Susy",
"Score":"77"
}]
}
What things you need to install the software and how to install them
Visual Studio 15.5.7
Postman v6.0.7
Download Postman v6.0.7
A step by step series of examples that tell you have to get a development env running
Say what the step will be
Give the example
And repeat
until finished
End with an example of getting some data out of the system or using it for a little demo
Explain how to run the automated tests for this system
These variables used in the Postman tests.
url - This is the URI of the Azure Function that you have published to or use for local testing. (ie: localhost:7071) This variable is in each test collection. Be sure to update them both.
"variable": [
{
"id": "10226ddb-3af2-4d88-a4da-efff2ffed2b3",
"key": "url",
"value": "localhost:7071",
"type": "text",
"description": ""
}
]
Or
variable": [
{
"id": "34f41e2c-c10f-46e8-b08f-bb1d3be4b714",
"key": "url",
"value": "YOUR_FUNCTION_APP_NAME_GOES_HERE.azurewebsites.net",
"type": "text",
"description": ""
}
functions-key - This is the Function Key that you can use to limit access to your Azure functions. This variable is only in the Azure test collection.
"variable": [
....,
{
"id": "f7592072-64a6-4a15-b5d5-77c13b06487c",
"key": "functions-key",
"value": "YOUR_KEY_GOES_HERE",
"type": "string",
"description": ""
}
]
There are two Postman collections that cover the testing of the CSVToJSON function. One for local testing and the other for Azure testing.
FunctionAppCSVToJSON
POST to CSVToJSON with Windows text file contents
Negative Test: POST to CSVToJSON with Windows text file contents, no filename
FunctionAppCSVToJSON Azure
POST to CSVToJSON with Windows text file contents
Negative Test: POST to CSVToJSON with Windows text file contents, no filename
CSVToJSON swagger
Add additional notes about how to deploy this on a live system
Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.
We use SemVer for versioning. For the versions available, see the tags on this repository.
- Aaron Ralls - Initial work - Aaron Ralls
See also the list of contributors who participated in this project.
This project is licensed under the MIT License - see the LICENSE.md file for details
-
This is based on the CSVToJSON API created by Jeff Hollan
-
etc
- Azure Functions Documentation
- Contact me on twitter @cajunAA