Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop #3

Open
wants to merge 195 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
195 commits
Select commit Hold shift + click to select a range
7ea2f97
Merge pull request #2 from fabricioasn/master
fabricioasn Apr 9, 2021
461cfc8
Analysis services pre-processing and data cleaning initiated
fabricioasn Apr 11, 2021
b40fa66
Data cleaning and pre-processing update for discretization
fabricioasn Apr 15, 2021
cf339b5
Transformation data
fabricioasn Apr 24, 2021
635d348
Adding integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 4, 2021
f30a6f8
Adding linkedService: AzureDataLakeStore1
fabricioasn May 4, 2021
05198c8
Adding managedPrivateEndpoint: AzureSqlDatabase1
fabricioasn May 4, 2021
2fea685
Adding linkedService: AzureSqlDatabaseCopyData
fabricioasn May 4, 2021
a68a04b
Adding dataset: DestinationDataset_0oj
fabricioasn May 4, 2021
0f37ef5
Adding pipeline: CopyPipeline_0oj
fabricioasn May 4, 2021
e43c2b4
Updating dataset: SourceDataset_0oj
fabricioasn May 4, 2021
6ac7e25
Updating dataset: DestinationDataset_0oj
fabricioasn May 4, 2021
60eca5b
Updating dataset: DestinationDataset_0oj
fabricioasn May 4, 2021
81813d1
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 4, 2021
2319e9b
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 4, 2021
a92d2bf
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 4, 2021
bfb1b21
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 4, 2021
0806960
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 4, 2021
d39bde3
Updating pipeline: CopyPipeline_0oj
fabricioasn May 4, 2021
3c09a0f
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 4, 2021
2f153cd
Updating linkedService: AzureSqlDatabaseCopyData
fabricioasn May 4, 2021
c1bdbad
Updating dataset: DestinationDataset_0oj
fabricioasn May 4, 2021
d1eb5b7
Updating dataset: DestinationDataset_0oj
fabricioasn May 4, 2021
f95e9d2
Updating dataset: DestinationDataset_0oj
fabricioasn May 4, 2021
336e8ef
Updating pipeline: CopyPipeline_0oj
fabricioasn May 4, 2021
7843f33
Updating dataset: SourceDataset_0oj
fabricioasn May 4, 2021
30278d8
Updating dataset: SourceDataset_0oj
fabricioasn May 4, 2021
6f54946
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 4, 2021
8c9d4e0
Adding dataset: DestinationDataset_0s6
fabricioasn May 4, 2021
7987e4e
Adding pipeline: CopyDataFromRawToRDB
fabricioasn May 4, 2021
b9b71b9
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 4, 2021
1afea6b
Updating pipeline: CopyDataFromRawToRDB
fabricioasn May 4, 2021
542f3fe
Updating pipeline: CopyDataFromRawToRDB
fabricioasn May 4, 2021
06922c6
Deleting pipeline: CopyPipeline_0oj
fabricioasn May 4, 2021
e875cc4
Updating pipeline: CopyDataFromRawToRDB
fabricioasn May 4, 2021
3f65d3a
Updating dataset: SourceDataset_0oj
fabricioasn May 4, 2021
2b41b10
Updating dataset: DestinationDataset_0oj
fabricioasn May 4, 2021
aacd51a
Updating linkedService: AzureSqlDatabaseCopyData
fabricioasn May 4, 2021
231042d
Adding dataset: DestinationDataset_ydg
fabricioasn May 4, 2021
8946890
Adding pipeline: CopyPipeline_EnemRDB
fabricioasn May 4, 2021
bd8201c
Updating linkedService: AzureSqlDatabaseCopyData
fabricioasn May 4, 2021
2d26ae1
Adding dataset: SourceDataset_bjy
fabricioasn May 4, 2021
73d070e
Adding pipeline: CopyDaraSourceToSqlRDB_Azure
fabricioasn May 4, 2021
695cc29
HMG versions of sql rdb on azure sql database from data source adlg1
fabricioasn May 5, 2021
09c016e
Merge branch 'develop' of github.com:fabricioasn/EnemInData_TCC_Unica…
fabricioasn May 5, 2021
66dbbe0
Scripts sql for create hmg and prod
fabricioasn May 5, 2021
674a5d9
R arules tests on sample source
fabricioasn May 5, 2021
b3dfae0
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 6, 2021
3d0e2c8
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 6, 2021
2712f40
Updating linkedService: AzureSqlDatabaseCopyData
fabricioasn May 6, 2021
ae3ba05
Updating dataset: DestinationDataset_ydg
fabricioasn May 6, 2021
7668ff2
Adding dataset: DestinationDataset_xe7
fabricioasn May 6, 2021
10503c2
Adding pipeline: EnemFactTableCopyData
fabricioasn May 6, 2021
173dedd
Adding dataset: DestinationDatasetInfoPessoal
fabricioasn May 6, 2021
ccb06db
Adding dataset: DestinationDataset_18t
fabricioasn May 6, 2021
fbb9c6d
Adding pipeline: CopyDataDimTablePessoais
fabricioasn May 6, 2021
19f8a45
Adding dataset: AzureSqlTable1
fabricioasn May 6, 2021
001d3ff
Adding dataset: DestinationDatasetSTEM
fabricioasn May 6, 2021
2af8beb
Adding dataset: DestinationDataset_kas
fabricioasn May 6, 2021
609d14d
Adding pipeline: CopyPipeline_InfoProva
fabricioasn May 6, 2021
ca26ef5
Adding dataset: SourceDataset_dev
fabricioasn May 6, 2021
ec8c3ae
Adding pipeline: CopyPipeline_Inclusao
fabricioasn May 6, 2021
a694d89
Adding dataset: DestinationDataset_5hn
fabricioasn May 6, 2021
c92e8e7
Adding pipeline: CopyPipeline_ST_EM
fabricioasn May 6, 2021
0a00b6d
Adding dataset: SourceDataset_vko
fabricioasn May 6, 2021
8ef43d2
Adding pipeline: CopyPipeline_QSE
fabricioasn May 6, 2021
eeb8c43
Adding dataset: SourceDataset_wig
fabricioasn May 6, 2021
eebf687
Adding pipeline: CopyPipeline_QSE
fabricioasn May 6, 2021
1cafbaa
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 6, 2021
8822a88
Deleting dataset: DelimitedText1
fabricioasn May 6, 2021
5f589f2
Deleting dataset: DelimitedText2
fabricioasn May 6, 2021
fad3c0b
Azure SQL Db Relational DataWarehouse Model of dataset
fabricioasn May 6, 2021
b8111f5
Azure SQLDB DW scripts query and create
fabricioasn May 6, 2021
1715669
Scripts select all rows
fabricioasn May 6, 2021
1ea4581
Data Dictionary and SQL Scripts
fabricioasn May 6, 2021
c41a405
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 7, 2021
4abdb77
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 7, 2021
9fcb4b6
Adding dataset: DestinationDataset_khs
fabricioasn May 7, 2021
bf612c1
Adding pipeline: CopyPipeline_JFC_InfoPessoal
fabricioasn May 7, 2021
ba7c266
Adding dataset: SourceDataset_npo
fabricioasn May 7, 2021
5cb5a26
Adding pipeline: CopyPipeline_JFT_InfoProva
fabricioasn May 7, 2021
0d52d96
Adding dataset: SourceDataset_3wf
fabricioasn May 7, 2021
c8cc87d
Adding pipeline: CopyPipeline_JFT_QSE
fabricioasn May 7, 2021
641179a
Adding dataset: DestinationDataset_r9e
fabricioasn May 7, 2021
29f291d
Adding pipeline: CopyPipeline_JFT_STEM
fabricioasn May 7, 2021
251e9b6
Adding dataset: SourceDataset_lru
fabricioasn May 7, 2021
ee705c0
Adding pipeline: CopyPipeline_JDT_QSE_InfoPessoa
fabricioasn May 7, 2021
23f7a9d
Adding dataset: DestinationDataset_2z2
fabricioasn May 7, 2021
95de12c
Adding pipeline: CopyPipeline_JDT_QSE_InfoProva
fabricioasn May 7, 2021
44c0707
Adding dataset: SourceDataset_8f4
fabricioasn May 7, 2021
352e7cd
Adding pipeline: CopyPipeline_JDT_QSE_STEM
fabricioasn May 7, 2021
02a344d
Join Dimensions and fact tables without inclusion
fabricioasn May 7, 2021
fc94597
Adding dataset: DestinationDataset_ov4
fabricioasn May 7, 2021
70c024f
Adding pipeline: CopyPipeline_JFT_Inclusao
fabricioasn May 7, 2021
c7d5a57
Adding dataset: DestinationDataset_rak
fabricioasn May 7, 2021
f2b328a
Adding pipeline: CopyPipeline_JFT_AllTablesExceptDimInclusao
fabricioasn May 7, 2021
65f28a8
Deleting pipeline: CopyPipeline_JDT_QSE_InfoProva
fabricioasn May 7, 2021
c72fa4c
Join Tables including DimInclusal and Dim AllTablesExceptInclusao
fabricioasn May 7, 2021
94cbd8f
Merge branch 'develop' of github.com:fabricioasn/EnemInData_TCC_Unica…
fabricioasn May 7, 2021
6b253b9
Adding dataset: DestinationDataset_9ij
fabricioasn May 8, 2021
46a81ae
Adding pipeline: CopyPipeline_JFT_FactDim_AllTablesWithoutInclusao
fabricioasn May 8, 2021
c27556b
Adding pipeline: pipeline1
fabricioasn May 8, 2021
4bde57e
Updating dataset: DestinationDataset_9ij
fabricioasn May 8, 2021
3370942
Deleting pipeline: pipeline1
fabricioasn May 8, 2021
602f1b2
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 8, 2021
3a174d8
Adding dataset: SourceDataset_kuc
fabricioasn May 8, 2021
bd018ce
Adding pipeline: CopyPipeline_JFT_FactDim_Pessoa_Prova_QSE_STEM
fabricioasn May 8, 2021
5c0d5c8
Updating dataset: DestinationDataset_9ij
fabricioasn May 8, 2021
7fae9d0
Updating pipeline: CopyPipeline_JFT_AllTablesExceptDimInclusao
fabricioasn May 8, 2021
286ae09
Deleting pipeline: CopyPipeline_JFT_FactDim_AllTablesWithoutInclusao
fabricioasn May 8, 2021
cd283a2
R scripts for AAR algorithm test on untransformed dataset
fabricioasn May 9, 2021
58cb55e
Merge branch 'develop' of github.com:fabricioasn/EnemInData_TCC_Unica…
fabricioasn May 9, 2021
a2260a2
R script for AAR algorithm test on untransformed dataset
fabricioasn May 9, 2021
c678e0c
R script for AAR algorithm test on untransformed dataset inclusion an…
fabricioasn May 9, 2021
2a09619
All fullscan tables in model mined by Apriori in R hmg not transformed
fabricioasn May 11, 2021
b3995a2
Modeled Data Dictionary for pre-processing and final cleaning
fabricioasn May 11, 2021
5410be3
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 13, 2021
a1a82ec
Adding integrationRuntime: SelfHostedintegrationRuntime1
fabricioasn May 13, 2021
d12d5ea
Adding linkedService: SqlServer1
fabricioasn May 13, 2021
a67b941
Adding linkedService: AzureDataLakeStoreForEnemv2
fabricioasn May 13, 2021
c89e787
Adding pipeline: pipeline1
fabricioasn May 13, 2021
802a2d1
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 13, 2021
53ac236
Adding dataset: DestinationDataset_v7x
fabricioasn May 13, 2021
be7e955
Adding pipeline: CopyPipeline_CopyDataFromADLSG1toSelfHostedSqlServe_…
fabricioasn May 13, 2021
52148ef
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 13, 2021
22d075f
Adding dataset: MicroDadosEnemCSV
fabricioasn May 14, 2021
727df8e
Updating pipeline: CopyPipeline_JFT_FactDim_Pessoa_Prova_QSE_STEM
fabricioasn May 14, 2021
7c83dcd
Adding dataset: SourceDataset_od7
fabricioasn May 14, 2021
545666f
Adding pipeline: CopyPipeline_EnemFactTable_Prodution_SqlServerOnPrem…
fabricioasn May 14, 2021
daf26ec
Deleting pipeline: pipeline1
fabricioasn May 14, 2021
cacc1f1
Adding dataset: SourceDataset_xjk
fabricioasn May 14, 2021
511a716
Adding pipeline: CopyPipeline_EnemDimTables_InfoProva_Production
fabricioasn May 14, 2021
0cf80da
Updating dataset: DestinationDataset_od7
fabricioasn May 14, 2021
60cb546
Adding dataset: DestinationDataset_8xf
fabricioasn May 14, 2021
b1fedfd
Adding pipeline: CopyPipeline_EnemDimTables_InfoPessoal_Production
fabricioasn May 14, 2021
3605290
Updating dataset: DestinationDataset_xjk
fabricioasn May 14, 2021
29b0fe8
Adding dataset: SourceDataset_fpo
fabricioasn May 14, 2021
6922d7f
Adding pipeline: CopyPipeline_EnemDimTables_QSE_Production
fabricioasn May 14, 2021
b088814
Updating dataset: DestinationDataset_8xf
fabricioasn May 14, 2021
3d9f84c
Adding dataset: DestinationDataset_od6
fabricioasn May 14, 2021
a694f68
Adding pipeline: CopyPipeline_EnemDimTables_STEM_Production
fabricioasn May 14, 2021
db85f43
Adding dataset: SourceDataset_2du
fabricioasn May 14, 2021
c12a93d
Adding pipeline: CopyPipeline_EnemDimTables_Inclusao_Production
fabricioasn May 14, 2021
82858ea
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 14, 2021
4d9cf04
Deleting pipeline: CopyPipeline_CopyDataFromADLSG1toSelfHostedSqlServ…
fabricioasn May 14, 2021
d4a546e
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 14, 2021
d4e127f
Adding dataset: SourceDataset_qqg
fabricioasn May 14, 2021
672032f
Adding pipeline: CopyPipeline_JoinEnemFactDims_NoInclusao_Production
fabricioasn May 14, 2021
b62ffc3
Adding dataset: SourceDataset_u8i
fabricioasn May 14, 2021
329068e
Adding pipeline: CopyPipeline_JoinEnemDimAllTables_Production
fabricioasn May 14, 2021
52af261
Updating dataset: DestinationDataset_qqg
fabricioasn May 14, 2021
558f85b
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 14, 2021
fccd674
Updating integrationRuntime: AutoResolveIntegrationRuntime
fabricioasn May 14, 2021
4d24198
Deleting linkedService: AzureSqlDatabaseCopyData
fabricioasn May 14, 2021
e0202c9
Updating dataset: AzureSqlTable1
fabricioasn May 14, 2021
3cadc82
Deleting dataset: DestinationDataset_0oj
fabricioasn May 14, 2021
14383b6
Deleting dataset: DestinationDataset_0s6
fabricioasn May 14, 2021
f6186fc
Deleting dataset: DestinationDataset_2du
fabricioasn May 14, 2021
e6078f7
Deleting dataset: DestinationDataset_2z2
fabricioasn May 14, 2021
178f08f
Deleting dataset: DestinationDataset_3wf
fabricioasn May 14, 2021
887921e
Updating pipeline: CopyPipeline_JoinEnemDimAllTables_Production
fabricioasn May 14, 2021
ade59e9
Updating dataset: DestinationDataset_qqg
fabricioasn May 14, 2021
63235ae
Adding dataset: DataFlowJoinFactAllDimsNoInclusao
fabricioasn May 14, 2021
3421cd7
Adding dataflow: DataFlowHMG
fabricioasn May 14, 2021
93fa62a
Updating dataflow: DataFlowHMG
fabricioasn May 14, 2021
1e098d4
Relational Data Mart Modeled with Joins
fabricioasn May 15, 2021
64ae2e7
Updating dataset: DataFlowJoinFactAllDimsNoInclusao
fabricioasn May 18, 2021
0c1f93d
Updating dataset: DataFlowJoinFactAllDimsNoInclusao
fabricioasn May 18, 2021
2bd4f13
Updating dataflow: DataFlowHMG
fabricioasn May 18, 2021
29eb9bc
base pre-processing scripts
fabricioasn May 19, 2021
8f1c142
Merge branch 'develop' of github.com:fabricioasn/EnemInData_TCC_Unica…
fabricioasn May 19, 2021
b5fafe5
Adding dataset: SourceDataset_14f
fabricioasn May 19, 2021
10ec0ff
Adding pipeline: CopyPipeline_JFT_FactAndDimensions_WithoutInclusao
fabricioasn May 19, 2021
48dfab1
Relational Data Mart Staging PreProcessing
fabricioasn May 20, 2021
9539c7c
Merge branch 'develop' of github.com:fabricioasn/EnemInData_TCC_Unica…
fabricioasn May 20, 2021
ae38a7c
Adding dataset: DestinationDataset_how
fabricioasn May 20, 2021
ab8d4d5
Adding pipeline: CopyPipeline_CopyQSEtoDim
fabricioasn May 20, 2021
7e6d502
Deleting pipeline: CopyPipeline_CopyQSEtoDim
fabricioasn May 20, 2021
c0e770e
Adding dataset: SourceDataset_9kr
fabricioasn May 20, 2021
de0221c
Adding pipeline: CopyPipeline_JFT_AllTables_NoInclusao_Final
fabricioasn May 20, 2021
bb4490c
Adding dataset: SourceDataset_w8k
fabricioasn May 20, 2021
08b5ef4
Adding pipeline: CopyPipeline_DimTableQSE_Final
fabricioasn May 20, 2021
f88203a
Pre processing terminated scripts
fabricioasn May 21, 2021
72d30b8
Merge branch 'develop' of github.com:fabricioasn/EnemInData_TCC_Unica…
fabricioasn May 21, 2021
6498275
Adding linkedService: SqlServerDMLS
fabricioasn May 21, 2021
cb9f18f
PreProcessing completed Transformation initiated = Nulls Transformed
fabricioasn May 23, 2021
f7616df
All nulls removed in transformation. Grades discretization and avg pe…
fabricioasn May 23, 2021
927935a
Complete project update
fabricioasn Jun 6, 2021
d42dc44
Enem in data Project almost finished
fabricioasn Jun 6, 2021
61b7453
Enem in data Project almost finished
fabricioasn Jun 6, 2021
6829c6f
Create README.md
fabricioasn Jun 9, 2021
b254378
Update README.md
fabricioasn Jun 9, 2021
c0e2eca
Final monography project update
fabricioasn Jun 20, 2021
3c26051
Merge branch 'develop' of github.com:fabricioasn/EnemInData_TCC_Unica…
fabricioasn Jun 20, 2021
fbcef3a
Monography documentation upload
fabricioasn Jun 24, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
*.dictionary filter=lfs diff=lfs merge=lfs -text
*.hidx filter=lfs diff=lfs merge=lfs -text
*.idf filter=lfs diff=lfs merge=lfs -text
*.sql linguist-detectable=true
*.sql linguist-language=sql
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.Rproj.user
.Rhistory
.RData
.Ruserdata
Binary file not shown.
3 changes: 0 additions & 3 deletions Azure_Analysis_Services/EnemVS/.vs/ProjectSettings.json

This file was deleted.

7 changes: 0 additions & 7 deletions Azure_Analysis_Services/EnemVS/.vs/VSWorkspaceState.json

This file was deleted.

Binary file removed Azure_Analysis_Services/EnemVS/.vs/slnx.sqlite
Binary file not shown.
23 changes: 0 additions & 23 deletions Azure_Analysis_Services/EnemVS/EnemVS.sln

This file was deleted.

45 changes: 0 additions & 45 deletions Azure_Analysis_Services/EnemVS/EnemVS.smproj

This file was deleted.

Loading