Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(pipeline): add odspep mapping for services #27 #31

Merged
merged 21 commits into from
Feb 24, 2023
Merged

feat(pipeline): add odspep mapping for services #27 #31

merged 21 commits into from
Feb 24, 2023

Conversation

ColinMaudry
Copy link
Contributor

@ColinMaudry ColinMaudry commented Feb 20, 2023 •
edited
Loading

Mapping is not complete

@ColinMaudry ColinMaudry marked this pull request as draft February 20, 2023 15:43
@ColinMaudry ColinMaudry self-assigned this Feb 21, 2023
This was linked to issues Feb 21, 2023
Add geographic availability to services #30
Closed
ODSPEP pipeline with dbt #27
Closed
Mapping champs ODSPEP 2023 <=> format intermédiaire #17
Closed
@ColinMaudry ColinMaudry marked this pull request as ready for review February 21, 2023 21:14
@ColinMaudry ColinMaudry changed the title feature(pipeline): add odspep mapping for services #27 feat(pipeline): add odspep mapping for services #27 Feb 22, 2023
vmttn
vmttn reviewed Feb 22, 2023
View reviewed changes
Copy link
Contributor

@vmttn vmttn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

top ça fait plaisir de voir ta première contribution aux mappings ! 😀

pipeline/dbt/models/intermediate/odspep/_odspep__models.yml Outdated Show resolved Hide resolved
pipeline/dbt/models/intermediate/odspep/int_odspep__aggregated_ressource.sql Outdated Show resolved Hide resolved
pipeline/dbt/models/intermediate/odspep/int_odspep__aggregated_ressource.sql Outdated Show resolved Hide resolved
pipeline/dbt/models/intermediate/odspep/int_odspep__aggregated_ressource.sql Outdated Show resolved Hide resolved
pipeline/dbt/models/intermediate/odspep/int_odspep__enhanced_res_partenariales.sql Outdated Show resolved Hide resolved
pipeline/dbt/models/intermediate/odspep/int_odspep__services.sql Outdated Show resolved Hide resolved
pipeline/dbt/models/sources/odspep/_odspep__models.yml Show resolved Hide resolved
pipeline/dbt/models/intermediate/odspep/int_odspep__services.sql Outdated Show resolved Hide resolved
pipeline/dbt/models/sources/odspep/_odspep__models.yml Show resolved Hide resolved
ColinMaudry and others added 9 commits February 22, 2023 15:04
@ColinMaudry @vmttn
doc(dbt): add table documentation in _odspep__models.yml
2b4f360 
Co-authored-by: vmttn <taq@protonmail.ch>
@ColinMaudry @vmttn
docs(dbt): rephrase table description
f4b177a 
Co-authored-by: vmttn <taq@protonmail.ch>
@ColinMaudry @vmttn
chore(odspep): use double quotes when renaming columns (vs. transform…
d51cb6e 
…ations)

Co-authored-by: vmttn <taq@protonmail.ch>
@ColinMaudry
feat(odspep): add surrogate key to *_ressource data
c395f7c 
in order to have unique id in aggregate_ressource table #30
@ColinMaudry
feat(odspep): test id uniqueness in int services and int aggregated_r…
9e9ed55 
…essource #30
@ColinMaudry
chore(odspep): merge changes from the review
8a601ed
@ColinMaudry
feat(odspep): enable unique service id using unique zone code #30
5cd9ea6
@ColinMaudry
fix(odspep): unique ressource and service id (dbt tested) #30
afb8878
@ColinMaudry
feat(odspep): disabled dedup structures, removed aggr ressource id #30
d61cd1c
ColinMaudry
ColinMaudry commented Feb 23, 2023
View reviewed changes
Copy link
Contributor Author

@ColinMaudry ColinMaudry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Je pense avoir tout traité

pipeline/dbt/models/intermediate/odspep/_odspep__models.yml Outdated Show resolved Hide resolved
pipeline/dbt/models/intermediate/odspep/_odspep__models.yml Outdated Show resolved Hide resolved
@vmttn
Copy link
Contributor

vmttn commented Feb 24, 2023

J'ai fait 4 petites modifs :

  • changé l'ordre pour satifsfaire le linter. J'ai l'impression que la désactivation de L034 ne fonctionne pas sur une seule ligne. Pas le temps de creuser.
  • SELECT "service" AS -> SELECT service AS
  • j'ai mis en place une simple déduplication des id_res pour les structures, qui vise à enlever la rédondance qu'on a créée sur la jointure avec les zone de diffusion. Dans le modèle int_odspep__structures, je ne sélectionne qu'une ligne par id_res. Ca différe de la déduplication qu'on avait avant (qui elle retirait des id_res pour dédupliquer les structures sous-jacentes)
  • mis en place le petit bout de code qu'on avait discuté dans la PR sur le mapping des thématiques

Allez ça part!

Merci d'avoir update la branche et pour tout le boulot !

@vmttn vmttn merged commit 946ffcf into main Feb 24, 2023
@ColinMaudry ColinMaudry deleted the odspep branch March 29, 2023 07:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vmttn vmttn vmttn left review comments

Assignees

@ColinMaudry ColinMaudry

Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@ColinMaudry @vmttn