Skip to content

Commit 12965ba

Browse files
linogalianagithub-actions[bot]
linogaliana
and
github-actions[bot]
authored
:launch: Bascule vers quarto (#226)
* premeir test quarto * validate yaml * Continue avec quarto * modification badge vscode * passage sous forme de projet quarto * commence à tatonner sur action * hugo build * update * Ajoute des arguments au build * ajoute l'option par défaut wrap * modif * Ajoute une exception pour build * Workaround actions/checkout#760 * restructure page bundle * rename * retire bout problématique * Force rendering _index.qmd * mathjax * gère chemin images * ajoute image reshape * ajoute image join * Delete pandas_join.png * mv file * Ajoute le TP pandas * quarto * adapte le notebook * retire fix * file * correction * ajoute image * déplace deux TP * rename file * rename * typo folium * corrige typo dans chapitre geopandas * retour titre * output asis * corrige erreur * corrige erreur * dep * dep * surface * corrige des typos * carte arrondissements * figures by pandas * finalise cleaning gpd * arrange * Ajoute des ancres * essaie à nouveau la construction des notebooks * essaie à nouveau la construction des notebooks * change artifacts dir * change section name * ajoute corrections au worfjlow * ajoute corrections au worfjlow * Qmd maps * corrige clé * convert matplotlib tuto * clean qmd * echo false * feature engineering * ajoute index * Modélisation chapitre 2 * update * tp regression * ajoute feature selection * clustering * création fichier pipeline * finit le fichier * prb exec * corrige erreurs * import pandas * no more r * update * ajoute image * ajoute index * ajoute image scikit * typo * duplication image * duplication image * les deux premiers chapittes NLP * ajoute download * python LDA * error * Exos supp * chapitre elastic * chapitre elastic * rename utils * rename * refactorise * introgit * change anem * refactorise * refactorise * S3 * shell * typo * liste TP supprimée * update * update wordcloud * retour du wordcloud * erreur nom badge * update * raw shortcode * ajoute hack shortcode * update * en eval false * retire import brut image * tente un seul job pour les notebooks * nettoie tous les shortcodes * retire dossier notebooks * force * Automated changes * script to hack names * hack names * add move files * dir create * Automated changes * corrige typo * Automated changes * Automated changes * force commit * Automated changes * Automated changes * test ligne à ligne * Automated changes * Automated changes * trying differently * Automated changes * change git add * Automated changes * Automated changes * widlcard aussi là * updae * Automated changes * Automated changes * qmdise * tuto word2vec * les derniers TP * retire R * eval false pour tester * retour * eval false * eval false * retire * Automated changes * simplifie elastic * master * Automated changes * Automated changes * rewrite * Automated changes * Automated changes * clean * Automated changes * resub * Automated changes * Automated changes Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
1 parent aa945cb commit 12965ba

File tree

54 files changed

+1673
-2628
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

54 files changed

+1673
-2628
lines changed

‎.github/workflows/netlify-test.yaml‎

Lines changed: 107 additions & 89 deletions
Original file line numberDiff line numberDiff line change
@@ -11,48 +11,40 @@ jobs:
1111
blogdown:
1212
name: Render-Blog
1313
runs-on: ubuntu-latest
14-
container: linogaliana/python-datascientist:latest
14+
container: linogaliana/python-datascientist-vstudio:quarto
1515
steps:
1616
- uses: actions/checkout@v2
1717
with:
1818
submodules: true
1919
fetch-depth: 0
2020
ref: ${{ github.event.pull_request.head.ref }}
2121
repository: ${{github.event.pull_request.head.repo.full_name}}
22-
- name: Install rmarkdown
23-
env:
24-
GITHUB_PAT: ${{ secrets.PAT_GITHUB }}
25-
run: |
26-
Rscript -e 'remotes::install_github("yihui/xfun")'
27-
Rscript -e 'remotes::install_github("yihui/knitr")'
28-
Rscript -e 'install.packages(c("rmarkdown", "here"))'
29-
- name: install hugo
30-
run: Rscript -e 'blogdown::install_hugo("0.83.0", force = TRUE)'
31-
- name: Install Python
32-
run: |
33-
Rscript -e 'print(blogdown:::hugo_version())'
34-
Rscript -e "install.packages(c('remotes', 'reticulate'))"
35-
Rscript -e "install.packages(c('here'))"
36-
- shell: bash -l {0}
22+
- name: Configure safe.directory # Workaround for actions/checkout#760
23+
run: git config --global --add safe.directory /__w/python-datascientist/python-datascientist
24+
- shell: bash
3725
run: |
3826
conda info
3927
conda list
4028
- name: Build to md
4129
run: |
42-
git diff --name-only origin/master origin/${GITHUB_HEAD_REF} >> diff.txt
43-
cat diff.txt
44-
Rscript -e 'source("./build/build_light.R")'
30+
quarto render --to hugo
31+
mv content/course/manipulation/index.md content/course/manipulation/_index.md
32+
mv content/course/visualisation/index.md content/course/visualisation/_index.md
33+
mv content/course/modelisation/index.md content/course/modelisation/_index.md
34+
#git diff --name-only origin/master origin/${GITHUB_HEAD_REF} >> diff.txt
35+
#cat diff.txt
36+
#Rscript -e 'source("./build/build_light.R")'
37+
hugo mod graph
38+
hugo -D --themesDir themes -t github.com
4539
- name: Clean files with Python function
4640
run: |
47-
rm -rf "./temp"
48-
mkdir -p temp
49-
mkdir -p notebooks
5041
pip install matplotlib wordcloud nltk
51-
python build/cleanmd.py
5242
python build/wc_website.py
53-
- name: Render blog
43+
python build/post-build.py
44+
- name: Copy notebooks in dedicated directory
5445
run: |
55-
Rscript -e 'source("./build/netlify.R")'
46+
mkdir -p notebooks
47+
python build/move_files.py notebooks
5648
- name: Install npm
5749
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
5850
uses: actions/setup-node@v2
@@ -74,28 +66,10 @@ jobs:
7466
with:
7567
name: Website
7668
path: public/
77-
- name: Convert in ipynb with Jupytext
78-
run: |
79-
pip install jupytext
80-
cd ./temp
81-
for i in $(find . -type f \( -iname "*.Rmd" \)); do \
82-
j="${i%.Rmd}.md" ;
83-
echo "$j" ;
84-
if [ -e $j ]
85-
then
86-
echo "Converting $j"; \
87-
#jupytext --to py --execute "$i"
88-
jupytext --to ipynb "$j" ;
89-
k="${j%.md}.ipynb" ;
90-
fi
91-
done
92-
mkdir -p ../notebooks
93-
cp -R . ../notebooks
94-
cd ..
9569
- uses: actions/upload-artifact@v2
9670
with:
9771
name: Notebooks
98-
path: temp/
72+
path: notebooks/
9973
- name: Commit new notebooks
10074
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
10175
env:
@@ -106,67 +80,32 @@ jobs:
10680
# git config user.email github-actions@github.com
10781
#git checkout ${BRANCHE_REF}
10882
git status
109-
git add notebooks/**/*.ipynb;
83+
find notebooks -type f -name "*.ipynb" | xargs git add
11084
git commit -m "Automated changes";
11185
git pull
11286
git push
11387
corrections:
88+
name: Render-Blog
11489
runs-on: ubuntu-latest
115-
container: linogaliana/python-datascientist:latest
90+
container: linogaliana/python-datascientist-vstudio:quarto
11691
steps:
11792
- uses: actions/checkout@v2
11893
with:
11994
submodules: true
12095
fetch-depth: 0
12196
ref: ${{ github.event.pull_request.head.ref }}
12297
repository: ${{github.event.pull_request.head.repo.full_name}}
123-
- name: Install rmarkdown
124-
env:
125-
GITHUB_PAT: ${{ secrets.PAT_GITHUB }}
126-
run: |
127-
Rscript -e 'remotes::install_github("yihui/xfun")'
128-
Rscript -e 'remotes::install_github("yihui/knitr")'
129-
Rscript -e 'install.packages(c("rmarkdown"))'
130-
- name: Install Python
131-
run: |
132-
Rscript -e "install.packages(c('remotes', 'reticulate'))"
133-
Rscript -e "install.packages(c('here'))"
134-
- shell: bash -l {0}
98+
- name: Configure safe.directory # Workaround for actions/checkout#760
99+
run: git config --global --add safe.directory /__w/python-datascientist/python-datascientist
100+
- shell: bash
135101
run: |
136102
conda info
137103
conda list
138-
- name: Modif Rmd echo status
139-
run: |
140-
python build/modifmd.py
141-
- name: Build to md
142-
run: |
143-
git diff --name-only origin/master origin/${GITHUB_HEAD_REF} >> diff.txt
144-
cat diff.txt
145-
Rscript -e 'source("./build/build_light.R")'
146-
- name: Clean files with Python function
147-
run: |
148-
rm -rf "./temp"
149-
mkdir -p temp
150-
mkdir -p corrections
151-
python build/cleanmd.py
152104
- name: Convert in ipynb with Jupytext
153105
run: |
154-
pip install jupytext
155-
cd ./temp
156-
for i in $(find . -type f \( -iname "*.Rmd" \)); do \
157-
j="${i%.Rmd}.md" ;
158-
echo "$j" ;
159-
if [ -e $j ]
160-
then
161-
echo "Converting $j"; \
162-
#jupytext --to py --execute "$i"
163-
jupytext --to ipynb "$j" ;
164-
k="${j%.md}.ipynb" ;
165-
fi
166-
done
167-
mkdir -p ../corrections
168-
cp -R . ../corrections
169-
cd ..
106+
quarto render content --to ipynb --execute
107+
mkdir -p corrections
108+
python build/move_files.py corrections
170109
- uses: actions/upload-artifact@v2
171110
with:
172111
name: Corrections
@@ -181,8 +120,87 @@ jobs:
181120
# git config user.email github-actions@github.com
182121
#git checkout ${BRANCHE_REF}
183122
git status
184-
git add corrections/**/*.ipynb;
123+
find corrections -type f -name "*.ipynb" | xargs git add
185124
git commit -m "Automated changes";
186125
git pull
187126
git push
188127
128+
129+
130+
131+
# corrections:
132+
# runs-on: ubuntu-latest
133+
# container: linogaliana/python-datascientist:latest
134+
# steps:
135+
# - uses: actions/checkout@v2
136+
# with:
137+
# submodules: true
138+
# fetch-depth: 0
139+
# ref: ${{ github.event.pull_request.head.ref }}
140+
# repository: ${{github.event.pull_request.head.repo.full_name}}
141+
# - name: Install rmarkdown
142+
# env:
143+
# GITHUB_PAT: ${{ secrets.PAT_GITHUB }}
144+
# run: |
145+
# Rscript -e 'remotes::install_github("yihui/xfun")'
146+
# Rscript -e 'remotes::install_github("yihui/knitr")'
147+
# Rscript -e 'install.packages(c("rmarkdown"))'
148+
# - name: Install Python
149+
# run: |
150+
# Rscript -e "install.packages(c('remotes', 'reticulate'))"
151+
# Rscript -e "install.packages(c('here'))"
152+
# - shell: bash -l {0}
153+
# run: |
154+
# conda info
155+
# conda list
156+
# - name: Modif Rmd echo status
157+
# run: |
158+
# python build/modifmd.py
159+
# - name: Build to md
160+
# run: |
161+
# git diff --name-only origin/master origin/${GITHUB_HEAD_REF} >> diff.txt
162+
# cat diff.txt
163+
# Rscript -e 'source("./build/build_light.R")'
164+
# - name: Clean files with Python function
165+
# run: |
166+
# rm -rf "./temp"
167+
# mkdir -p temp
168+
# mkdir -p corrections
169+
# python build/cleanmd.py
170+
# - name: Convert in ipynb with Jupytext
171+
# run: |
172+
# pip install jupytext
173+
# cd ./temp
174+
# for i in $(find . -type f \( -iname "*.Rmd" \)); do \
175+
# j="${i%.Rmd}.md" ;
176+
# echo "$j" ;
177+
# if [ -e $j ]
178+
# then
179+
# echo "Converting $j"; \
180+
# #jupytext --to py --execute "$i"
181+
# jupytext --to ipynb "$j" ;
182+
# k="${j%.md}.ipynb" ;
183+
# fi
184+
# done
185+
# mkdir -p ../corrections
186+
# cp -R . ../corrections
187+
# cd ..
188+
# - uses: actions/upload-artifact@v2
189+
# with:
190+
# name: Corrections
191+
# path: corrections/
192+
# - name: Commit new notebooks
193+
# if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
194+
# env:
195+
# BRANCHE_REF: ${{ github.event.pull_request.head.ref }}
196+
# run: |
197+
# git config user.name 'github-actions[bot]'
198+
# git config user.email 'github-actions[bot]@users.noreply.github.com'
199+
# # git config user.email github-actions@github.com
200+
# #git checkout ${BRANCHE_REF}
201+
# git status
202+
# git add corrections/**/*.ipynb;
203+
# git commit -m "Automated changes";
204+
# git pull
205+
# git push
206+

‎.github/workflows/prod.yml‎

Lines changed: 26 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -10,51 +10,47 @@ jobs:
1010
blogdown:
1111
name: Render-Blog
1212
runs-on: ubuntu-latest
13-
container: linogaliana/python-datascientist:latest
14-
if: "!contains(github.event.commits[0].message, '[skip ci]')"
13+
container: linogaliana/python-datascientist-vstudio:quarto
1514
steps:
16-
- uses: actions/checkout@v1
15+
- uses: actions/checkout@v2
1716
with:
1817
submodules: true
19-
- name: Install rmarkdown
20-
env:
21-
GITHUB_PAT: ${{ secrets.PAT_GITHUB }}
22-
run: |
23-
Rscript -e 'remotes::install_github("yihui/xfun")'
24-
Rscript -e 'remotes::install_github("yihui/knitr")'
25-
Rscript -e 'install.packages(c("rmarkdown", "here"))'
26-
- name: install hugo
27-
run: Rscript -e 'blogdown::install_hugo("0.83.0", force = TRUE)'
28-
- name: Install Python
29-
run: |
30-
Rscript -e 'print(blogdown:::hugo_version())'
31-
Rscript -e "install.packages(c('remotes', 'reticulate'))"
32-
Rscript -e "install.packages(c('here'))"
33-
- shell: bash -l {0}
18+
fetch-depth: 0
19+
ref: ${{ github.event.pull_request.head.ref }}
20+
repository: ${{github.event.pull_request.head.repo.full_name}}
21+
- name: Configure safe.directory # Workaround for actions/checkout#760
22+
run: git config --global --add safe.directory /__w/python-datascientist/python-datascientist
23+
- shell: bash
3424
run: |
3525
conda info
3626
conda list
37-
- name: Render blog
27+
- name: Build to md
3828
run: |
39-
Rscript -e 'source("./build/build.R")'
40-
- name: Create
29+
quarto render --to hugo
30+
mv content/course/manipulation/index.md content/course/manipulation/_index.md
31+
mv content/course/visualisation/index.md content/course/visualisation/_index.md
32+
mv content/course/modelisation/index.md content/course/modelisation/_index.md
33+
#git diff --name-only origin/master origin/${GITHUB_HEAD_REF} >> diff.txt
34+
#cat diff.txt
35+
#Rscript -e 'source("./build/build_light.R")'
36+
hugo mod graph
37+
hugo -D --themesDir themes -t github.com
38+
- name: Clean files with Python function
4139
run: |
42-
rm -rf "./temp"
43-
mkdir -p temp
44-
mkdir -p notebooks
4540
pip install matplotlib wordcloud nltk
46-
python build/cleanmd.py
4741
python build/wc_website.py
48-
- name: Render blog
42+
python build/post-build.py
43+
- name: Copy notebooks in dedicated directory
4944
run: |
50-
Rscript -e 'source("./build/netlify.R")'
45+
mkdir -p notebooks
46+
python build/move_files.py notebooks
5147
- name: Install npm
52-
if: ${{ github.repository == 'linogaliana/python-datascientist' }}
48+
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
5349
uses: actions/setup-node@v2
5450
with:
5551
node-version: '12'
5652
- name: Deploy to Netlify
57-
if: ${{ github.repository == 'linogaliana/python-datascientist' }}
53+
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
5854
# NETLIFY_AUTH_TOKEN and NETLIFY_SITE_ID added in the repo's secrets
5955
env:
6056
NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
@@ -63,6 +59,7 @@ jobs:
6359
run: |
6460
npm init -y
6561
npm install --unsafe-perm=true netlify-cli -g
62+
netlify init
6663
netlify deploy --prod --dir="public" --message "Deploy master"
6764
- uses: actions/upload-artifact@v1
6865
with:

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,3 +141,5 @@ pip-selfcheck.json
141141
# End of https://www.toptal.com/developers/gitignore/api/pycharm,venv
142142
.Rproj.user
143143
.Rhistory
144+
145+
/.quarto/

‎_quarto.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
project:
2+
title: "python-datascientist"
3+
pre-render: utils.py
4+
render:
5+
- content/**/*.qmd
6+
7+
validate-yaml: false
8+
jupyter: python3
9+
10+
wrap: preserve
11+
keep-ipynb: true
12+
cache: true
13+
14+
format:
15+
hugo:
16+
html-math-method:
17+
method: mathjax
18+
ipynb: default
19+
20+
#bibliography: references.bib
21+

0 commit comments

Comments
 (0)