Data Scraping all years

[lopezpedres]

The last number in the link represents the school id. There are 4 schools:
15,25,35,45
Here is the link for 2021:
https://www.dgae.unam.mx/Licenciatura2021/resultados/15.html
Here is the link for 2017:
https://www.dgae.unam.mx/Febrero2017/resultados/35.html
Here is the link for 2016:
https://servicios.dgae.unam.mx/Febrero2016/resultados/35.html
Here is the link for 2015:
https://servicios.dgae.unam.mx/Febrero2015/resultados/15.html
Here is the link for 2014:
https://servicios.dgae.unam.mx/Febrero2014/resultados/15.html
Here is the link for 2013:
https://servicios.dgae.unam.mx/Febrero2013/resultados/25.html

[mate-h]

Data sources from 1996 to present, HTML format:

https://web.archive.org/web/19970329111029/http://www.dgae.unam.mx/
https://www.dgae.unam.mx/admision/
https://web.archive.org/web/*/http://www.dgae.unam.mx*

Archival statistical agenda data source from 1959 to present, PDF format:
http://agendas.planeacion.unam.mx/

[mate-h]

Collected all the available links from the Web Archive here:
https://storage.googleapis.com/mate-h.appspot.com/archive-links.csv

[mate-h]

Need to gather all of working links. Link example:

https://www.dgae.unam.mx/Licenciatura2021/resultados/1/10400035.html

Schematic link:

https://www.dgae.unam.mx/:term/resultados/:areaId/:facultyId:buildingId.html
https://servicios.dgae.unam.mx/:term/resultados/:areaId/:facultyId:buildingId.html

1. step one is to scrape all of the identifiers for:

• Terms
• Areas
• Facilties
• Buildings

2. Construct all of the possible URLs using the ids and the link schema above. Link schema version depends on the term.
3. Download the one by one using lynx and discard the documents that were not resolved. Take all of the tags into account (Special link in 1996 and Archive links, Servicio subdomain, etc.)
4. Write scripts to process that data.