Automatically detect latest flight data available

[rafapereirabr]

ANAC adds new flight data sets to their website almost every month. Ideally, we would like to automatically detect the latest flight data available . There should be a way to list the files on ANAC url using curl or httr. Any help / contributions / suggestions would be much appreciated!

[rafapereirabr]

This function should do the trick of determining all dates available for flights data

#' Retrieve from ANAC website all dates available for flights data
#'
#' @return Numeric vector.
#' @keywords internal
#' @examples \dontrun{ if (interactive()) {
#' # check dates
#' a <- get_all_dates_available()
#'}}
get_all_dates_available <- function() {

  # read html table
  url = 'https://www.gov.br/anac/pt-br/assuntos/regulados/empresas-aereas/envio-de-informacoes/microdados/'
  h <- rvest::read_html(url)
  elements <- rvest::html_elements(h, "a")

  # filter elements of basica data
  basica_urls <- elements[elements %like% '/basica']
  basica_urls <- lapply(X=basica_urls, FUN=function(i){rvest::html_attr(i,"href")})

  # get all dates available
  all_dates <- substr(basica_urls, (nchar(basica_urls) + 1) -11, nchar(basica_urls)-4 )
  all_dates <- gsub("[-]", "", all_dates)
  all_dates <- as.numeric(all_dates)
  return(all_dates)
}