Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sunsetting Dropbox Forms #7378

Open
2 of 8 tasks
Betree opened this issue Apr 19, 2024 · 0 comments
Open
2 of 8 tasks

Sunsetting Dropbox Forms #7378

Betree opened this issue Apr 19, 2024 · 0 comments
Labels
api Issues that require some work on the API (https://github.com/opencollective/opencollective-api) task team Enhancements for the team to take on. technical-debt Deprecated code to migrate and other necessary refactors
Projects

Comments

@Betree
Copy link
Member

Betree commented Apr 19, 2024

Tasks

  1. 6 of 6
    Betree
  2. api task team
    Betree

To backup data from Dropbox Forms:

start_date="2019-01-01T00:00:00.000Z"
end_date="2019-05-01T23:59:59.000Z"
end_loop_date=$(date -u +"%Y-%m-%dT00:00:00.000Z")
interval="4 months"
workflows="&guid[]=MfmOZErmhz1qPgMp&guid[]=qdUbX5nw8sMZzykz&guid[]=MkFBvG39RIA61OnD"

# Download
while [[ "$start_date" < "$end_loop_date" ]]; do
  # Calculate the end date for this period
  period_end_date=$(date -d "$start_date + $interval" "+%Y-%m-%dT%H:%M:%S.000Z")

  # Call the curl command with the current date range
  # curl "http://my-url.com?start_date=$start_date&end_date=$period_end_date"
  curl --output result-$start_date-$period_end_date.zip "https://portal.helloworks.com/workflow/transactions/detail/?mode=live&$worflows&start_date=$start_date&end_date=$period_end_date"

  # Update the start date for the next period
  start_date=$(date -d "$period_end_date + 1 second" "+%Y-%m-%dT%H:%M:%S.000Z")
done

# Extract all CSV files from the zip files
for f in *.zip; do unzip -B -j "$f" "*.csv"; done

# Merge all CSV files into one
csvstack Open*.csv* >all.csv
@Betree Betree added team Enhancements for the team to take on. api Issues that require some work on the API (https://github.com/opencollective/opencollective-api) technical-debt Deprecated code to migrate and other necessary refactors task labels Apr 19, 2024
@Betree Betree added this to To do in Tax forms via automation Apr 19, 2024
@Betree Betree moved this from To do to In progress in Tax forms Apr 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Issues that require some work on the API (https://github.com/opencollective/opencollective-api) task team Enhancements for the team to take on. technical-debt Deprecated code to migrate and other necessary refactors
Projects
Tax forms
In progress
Development

No branches or pull requests

1 participant