Downloading using ee_to_csv slow/breaks

[Akramz]

Hello,

I've been using geemap for my work recently, really enjoying it :)

The only issue that I'm facing is around downloading processed datasets. I usually try to export data in CSV using the ee_to_csv function but it just hangs in Please wait ...

When I try to split the downloading into multiple exports, it randomly hangs in some requests. Here is an example of one attempt (daily climate reductions over 438 US counties):

def year_aggregates(year, roi, counties):
    """Builds features representing county-aggregates for a specific `year` for an `imageCollection`.

    # Params
        year (int): the year of interest.
        roi (ee.FeatureCollection): the area of interest for filtering (from: `FAO/GAUL/2015/level1`).
        counties (ee.FeatureCollection): the counties of interest (from: `TIGER/2018/Counties`).

    # Returns
        features (ee.FeatureCollection): county-aggregates of temperature and precipitation.
    """
    
    # Set the spatio-temporal filter
    start_date = f"{year}-04-01"
    end_date = f"{year}-11-01"
    
    # Filter PRISM's collection using our filters
    imgs = ee.ImageCollection("OREGONSTATE/PRISM/AN81d")\
             .select(["ppt", "tmean", "tmin", "tmax"])\
             .filterBounds(roi)\
             .filterDate(start_date, end_date)
    
    # Map to create daily county-aggregates
    features = imgs.map(lambda img: img.reduceRegions(counties, ee.Reducer.mean(), scale=4000)\
                        .map(lambda f: f.set("date", img.date()))).flatten()
    return features

# Set the temporal range
years = list(range(2018, 2021))

# Select feature of interest
cols = ["NAME", "STATEFP", "COUNTYFP", "date", "ppt", "tmean", "tmin", "tmax"]

for year in years:
    
    # Get the daily-scale county-aggregated data
    aggs = year_aggregates(year, states, counties)
    
    # Download it
    geemap.ee_to_csv(aggs, f"data/daily_{year}.csv", cols)
    
    # Log!
    print(f"YEAR {year} DOWNLOADED!")

I'm looking for ways to download data more efficiently. Do you have any recommendations?

Thank you,

[giswqs]

This is more likely a computation issue rather than a download issue. It seems you are doing some computationally intensive analysis at the global scale. If the year_aggregates() function fails to generate the resulting FeatureCollection for a certain year, the ee_to_csv function will certainly fail to download it. One way to test this is to add the resulting FeatureCollection to the map. If it can't be added to the map or used with .getInfo(), it can't be downloaded either. If this is the case, consider reducing the computation by using a larger scale value and simpler geometries. FAO countries and US Census data are complicated geometries with numerous vertices, which can often cause computation timeout or memory issues.

[giswqs]

You can also check out the zonal statistics examples.

• https://geemap.org/notebooks/12_zonal_statistics
• https://geemap.org/notebooks/13_zonal_statistics_by_group