Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include information about download endpoint #12

Open
niwla23 opened this issue Sep 18, 2022 · 1 comment
Open

Include information about download endpoint #12

niwla23 opened this issue Sep 18, 2022 · 1 comment

Comments

@niwla23
Copy link

niwla23 commented Sep 18, 2022

There is another way to fetch data from the API as CSV. This is probably more useful for some usecases. The endpoint URL is https://www.smard.de/nip-download-manager/nip/download/market-data . It takes only Post requests with this body:

{
	"request_form": [
		{
			"format": "CSV",
			"moduleIds": [
				2000122,
				2005097,
				2000715,
				2000125,
				2003791,
				2000123
			],
			"region": "DE",
			"timestamp_from": 1663452000000,
			"timestamp_to": 16634916000000,
			"type": "discrete",
			"language": "de"
		}
	]
}

You can get the moduleIds from the "Daten herunterladen" Page on smard.de . For some weird reason only the moduleIds used by the download form there work together. For example,. you can't combine consumption and production data into one request.

Maybe you want to include this endpoint in the docs?

I have also built a currently incomplete (supports only production, forecast and consumption) go client for it: https://github.com/niwla23/smard-go

EDIT:
these are the id sets that work together:

productionModuleIds := []int{1001224, 1004066, 1004067, 1004068, 1001223, 1004069, 1004071, 1004070, 1001226, 1001228, 1001227, 1001225}
productionForecastIds := []int{2000122, 2005097, 2000715, 2000125, 2003791, 2000123}
consumptionModuleIds := []int{5000410, 5004387, 5004359}
@ThomasRegier
Copy link

I'm usiung this strategy as well. Some more useful stuff which I realized quick and dirty. As well as the moduleIds for Trade (import / Export) and Market (prices / kwh).

def get_time_in_mil(date_end = dt.datetime.now(),days = 8):
    """Gets time_start and time_end in milli seconds for downloads from the page: 'https://www.smard.de/nip-download-manager/nip/download/market-data'. Download will be for data from today backwards for the number of days specified in 'days' """
    days_dif = (date_end - dt.datetime(2022, 8, 1)).days
    # time in milli seconds. 1 / 1000 seconds.
    a = 1659304800000 # 1.8.22 day start
    b = 1659391199999 # 1.8.22 day end
    day_length = 60**2*24*1000 # day_length = (b-a+1)
    
    time_start = a + (days_dif - days + 1) * day_length
    time_end = a + days_dif * day_length + day_length - 1
    return time_start, time_end

time_start, time_end = get_time_in_mil(date_end = dt.datetime.now(),days = 3653)

url = 'https://www.smard.de/nip-download-manager/nip/download/market-data'

moduleIds_market = [8004169,8004170,8000251,8005078,8000252,8000253,8000254,8000255,8000256,8000257,8000258,8000259,8000260,8000261,8000262,8004996,8004997]

moduleIds_trade = [22004629,22004722,22004724,22004998,22004712,22004404,22004409,22004545,22004546,22004548,22004550,22004551,22004552,22004405,22004547,22004403,22004406,22004407,22004408,22004410,22004412,22004549,22004553]

moduleIds=moduleIds_trade
payload = {"request_form":[{"format":"CSV","moduleIds":moduleIds,"region":"DE","timestamp_from":time_start,"timestamp_to":time_end,"type":"discrete","language":"de"}]}
  
y = requests.post(url, json = payload_market)
with open(f'energy_market.csv', 'wb') as f:
    f.write(y.content)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants