New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Normalised USD values available for budgets but not for transactions #2219
Comments
Thanks @JoshStanleyDevInit - NB the values of other currencies in budgets all have the value |
@JoshStanleyDevInit - Is there going to be a way that users can know the exchange rates (and source) that were used for these normalisations - that tends to be the most common question that I get asked when showing d-portal normalised figures. If this were d-portal you could have a link to the source, but in the datastore I guess something could be inserted in the output and I guess also at least in the query builder (i.e. not just a 'select output currency' dropdown but a link and an explanation of the methodology, and a way to identify which data returned was originally in USD [edit: I see this already exists as the normalised data uses a new element - I guess the new element should be within the datastore specific namespace though]? I am worried that users might mistake the USD figure for the data published by the publisher? Also what is going to be done for amount that are in the future, or which span long time periods - is there a discussion on the methodology somewhere? Also, why stop at USD, if we are building in currency conversion, persumably many other currencies are easily done? |
As per slack conversation this issue needs to be reopened as it is not currently working as intended. |
@siemvaessen @luminhan can this issue be reopened? |
Adding in comment from other linked issue #2397: Once #2219 is done, both budgets and transactions are available in USD. The currency conversion is based on the IMF rates . We would like to add the actual conversion used for any budget and/or transaction in the element and a link to raw source for conversion (IMF). Example: Budget or transaction, original currency EUR, amount: EUR1.000 on 01/01/2015 (monthly average is used) I would expect the Budget or transaction to include raw EUR, converted USD amount, conversion nr + link to IMF currency data report. |
Spec for getting USD conversion into the Query Builder is here: https://docs.google.com/document/d/1f-BhnkvRsooIKo5COu9g-fYFbYK2tmvgewfiVsKGY1c/edit Impact assessment by Zimmerman is here: https://docs.google.com/document/d/1gKxQgjKctR9OA_9wJ-WsDCUXz0-Eujnj0XPgkYEvqw0/edit?usp=sharing Estimation is that this will take 9 days of work and can be completed by the end of the month. |
[FORMAL]: TOTALY DAYS REQUIRED: 9 DAYS (IMPLEMENTATION) AS PART OF PHASE 5. DELIVERY END OF MONTH INLCUDING NEW INDEXING/PARSE |
Testing on this work is complete. Happy with how the currency conversion is working. This can be deployed. |
Hi - how exciting - is there a sample of the output that we can see e.g. to setup parsers for the new USD conversion namespace? |
Hi @matmaxgeds, do you want to send a query and I'll return the test version? Note, that we've only got it tested on a handful of publishers so far. |
Any random single activity would be great @amy-silcock thanks! |
|
Thanks @amy-silcock - that looks very IATI 3.0! and I am always happy when the namespace element is being used - and this is a great use-case for it!
|
Good to hear your views @matmaxgeds. We'll be updating the user guide with the conversion details and methodology. @luminhan please do correct me if I'm wrong, the url is what's being used to programmatically grab the conversion rates. As you say, not so workable for users. Matt, if you've got a suggestion of another URL to use please do feel free to share. |
Cool, putting the actual rate returned by the IMF query would also be helpful The user guide sounds like what the namespace url should link to then, I don't think we can assume all users of the datastore (especially when they access IATI data via a third system that is using the datastore as a source) will know where to look to find this stuff out - hence if you provide the link, this can in turn be provided to the end user by the system consuming the data |
Is the exchange rate to 5 decimal points not enough? |
Definitely! Sorry that I missed it hidden in the opening element :) |
@matmaxgets The imf urls work well.
|
@siemvaessen please can we keep this open until it's working in live. Testing this morning and there are no USD conversions when using the Query Builder |
@amy-silcock could you let us know exactly what it is you were testing? USD conversion works right? Just tested it on production as the Fresh.Parse/Index finished yesterday. |
I've had another look and it's now there! I probs tested it a little too soon yesterday. I'll do a full test when I'm back in on Monday. |
We've tested this, the conversion is working as expected. |
With IATI publishers reporting budgets and transactions in various currencies, comparison and aggrgation is made much easier when the values are converted to a normalised currency (particularly USD).
When retrieving activity information, budgets are indeed converted into USD and other currencies.
However, the same conversion is not done for transactions.
See the following query as an example:
https://iati.cloud/search/activity?q=(title_narrative:%22COVID%22%20OR%20description_narrative:%22COVID%22%20OR%20humanitarian_scope_code:%22EP-2020-000012-001%22%20OR%20humanitarian_scope_code:%22HCOVD20%22%20OR%20tag_code:%22EP-2020-000012-001%22)&wt=json&rows=500000
Example Budget Result
Example Transactions Result
@markbrough feel free to add anything to this!
The text was updated successfully, but these errors were encountered: