Feedback on the FTS feature/DDW

[daninmilani]

Hi team,

I hope you are doing well.

Over the last few days, I was updating the FTS coding lists in the DDW and downloading the Financial Tracking Service (precoded). I've only found two issues with the system:

1. I was not able to select the CVS in the Update Data Source. Edwin removed CSV check-in CSV to JSON converter yesterday, and this issue was sorted out (many thanks!).

2. Looking at a new download of the FTS (precoded) today, I noticed that the new Donors' codenames that I updated yesterday were not included in the most recent download. Do you know why it is happening? How long is the length of time between the upload of the coding lists and the update of the database?

I am still manually coding some lists, but, apart from the second issue, everything is working as expected.

Once again, many thanks,

Daniele.

[k8hughes]

It sounds like point 2 here is either a bug or a user issue - I think we need to start by seeing if we can reproduce the issue. @wakibi will look at this on Friday.

[edwinmp]

@daninmilani the upload (through the Update Data Source feature is immediate. Is the functionality mentioned in issue 2 supposed to be automated (part of the scheduled events)? The scheduled events feature is currently not functioning as expected, so automated updates are not being done. I'm dedicating tomorrow to fixing it.

[edwinmp]

@daninmilani there was FTS update yesterday... could you please test again

[edwinmp]

Hold that thought ... it seems adding aliases broke the table download APIs https://ddw.devinit.org/api/tables/download/fts_codenames/ ...

[daninmilani]

Hi @edwinmp. I downloaded a new database (FTS (precoded) database in the 'Query Builder') this morning and the second issue still persist. I am not able to update a new coding list in the 'Update Data Sources' now because when I click in 'download current data' the site shows the following message: 'Internal Server Error'.

Let me know when you fix the system and I check it again.

[edwinmp]

@wakibi the codenames updated through the Update Data Source feature don't reflect in their respective main data sources ... e.g A codename was added to the Queen Silvia Foundation donor, but when querying the FTS (Precoded) table, the codename is missing.

https://ddw.devinit.org/queries/data/646

[wakibi]

@edwinmp I am handling this with high priority

[wakibi]

This is a documentation issue. We need to run the fts_precode.sh and fts_diff.sh scripts every time an update is made to the coding lists. I shall work on updating the documentation tomorrow. @daninmilani I have run them manually, kindly download the dataset and confirm if it's working as expected. Cheers.