Calculation of net_value

[c4rl0sm3nd3s]

As discussed on Telegram, there are cases where the net_value is zero but when compared on the result from the website, it shows a value different from zero.

For example, this case: http://www.camara.gov.br/cota-parlamentar/documento?nuDeputadoId=1074&numMes=6&numAno=2016&despesa=5&cnpjFornecedor=14415344000176&idDocumento=0000000068

[c4rl0sm3nd3s]

One way to fix that is to calculate the net_value. First compare the reimbursement_value, if it is not zero, check if the net_value is equals to zero, then calculate the difference between reimbursement_value with document_value.

[cuducos]

Great topic, thanks, @c4rl0sm3nd3s.

@Irio, any advice on how to proceed in terms of best practices for data science projects? Just updating the dataset seems quite aggressive in terms of reproducibility — does adding calculated_net_value to the big CSV make sense?

[c4rl0sm3nd3s]

maybe use temporary variable during the process, no need to save it in the CSV.

[pmargreff]

Hi, I bit in the same problem, the columns value aren't all consistent. You can create a new column and fill that in one line using Julia map function.

The result is something like:

table[:calculated_net_value] = map((x,y) -> x - y, table[:document_value], table[:reimbursement_value])

[cuducos]

Just as an option, I would suggest asking the Lower House for clarification. They have a contact form that generates a protocol (we can make some pressure with the protocol if they don't reply) and (from my experience) they are very thoughtful in responding to the public.

[Irio]

Today I am pairing with @cuducos to solve this issue.

There are a few things we already discovered as sources of these inconsistencies:

1. Flight tickets issue for multiple passengers (where a document corresponds to the expense for a single person) and ticket compensation (when the airline gives a voucher in specific cases).
2. A single document being reimbursed in more than 1 payment (check http://jarbas.datasciencebr.com/#/document_id/5914504. 2 payments for the expense).

[cuducos]

Fixed in YYYY-MM-DD-reimbursements.xz dataset ; )