Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suwinet prefill #586

Closed
VerbeekIT opened this issue Aug 23, 2021 · 2 comments · Fixed by #3265 or #3490
Closed

Suwinet prefill #586

VerbeekIT opened this issue Aug 23, 2021 · 2 comments · Fixed by #3265 or #3490

Comments

@VerbeekIT
Copy link

VerbeekIT commented Aug 23, 2021

Als burger van de gemeente wil ik graag dat mijn aanvraagformulieren zoveel mogelijk voor ingevuld worden. Voor het ophalen van data bij de Belastingdienst of bij Suwinet, moet ik hiervoor een juist subOIN meegeven met de serviceaanroep. De aangeroepen servive kan dan controleren of dit verzoek om uitwisseling van data gehonoreerd mag worden.

There are many different APIs in Suwinet:

  • Bijstandsregelingen
  • BRPDossierPersoonGSD
  • DUODossierPersoonGSD
  • DUODossierStudiefinancieringGSD
  • GSDDossierReintegratie
  • IBVerwijsindex
  • KadasterDossierGSD
  • RDWDossierDigitaleDiensten
  • RDWDossierGSD
  • SVBDossierPersoonGSD
  • UWVDossierAanvraagUitkeringStatusGSD
  • UWVDossierInkomstenGSDDigitaleDiensten
  • UWVDossierInkomstenGSD
  • UWVDossierQuotumArbeidsbeperktenGSD
  • UWVDossierWerknemersverzekeringenGSDDigitaleDiensten
  • UWVDossierWerknemersverzekeringenGSD
  • UWVWbDossierPersoonGSD

The responses can contain many nested keys. For example, a person can have multiple salaries + benefits (so the responses would correspond to "arrays of objects" if turned into JSON).

This is problematic because the form designer does not know in advance how much data will come back.

To discuss:

  • How will the form designer configure how a component will be prefilled?
  • How are we going to deal with "prefilling repeating groups"?
  • How do we deal with the many APIs of Suwinet?
  • Do we need to do extra processing of the data? For example, convenience method to calculate the total income or something.
@SilviaAmAm SilviaAmAm changed the title Als burger van de gemeente wil ik graag dat mijn aanvraagformulieren zoveel mogelijk voor ingevuld worden. Voor het ophalen van data bij de Belastingdienst of bij Suwinet, moet ik hiervoor een juist subOIN meegeven met de serviceaanroep. De aangeroepen servive kan dan controleren of dit verzoek om uitwisseling van data gehonoreerd mag worden. Suwinet prefill Jul 11, 2023
@SilviaAmAm SilviaAmAm self-assigned this Jul 11, 2023
@joeribekker
Copy link
Contributor

joeribekker commented Jul 13, 2023

Refinement: For now, we will just get the data exposed as used-defined variable in Open Forms. You can select "Suwinet" as Prefill plugin. And as attribute, you can select an operation (one of the 16 SOAP-services, which all have 1 operation).

Schermafbeelding 2023-07-13 110234

...that can be used in Content-components:

Schermafbeelding 2023-07-13 110049

The SOAP-calls require 2 attributes: BSN and period. The BSN can be obtained from DigiD but the date is a different matter. I think it's unavoidable to create "settings" for prefill attributes but maybe these settings can be determined during the form... so it takes variables again.

Let's postponed this last issue and start with:

  • Add Suwinet plugin
  • Create 16 SOAP-services and expose them as "plugin attribute"
  • Objectify the SOAP-responses so they are accessible as JSON-path in the variable (ie. inkomens.0.waarde)
  • Determine the datatype automatically based on the selected attribute (probably they are all lists or objects).
  • Make the Suwinet plugin available only to the user defined variable prefill plugin list, and not the component prefill plugins

SilviaAmAm added a commit that referenced this issue Jul 18, 2023
SilviaAmAm added a commit that referenced this issue Jul 18, 2023
SilviaAmAm added a commit that referenced this issue Jul 18, 2023
sergei-maertens added a commit that referenced this issue Jul 18, 2023
…-services

[#586] Move SOAP services to own module
@SilviaAmAm SilviaAmAm reopened this Jul 19, 2023
@joeribekker joeribekker assigned CharString and unassigned SilviaAmAm Jul 25, 2023
@sergei-maertens sergei-maertens added this to the Release 2.3.0 milestone Jul 26, 2023
ErhanCitil pushed a commit to maykinmedia/open-forms that referenced this issue Aug 7, 2023
ErhanCitil pushed a commit to maykinmedia/open-forms that referenced this issue Aug 7, 2023
ErhanCitil pushed a commit to maykinmedia/open-forms that referenced this issue Aug 7, 2023
@joeribekker
Copy link
Contributor

Refinement: We scoped this ticket to only work with a few of the Suwinet (DKD) requests that can work as prefill, meaning, it can get the information based on the BSN provided by a DigiD login. Specifically, we target the income-related calls.

The calls that require (so, when not optional) additional inputs (start date, end date, postal code, etc.) are out of scope. These could be tackled later but require an additional mechanism that doesn't fit prefilling. Rather, it requires "if all inputs are available" then perform the call.

CharString added a commit that referenced this issue Oct 16, 2023
CharString added a commit that referenced this issue Oct 18, 2023
CharString added a commit that referenced this issue Oct 18, 2023
Trusting the main prefill integration test the binding to variables.
And this testing of plugin api is enough.
Also dropped the failing test.

There is something wonky going on resource leakage due to, I think,
threading (requests in a threadpool)/multi processing. Running the whole
test suite in reverse with 2 testrunners will trip it sooner than what
origin/master shows. raising to 11 runners and random order, lowers the
chance of tripping it.

I'm hitting the postgres connection limit. And it's unclear to me if
this is just due to the way we run the test or if this could happen in
production. (no signs of it in Sentry of our OF instances)

I'm giving up, kicking the pebble up the road to #3242. Might run a
bisect this weekend, when I don't need my machine.
CharString added a commit that referenced this issue Oct 18, 2023
Trusting the main prefill integration test the binding to variables.
And this testing of plugin api is enough.
Also dropped the failing test.

There is something wonky going on resource leakage due to, I think,
threading (requests in a threadpool)/multi processing. Running the whole
test suite in reverse with 2 testrunners will trip it sooner than what
origin/master shows. raising to 11 runners and random order, lowers the
chance of tripping it.

I'm hitting the postgres connection limit. And it's unclear to me if
this is just due to the way we run the test or if this could happen in
production. (no signs of it in Sentry of our OF instances)

I'm giving up, kicking the pebble up the road to #3242. Might run a
bisect this weekend, when I don't need my machine.
CharString added a commit that referenced this issue Oct 21, 2023
CharString added a commit that referenced this issue Oct 23, 2023
About the keymaterial from a path in .env: should we consider
using the keychain package, so each platform can use their proper key
storage?
CharString added a commit that referenced this issue Oct 23, 2023
About the keymaterial from a path in .env: should we consider
using the keychain package, so each platform can use their proper key
storage?
CharString added a commit that referenced this issue Oct 23, 2023
- An unresponsive service should not hang the client.
CharString added a commit that referenced this issue Oct 23, 2023
Uses create for the Certificate fields because FactoryBoy build strategy
results in unreadable file fields and zeep wsse reads the key and cert
material.
CharString added a commit that referenced this issue Oct 23, 2023
CharString added a commit that referenced this issue Oct 23, 2023
About the keymaterial from a path in .env: should we consider
using the keychain package, so each platform can use their proper key
storage?
CharString added a commit that referenced this issue Oct 23, 2023
Uses create for the Certificate fields because FactoryBoy build strategy
results in unreadable file fields and zeep wsse reads the key and cert
material.
CharString added a commit that referenced this issue Oct 23, 2023
CharString added a commit that referenced this issue Oct 23, 2023
About the keymaterial from a path in .env: should we consider
using the keychain package, so each platform can use their proper key
storage?
CharString added a commit that referenced this issue Oct 23, 2023
CharString added a commit that referenced this issue Oct 23, 2023
About the keymaterial from a path in .env: should we consider
using the keychain package, so each platform can use their proper key
storage?
CharString added a commit that referenced this issue Oct 27, 2023
CharString added a commit that referenced this issue Oct 27, 2023
About the keymaterial from a path in .env: should we consider
using the keychain package, so each platform can use their proper key
storage?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment