Document application architecture and interfaces

[cecilia-donnelly]

• psm-II-4.2 The PSM shall document all interfaces in an Interface Control Document (ICD), along with how those interfaces are maintained.
• psm-SA-4.3 The PSM design documents shall utilize a widely supported modeling language (e.g., UML, BPMN).
• psm-IU-4.3 The PSM shall provide an architecture diagram depicting how it is technically structured.
• psm-SA-3.1 The PSM shall support the architecture adopted to preserve the ability to efficiently, effectively, and appropriately exchange data with other participants in the health and human services enterprise.
• psm-IA-4.1 The PSM shall support a Logical Data Model (LDM) in the identification of data classes, attributes, relationships, standards, and code sets for intrastate exchange.

I'm grouping these together for now, but we can certainly split them into different issues as needed.

[brainwane]

I'm beginning to work with Umbrello, read UML basics, and figure out whether we need to use UML in order to make/support an LDM.

[brainwane]

Basically as I see it we have two choices regarding the Interface Control Document (ICD) & Logical Data Model (LDM) requirements. We can either write them manually and have a design or release checklist to ensure we keep code and documentation in sync, or we can figure out how to generate them from the PSM (in the case of the LDM, from Postgres). I'm poking at these possibilities now.

[brainwane]

@jasonaowen I see a bunch of options mentioned (e.g., in https://stackoverflow.com/questions/3223770/tools-to-generate-database-tables-diagram-with-postgresql and https://softwarerecs.stackexchange.com/questions/34552/generate-database-table-diagrams-from-a-postgresql-database & https://wiki.postgresql.org/wiki/GUI_Database_Design_Tools ) to generate diagrams directly from Postgres. If you could take 10 min to look through those and identify 2-3 for me to test out (excluding ones that are unworkable with our version of Postgres, or use plugins we wouldn't want to touch), I'll test them out to see whether they'd suit our needs for the LDM.

[cecilia-donnelly]

@brainwane, it looks like those Postgres-generated options are ERD-like. An ERD is something we've been looking into because @chj124 asked about it, so thank you for re-raising it!

[brainwane]

@cecilia-donnelly here's my take and some things I'd love the stakeholders to help us with.

• psm-II-4.2 The PSM shall document all interfaces in an Interface Control Document (ICD), along with how those interfaces are maintained.

An ICD would be meant to document and track possible inputs and outputs for the PSM. It would describe data interchange between the application and the database, and between the application and other applications (via application programming interfaces). It would NOT describe data interchange within the PSM (between and among different components of the application).

https://www.cms.gov/Research-Statistics-Data-and-Systems/CMS-Information-Technology/XLC/Downloads/InterfaceControlDocument.docx is the template we would use for this, unless told otherwise.

• psm-IA-4.1 The PSM shall support a Logical Data Model (LDM) in the identification of data classes, attributes, relationships, standards, and code sets for intrastate exchange.

An LDM would represent the specific entities, attributes, and relationships involved in a business function’s view of information, per the guidance at https://www.cms.gov/Research-Statistics-Data-and-Systems/CMS-Information-Technology/DataAdmin/LogicalDataDesign.html and based, as much as possible, on the Enterprise Logical Data Model (ELDM). Therefore we need CMS's ELDM to fulfill this requirement.

[brainwane]

@cecilia-donnelly and I just spoke. A few things we figured out/decided/shared:

• Right now we will basically say that "architecture diagram" is equivalent to "LDM," in terms of what we need to deliver to fulfill this requirement, and that "architecture diagram" here means a boxes-and-arrows diagram plus necessary prose annotations. Also, our focus in the architecture diagram will not be "you're a developer and here's what you'd want to install/change" -- rather it's "here's the flow of data through the PSM, from user input through the application to the database, from APIs like our LEIE interface, and to users via the front end or via APIs/export".

• It's useful for Sumana to think of a Logical Data Model, in general, as a sort of dictionary of possible objects and relationships, a structured data reference.

• It's useful for Sumana to think of the audience for the architecture diagram and for the ICD as Project Poplin, to help them build their reference architecture.

• "Enterprise" stuff in CMS, as in the reference I found to an ELDM, is more for overall frameworks/architectures that have lots of modules in them. We do not have an Enterprise LDM and will proceed on the understanding that we are creating an LDM that Project Poplin and others will use, along with other LDMs, to gather and create a larger LDM for multistate use.

Next up for me (basically my task through the rest of today): see what I can autogenerate.

[brainwane]

Today I discussed, a little, the possibility of autogenerating UML or similar diagrams from our code -- @jasonaowen has reservations about how duplicative and uninformative it might be to, for instance, generate this sort of illustration from our Javadoc. Something like Class Visualizer might be worth investigating.

Also today I tried autogenerating what I could in terms of our database -- objects and relationships. I used pgmodeler.

The version of pgmodeler available in Debian currently does not support Postgres 9.6, so, with a lot of help from @jasonaowen, I installed from source.

Gotchas:

• remember that you'll need to at some point, you know, get the source :)
• you may need a lot of dependencies in order to build this thing from source! If you're like me and rarely build from source, use apt build-dep pgmodeler to get dependencies installed first.
• get around the "wait, which qt, 4 or 5?!" by telling qmake to use Qt 5 when you invoke it: qmake -qt=qt5 pgmodeler.pro
• Jason suggested I not run that final make install step, and just run pgmodeler from the local directory. The start-pgmodeler.sh script seemed to get confused about where the binary lived and where the $INSTALL_ROOT was, and running source pgmodeler.vars did not entirely fix this. So I made a pgmodeler subdirectory and ran INSTALL_ROOT=/home/sumanah/test/pgmodeler/pgmodeler make install. Further fiddling required export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/sumanah/test/pgmodeler/pgmodeler/usr/local/lib/pgmodeler after which this worked:

~/test/pgmodeler/pgmodeler/usr/local$ ./bin/pgmodeler

• pgmodeler will want the psm Postgres user's password, which, if you've set it up this way, is in ~/.pgpass
• if you run into strange window-scaling display issues on Linux, check this Qt invocation which helped me:

$ QT_AUTO_SCREEN_SCALE_FACTOR=0 QT_SCALE_FACTOR=1 ./bin/pgmodeler

[brainwane]

Here's what I was able to output from pgmodeler -- first by default, then by asking it to arrange things hierarchically. It outputs SQL, SVG, and PNG -- not UML.

[jasonaowen]

Wow, I don't think I saw the full, exported version when we were working together last week! There's a lot going on there.

Note that our database includes all the jBPM tables, which are separate but alongside the tables used directly by the PSM. If this type of image is useful, it is probably worth generating two, one for the tables listed in seed.sql and one for the tables listed in jbpm.sql. I don't know how easy pgModeler makes that!

[cecilia-donnelly]

Hey, @cjh124, check out the diagrams @brainwane generated -- are those what you're looking for in an ERD?

Thanks so much for writing this up, @brainwane! Looks super helpful.

[chj124]

looks good first glance, let you know if i have any questions, thanks!