Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
OSDI action handler and server for import and people, questions, answers, messages #1166
This PR contains an OSDI action handler to submit canvass responses like Question Answers, Tags (aka in VAN terms Survey Question Responses, Activist Codes). The handler will automatically download the available questions/SQs and tags/ACs and show them in the interaction script editing form as actions.
This also contains OSDI Server functionality that lets other OSDI clients read data from Spoke in OSDI format, as well as import contacts with batch mode.
This pull request is more than 300 lines of code. This is due to the functionality added, as well as incorporating several rounds of feedback, and customer requests.
@joemcl @lperson I ended up creating a more expansive OSDI implementation. In addition to supporting people import and collection browsing, I mapped spoke interaction steps, question_response into OSDI Questions and Answers. I also expose the messages collection, assignments, and users. as well as the relationship links between them.
In addition to automating import, this can be used to simplify integrations moving Q&A data into other systems like VAN et al. A script in a nightly cron job could suck down new answers, associated people, and do the mapping between the spoke Q&A and the VAN survey questions, or other logic and push them into VAN et al.
I set up a test server with some sample data and a public API for people to play with here:
Browse to http://spoke.dev.joshco.org/osdi
While looking at the HAL browser, try the following walkthrough
More docs here: https://github.com/joshco/Spoke/blob/osdi_key/docs/OSDI.md
I created a free script that automates moving recent question_responses into VAN with a customization mapping between Spoke and VAN Q&A. You can put it in a nightly cron job on your Spoke server to update VAN on the day's work.
schuyler1d left a comment
I haven't reviewed the entire API surface yet, but stopped with some early feedback.
This PR creates a fantastic external API that will help other programs interface with Spoke -- it's breadth is excellent, but also means we'll have to be very careful about the additional security surface it exposes.
@schuyler1d Thanks for the kind remark. I've reviewed your feedback and agree. I've also discussed the issue with other OSDI stakeholders, and have a proposal for next steps.
The current pull request includes some functionality for vision's sake. It also includes the bypass and public API switches which were meant to reduce friction for those who wanted to experience it for themselves using the demo server. I'll remove those, or refactor it as skyler suggested.
The current PR also goes beyond the settled resources in OSDI based on my own improvisations of assignments and messages. I'll remove these resources and produce an updated, more constrained PR, incorporating Sky and others' feedback. This way we could have working code this cycle.
In parallel, settling on the definition of what these new resources should be a democratic consensus among OSDI stakeholders, with leadership from those who build products that implement these features.
Augustus Franklin @augfrank , from CallHub (which also implements similar features) has offered to serve as a convener of an OSDI a breakout group on these resources. It would be great if someone from Spoke would participate as well as Hustle, Relay and any other implementers.
Latest update. @schuyler1d @ibrand I've added the global ENV var, improved the settings dialog. I've also removed the git submodule for the osdi browser, and instead snapshotted the browser code and am using webpack copy plugin. This will put it in the webpack, and also drop it in the assets directory which will be picked up by the S3 upload script.
@joemcl Thanks for passing this along; I doubt my response will surprise you. Approaches like this that encourage the use of proprietary APIs disproportionately harm the most marginalized communities in the progressive movement. The result is vendor lock-in where vendors will win sales based on avoiding the higher cost of integration with other products, rather than the merits of the product itself. This helps the big get bigger due to network effect. Larger customer organizations, or larger campaigns (like a presidential or $12m statewide gay marriage campaign) can overcome these costs with money or volunteer brute-force. However, smaller and more marginalized campaigns and organisations will not be able to do this as easily, and will stick with the larger "suite" type vendors. This further disadvantages products from marginalized innovators, because the additional cost of integrating those products will dis-incentivize customers from choosing one of these products based on merit and solidarity.
Given the recent consolidation in our marketplace, IMHO this will make our problems worse not better.
I've updated the branch with 2 OSDI test cases.
One weird thing I'm seeing in the tests is that intermittently, I'm getting an error in test_helpers. It seems to happen when I run the tests for the first time after time has passed. But then it works.
@schuyler1d Any ideas why? Who's the most familiar with the tests?
@schuyler1d @ibrand @joemcl I've added outbound OSDI (like action handlers) for interaction steps to submit survey question answers or apply tags/activist codes. By default, it works with VAN / EveryAction unless a different AEP is configured.
I'm looking for dogfooders who would like to give it a spin.
From the updated documentation:
Outbound OSDI Actions
Outbound OSDI is configured with these environment variables
By default, if you configured an API TOKEN but no AEP, Spoke will push to VAN / EveryAction.
Once configured, when you are editing your interaction steps, Spoke will download the available survey questions and tags (aka Activist Codes in VAN)
In the drop down for actions, you'll see options to apply tags/activist codes or submit responses to survey questions.
Dropdown of Choices
By default, when pushing via OSDI, Spoke will invoke the
Once matched, Spoke will invoke
Spoke will also save the remote OSDI Identifier in a contact custom field named
To improve matching, when you import into spoke, include
If you are importing contacts that you got from the OSDI system itself (eg VAN), include the OSDI Identifier itself (eg VANID) in a custom field named
# Conflicts: # __test__/test_helpers.js # app.json # jest.config.js # src/api/schema.js # src/components/CampaignInteractionStepsForm.jsx # src/containers/Settings.jsx # src/lib/index.js # src/lib/zip-format.js # src/server/api/organization.js # src/server/api/schema.js # src/server/index.js # src/server/middleware/render-index.js # src/workers/jobs.js # webpack/config.js
So I deployed @joshco 's
@joemcl I did some testing with the other CRM and made an update which pertains to you. You'll want to pull this.
@joemcl added odata date filtering, so queersync can be more efficient. I've also updated the civi extension to improve the AEP and PSH endpoint.
Updated queersync to run as signup only (when survey questions aren't needed in other systems)
You should now be able to have it autopush to VAN via interaction steps/scripts, and sync nightly with Civi via queersync.
Improvements based on customer testing and feedback to make it easier to use and set up. Thanks to @joemcl
@lperson Can we update the google doc script importer so the customer can put a token for an