Update the GraphQL queries with new Cassandra schema #87
Conversation
`conjunctiontopics` tuple will always be in alphabetical order for records for the main edge is represented as `(isis, None, None)` then conjunction selected of `isis->car->fear` is represented as `(car,fear,isis)` isis->car` is represented as `(car,isis,None)`
This is the backend change for the following frontend change: CatalystCode/project-fortis-interfaces#15
This is the backend change for the frontend change in CatalystCode/project-fortis-interfaces#16
This is the backend change for the frontend change in CatalystCode/project-fortis-interfaces#18
This is the backend change for the frontend change in CatalystCode/project-fortis-interfaces#19
c-w
force-pushed
the
implement-tiles-resolver
branch
from
94bbd52 to
8ea73a2
Compare
By adding non-queryable columns to the events table that contain all of an events places and topics, we can re-build a full event just from the data that we get back from that table. This makes a number of queries easier and more performant, including the MessagesSchema.event query.
See rationale in comment CatalystCode/project-fortis-pipeline#95 (comment)
| cassandraConnector.executeQuery(query, params) | ||
| .then(rows => { | ||
| const keywords = makeSet(rows.map(row => row.topics), topic => topic); | ||
| const keywords = makeSet(rows, row => row.topic); |
There was a problem hiding this comment.
Rather than const keywords maybe it would be clearer to use const topics so there's one term for topic rather than a topic's name either being keyword or topic?
erikschlegel
left a comment
erikschlegel
left a comment
There was a problem hiding this comment.
Looks great. Sorry for the delay. Please opine to the comments below.
| AND periodtype = ? | ||
| AND pipelinekey = ? | ||
| AND externalsourceid = ? | ||
| AND conjunctiontopics = ? |
There was a problem hiding this comment.
we've switched conjunctivetopic1, conjunctivetopic2, conjunctivetopic3
| AND externalsourceid = ? | ||
| AND event_time <= ? | ||
| AND event_time >= ? | ||
| `.trim(); |
There was a problem hiding this comment.
we should probably use LIMIT here, as this result set can be large. We only care about the events that display on the news feed which are pages of 15 events.
| SELECT * | ||
| FROM fortis.events | ||
| WHERE pipelinekey = ? | ||
| AND eventid IN ? |
There was a problem hiding this comment.
apparently the limit of items you can include in a C* IN clause is 65535. We should be fine as long as we cap the number of filtered eventid based on a LIMIT threshold from ^^^.
| const eventsParams = [ | ||
| toPipelineKey(args.sourceFilter), | ||
| eventIds, | ||
| `%${args.fulltextTerm}%` |
| AND periodtype = ? | ||
| AND pipelinekey = ? | ||
| AND externalsourceid = ? | ||
| AND conjunctiontopic1 = ? |
There was a problem hiding this comment.
I know this was a change for CatalystCode/project-fortis-pipeline@f869c20, but conjunctiontopic1 doesn't seem so clear to me just with the 1 appended. I see that it is derived from the first parameter mainEdge in toConjunctionTopics, so maybe there could be a better name for this? But then doing so conjunctiontopic2 and conjunctiontopic3 would have to be changed as well to reflect a change in name of conjunctiontopic1.
There was a problem hiding this comment.
This is one for @erikschlegel. I don't particularly care about the naming here because we have quite a few things in the database that require documentation already.
Just to get us started, still some things to implement (e.g. extra data needs to get wired from the frontend).