Accelerator 1st Meetup MVP discussion

charlottebrf edited this page Aug 25, 2016 · 1 revision

Challenges

Wednesday 24th August 2016

  • Define parameters for data (e.g. collect data from date since Cameron announced referendum up to referendum result)
  • Aim to automate as much of the data collection as possible, difficulty of inconsistent files & formats (CSV & PDF for example)
  • Paperless app (Daniel) to put PDF into folder
  • Sunlight Foundation could be a useful resource
  • Issue of consistency of tags - manual process, human subjectivity
  • Stack Overflow could be useful guide for adding tags & tag alias & merge tags
  • Gathering data one of biggest tasks. Not very interesting task, good to have the whole team involved in this.
  • Natural language processing could be useful to make gathering less manual
  • Key challenge is the data collection, the technical challenge after this would be to reformat CSV to use clean formatted data present an HTML table using Javascript,then D3/ C3 force chart - could take approximately 1 month.
  • Do we still continue to use Neo4J? Or just switch entirely to D3/ C3 force chart
  • Need to link new team up with Eli for technical insights on work already done

Epics for phase 1 (3 months)

  • Work out how to get data from meetings into the correct format (what can be automated, what has to be done manually)
  • Small sample of data (5-10 meetings) to create a common format to apply to all data
  • Collect all data on meetings in consistent format
  • Define categories for tags
  • Present HTML tables
  • Create D3 or C3 Force chart