What is the recommended workflow to check completion codes?

[smonsays]

Smile does not to record the completionCode to Firebase which I assume is due to risk of it being manipulated by a participant. This means we have to rehash participants' data to compute their completion codes later on for verification against recruitment services. I am wondering what the recommended workflow would be here? Would it make sense to automatically compute completion codes in download_data.mjs for instance?

[gureckis]

Yeah, that feature is kind of up to you. Smile doesn't store the code itself because the idea is that it can act as some kind of validation. So for instance, if you poke around inside the Prolific code, you can see that the code that gets passed back on completion to Prolific is a hash of the entire dataset from the subject. Using the same logic, you could in your analysis script hash the data the same way. If the subjects compeltion code matched you hash you at least know that they actually did the task and didn't make up a code.

[smonsays]

Yes that makes sense, thank you! The way the code is currently structured leads me to think that I would have to (a) reimplement computeCompletionCode since the version in useAPI.js relies on internal state (this.store.data) and (b) do this in javascript (as opposed to in python) to make sure the hashing algorithm works exactly the same. It might worth considering to slightly refactor computeCompletionCode to not rely on state, then it could just be reused in another script. Currently I am reimplementing it as follows inside of download_data.mjs:

const computeCompletionCode = (docData) => {
  const pageDataFields = {}
  for (const key in docData) {
    if (key.startsWith('pageData_')) {
      pageDataFields[key] = docData[key]
    }
  }

  let dataToHash
  if (Object.keys(pageDataFields).length > 0) {
    dataToHash = stringify(pageDataFields)
  } else {
    dataToHash = stringify(docData.studyData)
  }

  const hashDigest = Base64url.stringify(sha256(dataToHash))

  let endCode = ''
  if (docData.withdrawn) {
    endCode = 'xx'
  } else if (docData.done) {
    endCode = 'oo'
  }
  return hashDigest.slice(0, 20) + endCode
}

[gureckis]

Yes, internally we have made a dashboard which lets us rehash the data and compare the submitted code but that feature is still in development. When done it should make life easier than download data files. Of course you can copy the javascript logic and run it on your dataset after download data files. Or the python equivalent might output the same hash. Seems something a coding agent would love to help with! :)

[smonsays]

Alright, if the feature is on the roadmap with the dashboard, I'll roll my own implementation in the mean time. Thanks a lot for your help :)