Enketo submissions don't work for forms with space in formid that are in projects with project-level encryption

[lognaturel]

+4da6d37bb30dd44716ef68f279669672d279189a server (v1.1.1-138-g4da6d37)

1. In a project with project-level encryption, create a form with a formid with spaces (e.g. Geotrace and geoshape
2. Try to make an Enketo submission (e.g. here)
3. See Submission service on data server not found. (404)

I'm pretty sure this is related to the spaces in formid and that it's a regression. Another interesting thing about this form is that it has a blank version but I've tried another form with a blank version without problem.

I get the same behavior with a published form on v1.1.1 but on a draft on v1.1.1 I get a different error message "Unexpected form id value Geotrace_and_shape; did not match the form ID in the URL"

[lognaturel]

Likely root cause of https://forum.getodk.org/t/odk-central-change-to-encrypted-form-once-already-started-data-collection-breaks-submissions/33082

[issa-tseng]

i wrote a test that does all of the above except use enketo, and it passes. so it must be something enketo is doing. but i can't debug it without access to that server to see the enketo logs.

    it.only('should not fail given encryption and spaces in the name', testService((service, { all }) =>
      service.login('alice', (asAlice) =>
        asAlice.post('/v1/projects/1/key')
          .send({ passphrase: 'test' })
          .expect(200)
          .then(() => asAlice.post('/v1/projects/1/forms?publish=true')
            .set('Content-Type', 'application/xml')
            .send(testData.forms.simple.replace('id="simple"', 'id="sim ple"'))
            .expect(200))
            .then(() => asAlice.get('/v1/projects/1/forms/sim%20ple')
              .expect(200)
              .then(({ body }) => asAlice.post('/v1/projects/1/submission')
                .set('X-OpenRosa-Version', '1.0')
                .attach('xml_submission_file', Buffer.from(testData.instances.simple.one
                  .replace('id="simple"', `id="sim ple" version="${body.version}"`)), { filename: 'data.xml' })
                .expect(201))))));

[issa-tseng]

my best guess enketo is overzealously encoding spaces somewhere?

[issa-tseng]

@lognaturel does this line mean anything to you?
https://github.com/enketo/enketo-express/blob/3f353ea44934045e185db71e26ec331146ec2610/public/js/src/module/controller-webform.js#L268

[issa-tseng]

and in particular why it differs from line 261 above which references form.instanceID. i would bet the difference here is the problem.

because the instanceID is taken as xml attribute text, i would not expect to see it url-escaped ever.

[lognaturel]

Line doesn't mean anything to me but I agree it looks like a very likely culprit. It's definitely something Enketo-specific. I can't see anything obviously related in issues for either enketo-express or enketo-core.

I looked at logs and all that's in there since the last restart is ERROR: Submissions element(s) not supported. They're not verbose logs.

From nginx: "POST /v1/projects/275/submission HTTP/1.1" 404 183 "https://test.getodk.cloud/-/mNI13cGzUkxcIfvQOgheKuWvocUjauz"

Nothing useful in the redis DB I don't think:

127.0.0.1:6379> hgetall id:mNI13cGzUkxcIfvQOgheKuWvocUjauz
 1) "openRosaServer"
 2) "https://test.getodk.cloud/v1/projects/275"
 3) "openRosaId"
 4) "Geotrace and shape"
 5) "submissions"
 6) "0"
 7) "launchDate"
 8) "2021-04-26T18:10:37.413Z"
 9) "active"
10) "true"
11) "theme"
12) ""
13) "lastAccessed"
14) "2021-04-26T20:59:06.759Z"

You can ssh ubuntu@test.getodk.cloud if you want more.

I think next steps are to file an Enketo issue and/or try locally.

[issa-tseng]

i logged on but.. couldn't find where anything is. the thing i would like to see is enketo trying to talk to central, which i don't see in here. (edit: and actually what i'd need to see is the payload of that request which won't be in a log anywhere.)

[lognaturel]

There's no logging to that effect other than at the nginx layer. So I can't think of a way to figure out what formID Enketo is trying to use without doing it locally.

[lognaturel]

Closing in favor of enketo/enketo#1079 since there's no action to take on the Central end.