Skip to content

Issue 52827: File/attachment fields with special characters#6580

Merged
XingY merged 15 commits intodevelopfrom
fb_issue52827
May 1, 2025
Merged

Issue 52827: File/attachment fields with special characters#6580
XingY merged 15 commits intodevelopfrom
fb_issue52827

Conversation

@XingY
Copy link
Copy Markdown
Contributor

@XingY XingY commented Apr 17, 2025

Rationale

  • "multipart/form-data" is used when submitting a html form with file, instead of "application/x-www-form-urlencoded". When submitted as form, form parameters with " are converted to %22 since " is a special character for URL. This is resulting in the parameter being ignored since it cannot be matched to an existing table column.
  • :: is used as delimiter for file parameters. The logic to extract parts from the delimiter is flawed in that it doesn't account for fields that contains such substring.

Related Pull Requests

Changes

  • Added MatchType.mutiPartFormData to relax column matching with encoded "
  • fix logic to extract field name from file parameters

XingY added 2 commits April 16, 2025 21:16
…ins tricky characters (unclosed " or containing ::)
# Conflicts:
#	assay/src/org/labkey/assay/actions/ImportRunApiAction.java
@XingY XingY requested review from cnathe and labkey-matthewb April 28, 2025 18:37
if (!fieldCommandIndex.equals(commandIndex+""))
continue;
}
else
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the correct fix is to remove the encoded characters here (early) rather than trying to match them later. Ideally the encoded chars shouldn't "leak" out of code that is pulling values out of the request object.

Then we wouldn't have to handle these characters later.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The challenge here is that a field might actually contain "%22". Both field" and field%22 will come in as field%22 and we don't know when to decode vs not here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤯

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really an HTTP bug (it's not OK to escape " to %22 without escaping %). So this would break any column with " in its name in a type with a file column then. Not just the file column.

Since these columns could never map to a java property. I'd propose changing BaseColumnInfo.getPropertyName() to replace " with %QUOTE% or something. That should solve the problem by never using the " in the input name. That would fix the dataregion case, anyway. I'm not sure what the editable grid does.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Editable grid in apps currently doesn't support file fields. The paths supported in Apps are editing detail editing and bulk ediitng. Detail editing uses updateRows api with formData (file, plus json data for other row data) while bulk update uses saveRows. For Apps, only the file field is impacted since other fields are in json. This approach of replace " with %QUOTE% could work for Apps too, but it's also more error prone. Also, it's hard to pick a substitute that absolutely won't collide with real field names, a field might just actually contain %QUOTE%.

Comment thread api/src/org/labkey/api/dataiterator/DataIteratorUtil.java Outdated
Copy link
Copy Markdown
Contributor

@labkey-matthewb labkey-matthewb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can use your approach, but I would add a longer explanation about the problem with the form encoding. We might also consider removing double-quote from generated property name. BaseColumnInfo.getPropertyName().

@XingY XingY changed the base branch from develop to release25.4-SNAPSHOT April 30, 2025 20:05
@XingY XingY changed the base branch from release25.4-SNAPSHOT to develop April 30, 2025 20:05
cnathe and others added 3 commits April 30, 2025 13:09
…126, assay unresolved sample lookup issue 52745) (#6609)

- AssayResultUpdateService to call syncLineage() on assay result delete rows when sample lookup column exists
- ExprColumn null check for various sample lookup fields (IsAliquot, IsPlated, StorageStatus)
# Conflicts:
#	api/src/org/labkey/api/exp/property/DomainUtil.java
Comment thread api/src/org/labkey/api/data/TableViewForm.java Outdated
Comment on lines +583 to +595
MultipartHttpServletRequest request = (MultipartHttpServletRequest) getRequest();
MultipartFile f = request.getFile(quoteEncodedFieldName);
isFileColFileRemoved = f != null && (f.getOriginalFilename() == null || f.getOriginalFilename().isEmpty());
}

if (isFileColFileRemoved)
values.put(column.getName(), null);
else
{
Object value = getTypedValues().get(quoteEncodedFieldName);
if (value != null)
values.put(column.getName(), value);
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems like a lot of duplicate code from what is already there below for handling the non-slash column name case (lines 599-607). it seems like the main part is the field name to use in request.getFile(). Could these blocks of code be collapsed and have the conditional part be using either getMultiPartFormFieldName or getFormFieldName? Then after we have the MultipartFile, the rest of the code seems pretty similar.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was how it was written in a previous commit, but that broke LKS attachment update. The different part is what value to use for file field: string or File. The only shared code is the following 2 lines so I removed the shared util in the last commit.
MultipartHttpServletRequest request = (MultipartHttpServletRequest) getRequest(); MultipartFile f = request.getFile(quoteEncodedFieldName);

Comment thread api/src/org/labkey/api/dataiterator/DataIteratorUtil.java
@XingY XingY merged commit 71a9268 into develop May 1, 2025
2 of 5 checks passed
@XingY XingY deleted the fb_issue52827 branch May 1, 2025 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants