Add support for upserting multiple document types #96
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #62
This PR adds support for text content in addition to files and URLs to the
Document
model.Previously, the API required users to upload a file for every document. Now, users can either upload a file or provide text content for the document. The
Document
model now has a new fielddocument_type
which specifies the type of theDocument
. It can now be either a file, URL or text.Additionally, a new field
text_content
has been added to the model which is used to store the text content of the document whendocument_type
is set to text or the URL of the webpage whendocument_type
is set to url.The
DocumentSerializer
has been updated to include the new fields when serializing aDocument
object. It now also checks if eitherfile
ortext_content
is provided during creation and raises a validation error if neither field is present.The
DocumentViewSet
has also been updated to use thedocument_type
field when uploading a new document. It now calls theupsert()
function with the appropriate parameters depending on thedocument_type
of the uploadedDocument
.A new library
beautifulsoup4
has been added to support web scraping and parsing when providing a URL as thedocument_type
. This is required for parsing the HTML of the website.Finally, the
SPECTACULAR_SETTINGS
setting has been added to displayFileField
as a file upload input in the Swagger UI.Tested with all the document types,
file
,url
andtext
.Upserting a pdf file: sample.pdf
Upserting https://react.dev/reference/react/useEffect :
Upserting raw text: