Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ImageUrlField is causing an error #287

Closed
ghost opened this issue Mar 26, 2022 · 13 comments
Closed

ImageUrlField is causing an error #287

ghost opened this issue Mar 26, 2022 · 13 comments
Labels

Comments

@ghost
Copy link

ghost commented Mar 26, 2022

when I try to create a category with an image I get the following error:

Environment:`

Request Method: PUT
Request URL: http://127.0.0.1:8000/api/admin/categories/8/

Django Version: 3.2.12
Python Version: 3.8.10
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.sites',
'django.contrib.flatpages',
'oscar.config.Shop',
'oscar.apps.analytics.apps.AnalyticsConfig',
'oscar.apps.checkout.apps.CheckoutConfig',
'oscar.apps.address.apps.AddressConfig',
'oscar.apps.shipping.apps.ShippingConfig',
'oscar.apps.catalogue.apps.CatalogueConfig',
'oscar.apps.catalogue.reviews.apps.CatalogueReviewsConfig',
'oscar.apps.communication.apps.CommunicationConfig',
'oscar.apps.partner.apps.PartnerConfig',
'oscar.apps.basket.apps.BasketConfig',
'oscar.apps.payment.apps.PaymentConfig',
'oscar.apps.offer.apps.OfferConfig',
'oscar.apps.order.apps.OrderConfig',
'oscar.apps.customer.apps.CustomerConfig',
'oscar.apps.search.apps.SearchConfig',
'oscar.apps.voucher.apps.VoucherConfig',
'oscar.apps.wishlists.apps.WishlistsConfig',
'oscar.apps.dashboard.apps.DashboardConfig',
'oscar.apps.dashboard.reports.apps.ReportsDashboardConfig',
'oscar.apps.dashboard.users.apps.UsersDashboardConfig',
'oscar.apps.dashboard.orders.apps.OrdersDashboardConfig',
'oscar.apps.dashboard.catalogue.apps.CatalogueDashboardConfig',
'oscar.apps.dashboard.offers.apps.OffersDashboardConfig',
'oscar.apps.dashboard.partners.apps.PartnersDashboardConfig',
'oscar.apps.dashboard.pages.apps.PagesDashboardConfig',
'oscar.apps.dashboard.ranges.apps.RangesDashboardConfig',
'oscar.apps.dashboard.reviews.apps.ReviewsDashboardConfig',
'oscar.apps.dashboard.vouchers.apps.VouchersDashboardConfig',
'oscar.apps.dashboard.communications.apps.CommunicationsDashboardConfig',
'oscar.apps.dashboard.shipping.apps.ShippingDashboardConfig',
'widget_tweaks',
'haystack',
'treebeard',
'sorl.thumbnail',
'easy_thumbnails',
'django_tables2',
'django.contrib.sitemaps',
'django_extensions',
'debug_toolbar',
'oscarapi',
'rest_framework']
Installed Middleware:
['debug_toolbar.middleware.DebugToolbarMiddleware',
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.middleware.http.ConditionalGetMiddleware',
'django.middleware.common.CommonMiddleware',
'oscar.apps.basket.middleware.BasketMiddleware',
'oscarapi.middleware.HeaderSessionMiddleware']

Traceback (most recent call last):
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
response = self.handle_exception(exc)
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
self.raise_uncaught_exception(exc)
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
raise exc
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/rest_framework/generics.py", line 285, in put
return self.update(request, *args, **kwargs)
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/rest_framework/mixins.py", line 67, in update
serializer.is_valid(raise_exception=True)
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/rest_framework/serializers.py", line 227, in is_valid
self._validated_data = self.run_validation(self.initial_data)
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/rest_framework/serializers.py", line 426, in run_validation
value = self.to_internal_value(data)
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/rest_framework/serializers.py", line 483, in to_internal_value
validated_value = field.run_validation(primitive_value)
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/rest_framework/fields.py", line 568, in run_validation
value = self.to_internal_value(data)
File "/home/mhb/Documents/django-oscar/venv/lib/python3.8/site-packages/oscarapi/serializers/fields.py", line 332, in to_internal_value
http_prefix = data.startswith(("http:", "https:"))

Exception Type: AttributeError at /api/admin/categories/8/
Exception Value: 'InMemoryUploadedFile' object has no attribute 'startswith'

can someone also tell me what is the purpose of this ImageUrlField and why it replaces the rest_frameworks ImageField?

@maerteijn
Copy link
Member

It is there so you can also provide an external url so oscarapi will download this instead of providing the image itself.

Can you provide the request data as well?

@maerteijn
Copy link
Member

@tofubit See my question above

@Jimmar
Copy link

Jimmar commented Jun 13, 2022

I'm facing the same issue too, data in this context seem to be a InMemoryUploadedFile type object (or TemporaryUploadedFile if it was more than 2.5MB).
neither of which has the method startWith.

I had a working api before adding django-oscar and django-oscar-api and by just adding them it stared to crash on this same line.

I'm sending the request image as a form data from my frontend code

const formData = new FormData();

const headers = { "Content-Type": "multipart/form-data" };
formData.append("img", blob, file_name);

return await $axios.post(url, formData, { headers });

if I print the request.data inside django-oscar-api I get this

<QueryDict: {'img': [<InMemoryUploadedFile: photo-1598411072028-c4642d98352c.jpeg (image/jpeg)>]}>

note that I'm also using django-storages to upload the images to s3.

is there a way to at least disable this override ?

@maerteijn if there are any more info you'd like I'd be happy to provide them.

@maerteijn
Copy link
Member

Can you provide the url you are posting to (url in your example), the complete contents offormData and the Django traceback?

@Jimmar
Copy link

Jimmar commented Jun 14, 2022

I couldn't share the project that I was working on, but I created a minimal project that shows this issue and deployed it on heroku.

https://oscar-api-test.herokuapp.com/mapi/store-image/
if you go to this url and use the DRF form, you can get the same error

the project is here
https://github.com/Jimmar/oscarApiTest

if I just remove the line

path("api/", include("oscarapi.urls")),

from urls.py then this issue goes away and the upload works fine

@specialunderwear
Copy link
Member

the ImageUrlField is there because there is no multipart form-data in a json api. So you can't upload images you have to put a url of an image there, oscarapi will download the image from that url.

@specialunderwear
Copy link
Member

Maybe you are trying to use oscarapi to build a javascript frontend for the admin maybe? In that case you need to implement your own api for uploading images. the admin api is build to synchronise data to oscar from an external application via json/rest.

@Jimmar
Copy link

Jimmar commented Jun 14, 2022

@specialunderwear I'm not even trying to use oscar api yet.

I currently have an API endpoint that works fine to upload images, just adding oscar api to the project breaks that.
I'm not trying to use oscar api to upload images.

in the project that I just linked, if you removed that line from the urls.py then the image upload works fine.

@maerteijn
Copy link
Member

@specialunderwear I'm not even trying to use oscar api yet.

I currently have an API endpoint that works fine to upload images, just adding oscar api to the project breaks that. I'm not trying to use oscar api to upload images.

in the project that I just linked, if you removed that line from the urls.py then the image upload works fine.

@Jimmar So what you are saying is that by just installing / enabling django-oscar-api, the regular Django Rest Framework ImageField gets broken?

@Jimmar
Copy link

Jimmar commented Aug 8, 2022

@specialunderwear I'm not even trying to use oscar api yet.
I currently have an API endpoint that works fine to upload images, just adding oscar api to the project breaks that. I'm not trying to use oscar api to upload images.
in the project that I just linked, if you removed that line from the urls.py then the image upload works fine.

@Jimmar So what you are saying is that by just installing / enabling django-oscar-api, the regular Django Rest Framework ImageField gets broken?

basically yes, doesn't to work well with formData, the serializer is expecting data to be a string url but it's actually being handled as an InMemoryUploadedFile (or TemporaryUploadedFile if it was more than 2.5MB).

if you take a look at this comment that I wrote here
#287 (comment)

it has a link to a sample project that I made to showcase this issue and a url to test it (deployed on Heroku)

@maerteijn maerteijn added the bug label Aug 9, 2022
@maerteijn
Copy link
Member

maerteijn commented Aug 9, 2022

Ok confirmed,

This is because we patch rest framework to have the ImageUrlField as a default field for models.ImageField. This happens in the OscarSerializer and is implicitly monkey patching DRF:

def expand_field_mapping(extra_fields):
    # This doesn't make a copy
    field_mapping = serializers.ModelSerializer.serializer_field_mapping   # <-- monkeypaych
    field_mapping.update(extra_fields)
    return field_mapping


class OscarSerializer(object):
    field_mapping = expand_field_mapping(
        {
            oscar.models.fields.NullCharField: serializers.CharField,
            models.ImageField: ImageUrlField,
        }
    )

This, of course, we shouldn't do.

Fastest solution for now: explicity define the DRF ImageField on the serializer while we think of a solution.

@maerteijn
Copy link
Member

@Jimmar We committed a fix to the main branch. Could you install the package from github in your local environment:

pip install git+https://github.com/django-oscar/django-oscar-api.git

and verify this fix works for your scenario?

@Jimmar
Copy link

Jimmar commented Aug 11, 2022

@Jimmar We committed a fix to the main branch. Could you install the package from github in your local environment:

pip install git+https://github.com/django-oscar/django-oscar-api.git

and verify this fix works for your scenario?

I confirm that this fixes the issue, thank you !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants