protocol Jason.Encoder not implemented for %Plug.Upload

[thiagogsr]

Hello

I'm upgrading my Phoenix to 1.4 and it is raising error in controllers that I use %Plug.Upload{}:

** (Protocol.UndefinedError) protocol Jason.Encoder not implemented for %Plug.Upload{content_type: "jpg", filename: "bar.jpg", path: "/foo/bar.jpg"}, Jason.Encoder protocol must always be explicitly implemented.

Shouldn't it be implemented by the plug or we should define it in my app using Protocol.derive(Jason.Encoder, Plug.Upload)?

[OvermindDL1]

How would you implement Plug.Upload into a Jason.Encoder, it is a structure that is pointing to an entirely temporary file on the filesystem that will be wiped at the end of the request. I'm not sure how you would serialize that into JSON in a way that would make sense?

[josevalim]

Plug doesn't know anything about Jason. So it is either up to your app to do it. You can implement it yourself with the semantics you find better, if you can find any, per the reasons mentioned by @OvermindDL1.

[thiagogsr]

The problem occurs when I try to test my controller.

test "returns the image url when the upload has successful", %{conn: conn} do
  upload = %Plug.Upload{path: "/foo/bar.jpg", filename: "bar.jpg", content_type: "jpg"}
  conn = post(conn, image_path(conn, :create), %{"image" => upload})

  assert %{"data" => %{"link" => "url_to_image"}} = json_response(conn, 200)
end

I guess the test try to encode the upload variable. I just changed it to send a simple map and it worked 👍

[OvermindDL1]

@thiagogsr For a test like that I wouldn't use an endpoint test, I'd test the controller straight, which would accept the structure fine and it would more properly and specifically test the specific functionality that the test is trying to test for instead of testing the whole pipeline. :-)

[thiagogsr]

@OvermindDL1 thank you. I'm not sure if I know how to do it, I'm using mox library to mock the upload and I'm testing if the response is the expected. Couldn't you send some example or doc url?

[OvermindDL1]

@thiagogsr I'd recommend asking about how to handle that test best on the ElixirForums, you'll get more detailed and better answers there. It doesn't really belong in this github issue and I don't want to pollute it. :-)

[ngdelcastillo]

I also got this error when I'm using Arc in my Phoenix app for uploading files. I was able to fix it by adding this code within my Phoenix app:

  # lib/jason_encoder.ex

  require Protocol
  Protocol.derive(Jason.Encoder, Plug.Upload)

[OvermindDL1]

@ngdelcastillo Protocol.derive(Jason.Encoder, Plug.Upload) sounds exceptionally dangerous because Plug.Upload is pointing to a very transient file on the filesystem itself and thus becomes invalid immediately after the request ends. I use Arc with no issue and I would highly highly recommend not having that line.

[ngdelcastillo]

I see.. thanks for that @OvermindDL1 . This is a weird issue that happened to me recently. My app was working before using Arc. But when I did some db migrations that includes using postgis it suddenly appeared.

** (exit) an exception was raised:
    ** (Protocol.UndefinedError) protocol Jason.Encoder not implemented for %Plug.Upload{content_type: "image/jpeg", filename: "dd.jpg", path: "/var/folders/pf/sf8lnprx11j14sgkdspqym100000gn/T//plug-1550/multipart-1550244451-968929486672964-1"}, Jason.Encoder protocol must always be explicitly implemented.

If you own the struct, you can derive the implementation specifying which fields should be encoded to JSON:

    @derive {Jason.Encoder, only: [....]}
    defstruct ...

It is also possible to encode all fields, although this should be used carefully to avoid accidentally leaking private information when new fields are added:

    @derive Jason.Encoder
    defstruct ...

Finally, if you don't own the struct you want to encode to JSON, you may use Protocol.derive/3 placed outside of any module:

    Protocol.derive(Jason.Encoder, NameOfTheStruct, only: [...])
    Protocol.derive(Jason.Encoder, NameOfTheStruct)
. This protocol is implemented for: Ecto.Association.NotLoaded, Ecto.Schema.Metadata, Date, BitString, Jason.Fragment, Any, Map, NaiveDateTime, List, Integer, Time, DateTime, Decimal, Atom, Float
        (jason) lib/jason.ex:199: Jason.encode_to_iodata!/2
        (postgrex) lib/postgrex/type_module.ex:713: Postgrex.DefaultTypes.encode_params/3
        (postgrex) lib/postgrex/query.ex:62: DBConnection.Query.Postgrex.Query.encode/3
        (db_connection) lib/db_connection.ex:1088: DBConnection.encode/5
        (db_connection) lib/db_connection.ex:1186: DBConnection.run_prepare_execute/5
        (db_connection) lib/db_connection.ex:1282: DBConnection.run/6
        (db_connection) lib/db_connection.ex:504: DBConnection.parsed_prepare_execute/5
        (db_connection) lib/db_connection.ex:497: DBConnection.prepare_execute/4
        (postgrex) lib/postgrex.ex:161: Postgrex.query/4
        (ecto_sql) lib/ecto/adapters/sql.ex:657: Ecto.Adapters.SQL.struct/10
        (ecto) lib/ecto/repo/schema.ex:651: Ecto.Repo.Schema.apply/4
        (ecto) lib/ecto/repo/schema.ex:264: anonymous fn/15 in Ecto.Repo.Schema.do_insert/3

Maybe I'll just backtrack what caused of it. I wondered why the error message suggested to use the Protocol.derive....

[OvermindDL1]

I wondered why the error message suggested to use the Protocol.derive....

Jason assumes whatever was passed in was 'meant' to be turned into JSON, so it tries to be helpful. :-)

As always, this is better discussed on the elixir forums, but from what I see:

        (jason) lib/jason.ex:199: Jason.encode_to_iodata!/2
        (postgrex) lib/postgrex/type_module.ex:713: Postgrex.DefaultTypes.encode_params/3

This implies that the upload passed into an ecto schema somewhere, though without the rest of the stacktrace I couldn't tell you where. Check those first and see if you are accidentally putting an upload structure into an ecto schema that is then submitted into the database. An upload schema definitely should not end up in the database since it is a very temporary file definition (that single request only, it would be like uploading the Conn into the database, doesn't make sense).

If you are wanting to take an upload structure to take the on-disk file that it points to and save it somewhere permanent and then store 'that' link into the database then you can do that manually, or use arc_ecto that automates 'most' of it (unless you need to do a few specific things that it doesn't support but they are easy enough to work around). :-)