Reimplement Pict-RS #43

tituspijean · 2023-06-19T22:22:28Z

Closes #31

Reimplements Pict-RS, while avoiding build time by extracting builds from Docker images, as for Lemmy and Lemmy UI themselves.

PR Status

Code finished and ready to be reviewed/tested
The fix/enhancement were manually tested (if applicable)

Automatic tests

Automatic tests can be triggered on https://ci-apps-dev.yunohost.org/ after creating the PR, by commenting "!testme", "!gogogadgetoci" or "By the power of systemd, I invoke The Great App CI to test this Pull Request!". (N.B. : for this to work you need to be a member of the Yunohost-Apps organization)

tituspijean · 2023-06-19T22:23:01Z

Status: file upload fails with error 500 and the following error:

(stacktrace enabled by adding Environment=RUST_BACKTRACE=full in /etc/systemd/system/lemmy-pict-rs.service)

# cat /var/log/lemmy/lemmy-pict-rs.log
2023-06-19T22:16:50.414812Z  INFO actix_server::builder: starting 4 workers
2023-06-19T22:16:50.414842Z  INFO actix_server::server: Actix runtime found; starting in Actix runtime
2023-06-19T22:16:53.532435Z  INFO HTTP request{http.method=POST http.route=/image http.flavor=1.1 http.scheme=http http.host=127.0.0.1:8081 http.client_ip=172.19.112.1 http.user_agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0 http.target=/image otel.name=HTTP POST /image otel.kind="server" request_id=169fc643-1d45-42ba-b5ac-bdf9850db875}: tracing_actix_web::root_span_builder: new
2023-06-19T22:16:53.630965Z  INFO HTTP request{http.method=POST http.route=/image http.flavor=1.1 http.scheme=http http.host=127.0.0.1:8081 http.client_ip=172.19.112.1 http.user_agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0 http.target=/image otel.name=HTTP POST /image otel.kind="server" request_id=169fc643-1d45-42ba-b5ac-bdf9850db875 trace_id=00000000000000000000000000000000 exception.message=Error interacting with filesystem exception.details=
   0: Error interacting with filesystem
   1: No such file or directory (os error 2)

Location:
   /drone/src/src/error.rs:38

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: pict_rs::magick::details_bytes with hint=None
      at src/magick.rs:135
   1: pict_rs::validate::validate_bytes
      at src/validate.rs:38
   2: pict_rs::ingest::ingest with declared_alias=None should_validate=true
      at src/ingest.rs:44
   3: pict_rs::file-upload with filename="ynh_logo_white_300dpi.png"
      at src/lib.rs:154
   4: tracing_actix_web::root_span_builder::HTTP request with http.method=POST http.route=/image http.flavor=1.1 http.scheme=http http.host=127.0.0.1:8081 http.client_ip=172.19.112.1 http.user_agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0 http.target=/image otel.name=HTTP POST /image otel.kind="server" request_id=169fc643-1d45-42ba-b5ac-bdf9850db875 trace_id=00000000000000000000000000000000
      at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-actix-web-0.7.5/src/root_span_builder.rs:41
   5: tokio::task::runtime.spawn with kind=local task.name= task.id=46 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-server-2.2.0/src/service.rs" loc.line=74 loc.col=17
      at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
   6: tokio::task::runtime.spawn with kind=local task.name= task.id=16 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-server-2.2.0/src/worker.rs" loc.line=457 loc.col=25
      at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
   7: tokio::task::runtime.spawn with kind=local task.name= task.id=12 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-server-2.2.0/src/worker.rs" loc.line=431 loc.col=21
      at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
   8: tokio::task::runtime.spawn with kind=local task.name= task.id=11 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-rt-2.8.0/src/arbiter.rs" loc.line=316 loc.col=25
      at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
   9: tokio::task::runtime.spawn with kind=block_on task.name= task.id=10 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-rt-2.8.0/src/arbiter.rs" loc.line=144 loc.col=24
      at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  <empty backtrace>

Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering. http.status_code=500 otel.status_code="ERROR"}: tracing_actix_web::root_span_builder: close time.busy=3.23ms time.idle=95.3ms

DrMxrcy · 2023-07-10T04:47:45Z

Status: file upload fails with error 500 and the following error:

(stacktrace enabled by adding Environment=RUST_BACKTRACE=full in /etc/systemd/system/lemmy-pict-rs.service)

# cat /var/log/lemmy/lemmy-pict-rs.log
2023-06-19T22:16:50.414812Z  INFO actix_server::builder: starting 4 workers
2023-06-19T22:16:50.414842Z  INFO actix_server::server: Actix runtime found; starting in Actix runtime
2023-06-19T22:16:53.532435Z  INFO HTTP request{http.method=POST http.route=/image http.flavor=1.1 http.scheme=http http.host=127.0.0.1:8081 http.client_ip=172.19.112.1 http.user_agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0 http.target=/image otel.name=HTTP POST /image otel.kind="server" request_id=169fc643-1d45-42ba-b5ac-bdf9850db875}: tracing_actix_web::root_span_builder: new
2023-06-19T22:16:53.630965Z  INFO HTTP request{http.method=POST http.route=/image http.flavor=1.1 http.scheme=http http.host=127.0.0.1:8081 http.client_ip=172.19.112.1 http.user_agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0 http.target=/image otel.name=HTTP POST /image otel.kind="server" request_id=169fc643-1d45-42ba-b5ac-bdf9850db875 trace_id=00000000000000000000000000000000 exception.message=Error interacting with filesystem exception.details=
   0: Error interacting with filesystem
   1: No such file or directory (os error 2)

Location:
   /drone/src/src/error.rs:38

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: pict_rs::magick::details_bytes with hint=None
      at src/magick.rs:135
   1: pict_rs::validate::validate_bytes
      at src/validate.rs:38
   2: pict_rs::ingest::ingest with declared_alias=None should_validate=true
      at src/ingest.rs:44
   3: pict_rs::file-upload with filename="ynh_logo_white_300dpi.png"
      at src/lib.rs:154
   4: tracing_actix_web::root_span_builder::HTTP request with http.method=POST http.route=/image http.flavor=1.1 http.scheme=http http.host=127.0.0.1:8081 http.client_ip=172.19.112.1 http.user_agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0 http.target=/image otel.name=HTTP POST /image otel.kind="server" request_id=169fc643-1d45-42ba-b5ac-bdf9850db875 trace_id=00000000000000000000000000000000
      at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-actix-web-0.7.5/src/root_span_builder.rs:41
   5: tokio::task::runtime.spawn with kind=local task.name= task.id=46 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-server-2.2.0/src/service.rs" loc.line=74 loc.col=17
      at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
   6: tokio::task::runtime.spawn with kind=local task.name= task.id=16 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-server-2.2.0/src/worker.rs" loc.line=457 loc.col=25
      at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
   7: tokio::task::runtime.spawn with kind=local task.name= task.id=12 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-server-2.2.0/src/worker.rs" loc.line=431 loc.col=21
      at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
   8: tokio::task::runtime.spawn with kind=local task.name= task.id=11 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-rt-2.8.0/src/arbiter.rs" loc.line=316 loc.col=25
      at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
   9: tokio::task::runtime.spawn with kind=block_on task.name= task.id=10 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-rt-2.8.0/src/arbiter.rs" loc.line=144 loc.col=24
      at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  <empty backtrace>

Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering. http.status_code=500 otel.status_code="ERROR"}: tracing_actix_web::root_span_builder: close time.busy=3.23ms time.idle=95.3ms

Getting the Same Error, Using local and S3 Paths.

ericgaspar · 2023-07-12T08:40:45Z

!testme

yunohost-bot · 2023-07-12T08:40:47Z

🚀

csolisr · 2023-07-17T14:08:42Z

Looks to me like this is missing some configuration files to be properly set - namely, the location of the folder for the uploads. As per the default settings, this defaults to /mnt/files.

scripts/install

ericgaspar · 2023-07-20T07:41:59Z

!testme

yunohost-bot · 2023-07-20T07:42:02Z

May the CI gods be with you!

csolisr · 2023-07-20T16:28:14Z

So! Is it working now with this fix?

yalh76

Very nice work

DrMxrcy · 2023-07-28T20:13:00Z

So! Is it working now with this fix?

Looks like it was approved!

csolisr · 2023-07-28T20:15:22Z

And just in time for the 0.18.3 update I guess! Can somebody merge it please?

ericgaspar · 2023-07-29T11:28:09Z

before merging, did someone tested this?

tituspijean · 2023-07-30T10:57:57Z

!testme

yunohost-bot · 2023-07-30T10:57:59Z

Alrighty!

tituspijean · 2023-08-08T19:09:56Z

If it could help ImageMagick was implemented and build in lemmy_ynh and removed at #30

Fixed by unpacking the AppImage :)

Ready to merge!

yalh76 · 2023-08-09T19:34:21Z

!testme
!bookwormtestme

yunohost-bot · 2023-08-09T19:34:23Z

📚 🪱

yunohost-bot · 2023-08-09T19:34:24Z

🚀

tituspijean · 2023-08-10T07:38:47Z

CI test on YunoHost 11 is experiencing the issue mentioned there: #69 (comment)

ericgaspar · 2023-12-06T16:32:15Z

!testme

yunohost-bot · 2023-12-06T16:32:19Z

Meow 🐈

Reimplement Pict-RS from Docker extract

2ff94ff

ericgaspar added 2 commits July 12, 2023 10:38

Merge branch 'testing' into pictrs

acbde41

Update _common.sh

5b20248

ericgaspar added 7 commits July 12, 2023 10:50

start pict-rs service

ae9fc4f

Update upgrade

2648c37

Update upgrade

ede495b

cleaning

26eb2c3

Update change_url

b927bd5

fix

883c2f0

Update upgrade

6e0b4fb

csolisr reviewed Jul 17, 2023

View reviewed changes

scripts/install Show resolved Hide resolved

ericgaspar added 3 commits July 17, 2023 17:58

Update manifest.toml

6c87f53

Create pict-rs.toml

21d5464

Update install

59a34c6

ericgaspar added 4 commits July 20, 2023 09:42

Update pict-rs.toml

911335d

Update pict-rs.toml

60bba09

fix

b858c09

Update pict-rs.toml

6ce80fc

yalh76 approved these changes Jul 27, 2023

View reviewed changes

tituspijean added 2 commits July 29, 2023 21:01

Add pict-rs.toml during upgrade too

62411c1

Fix Pict-RS config and include ImageMagick

4ab3b02

This comment was marked as outdated.

Sign in to view

tituspijean added 2 commits July 30, 2023 12:26

Fix Pict-RS by loosening systemd restrictions, add exiftools dependency

76b2a65

Implement ImageMagick policy

8757210

This comment was marked as outdated.

Sign in to view

Remove not needed ImageMagick cache secret

cf18384

tituspijean added 3 commits July 30, 2023 13:05

Fix ImageMagick download

81facf9

Fix ImageMagick download²

7ffd19d

Fix ImageMagick download

ad04f17

tituspijean marked this pull request as ready for review July 30, 2023 11:34

This comment was marked as outdated.

Sign in to view

tituspijean and others added 3 commits August 8, 2023 21:03

Deploy ImageMagick's AppImage

f7a2c2c

Do not remove Pict-RS as backward compatibility step

0a1b64b

Merge branch 'testing' into pictrs

a1dffb8

This comment was marked as duplicate.

Sign in to view

ericgaspar and others added 2 commits December 6, 2023 17:27

Merge branch 'testing' into pictrs

92150f9

Auto-update README

509b46e

Gildas-GH mentioned this pull request Dec 23, 2023

Bugfix and reimplement Pictrs #76

Merged

2 tasks

ericgaspar merged commit 509b46e into testing Dec 26, 2023

Salamandar deleted the pictrs branch July 14, 2024 16:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reimplement Pict-RS #43

Reimplement Pict-RS #43

tituspijean commented Jun 19, 2023 •

edited

Loading

tituspijean commented Jun 19, 2023 •

edited

Loading

DrMxrcy commented Jul 10, 2023

ericgaspar commented Jul 12, 2023

yunohost-bot commented Jul 12, 2023

csolisr commented Jul 17, 2023

ericgaspar commented Jul 20, 2023

yunohost-bot commented Jul 20, 2023

csolisr commented Jul 20, 2023

yalh76 left a comment

DrMxrcy commented Jul 28, 2023

csolisr commented Jul 28, 2023

ericgaspar commented Jul 29, 2023

This comment was marked as outdated.

This comment was marked as outdated.

tituspijean commented Jul 30, 2023

yunohost-bot commented Jul 30, 2023

This comment was marked as outdated.

tituspijean commented Aug 8, 2023

yalh76 commented Aug 9, 2023

yunohost-bot commented Aug 9, 2023

yunohost-bot commented Aug 9, 2023

tituspijean commented Aug 10, 2023

This comment was marked as duplicate.

ericgaspar commented Dec 6, 2023

yunohost-bot commented Dec 6, 2023

Reimplement Pict-RS #43

Reimplement Pict-RS #43

Conversation

tituspijean commented Jun 19, 2023 • edited Loading

PR Status

Automatic tests

tituspijean commented Jun 19, 2023 • edited Loading

DrMxrcy commented Jul 10, 2023

ericgaspar commented Jul 12, 2023

yunohost-bot commented Jul 12, 2023

csolisr commented Jul 17, 2023

ericgaspar commented Jul 20, 2023

yunohost-bot commented Jul 20, 2023

csolisr commented Jul 20, 2023

yalh76 left a comment

Choose a reason for hiding this comment

DrMxrcy commented Jul 28, 2023

csolisr commented Jul 28, 2023

ericgaspar commented Jul 29, 2023

This comment was marked as outdated.

This comment was marked as outdated.

tituspijean commented Jul 30, 2023

yunohost-bot commented Jul 30, 2023

This comment was marked as outdated.

tituspijean commented Aug 8, 2023

yalh76 commented Aug 9, 2023

yunohost-bot commented Aug 9, 2023

yunohost-bot commented Aug 9, 2023

tituspijean commented Aug 10, 2023

This comment was marked as duplicate.

ericgaspar commented Dec 6, 2023

yunohost-bot commented Dec 6, 2023

tituspijean commented Jun 19, 2023 •

edited

Loading

tituspijean commented Jun 19, 2023 •

edited

Loading