Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add custom RAW format conversion tool support #929

Open
wants to merge 16 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 3 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,9 @@ ENV PHOTOVIEW_LISTEN_PORT 80

ENV PHOTOVIEW_SERVE_UI 1
ENV PHOTOVIEW_UI_PATH /ui
ENV PHOTOVIEW_RAW_PROCESSING_EXECUTABLE="darktable-cli"
ENV PHOTOVIEW_RAW_PROCESSING_EXECUTABLE_CHECK="--version"
ENV PHOTOVIEW_RAW_PROCESSING_ARGS="{{ .InputPath }} {{ .OutputPath }} --core --conf plugins/imageio/format/jpeg/quality={{ .Quality }} --configdir /tmp/photoview-convert"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this lead to a command injection attack, by crafting a malicious environment variable?

And if so is it a problem?

Just wanted to mention it to hear what you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@viktorstrate, This is a great catch!
Yes, from a security point of view, this is a critical vulnerability, as we cannot know in advance which executable should be called by the Photoview and rely on the external config (environment variable in this case), which can be modified.
To mitigate this we need to statically resolve the path to the executable at the build time and accept later only the commandline parameters from the variables. This also means that we will need to preinstall all supported executables inside the same image and define paths to them in Dockerfile providing the ability to choose an executable from the predefined list in the scanner options on the Settings page. A similar approach might work for a directly hosted app, where all paths should be provided in some form at the build time, so they are available on the Settings later. If a user wants to add a custom executable (a self-developed one or just something, not tested by us and not supported from the project perspective) in the case of deployment with docker, a local rebuild of the Photoview image will be required.
An alternative way would be to define the AppArmor profile for the Photoview image and allow only a predefined list of executables to be executed inside the container. However, this is more difficult to maintain and would require strong Linux admin skills for a user in the case of direct deployment to a host instead of using docker (which, I believe, is a stop-factor for us, as our target audience is not only Linux gurus but also regular users), while doesn't provide much more flexibility, as we just shift the place of executables hardcoding from the code to the AppArmor profile, so to add a custom executable, the profile should be changed, which requires an image rebuild as well.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this issue is within the scope of consideration. The prerequisite for attacking through custom environment variables is that the attacker has obtained the permissions of the machine. If the attacker has obtained the permissions of the machine, then the security issue does not require us to consider it. In addition, regarding the docker image, the program and environment variables used for RAW conversion have been pre-installed in the official image. If the program or the value can be artificially modified, the attacker still needs to obtain the permissions of the machine. From what perspective, this should not be an issue that ordinary applications need to consider.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An attacker might get access to the docker container and put there a new malicious executable, then manipulate with env variables to replace the RAW processing one, so that the malicious executable is called instead. In this case, there is no need to have access to the host.
If Photoview is deployed on the host directly, this is the same system, so an attacker has access to the host, as a 1st step.
But in both cases, there are different levels of access: to put a file to the home folder and modify an env variable you don't need admin permissions. The photoview user can do that. While to modify an existing executable in the system (owned by root), which is registered in the Photoview app, an attacker should become an admin 1st.
Please see the PR #863, where I implemented many security enhancements for our docker image, so now the Photoview app is executed there under non-root user, which has write access to his home folder and mounted app cache (which by default is also inside the home folder). All executables remain owned by root and cannot be modified without root permissions.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If an attacker can modify environment variables and restart photoview, he also has permission to directly execute the malicious program. In addition, environment variables can be changed, so even under existing circumstances, the default RAW converter and video converter will still be replaced without requiring root permission. Just modify the PATH environment variable

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition, environment variables can be changed, so even under existing circumstances, the default RAW converter and video converter will still be replaced without requiring root permission. Just modify the PATH environment variable

I really like this argument, this PR would not make Photoview any less secure because of this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, we need to prepare an AppArmor profile, which will at least forbid the execution of any apps from non-standard locations, or allow the execution of only allowed binaries - I need to think about this more...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and video converter

BTW, if we provide a way to register any external RAW processing app, do we want to provide the same approach for video-converter integration as well?
How does the scanner create thumbnails for non-RAW images: does it use the same RAW-processing app integration? if not, probably, we need to change that as well? I assume that there is a low probability that a user wants to use a non-default app to process RAW files while keeping the standard one for regular images.
@viktorstrate, @jordy2254, @fierceX, what do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the first message I don't think it's something that we need to do unless anyone raises a concern. Ultimatly the attack vector is minimised in a docker environment and on a bare metal install I'd expect people to manage it as they see fit based on there use case.
For the latter message In my opinion out of scope for this piece of work. lets keep things in scope and make additional issues/PRs for them as needed to try and keep risk to a minimum and follow CI/CD better.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the latter message In my opinion out of scope for this piece of work

If you refer to the "do we want to provide the same approach for video-converter integration as well?" message, sure, I agree that this might be a new PR. My question was just from a product functionality perspective: do you see the inconsistency between the flexibility of the RAW processing app integration and video / thumbnails converters, possibly some other workers?
This is a good approach to give the user an opportunity to redefine what a 3rd-party app should process the incoming media. I'd like to have it available for all types of workers at the final stage (after all corresponding PRs are merged)


EXPOSE 80

Expand Down
4 changes: 2 additions & 2 deletions api/scanner/media_encoding/encode_photo.go
Original file line number Diff line number Diff line change
Expand Up @@ -110,8 +110,8 @@ func (img *EncodeMediaData) EncodeHighRes(outputPath string) error {

// Use darktable if there is no counterpart JPEG file to use instead
if contentType.IsRaw() && img.CounterpartPath == nil {
if executable_worker.DarktableCli.IsInstalled() {
err := executable_worker.DarktableCli.EncodeJpeg(img.Media.Path, outputPath, 70)
if executable_worker.CustomRawCli.IsInstalled() {
err := executable_worker.CustomRawCli.EncodeJpeg(img.Media.Path, outputPath, 70)
if err != nil {
return err
}
Expand Down
66 changes: 42 additions & 24 deletions api/scanner/media_encoding/executable_worker/executable_worker.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,52 +7,68 @@ import (
"os"
"os/exec"
"strings"
"text/template"
"bytes"


"github.com/photoview/photoview/api/utils"
"github.com/pkg/errors"
"gopkg.in/vansante/go-ffprobe.v2"
)

func InitializeExecutableWorkers() {
DarktableCli = newDarktableWorker()
CustomRawCli = newCustomRawWorker()
FfmpegCli = newFfmpegWorker()
}

var DarktableCli *DarktableWorker = nil
var CustomRawCli *CustomRawWorker = nil
var FfmpegCli *FfmpegWorker = nil

type ExecutableWorker interface {
Path() string
}

type DarktableWorker struct {
type FfmpegWorker struct {
path string
}

type FfmpegWorker struct {
type CustomRawWorker struct {
path string
args string
}

func newDarktableWorker() *DarktableWorker {
type RawArgs struct {
InputPath string
OutputPath string
Quality int
}


func newCustomRawWorker() *CustomRawWorker {
if utils.EnvDisableRawProcessing.GetBool() {
log.Printf("Executable worker disabled (%s=1): darktable\n", utils.EnvDisableRawProcessing.GetName())
log.Printf("Executable worker disabled (%s=1) \n", utils.EnvDisableRawProcessing.GetName())
jordy2254 marked this conversation as resolved.
Show resolved Hide resolved
return nil
}

path, err := exec.LookPath("darktable-cli")
var _check = utils.EnvRawProcessingCheck.GetValue()
var _exec = utils.EnvRawProcessing.GetValue()
var _args = utils.EnvRawProcessingArgs.GetValue()

path, err := exec.LookPath(_exec)
if err != nil {
log.Println("Executable worker not found: darktable")
log.Printf("Executable worker not found: %s",_exec)
} else {
version, err := exec.Command(path, "--version").Output()
version, err := exec.Command(path, _check).Output()
if err != nil {
log.Printf("Error getting version of darktable: %s\n", err)
log.Printf("Error getting version of %s: %s\n",_exec, err)
return nil
}

log.Printf("Found executable worker: darktable (%s)\n", strings.Split(string(version), "\n")[0])
log.Printf("Found executable worker: %s (%s)\n", _exec, strings.Split(string(version), "\n")[0])

return &DarktableWorker{
return &CustomRawWorker{
path: path,
args: _args,
}
}

Expand Down Expand Up @@ -85,30 +101,32 @@ func newFfmpegWorker() *FfmpegWorker {
return nil
}

func (worker *DarktableWorker) IsInstalled() bool {
func (worker *CustomRawWorker) IsInstalled() bool {
return worker != nil
}

func (worker *FfmpegWorker) IsInstalled() bool {
return worker != nil
}

func (worker *DarktableWorker) EncodeJpeg(inputPath string, outputPath string, jpegQuality int) error {
tmpDir, err := ioutil.TempDir("/tmp", "photoview-darktable")
func (worker *CustomRawWorker) EncodeJpeg(inputPath string, outputPath string, jpegQuality int) error {
tmpDir, err := ioutil.TempDir("/tmp", "photoview-convert")
if err != nil {
log.Fatal(err)
}
defer os.RemoveAll(tmpDir)

args := []string{
inputPath,
outputPath,
"--core",
"--conf",
fmt.Sprintf("plugins/imageio/format/jpeg/quality=%d", jpegQuality),
"--configdir",
tmpDir,
}
tmpl, err := template.New("RawArgs").Parse(worker.args)
if err != nil {
log.Fatal(err)
}
_args := RawArgs{inputPath,outputPath,jpegQuality}
var buf bytes.Buffer
err = tmpl.Execute(&buf,_args)
if err != nil {
log.Fatal(err)
}
args := strings.Split(buf.String()," ")

cmd := exec.Command(worker.path, args...)

Expand Down
2 changes: 1 addition & 1 deletion api/scanner/media_type/media_type.go
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,7 @@ func (imgType *MediaType) IsSupported() bool {
return true
}

if executable_worker.DarktableCli.IsInstalled() && imgType.IsRaw() {
if executable_worker.CustomRawCli.IsInstalled() && imgType.IsRaw() {
return true
}

Expand Down
3 changes: 3 additions & 0 deletions api/utils/environment_variables.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ const (
EnvDisableFaceRecognition EnvironmentVariable = "PHOTOVIEW_DISABLE_FACE_RECOGNITION"
EnvDisableVideoEncoding EnvironmentVariable = "PHOTOVIEW_DISABLE_VIDEO_ENCODING"
EnvDisableRawProcessing EnvironmentVariable = "PHOTOVIEW_DISABLE_RAW_PROCESSING"
EnvRawProcessing EnvironmentVariable = "PHOTOVIEW_RAW_PROCESSING_EXECUTABLE"
EnvRawProcessingCheck EnvironmentVariable = "PHOTOVIEW_RAW_PROCESSING_EXECUTABLE_CHECK"
EnvRawProcessingArgs EnvironmentVariable = "PHOTOVIEW_RAW_PROCESSING_ARGS"
)

// GetName returns the name of the environment variable itself
Expand Down