Ideas for a default profile to produce predictable "camera-like" output

[Beep6581]

Originally reported on Google Code with ID 2071

Autolevels are today mainly used to make the first open of an image to look good (it's
used in the default profile). I'd prefer if first open should would just look similar
to a standard in-camera jpeg, ie just a tonecurve to brighten and compress highlights,
and that's it.

I've attached how such a tonecurve looks. Not sure if it can be achieved with the Exposure
box sliders. Try open a file, select the neutral profile (make sure highlight recovery
is also at 0 if you don't have bleeding edge when this is fixed), and then apply this
type of curve -- you'll see how very similar it becomes to the camera's JPEG output.

I think autolevels should not be included in any of the bundled profiles, and maybe
not even be saved into the .pp3 when enabled. Instead just seen as a settings "assistant",
you press once and get some values as a starting point for more advanced processing.
I could be convinced otherwise though, but I certainly don't think it's good to have
in the Default profile.

Why do I want this? Because ideally I'd like that you could dump all your raw photos
from a shoot on RT and do nothing and get predictable output. Autolevels is not good
to have in such a situation, it will try to correct an underexposed image, expand contrast
in low contrast conditions etc. A fixed curve (or corresponding) will give you something
that looks like normal "unprocessed" camera output.

It's also what basically all other raw converters do, default profile is something
stable similar to default camera jpegs. A few converters do have an "autolevels" function
(Lightroom has it for example), but it's not something that gets enabled per default,
as the results generally require further tuning by a human operator.

What do you think?

Reported by torger@ludd.ltu.se on 2013-11-20 19:25:54

[Beep6581]

My idea of Auto Levels is a tool that is supposed to get the darkest pixels to 0%, the
lightest to 100%, and to fiddle with the info in between to get it into the right ballpark.
Of course I can just move the bottom and top end points of a curve, or use the exposure
and black point sliders to do it, but why waste time when I can have RT do it for me?
Having the image in the correct ballpark, I can now use the other tools to fine tune
(how can you possibly use RT's Tone Mapping if you didn't first set proper black and
white points?) Likewise, bundled profiles can't possibly work well if the image is
not even in the right ballpark, which is why I think that for most of them Auto Levels
HAS TO be enabled.

The problem, however, is that currently I think auto levels is broken. Try with an
underexposed image. Jimr in the forum sent me [url=http://rawtherapee.com/forum/viewtopic.php?p=34280#p34280]two
severely underexposed raws[/url], and Auto Levels made the main histogram look pretty
much like the one you attached. Still severely underexposed! But now I'm hijacking
your issue.

I think there most certainly is a reason to use AL in bundled PP3s, as long as it works
properly!

Reported by entertheyoni on 2013-11-20 23:51:10

[Beep6581]

Oops, no bbcode here. URL above is:
http://rawtherapee.com/forum/viewtopic.php?p=34280#p34280

Reported by entertheyoni on 2013-11-20 23:51:42

[Beep6581]

And "Default.pp3" is supposed to be a good starting point for RT, not "As Camera JPEG.pp3"
;]

Reported by entertheyoni on 2013-11-20 23:52:37

[Beep6581]

One problem is that expanding the range to 100% is not always "right". I shoot winter
landscapes in dusk and dawn quite often, these nearly always get wrong with autolevels,
as the scenes have a natural low contrast. I don't do much portraits, but I would think
that autolevels is quite poor for portraits too, increasing contrast too much.

The very nature of an autolevels function is that it cannot make the right decisions
always, as it's scene dependent. Our autolevels is still better than say Lightroom's
autotone, which makes more aggressive settings but more often very wrong.

When deviating from how "all other" raw converters work we should also have a very
good reason. All other raw converters try per default to show something that's similar
to what a film camera would produce, ie also what default camera jpeg does. That means
a fixed tone curve and nothing more. Predictable output. Any user that has used any
other raw converter will get surprised when it's not possible to get camera-like output
per default.

I just don't see any strong reason why default should be a risky autolevels profile
with unpredictable output, and getting a camera-like output should require you to make
an own custom profile.

If people really want this to be default I'd still like to have an easily accessible
profile that gives this camera-like output. It could be called "Camera.pp3" or something
and should be right up there beside Default, so those of us that want "as usual" behavior
can get that without needing to learn how to do it and make our own custom profile.

During the first years using RawTherapee I have many times struggled with this issue,
that default profile do not give predictable output but instead add high contrast "pop"
through autolevels. I'm today using Lightroom when I need that, usually for high volume
work like a sports shoot when all processing I do from camera-like image is cropping
and perhaps post-crop vignette, and exposure when camera's auto-exposure is a bit off,
but no contrast changes or curves etc. Earlier I thought it was an advanced task to
make a camera-like output so I did not even try to make a custom profile. Now when
I've done more photo programming I know better, it's basically just a tone-curve. It
does tell me though that you need to be a pretty advanced user to know how to make
a camera-like profile, and that RawTherapee lacking this feature per default makes
it hard to use for common workflows.

Reported by torger@ludd.ltu.se on 2013-11-21 08:35:41

[Beep6581]

Note that I do appreciate RT's design where "neutral" (ie no slider changes) means raw-file
as-is only color space conversion, often looking flat, dull and a bit underexposed,
ie not really usable as-is but gives you good understanding of how digital photography
works and what changes needed to make a good image.

Commercial raw converters "neutral" means camera-like output, ie they have this tone-curve
per default, often without possibility to turn it off. To me that is being too user-friendly,
taking away control and understanding of digital photography from the user, and not
something RT should do.

However, we can still provide camera-like output by having a profile that applies this
type of tone curve, and I think it would be best that this is the "Default" profile,
but I can live with having it on the side at the top level.

I've introduced RT to users of other raw converters, and that the first open gives
an unpredictable and often "over the top" result (nowadays autolevels is a bit better
than it was though) is not something that is appreciated. People are used to start
with a "camera-like" image and continue from there.

Sidenote: With DCPs this tone-curve is typically embedded in the DCP and all you need
to do to get camera-like output is to apply the DCP, with tone-curve enabled. The intention
of DNG/DCP is to give camera-like output per default. I don't think we should do it
like that though, I don't like hiding away changes from the user.

Reported by torger@ludd.ltu.se on 2013-11-21 08:51:32

[Beep6581]

What do you think of this default profile?
I personally never use auto levels, since it engages RGB Contrast and it induces hue
shifts that ruins skintones.

Attached profile has two tone curves - brightening and darkening, set in thier preferred
curve types to the colors clean.

What could be useful is another, simpler, auto levels that only sets the exposure &
highlight recovery while user-selected tone curves handle the contrast. We could have
an option (in preferences?) - the Auto-levels intent.

Reported by michaelezra000 on 2013-11-21 12:57:30

---

- _Attachment: [Default-Curve1.pp3](https://storage.googleapis.com/google-code-attachments/rawtherapee/issue-2071/comment-6/Default-Curve1.pp3)_

[Beep6581]

The problem IMO is that each camera/sensor needs a different curve to make the output
look good. So any "default" curves will look good for some models and terrible for
others. My Pentax DSLRs e.g. need a very strong and strangely shaped contrast curve
(see attached) to make them look like the camera JPG output while my Canon S110 needs
a completely different curve and looks terrible when I use the "Pentax" curve.

The Pentax files also look horrible with RT's current default profile by the way...
I think there's no really good solution for this problem because sensor data is so
different. 

Reported by stefan.ittner on 2013-11-21 13:30:43

---

- _Attachment: [PentaxCurve.rtc](https://storage.googleapis.com/google-code-attachments/rawtherapee/issue-2071/comment-7/PentaxCurve.rtc)_

[Beep6581]

Yes I like the concept. I'll return with further comments, I shall do some experiments
based on your profile.

Reported by torger@ludd.ltu.se on 2013-11-21 13:45:06

[Beep6581]

Yes, I realize now that there's more to it than just a fixed curve to make really good
support. I'll try to nail down the requirements for a "minimum" profile which can produce
*stable* "camera-like"/"film-like" results.

Probably we need more than just a profile, ie some sort of autolevels, Michael's "autolevels
intent" idea could fit. A global default camera curve that can be refined with a model-specific
curve in camconst.json could also be a part of it.

I think autolevels could work mostly on exposure, push under-exposed files a little.
Then we may need to apply a curve modifier to heavily clipped files, so we get the
desaturated film-like overexposure look that camera files have.

Just speculating now though, need to make further testing.

Reported by torger@ludd.ltu.se on 2013-11-21 14:11:01

[Beep6581]

Sounds like a good idea. I think it's worth a look what Adobe uses as "base" or default
curves in Lightroom for different camera models.
I agree that Auto levels should focus on exposure and maybe highlight recovery. I also
think it should take into account the applied tone curves in its calculations (if any
are present), otherwise the result will vary a lot depending on what kind of curves
are used in addition.

Reported by stefan.ittner on 2013-11-21 14:26:54

[Beep6581]

I've worked a lot with Adobe's DNG reference code recently in another project, and got
a bit inspired by that. The DNG conversion is very basic, but still yields a stable
result which looks camera-like. Actually I have not seen any autolevels tricks in the
DNG reference code, it applies the curve and a baseline exposure and that's it. The
curve and baseline exposure can vary between models though, as they make a DCP file
per model.

The curve in my initial post forgot about gamma so it's a bit wrong, tuned the curve
to better match a typical tonecurve, and played with some with Canon 5D Mark II files,
and for that camera it seems like in most cases just that tone curve is enough to produce
a result very similar to the camera's own JPEGs. The Film-like curve (which is Adobe
DNG style) is a bit too saturated though, I get better results with RT's "weighted"
curve.

I shall provide a .pp3 later when I've played around some more. Need to look at pictures
with skintones in them as those are most sensitive.

Different camera models strive for different looks in their jpegs. To say something
general I think that DSLRs strive for a natural look in the default settings, while
consumer compacts often strive to exaggerate saturation to get more pop. Probably one
should strive to primarily achieve the natural/neutral camera-like look first-hand,
rather than mimicking JPEG look exactly (the cameras often have various look modes
too of course) as that will be lots of extra work.

All camera sensors that we support today are linear light capture devices, there's
nothing magical about them, so I think that it should be possible to make some basic
profile and possibly support algorithm that produces a stable result. The main difference
between models I think is that cameras auto-exposure devices may expose the sensors
a bit differently, ie how much highlight headroom they choose to keep. The DNG spec
solves this by having a per-model baseline exposure, but as said otherwise no dynamic
autolevels type of function.

Camera JPEGs cut away lots of highlight detail, as doing highlight recovery tricks
is dangerous in terms of getting stable results. I don't think any highlight recovery
should be enabled in this type of profile, but one should be just as wasteful (and
thus stable) as the cameras are themselves. Recovering highlights and make the most
out of your image is something that require manual work.

Reported by torger@ludd.ltu.se on 2013-11-21 15:35:03