Plotting with negative x values on semilogx #396

Closed
nbigaouette opened this Issue Jul 11, 2011 · 11 comments

Comments

Projects
None yet
5 participants

I need to plot a semilogx graph but where x values are negative.

Using just plot() and a log scale using set_xscale('log'), values less then 0 are ignored. Using set_xscale('symlog', linthreshx = 1.0e-4) gives a weird result.

Here is a simple example:

import numpy
from matplotlib import pyplot

# Enable interactive mode
pyplot.ion()

# Draw the grid lines
pyplot.grid(True)

# Numbers from -50 to 50, with 0.1 as step
xdomain = numpy.arange(-50,50, 0.1)

# Plots a simple linear function 'f(x) = x'
pyplot.plot(xdomain, xdomain)
# Plots 'sin(x)'
pyplot.plot(xdomain, numpy.sin(xdomain))

pyplot.xscale('symlog', linthreshx=0.1)

In that example, the curves are ploted with increasing x values, then do a big jump to the left and increase x again...

I posted a simple example on the mailinglist but without an answer: http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg21189.html

Thanks

mdboom was assigned Jul 11, 2011

mdboom closed this in 0e6a2b6 Jul 25, 2011

Thanks Michael! I've tried it and it works.
It still seems broken at some places though. For example, the "x" value shown with mouse over is wrong, and changing the zoom will jump the "scene" to the wrong place. I think it's going to positive values of x, even though you zoomed in the (log) negative portion.

Owner

mdboom commented Jul 29, 2011

Hmm... I'm not seeing these problems you describe. Can you provide example code that reproduces the bug with instructions as to where to draw the zoom rectangle etc.?

My script is quite complex, so it'd be hard to share it. But I tried with the example I posted on stackoverflow (http://stackoverflow.com/questions/6681975/matplotlibs-symlog-option-how-to-prevent-curves-that-come-back/6849737) and I can see the behaviour.

  1. Placing the cursor between x = -10^-1 and 10^-1 (the linear region) shows correctly the x value in the lower right corner of the figure. But in the semilogx region, it's screwed. At x=10^-1, it show x=9, and moving the cursor in the positive x direction decreases the shown x value up to x = 1 == 10^0. Then it starts increasing again. Similar behaviour happens in the x<-10^-1 region.

  2. As for the zoom, I use the pan tool with right-click to stretch or compress the graph in the x direction. It's hard to explain... Normally, the figure will stay centered around the place where the click happened. But now it zoom in up to a certain point, start zooming out, then zoom in again, all when continuously moving the cursor to the right. I think the "jumpy" behaviour appears when the cursor crosses a certain point, but I can't identify "what" point (10^-1? 2x10^-1?) This happens when either clicking in the linear region or the semilogx region.

My script has a lot more data to plot, so problably the jumps in zoom/placement are bigger. But I can still reproduce it with the simple example. I'll post it here too just to make sure:

import numpy
from matplotlib import pyplot

# Enable interactive mode
pyplot.ion()

# Draw the grid lines
pyplot.grid(True)

# Numbers from -50 to 50, with 0.1 as step
xdomain = numpy.arange(-50,50, 0.1)

# Plots a simple linear function 'f(x) = x'
pyplot.plot(xdomain, xdomain)
# Plots 'sin(x)'
pyplot.plot(xdomain, numpy.sin(xdomain))

pyplot.axis([-60,60,-1.5,1.5])

pyplot.xscale('symlog', linthreshx=0.1)

ssyr commented Aug 23, 2011

The issue must be reopened. The fix in
https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/scale.py
is bad (( notations: y=SymmetricalLogTransform.transform(x), x=InvertedSymmetricalLogTransform.transform(y), L=linthresh>0, b=self.base>0 )):

  1. direct&inverse transformations do not match:
    the direct transformation is (removing all obfuscation)
    |x|>L: y = sign(x)_( 1 + log(|x|/L) / log(b) )
    |x|<L: y = x / L
    and x->(+L+/-0), y->(+1) ; x->(-L+/-0), y->(-1)
    while the inverse transformation
    |y|>1: sign(y) * exp( log(b) * |y - sign(y) * ( 1 - log(L) / log(b) ) | ) (Wrong)
    |y|<1: x_L (OK)
    as y->(+1+/-0), x->(+L) ; y->(-1+0), x->(-L) ; BUT as y->(-1-0), x->(-exp( log^2(b) / log(L) ) ) if L < b

  2. there is a kink in the transformation: the first derivative has a step at x=+/-L:
    x->(+L+0), y'->(1/log(b)) ; x->(+L-0), y'->(1/L)

The following patch cures both problems (patch was applied to matplotlib-py25 1.0.1-1)

--- matplotlib.orig/scale.py    2011-08-23 11:37:15.000000000 -0700
+++ matplotlib/scale.py 2011-08-23 12:08:06.000000000 -0700
@@ -326,10 +326,10 @@
             a = np.asarray(a)
             sign = np.sign(a)
             masked = ma.masked_inside(a, -self.linthresh, self.linthresh, copy=False)
-            log = sign * ma.log(np.abs(masked)) / self._log_base
+            log = sign * self.linthresh * (1 + ma.log(np.abs(masked) / self.linthresh))
             if masked.mask.any():
                 return np.asarray(ma.where(masked.mask,
-                                            a * self._linadjust,
+                                            a,
                                             log))
             else:
                 return np.asarray(log)
@@ -352,11 +352,15 @@

         def transform(self, a):
             a = np.asarray(a)
-            return np.where(a <= self._log_linthresh,
-                             np.where(a >= -self._log_linthresh,
-                                       a * self._linadjust,
-                                       -(np.power(self.base, -a))),
-                             np.power(self.base, a))
+            sign = np.sign(a)
+            masked = ma.masked_inside(a, -self.linthresh, self.linthresh, copy=False)
+            exp = sign * self.linthresh * ma.exp(sign * masked / self.linthresh - 1)
+            if masked.mask.any():
+                return np.asarray(ma.where(masked.mask,
+                                            a,
+                                            exp))
+            else:
+                return np.asarray(exp)

         def inverted(self):
             return SymmetricalLogScale.SymmetricalLogTransform(self.base)

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Aug 24, 2011

@mdboom mdboom Fix the inverse transform in symlog. Closes #396. cd3634b

mdboom reopened this Aug 24, 2011

Owner

mdboom commented Aug 24, 2011

@nbigaouette: Sorry this bug fell through the cracks for a few weeks. Indeed, there is a discrepancy between the forward and inverse transform, but only when linthresh < base. I think I have a fix for this in pull request #433. It seems to fix all of the "jumping around" you describe.

@ssyr: I'm not sure I follow. Your mathematical notation doesn't seem to quite match either the old (matplotlib 1.0.1) or new (git master) version so it's not clear which (or both) you are saying is broken. With my patch in #433, the forward and inverse transformations agree within floating-point rounding error. With your proposed changes, there are large differences between the forward and inverse transforms over the entire linear portion.

Here is the code I'm using to graph the differences:

from matplotlib import pyplot as plt
import numpy as np
from matplotlib import scale

data = np.arange(-100.0, 100.0, 0.1)

for base in (2., 10., 5.):
    for linthresh in (-2.5, -2, -1.5, -1, -0.5, 0.5, 1, 1.5, 2, 2.5):
        linthresh = np.power(base, linthresh)
        forward = scale.SymmetricalLogScale.SymmetricalLogTransform(base, linthresh)
        backward = forward.inverted()
        transformed = forward.transform(data)
        inverted = backward.transform(transformed)
        plt.figure()
        plt.plot(np.abs(data-inverted))
        plt.savefig("symlog_%d_%f.png" % (base, linthresh))

If you run that with your changes vs. the mdboom/matplotlib/symlog_bugs2 branch, you should be able to see the differences in agreement.

Additionally, the code you present seems to make the discontinuity in the forward transform at +/-L much worse than in master. See this image of symlog_demo.py with your code:

symlog discontinuous

vs master:

symlog continuous

That said, is there some reason (other than the bug addressed by pull #433) that you feel symlog does not do the right thing? Is it a matter of insufficient documentation? Is this a problem that presents itself with a certain kind of data? (Perhaps you could provide an example).

ssyr commented Aug 24, 2011

@mdboom:
It is strange, because I see my code working correctly:

  1. I have run your test example for |x| <= 100 and found
    | x - inverted(transform(x)) | <3e-13
    which is close to the roundoff error

  2. My symlog_demo.py result is
    ssyr
    which is also continuous AND smooth in addition to 'master' from your post:
    master
    A correct transformation should be (a) continuous and (b) monotonic; if it is also (c) smooth, it is neater than the one with kinks: a smooth line is transformed into another smooth line, which may be relevant when examining data.

  3. finally, the overall coefficient should not matter; in my transformation, 'transform(x) = x' in the linear region, but the whole thing may be rescaled with any factor>0, be it for consistency or any other reason.
    My transformation is
    |x| < L: transform(x) = x
    |x| > L: transform(x) = sign(x) * L * ( log(|x|/L) + 1 )
    and
    |y| < L: inverse(y) = y
    |y| > L: inverse(y) = sign(y) * L * exp(|y|/L - 1)
    which should work for any real number.

Perhaps, I have confused you with the version of matplotlib I use (fink-installed 'matplotlib-py25 1.0.1-1' @macosx 10.6.8), and the patch was not applied correctly. I have posted my (patched) version matplotlib/scale.py at http://dl.dropbox.com/u/39249666/scale.py and paste the relevant piece below:

# matplotlib/scale.py: replace two classes with vvvvvvvvvvvvvvvv
    class SymmetricalLogTransform(Transform):
        input_dims = 1
        output_dims = 1
        is_separable = True

        def __init__(self, base, linthresh):
            Transform.__init__(self)
            self.base = base
            self.linthresh = linthresh
            self._log_base = np.log(base)
            self._linadjust = (np.log(linthresh) / self._log_base) / linthresh

        def transform(self, a):
            a = np.asarray(a)
            sign = np.sign(a)
            masked = ma.masked_inside(a, -self.linthresh, self.linthresh, copy=False)
            log = sign * self.linthresh * (1 + ma.log(np.abs(masked) / self.linthresh))
            if masked.mask.any():
                return np.asarray(ma.where(masked.mask,
                                            a,
                                            log))
            else:
                return np.asarray(log)

        def inverted(self):
            return SymmetricalLogScale.InvertedSymmetricalLogTransform(self.base, self.linthresh)

    class InvertedSymmetricalLogTransform(Transform):
        input_dims = 1
        output_dims = 1
        is_separable = True

        def __init__(self, base, linthresh):
            Transform.__init__(self)
            self.base = base
            self.linthresh = linthresh
            self._log_base = np.log(base)
            self._log_linthresh = np.log(linthresh) / self._log_base
            self._linadjust = linthresh / (np.log(linthresh) / self._log_base)

        def transform(self, a):
            a = np.asarray(a)
            sign = np.sign(a)
            masked = ma.masked_inside(a, -self.linthresh, self.linthresh, copy=False)
            exp = sign * self.linthresh * ma.exp(sign * masked / self.linthresh - 1)
            if masked.mask.any():
                return np.asarray(ma.where(masked.mask,
                                            a,
                                            exp))
            else:
                return np.asarray(exp)
# end of copy-paste
Owner

mdboom commented Aug 25, 2011

I see what you're getting at now.

One problem with this patch is that it breaks ticking. (Compare the placement of ticks in the two images above). If that can be resolved, I think this is a very worthwhile improvement.

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Aug 30, 2011

@mdboom mdboom Implements a new version of symlog that is smooth around linthresh co…
…ntributed by ssyr. Also includes a cleaner implementation of the symlog ticker. Closes #396.
497681c
Member

dmcdougall commented Sep 24, 2012

Commit 497681c claims to close this.

@mdboom Let me know if this is not the case and we can re-open it.

dmcdougall closed this Sep 24, 2012

Member

pelson commented Sep 25, 2012

I'm not sure if this made it into matplotlib/matplotlib or not. Is there an easy way to find out?

Member

dmcdougall commented Sep 25, 2012

@pelson Click the commit number and look at the url.

Alternatively, git log 497681c should also work I think.

Member

dmcdougall commented Sep 25, 2012

@pelson Also, it didn't look like it was reverted: git log --grep="497681c*"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment