Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Candlestick shadow is drawn after candlestick body #2546

Closed
c0indev3l opened this issue Oct 25, 2013 · 40 comments
Closed

Candlestick shadow is drawn after candlestick body #2546

c0indev3l opened this issue Oct 25, 2013 · 40 comments
Milestone

Comments

@c0indev3l
Copy link

Hello,

in this example
http://matplotlib.org/examples/pylab_examples/finance_demo.html
there is a little display bug.
Candlestick shadows (vertical segment) should be drawn before candlestick body.
see http://en.wikipedia.org/wiki/Candlestick_chart

Kind regards

@pelson
Copy link
Member

pelson commented Oct 28, 2013

Looks like this could be a pretty accessible change. Are you interested in giving it a go, and I'lll happily review.

Cheers @c0indev3l!

@c0indev3l
Copy link
Author

@pelson that's not as easy as expected...

I tried to modify _candlestick function
https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/finance.py

My first idea was to return patches, lines instead of lines, patches
but it didn't worked

so I did a quite dirty fix...
I decomposed each shadow to 2 vline

There is probably a better way to do this...

def _candlestick(ax, quotes, width=0.2, colorup='k', colordown='r',
                    alpha=1.0, ochl=True):

    """
    Plot the time, open, high, low, close as a vertical line ranging
    from low to high.  Use a rectangular bar to represent the
    open-close span.  If close >= open, use colorup to color the bar,
    otherwise use colordown

    Parameters
    ----------
    ax : `Axes`
        an Axes instance to plot to
    quotes : sequence of quote sequences
        data to plot.  time must be in float date format - see date2num
        (time, open, high, low, close, ...) vs
        (time, open, close, high, low, ...)
        set by `ochl`
    width : float
        fraction of a day for the rectangle width
    colorup : color
        the color of the rectangle where close >= open
    colordown : color
         the color of the rectangle where close <  open
    alpha : float
        the rectangle alpha level
    ochl: bool
        argument to select between ochl and ohlc ordering of quotes

    Returns
    -------
    ret : tuple
        returns (lines, patches) where lines is a list of lines
        added and patches is a list of the rectangle patches added

    """

    OFFSET = width / 2.0

    lines = []
    patches = []
    for q in quotes: #[1:-1]:
        if ochl:
            t, open, close, high, low = q[:5]
        else:
            t, open, high, low, close = q[:5]

        if close >= open:
            color = colorup
            lower = open
            higher = close
            height = close - open
        else:
            color = colordown
            lower = close
            higher = open
            height = open - close

        vline = Line2D(
            xdata=(t, t), ydata=(low, lower),
            color='k',
            linewidth=0.5,
            antialiased=True,
        )

        vline2 = Line2D(
            xdata=(t, t), ydata=(higher, high),
            color='k',
            linewidth=0.5,
            antialiased=True,
        )

        rect = Rectangle(
            xy=(t - OFFSET, lower),
            width = width,
            height = height,
            facecolor = color,
            edgecolor = color,
        )
        rect.set_alpha(alpha)

        patches.append(rect)
        lines.append(vline)
        lines.append(vline2)
        ax.add_patch(rect)
        ax.add_line(vline)
        ax.add_line(vline2)
    ax.autoscale_view()

    return lines, patches

@tacaswell
Copy link
Member

That is the hard way to do it. Just change the zorder so that the shadow lines are behind the box.

@tacaswell
Copy link
Member

Something like https://github.com/tacaswell/matplotlib/tree/fix_candlestick_zorder (which I have not tested, and still needs a unit test)

@c0indev3l
Copy link
Author

should, at the same time, make the color shadows modifiable ?

change

    vline = Line2D(
        xdata=(t, t), ydata=(low, lower),
        color='k',
        linewidth=0.5,
        antialiased=True,
    )

to

    vline = Line2D(
        xdata=(t, t), ydata=(low, lower),
        color=colorshadow,
        linewidth=0.5,
        antialiased=True,
    )

It could be useful to be able to set this when drawing using inverted colors (black background, white foreground)

@tacaswell
Copy link
Member

@c0indev3l That sounds like a good idea. Could you put together a PR for each of these changes?

@c0indev3l
Copy link
Author

PR ? performance rating ?

@tacaswell
Copy link
Member

Pull request.

@c0indev3l
Copy link
Author

ok... but I'm quite busy...

I also have an other comment about this function (when using a black and white color theme with inverted colors ie black background) it's quite problematic

we should also pass coloredgeup and coloredgedown parameter

def _candlestick(ax, quotes, width=0.2, colorup='g', colordown='r',                
                alpha=1.0, ochl=True, colorshadow='k', coloredgeup=None, coloredgedown=None):                

    """                
    Plot the time, open, high, low, close as a vertical line ranging                
    from low to high.  Use a rectangular bar to represent the                
    open-close span.  If close >= open, use colorup to color the bar,                
    otherwise use colordown                

    Parameters                
    ----------                
    ax : `Axes`                
        an Axes instance to plot to                
    quotes : sequence of quote sequences                
        data to plot.  time must be in float date format - see date2num                
        (time, open, high, low, close, ...) vs                
        (time, open, close, high, low, ...)                
        set by `ochl`                
    width : float                
        fraction of a day for the rectangle width                
    colorup : color                
        the color of the rectangle where close >= open                
    colordown : color                
         the color of the rectangle where close <  open                
    alpha : float                
        the rectangle alpha level                
    ochl: bool                
        argument to select between ochl and ohlc ordering of quotes                

    Returns                
    -------                
    ret : tuple                
        returns (lines, patches) where lines is a list of lines                
        added and patches is a list of the rectangle patches added                

    """                
    if coloredgeup is None:                
        coloredgeup = colorup                

    if coloredgedown is None:                
        coloredgedown = colordown                

    OFFSET = width / 2.0                

    lines = []                
    patches = []                
    for q in quotes: #[1:-1]:                
        if ochl:                
            t, open, close, high, low = q[:5]                
        else:                
            t, open, high, low, close = q[:5]                

        if close >= open:                
            color = colorup                
            coloredge = coloredgeup                
            lower = open                
            higher = close                
            height = close - open                
        else:                
            color = colordown                
            coloredge = coloredgedown                
            lower = close                
            higher = open                
            height = open - close                

        vline = Line2D(                
            xdata=(t, t), ydata=(low, lower),                
            color=colorshadow,                
            linewidth=0.5,                
            antialiased=True,                
        )                

        vline2 = Line2D(                
            xdata=(t, t), ydata=(higher, high),                
            color=colorshadow,                
            linewidth=0.5,                
            antialiased=True,                
        )                

        rect = Rectangle(                
            xy=(t - OFFSET, lower),                
            width = width,                
            height = height,                
            facecolor = color,                
            edgecolor = coloredge,                
        )                
        rect.set_alpha(alpha)                

        patches.append(rect)                
        lines.append(vline)                
        lines.append(vline2)                
        ax.add_patch(rect)                
        ax.add_line(vline)                
        ax.add_line(vline2)                
    #ax.autoscale_view()                

    return lines, patches

@tacaswell
Copy link
Member

That also looks like a good idea, but these things are much easier to review from a pull request (fork matplotlib on github, create a new branch with your changes, and you can use the website to create the PR against master from your branch)

@c0indev3l
Copy link
Author

Ok I will send a pull request this week. There is also some other problems.
Default colorup should be green (not black).
open and close are python command... they should be changed for 'opn' and 'cls' (it works but I don't think it's a good idea to keep open and close as variable

@tacaswell
Copy link
Member

Changing internal variables is ok, changing the names of kwargs or the default value of anything is an api change and should be done very carefully.

@tacaswell
Copy link
Member

@c0indev3l Are you still interested in working on this issue?

@tacaswell
Copy link
Member

pushing to 1.4.x as @c0indev3l has gone silent.

@tacaswell tacaswell modified the milestones: v1.4.x, v1.3.x Feb 21, 2014
@sfroid
Copy link
Contributor

sfroid commented Mar 22, 2014

I see that this issue has been solved on the master branch. The approach to fix the problem is to change the stick color to match the candle color - then it doesn't matter which one is in the front.

Given that, I guess this issue can be closed.

@tacaswell
Copy link
Member

For reference, the color of the sticks was changed in PR #2207 . Closing.

@coralnut
Copy link

coralnut commented Apr 8, 2014

Sorry to be late to the party when it comes to contributing to this thread, but I've been working independently on this same problem, and I just noticed this thread, which seems to be implementing a color change that while logical, is completely unconventional when it comes to candlestick plotting.

Candlestock plotting began in Japan, where it was originally utilized to track price history on the rice exchange. Because the candlestick charts were originally drawn with black ink on white rice paper, the prototype method for drawing candlestick charts has always been to draw them in black ink on white paper, or using a computer, to draw them using black "ink" on a white background. In doing this, the boxes are always drawn with a black outline, and the handles are always drawn in black. If the interval price activity were upward (closing value exceeds opening value), then the candlestick would be drawn as only an outline in black ink on a white background, which produces an empty white candle that is circumscribed by black lines. If the interval price activity were downward (closing price is less than opening price) then the box would be colored-in with black ink.

In this fashion, the Japanese rice traders who created this style of plot originally drew their candlesticks either as white boxes with black handles, or as a black box with black handles. The handles were always black (they were using ink on paper, after all), and the candlestick bodies were always either black or white. This is the convention that remains standardized in all of the textbooks on technical analysis and stock charting, such as John J. Murphy's bestselling reference title: "Technical Analysis of the Financial Markets." This is also the convention that is utilized by Yahoo! Finance and many if not all of the online brokerages.

In an effort to Westernize this Japanese style of charting, some people have changed the default colors for upward price candlesticks to resemble the Stop/Go signals of a traffic light -- they have changed the colors for upward price candlesticks from white to green, and the colors for downard price candlesticks from black to red, while maintaining a black outline for the boxes and a black handle on the candle. In other words, the convention has aways been to retain a black color for outlining the box and for drawing the handles whenever the candlesticks are drawn on a white background -- even when the Japanese white/black candlesticks were Westernized (perverted) to make them correspond with the idiot-light color scheme of green/red.

In recent years, with the advent of the internet, charts with black backgrounds have become popular, and in those cases people will commonly outline the candlesticks, and draw the handles, in white (for the purpose of enhancing contrast against a dark background).

I have been using candlestick charts in technical analysis for decades, and I am unfamiliar with candlesticks that draw the handles in the same color as the candlestick body. Doing that would definitely allow people to add "pop!" to their charts, but it would also cause the color of the handles to change colors when the chart transitions from an up day to a down day, and that is definitely NOT the accepted convention. The accepted convention is always for the handle color and for the outline color of the candlestick to remain unchanged, while only the body of the candlestick changes color.

As an example of this: traditional candlesticks have always been drawn monochromatically, using black "ink" on a white "rice paper" background. Westernized versions of Candlesticks often do use Green or Red colors, as noted previously, but those have NEVER been conventional or standard. Similarly, coloring the handles to be the same color as the body would never be "conventional" unless the candle's body were black; and this would never be done with a white candle as it would make the candlestick disappear on a white background.

In the event that you should decide to implement a change in which the candlestick handles and outlines are given the same color as a candlestick body, I would recommend that you not do this as a default setting, or code the routine so that the user could not change the color pattern to revert to the conventional pattern, where the handle and the candlestick outline are NOT the same color as the candle body. Traditionally, Candlesticks have always been drawn in black and white on a white background, with black body outlines and black handles. More recently people have indeed begun to use green and red, with black outlines and handles, in an effort to make their charting stand out. The idea of color coding the handles to match the color of the candlestick body is a recent idea. It is not at all traditional, and it is definitely NOT an accepted standard.

That said, I can see how it would be valuable to allow the user to set each of these color options individually, to facilitate the use of non-conventional colors, especially in the case where the developer wants to implement a dark background, which seems to be all the rage in the internet era. But I think it would be a grave mistake to hard-code the color settings to force the box outlines and the handles to always be the same color as the candlestick body. Doing that would make white candlesticks disappear completely from a white background. It would force people who prefer the traditional color scheme to have to do a lot of unnecessary coding within the library to revert to normal charting behavior. That's more difficult than just coding color variables within a function call.

To make color changes easier, I would recommend that the user be granted the option of using the traditional black and white candlesticks with black outlines and black handles, and if the user should decide to over-ride the default colors, to allow the user to individually set four different color parameters: A) the upward price color for the candle body, B) the downward price color for the candle body, C) the candle outline color, and D) the candle handle color.

These four color settings would provide maximum flexibility and make everyone happy. Forcing the handles and the box outline to the same value as the box color, without allowing an easy method for the user to change back to a traditional color scheme would be, in my opinion, a very bad choice.

Maybe the guys who are new at this sort of thing will think otherwise, but the old-coot in me thinks that black and white on a white background is what candlesticks are all about, and that independent color selection via the 4 parameters above would provide maximum flexibility for customizing the charts while making the most people happy.

Thanks for your time.

@tacaswell
Copy link
Member

@coralnut Thank you for the interesting history lesson, I had not realized that candle stick plots had such a long history.

iirc none of the active devs have a background in finance and maintenance on the finance module has faltered. One of the consequences of this gap in expertise is things like the issue you raised here (if you note, I merged the offending commit). Due to this, the plan is to deprecate the finance module in 1.4 and move it to either mpl_toolkits or it's own repository in 1.5.

You seem to have a good deal of experience in this area. If you want to take charge of the finance module (either to re-visit the decision to deprecate it, which would need to be done 'soon' before we cut 1.4.0, or to take care of it in mpl_toolkits) it would be greatly appreciated.

@coralnut
Copy link

coralnut commented Apr 8, 2014

Yes, it's hard to maintain someone else's code when you're not familiar with what it's designed to do. I'm happy to try to help, but I need to warn you that I'm grossly unqualified to take charge of fixing this sort of thing. Although I've got plenty of experience using these types of charts, I'm a novice coder trying to learn to use python and matplotlib. Though I'd be happy to contribute whatever I can, I'm in no way qualified to spearhead such a project. I've been struggling just to get the candlestick() functions work in my application, and to be honest, I looked at the code in the library and I don't understand the data structures. :(

A lesser impediment is that I'm having quite a bit of trouble figuring out how this site works. I'm familiar with buzilla, but this place is like nothing I've ever seen, and it's quite a mystery to me. Is there a tutorial somewhere?

@jdgd1
Copy link

jdgd1 commented Apr 28, 2014

Not sure what has ultimately been done with this issues, but I created a modification of the candlestick function which addresses all the issues above, and allows for the use of different color themes.

def candlestick(ax, quotes, width=0.2, colorup='k', colordown='r',
            alpha=1.0, coloredge='k', theme='light'):

"""

quotes is a sequence of (time, open, close, high, low, ...) sequences.
As long as the first 5 elements are these values,
the record can be as long as you want (eg it may store volume).

time must be in float days format - see date2num

Plot the time, open, close, high, low as a vertical line ranging
from low to high.  Use a rectangular bar to represent the
open-close span.  If close >= open, use colorup to color the bar,
otherwise use colordown

ax          : an Axes instance to plot to
width       : fraction of a day for the rectangle width
colorup     : the color of the rectangle where close >= open
colordown   : the color of the rectangle where close <  open
alpha       : the rectangle alpha level
coloredge   : the color of the rectangle border
theme       : specify if using light theme or dark theme

return value is lines, patches where lines is a list of lines
added and patches is a list of the rectangle patches added

"""

OFFSET = width/2.0

if theme == 'light':
    coloredge = 'k'
    coloredgeup = 'k'
    coloredgedown = 'k'
else:
    coloredgeup = colorup
    coloredgedown = colordown

lines = []
patches = []
for q in quotes:
    t, open, close, high, low = q[:5]

    if close>=open :
        color = colorup
        lower = open
        height = close-open
        vline = Line2D(xdata=(t, t), ydata=(low, high), color=coloredgeup, linewidth=0.5, antialiased=True, zorder=0)
    else           :
        color = colordown
        lower = close
        height = open-close
        vline = Line2D(xdata=(t, t), ydata=(low, high), color=coloredgedown, linewidth=0.5, antialiased=True, zorder=0)

    """vline = Line2D(
        xdata=(t, t), ydata=(low, high),
        color='k',
        linewidth=0.5,
        antialiased=True,
        ) """

    # Should look for a better way to do this
    if theme != 'light':
        coloredge = color

    rect = Rectangle(
        xy    = (t-OFFSET, lower),
        width = width,
        height = height,
        facecolor = color,
        edgecolor = coloredge,
        zorder=1,
        )
    rect.set_alpha(alpha)


    lines.append(vline)
    patches.append(rect)
    ax.add_line(vline)
    ax.add_patch(rect)
ax.autoscale_view()

return lines, patches

@tacaswell
Copy link
Member

@jdgd1 Could you please put that in a PR?

jdgd1 added a commit to jdgd1/matplotlib that referenced this issue Apr 28, 2014
1. Added ability for the candlestick function to use a "light" and "dark" theme
2. Fill color of candlestick bodies can be user definite and is based upon stock price movement on each given period
3. In the light theme (which uses a white background) candlestick shadows are dark; ie: Follows traditional candlestick format
4. In dark theme (which uses black background) candlestick shadows match box colors
@jdgd1
Copy link

jdgd1 commented Apr 28, 2014

@tacaswell PR created. Let me know what ends up happening with these functions, as I use them heavily.

@lifetime42
Copy link

thank you very much for your function jdgd1. exactly what i was looking for. it didnt make it into the master branch? how come?

@efiring
Copy link
Member

efiring commented Sep 28, 2015

#3016 needed some revision, and seems to have been closed inadvertently, based on the last comment in it. We still need someone to maintain the finance module. I don't think we want to lose the apparent progress in that PR if we can avoid it.

@lifetime42
Copy link

Thank you for the link to the code discussion, I ended up altering the finance.py according to jdgd1s input; I dont use the dark/light scheme but a simple replacement of the edgecolor does work wonderfully with =None in constructor so the rest of the code can stay the same. The real issue here was the zorder and the vlines. Now the charts looks just as I wanted.
http://pastebin.com/fgTXiavs

@jdgd1
Copy link

jdgd1 commented Sep 29, 2015

@lifetkme42 Glad to hear the code worked for you. Let me know if any issues arise.

@efiring I will take a look at the code again today and try to clean it up a little. As I mentioned in the past I'm happy to help, but not as sure I have the time to responsible for maintaining it.

@tacaswell
Copy link
Member

@jdgd1 If you have any time or expertise it will be an improvement.

lifetime42 added a commit to lifetime42/matplotlib that referenced this issue Oct 1, 2015
This attempts to give the user the option to choose a color for the edge
of a candlestick body and the high-low bars (optional). If no argument
is given the behavior is just
like before. It also should fix the vertical line (high-low) zorder so
its behind the candlestickbody now.
Idea mainly by jdgd1. I tried to keep existing code working while adding
the feature.
matplotlib#3016
matplotlib#2546
Hopefully fixed the whitespaces.
@jdgd1
Copy link

jdgd1 commented Oct 1, 2015

@tacaswell ok, I will help as much as I can. Let me know where to start.

@tacaswell
Copy link
Member

@jdgd1 For starters, can you review #5160 ?

@ssanderson is a dev at Quantopian who has expressed (very very) lose interest in also helping with the finance module, it is probably worth getting in touch with him.

A survey of what is in the finance module would be good to do. Are the functions we have actually useful to a 'typical' finance type anymore? Are there obvious functions we are missing? A major struggle with that package right now is that none of the core-devs do finance so we lack the domain expertise to answer these questions.

Related, I don't think we have any prose documentation for the finance module, turning some of the examples in to more proper tutorials might be worth doing.

There has been some discussion of splitting the finance module out of core mpl (into a mpl-finance repo in the matplotlib gh org). If we did this, then it could pick up a pandas dependency which would definitely help increase the relevance to current finance users. This may be something that the pandas folks would be willing to help with.

I know this is a daunting list and by no means expect you to tackle all of it, but just trying to make the point that this is a domain where there is significant opportunity to have a major impact.

@jdgd1
Copy link

jdgd1 commented Oct 1, 2015

@tacaswell wow, you weren't kidding - that is quite the list. Will start with checking out #5160 and then trying to get in touch with @ssanderson. I've actually been spending a good bit of time lately working between Quantopian/zipline, so that could work out well.

Post those two things will start digging into the finance module and making my own list of what I think can go vs. what's missing.

@ssanderson
Copy link

@jdgd1, @tacaswell hello!

@ssanderson
Copy link

(@tacaswell I just realized I've owed you an email reply for like 3 months...sorry about that)

@tacaswell
Copy link
Member

@ssanderson Don't worry, that still leaves me way in the red in terms of net-emails-owed.

@ssanderson
Copy link

A couple notes on the discussion here:

  • I/we am/are interested in helping out with mpl.finance, but I'm not sure anyone has the bandwidth to make a sustained contribution right now. Zipline is basically being completely rewritten from the ground up on 2 different fronts right now, so we're stretched a bit thin.
  • I'm probably not any more qualified than the MPL devs to comment on things like how to display candlesticks properly, but I can at least help provide real-world user stories/issues from our user base. I suspect that our forums would also be an interesting place to ask for feedback/comments on this stuff.
  • Most of the interesting work we've done in financial visualization at Quantopian has been building visualizations for analyzing portfolio performance. @twiecki and co have been working on pyfolio, which is built primarily on top of pandas, matplotlib, and seaborn. I'm not sure what the right relationship would be between that project and mpl.finance.

@lifetime42
Copy link

I only picked up python recently and I just started using matplotlib, but I can tell you these charts are looking quite nice and easy to handle (I used jfreechart before)

Things the finance modul could use are eg.
-hollow/filled candlesticks (tc2000 style) adding additional information to one candle
-easy subchart addition right into the chart itself or below for indicators
-easy access to daw into the chart itself eg. support/resistance lines etc

to make @jdgd1 list even longer ;-)

@femtotrader
Copy link

@tacaswell @ssanderson +1 for mpl-finance as a separate repository with Numpy/Pandas as dependency.

@femtotrader
Copy link

Pinging some people from PyData / Pandas-DataReader and others who might be interested in a Matplotlib / Numpy / Pandas / Finance plotting lib @davidastephens @bashtage @hayd @jorisvandenbossche

@femtotrader
Copy link

The scope of such a lib could be (or not) to be able to display "realtime" plot. PyQtGraph is nice for realtime plot so pinging also @campagnola

See

http://www.pyqtgraph.org/
http://stackoverflow.com/questions/24670856/the-fastest-way-to-add-a-new-data-bar-with-pyqtgraph

@genliu777
Copy link

genliu777 commented May 20, 2016

so far i use hecandlestick2_ohlc, and come across the same shadow line through the real body! this question has been put here almost 3 years and not solved!!

@rkahun
Copy link

rkahun commented May 19, 2017

@genliu777 There is a quick resolving, use z-order as it had mentioned. It works for me without any side effect.
(By default lines have 2 and patches have 1 as z_order.
Of course, color of edges and shadows has not changed)

shadows, bodies = candlestick2_ohlc(ax, .... )
shadows.set_zorder(0)  # or 9
bodies.set_zorder(1)   # or 10 or so

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests