In Safari 9.1 on Mac OS X the color picker does not open when using the color picker in 'swatch only'-mode.
Reproduction scenario: open the demo page and set 'Swatch Only (swatch-only)' to Yes. Then try to open the color picker by clicking on the swatch.
After investigating the issue it seems that elements do not receive focus in Safari if either the width or the height of the element is 0. In my current project I have resolved the issue as follows:
.color-picker-wrapper.color-picker-swatch-only .color-picker-input {
width: 1px;
height: 1px;
}