Currently, it is practically impossible to use gulp-useref with gulp-sourcemaps, because gulp-sourcemaps does not support processing multiple file types in a single pipeline. This is something which @doxavore complains about in the bottom of #64. In other words, we end up with either js or css source maps, but not both.
The only way to get around this is introducing separate pipelines for js and css assets. I can try to implement this feature, if you agree with me.