Do we want to support NA, Inf, etc. as in scale_x_continuous(limits=c(NA, 5)) or coord_cartesian(xlim=c(NA, 5)) ?