New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FUTURE: Remove printer driver and raw queue support. #5271

Open
michaelrsweet opened this Issue Mar 19, 2018 · 49 comments

Comments

Projects
None yet
@michaelrsweet
Copy link
Collaborator

michaelrsweet commented Mar 19, 2018

Note: The goal of this issue is to have a place to discuss the eventual removal of printer driver
support and support for raw queues. This bug has not been scheduled for a specific release
milestone and this is not something we will implement immediately. Because most printer drivers
exist outside of CUPS this change needs to be discussed publicly.

Why do we want to do this?

  • Raw queues used for special-use printers require custom applications that know about printer capabilities and how to produce printer-ready (document) data. Using CUPS is a convenience, but plenty of applications talk directly to printers (think Point of Sale systems)
  • Raw queues used for shared printers require client software to talk directly to the server to get the printer capabilities, which breaks when sandboxing/AppArmor/SELinux is used
  • Almost every printer manufactured since 2010 supports IPP/2.0 with standard file formats
    • Holdouts are industrial label printers and certain vertical market printers
  • PPDs and drivers have been holding us back from offering better user experience (ready media, localization, full range of printer options/values), improved document processing, and improved accounting
  • PPDs and drivers are a security and distribution nightmare

We already have a replacement for raw queues for shared printers (local/temporary queues managed by cupsd), and raw queues for special-use printers already largely bypass CUPS and can use existing commands or character device files to communicate with those printers.

As for printer drivers, those few printers that "need" them could migrate to standalone applications/services using the CUPS API to provide an IPP Everywhere-compatible Printer instance, and both the one-file ippserver sample code and the larger implementation in the PWG IPP Sample code repository show this is feasible, even for existing CUPS-based drivers. And as an advantage of this approach drivers like Gutenprint will no longer be shackled by the limitations of PPD files.

@michaelrsweet michaelrsweet added this to the Future milestone Mar 19, 2018

@michaelrsweet michaelrsweet self-assigned this Mar 19, 2018

@jsmeix

This comment has been minimized.

Copy link

jsmeix commented Mar 20, 2018

This looks a bit scaring without a planned date or CUPS version
when printer driver and raw queue support will be actually removed.

In #5269 the commit
003c179
reads (excerpt):

The "raw" model is deprecated and will not be supported in a future version of CUPS.

In #5270 the commit
ada35e5
reads (excerpt):

PPD files and printer drivers are deprecated and will not be supported in a future version of CUPS.

I would like to know at least a currently estimated date or CUPS version
when PPD, printer driver, and raw queue support will be actually removed.

@imsedgar

This comment has been minimized.

Copy link

imsedgar commented Mar 20, 2018

In issue #5270 you wrote that

basically all printers sold in the last 8 years support IPP and standard file formats, we should formally deprecate printer drivers in CUPS
But printers are used longer that 8 years. For example, we use three printer HP LaserJet 4350 DN, which are first sold about 2004, and we have HP LaserJet 2300, first sold about 2003. They support no IPP Everywhere (at least as I know), and there may be many other printers around that need printer drivers.

Their may be also special printers that are supported only by a special (proprietary or custom) software, which cannot easily integrated into cups. Printer drivers should not disturb that printer code by changing it in any way - cups should just queue it and forward it unchanged to the printer. This is what raw queues are for.

The code for processing printer drivers and raw printers are already in cups, and it is working since many years. Why will you remove it and make many printers unusable, or force users to use old cups versions?

Please leave the code in cups, it already exists and is working, and I think that IPP Everywhere can coexists with the old code, as an alternative. You may document that the new interface should be used for new printers, that's ok.

Another possibility would be that the code for printer drivers and raw printers will be transferred to another (new?) project in a way that it can be easily integrated into cups, like an add-on or other printer drivers.

@michaelrsweet michaelrsweet changed the title Remove printer driver and raw queue support. FUTURE: Remove printer driver and raw queue support. Mar 20, 2018

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Mar 20, 2018

@jsmeix @imsedgar This is a future planning issue, not a notice of impending changes.

I will update this issue with some of the details ASAP, but we will be discussing this in detail at the May 2018 Open Printing Summit/PWG meeting.

@jsmeix

This comment has been minimized.

Copy link

jsmeix commented Mar 20, 2018

@michaelrsweet
many thanks for your prompt reply.

"future planning" looks much less scaring ;-)

Nevertheless I wonder if printing really could work
in practice "out there" in the foreseeable future
without any driver software i.e. without any
"device dependant additional software".

I guess manufacturers will always develop new kind of devices
or new kind of device functionality which needs device dependant
additional software to get the new stuff working.

By the way:
In Feb. 2018 there was a discussion on the Gimp-print-devel
i.e. the Gutenprint devel mailing list with subject
"OSX support (Was Re: CITIZEN photo printers)"
that wandered from the subject because I wrote something about
"PPD files are declared 'deprecated' at CUPS upstream"
and a Gutenprint developer replied that for printer driver developers
the PPD APIs are all they have to work with, see
https://sourceforge.net/p/gimp-print/mailman/gimp-print-devel/thread/20180219222444.GA18161%40shaftnet.org/#msg36231072

On Mon, Feb 19, 2018 at 11:37:23AM +0100, Johannes Meixner wrote:
> As far as I know PPD files are declared "deprecated" at CUPS upstream
> (but PPD files will still be supported by CUPS for a longer time)
> and current CUPS provides newer APIs for that functionality.
...
Unfortuntely, those "newer APIs" appear to only be suitable for 
printing *clients*.  For those like Gutenprint who are trying to write 
printing *drivers*, the PPD APIs are all we have to work with.

Solomon Peachy
@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Mar 20, 2018

@jsmeix It can and does work for billions of mobile devices in the wild today. And (2D) printing is not exactly a new industry - the vendors developing the printers are also the ones developing the standards we rely on, to the point that the IPP workgroup is left addressing edge cases, writing testing tools, and clarifying existing specifications.

As for Gutenprint, with all due respect I started that project and have done things before that Robert and others claimed were infeasible or impractical (Mini-XML to replace libxml2 plus the whole CUPS filter, backend, and driver interface to make Gutenprint a system-wide driver instead of just a Gimp plugin). Putting an ippserver wrapper around Gutenprint is trivial, the only "challenge" will be doing a decent UI for all of those controls.

@jsmeix

This comment has been minimized.

Copy link

jsmeix commented Mar 29, 2018

@michaelrsweet
I know that you started Gimp-Print that is now called Gutenprint
which is why I mentioned that particular Gutenprint driver discussion
as an example why even nowadays printer driver developers may
need some help to better understand how they could make
their device dependant additional software (e.g. "drivers")
in compliance with nowadays and future CUPS.

I have a general understanding question:

Is my understanding right that future CUPS (without built-in driver support)
requires that its recipient (i.e. whereto the cupsd will sent the print jobs)
is an "IPP Everywhere-compatible Printer instance" where such
an "IPP Everywhere-compatible Printer instance" can be basically
anything where the cupsd can communicate via IPP protocol and
which accepts the standard IPP Everywhere data formats?

If my understanding is right, it would mean that any physical device
that is not IPP Everywhere compatible could be made such
an "IPP Everywhere-compatible Printer instance" by some
device dependant additional software that implements
an "ippserver wrapper" around that physical device
so that it appears to the cupsd as such an
"IPP Everywhere-compatible Printer instance".

Is my understanding right?

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Mar 29, 2018

@jsmeix Yes, an IPP Everywhere printer application (or even a hardware box - several companies have them for AirPrint already) can provide a compatible interface for any CUPS 1.4 or later client.

@jsmeix

This comment has been minimized.

Copy link

jsmeix commented Apr 6, 2018

I have one more general understanding question:

As far as I understand it such an "IPP Everywhere-compatible Printer instance"
provides the traditional CUPS server functionality to run printer driver software
that converts commonly known application data formats like JPEG or PDF
into the final printing unit's native format like PCL or any other printing unit's
native format because the "IPP Everywhere-compatible Printer instance"
must accept PWG Raster Format and JPEG and should accept PDF.

Accordingly when CUPS requires that its recipient must be an
"IPP Everywhere-compatible Printer instance" future CUPS will act
only as an "IPP Everywhere Client" which matches what you wrote in
#5271 (comment)
"any CUPS 1.4 or later client".

But when CUPS acts only as a client I wonder if there is still a reason
for a cupsd?

If my understanding is right, for usual end-user use cases it should be
sufficient when applications only call CUPS library functions to submit
print jobs directly to an "IPP Everywhere-compatible Printer instance".

I could imagine a cupsd "in between" the printing application and the
"IPP Everywhere-compatible Printer instance" could be still useful
for high load use cases.
For example when the printing application is something like SAP
that may all of a sudden at 03:00 o'clock in the night submit
thousands of print jobs within a few minutes so that massive
queueing is needed on a traditional CUPS server "in between".

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Apr 9, 2018

@jsmeix Please do attend (either in person or by calling in) the 2018 Open Printing summit, where I will be talking about these issues.

Basically, CUPS can't just be the client library because we still need background spooling and file conversion, even with IPP Everywhere. And we don't want to keep PPD files and printer drivers around - yes it is technically possible, but it also retains all of the limitations we have today.

The SAP use case is not valid - even a simple client spooler could handle thousands of jobs in a short period of time, with IO speed being the primary constraint.

I do envision a dedicated CUPS server that is focused on enterprise functionality like access control, accounting, release printing, high-speed RIP capabilities, etc. This server would be the front-end for other IPP Everywhere printers that are not directly accessible from clients, and support things like the IPP Shared Infrastructure Extensions so that those printers could be anywhere in the world. But this server would only get used on servers, not clients...

@RobertKrawitz

This comment has been minimized.

Copy link

RobertKrawitz commented May 2, 2018

Gutenprint already has the old GTK-based UI (libgutenprintui2); perhaps it will be possible to use that at least for prototype purposes.

@UliWehner

This comment has been minimized.

Copy link

UliWehner commented May 21, 2018

i would like to see some more discussion about deprecating CUPS PPDs. Airprint has very limited functionality compared to an actual printer driver. For a mobile phone this may be acceptable (barely) but i don't foresee that users will accept this limited functionality. I did not make the printing summit but will be reading up on the topics. IPP Everywhere is apparently nowhere near providing the kind of functionality corporations are looking for. A lot of related software directly ties into PDLs and PPDs. Example: all print accounting software has to be able to parse the print data, most of that software runs on windows servers. How do you picture this will work without drivers and printer languages like Postscript, when 10s of thousands of MAC users that currently send Postscript jobs via SMB or LRP to a Windows server switch over to "Airprint"? or IPP Everywhere? I am pretty sure my customers have no idea this is coming....

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented May 21, 2018

@UliWehner AirPrint isn’t a public specification, but it is based on IPP and absolutely covers all important driver/printer features (just not “red eye removal” and other features that don’t belong in a driver anyways).

That said, this is a bug for tracking the future removal of driver support. PPDs have been deprecated for 10 years and drivers were just deprecated for CUPS 2.3 - we are not rushing into this and there will be ways to support legacy printers when CUPS stops supporting drivers, just not directly through CUPS.

@Michael-IDA

This comment has been minimized.

Copy link

Michael-IDA commented May 30, 2018

Printers can easily last 20 years, please do not implement this feature.

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Jun 4, 2018

@Michael-IDA We are not eliminating support for 20-year-old printers, we are just planning to move support for such printers to dedicated applications/services so that we don't have to drag around the 30+ year old PPD metadata that is preventing us from fully supporting modern (less than 10 year old) printers.

@Michael-IDA

This comment has been minimized.

Copy link

Michael-IDA commented Jun 4, 2018

Hi Michael, (@michaelrsweet)

Sorry for what probably came across as grumpy?

As pretty much an ignorant individual IRT CUPS, the synopsis at the top reads to me as:

A) CUPS is removing the code for PPDs and {whatever else?} is needed for printers not using {IPP?}.
B) CUPS is washing it’s hands of A) and someone else can create dedicated applications/services to service A).

If B) is actually the CUPS project’s intentions, then old printers are just ‘shit out of luck’ as no one is going to create dedicated applications/services for a few one off printers. Especially since these printer ‘use to have’ a methodology that made them functional. While installing old versions of CUPS is viable, at some point that will also entail the user having to install old version of the OS, with all the resultant security nightmares, to satisfy the old version of CUPS’ requirements.

If B) is not true, would you clarify in the synopsis that the CUPS project itself is the one who’s going to make these ‘dedicated applications/services?’

#

While it really does come across that B) is the CUPS project’s intentions, I guess really the lack of actual clarification of who is suppose to be making these dedicated applications/services is what prompted my post.

Best Regards,
Michael

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Jun 4, 2018

@Michael-IDA

First, please note the "FUTURE" in the title of this issue - this isn't something that will happen overnight.

Long-term, we do plan to drop PPD code from CUPS and rely entirely on IPP for supporting printer options, reporting status and configuration, and so forth. Support for using IPP for options, status, and configuration has been in CUPS since v1.0 (20 years ago...)

Before we drop PPDs, we will be working with developers to create those printer applications (particularly Gutenprint), and will likely end up creating our own printer application (based on the existing ippserver sample code) to support existing PostScript and CUPS raster drivers. So the goal is definitely not to leave people SOL, but rather to plan for a future where we aren't limited by the capabilities of the past.

@UliWehner

This comment has been minimized.

Copy link

UliWehner commented Jun 11, 2018

@michaelrsweet, you may remember that i am a great fan of driverless printing... right now it does not appear that the existing printer tools allow creation of an IPP everywhere queue when that queue can not be discovered? Am i wrong? many companies do not put printers and laptops on the same network. lpadmin works fine but is certainly not what users expect to use.

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Jun 11, 2018

@UliWehner Currently you can manually add a queue using the "everywhere" driver in the Web Interface or using lpadmin, on macOS you can add an "IP" printer and have it do the right thing, and I think the Linux printer configuration GUIs have similar functionality.

Longer term we want to add support for configuration profiles (pointing to printer(s) and/or server(s)) and LDAP. All completely transparent to the application/user.

@UliWehner

This comment has been minimized.

Copy link

UliWehner commented Jun 11, 2018

@michaelrsweet, on High Sierra with CUPS 2.2.5 this is certainly not the case. I have to choose a PPD, both in the cups webinterface as well as with the printer setup tool. Is this also a future feature for CUPS 2.3? I can use lpadmin just fine but i can not see a GUI tool that will do this on High Sierra with CUPS 2.2.5. I don't want to clog up this thread but i do not think it is possible today to manually create an IPP everywhere printer short of using lpadmin. Shoot me an email if you like

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Jun 11, 2018

On macOS, the printer needs to advertise AirPrint-specific attributes to be auto-configured from the Print & Scan preferences. Not sure why the "IPP Everywhere" model isn't showing up in the web interface, though...

@UliWehner

This comment has been minimized.

Copy link

UliWehner commented Jun 11, 2018

@michaelrsweet, that is exactly what i have been trying to say. My printers are airprint printers, this works perfectly with lpadmin, but i can not find a gui, or cups webinterface way to do this manually. Not in Mac OS X and not in OpenSuse Tumbleweed with 2.3b4.

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Jun 11, 2018

@UliWehner On macOS you can open an ipp: or ipps: URL and it will get added as AirPrint (if it is AirPrint). I filed Issue #5338 to track fixing the CUPS web interface.

@nahall

This comment has been minimized.

Copy link

nahall commented Jun 13, 2018

Our use case for RAW print queues (and is mostly the reason we use CUPS in the first place) is that we have several label printers that are printed to by a Windows application that we don't have the source to and unfortunately is pretty dumb so it can only be set to print to one printer.

So we created a virtual label printer using CUPS and share that with Samba so the Windows application can print to this virtual printer, and then have all the different label printers also set up in CUPS as RAW printers, and we have a little script using tea4cups that basically forwards the incoming print job to the different label printers based on some scriptable criteria that decides which printer to send it to.

It has worked for years and works very well. The Windows machine handles the driver so CUPS is just redirecting the data as it doesn't know what the print job is or care.

If I'm understanding you correctly your proposal will prevent us from using CUPS in this way in the future. Am I correct about this? Thanks.

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Jun 14, 2018

@nahall Yes, in the future there might not be a way to do this directly through CUPS. But since you are using Samba you can just have a printer share that runs the tea4cups script to "print" something and let Samba manage things for you. :)

@tresf

This comment has been minimized.

Copy link

tresf commented Jul 13, 2018

So we created a virtual label printer using CUPS and share that with Samba so the Windows application can print to this virtual printer, and then have all the different label printers also set up in CUPS as RAW printers, and we have a little script using tea4cups that basically forwards the incoming print job to the different label printers based on some scriptable criteria that decides which printer to send it to.

It has worked for years and works very well. The Windows machine handles the driver so CUPS is just redirecting the data as it doesn't know what the print job is or care.

If I'm understanding you correctly your proposal will prevent us from using CUPS in this way in the future. Am I correct about this? Thanks.

I'm in a similar boat, but without the tethered design. I run a project that hands-off raw commands to printers and we really only use CUPS to configure the raw queue so we have a way for the end-user to configure the printer while the application vendor controls the print sequences.

One of the design goals of CUPS was to eliminate the need for raw printing, and to provide a common interface and print file format to all applications so that UNIX applications and users could enjoy the same benefits as those on macOS and Microsoft Windows which likewise provide a common printing interface.

This is true for consumer printing, but for industry printing, raw is still king. But let's talk specifics... Some drivers on Windows (such as Epson TM-series and Zebra label printers -- both widely used in the industry) accept mixed content. They allow modern printing techniques as well as -- lightning fast -- raw commands and detects the print flavor and acts accordingly. When I say "lightning fast", I mean by a factor of 5x speed in some cases when compared to loading the libraries necessary to do PDF rendering combined with the spool time for the graphics and shapes. Furthermore, these printers are generally restricted by very low DPI (e.g. 180 for point-of-sales, 203 for shipping) which means the industry is often OK with a tiny chunk of text to represent a (relatively) large amount of pixel-perfect output. Placing PDFs or Images in the middle of that often doesn't scale well to the low-dpi printers, so -- not only speed -- but also quality tends to suffer.

Although the announcement states several times that this "isn't happening overnight", the blog states 2020, which is 2-years and is a timeline that can really rip out the Macs and Linux machines from the industry in a timeline that many of us need to prepare for.

Some questions...

  1. Will lpr -o raw ... and friends be removed? We use this technique as a fallback on occasion to use RAW without specifically using the RAW driver.
  2. Will whatever replacement to PPDs be able to support -- at a high level -- the ability to send raw commands if the driver manufacturers get on board?
    • Again... using specifics... CUPS bundles the Zebra ZPL and EPL drivers which leads me to believe regular driver enhancements from Zebra and friends don't make it in regularly. Is the plan to remove these drivers or have Apple work with the driver guys at Zebra to make new ones?

Before we drop PPDs, we will be working with developers to create those printer applications (particularly Gutenprint), and will likely end up creating our own printer application (based on the existing ippserver sample code) to support existing PostScript and CUPS raster drivers. So the goal is definitely not to leave people SOL, but rather to plan for a future where we aren't limited by the capabilities of the past.

This is comforting for sure. :)

Today the only printers that “need” a raw printing interface are dedicated label, receipt, or logging printers, and those same printers are poorly served by CUPS because the applications using them assume they have exclusive access to the printer, send a single “page” of arbitrary length that is not known ahead of time, and/or have specific timing requirements that CUPS cannot guarantee.

I can see the problem, and access a printer through a RAW technique definitely leaves a lot of the advanced messaging that a business relies on out of the picture, but RAW access to a printer that's listed in the System Preferences (regardless of whether or not CUPS is the communication channel) is very nice from a PC support perspective. We use this technique with some rather large clients and they like the flexibility of being able to support Mac and Linux just as well as Windows by setting up the RAW driver. If the driver manufacturers don't meet the same timeline as CUPS, it will force the ISVs to make middleware to fill this gap by pulling comm info from the printer mapping and trying to bypass the spooler entirely. This works well but seems like a regression. Please demystify any false assumptions I have here and P.S. thanks for CUPS. :)

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Jul 16, 2018

@tresf In answer to your questions:

  1. Will lpr -o raw ... and friends be removed? We use this technique as a fallback on occasion to use RAW without specifically using the RAW driver.

Yes, the "raw" option will no longer be supported. You might be able to specify the MIME media type for a printer's "native" (legacy) PDL, e.g., application/vnd.hp-pcl for PCL, but we won't be supporting "raw" printing like we used to (forces us to make too many compromises).

  1. Will whatever replacement to PPDs be able to support -- at a high level -- the ability to send raw commands if the driver manufacturers get on board?

IPP supports "raw" printing, meaning printing using some random vendor PDL, and that is the bare-bones minimum required for IPP/1.1 conformance. Prior to about 2009, that was pretty much all most vendors supported, which greatly limited what you could actually accomplish using just IPP and a printer.

...

All that said, it would certainly be possible to implement a raw-printing service (something you could use instead of/along side of CUPS) that just maintains queues for IP/USB printers and copies the print data directly without conversion/processing/translation. Such a service could be optimized for speed and simplicity while still providing the same sorts of convenience you get with "lp -oraw".

@tresf

This comment has been minimized.

Copy link

tresf commented Jul 23, 2018

All that said, it would certainly be possible to implement a raw-printing service (something you could use instead of/along side of CUPS) that just maintains queues for IP/USB printers and copies the print data directly without conversion/processing/translation. Such a service could be optimized for speed and simplicity while still providing the same sorts of convenience you get with "lp -oraw".

Speaking from a business that spends most of its time with raw devices if there's a relatively simple mechanism -- via configuration, programming or driver -- that allows (e.g. a Qt-based or a Java-based) application to talk the printer, that would suffice but what I think is not immediately obvious to the consumer -- by consumer, I mean a person consuming the CUPS as an interface from a 3rd-party API perspective -- what the elimination of raw means and if it will outright kill a product benefiting from this on a Unix-style platform.

When I say "kill", I'm embellishing on the fact that we'd otherwise need mechanisms for communicating directly with serial|parallel|USB|IP-based devices quickly. I realize CUPS really only offers a printer name in these instances, but these frameworks make the printer names iterable on Windows, Linux and MacOS which provides a direct path for raw communication. Java provides a raw byte array as part of their permanent API and for Qt we just use the Qt APIs to fetch the printer name and then manually call cupsStartDocument with the CUPS_FORMAT_RAW flag.

So from a C++ perspective, a replacement API should suffice, but it will require some type of negotiation and upgrade for the API calls -- of course the old API would fail to compile -- and I assume Java would be left in the dust for those applications that rely on raw printing from a Unix system. I'm not sure how Mono/.NET would be affected as I've never used their API.

So from a developer's perspective, I still am not sure what type of options we'll have if we plan to continue using Unix for the foreseeable future.

@Michael-IDA

This comment has been minimized.

Copy link

Michael-IDA commented Jul 25, 2018

A few lay thoughts on this:

A) Standard corporate reply is, “Go buy a NEW printer.” People who make statements that contradict that present themselves as either 1) woefully ignorant of reality or 2) that they believe their audience are such idiots that they can’t identify outright falsehoods.

B) A decent printer can last 20+ years.

C) From a consumer's perspective, CUPS (formerly an acronym for Common UNIX Printing System) has been jettisoning being “Common” capable for awhile.

Changes like up revving drivers/libraries that PPDs/filters are dependent upon have already removed those printers from CUPS’ common capability. [1]

There is no need that these printers can no longer work with CUPS, as software stacks handling these types of dependency requirements are routine. Example: Just as Apache can be told what version of PHP to use for a specific Website, CUPS can be told what Library version is needed for a specific Filter.

It’s also not like the PPDs and {everything else that “is preventing us from fully supporting modern (less than 10 year old) printers”} can’t be completely containerized within CUPS so it doesn’t affect anything else that allows CUPS to “fully support modern (less than 10 year old) printers.” Stuff like that is done all the time with a simple IF statement upon code entry.

@michaelrsweet commented on Jun 4
“Long-term, we do plan to drop PPD code from CUPS”

Then maybe you need to abandon the namespace CUPS? Really with the continued dropping of common capability this issue brings, shouldn’t CUPS be renamed APS?

Yes that blows a huge chunk of Apple’s “Good Will” that CUPS has given Apple over the last 11 years, but at least it’s an honest statement of intent.

#

Michael,

As opposed to trashing CUPS with this issue, why not actually embrace what the name came from?

A statement that, “CUPS will function for every PPD/filter/etc. for 20 years after that PPD/filter/etc. printer was manufactured,” is much more in line with the namespace of CUPS that you created.

While ultimately it’s up to your corporate bosses for funding for CUPS, aren’t you even willing to try? “Good Will” is a balance sheet item, raising their awareness that this issue is destroying it will probably get their attention.

Regards,
Michael

PS: Seriously man, it’s been your project since 1997, was the word ‘common’ just smoke and mirrors?

[1]
OpenPrinting/cups-filters#48

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Jul 26, 2018

@Michael-IDA "Common" in the original "Common UNIX Printing System" never meant "support every printer and application on the planet."

Common meant:

  • Using the same printing system and application/driver interfaces on multiple operating systems;
  • Using the same document format(s) for all printers, regardless of their actual capabilities (initially PostScript, later PDF) so that applications (and the toolkits they used) could produce pages using regular drawing functions rather than hardcoding file format support into every application;
  • Using the same option names for all printers, regardless of what the underlying driver/printer might call something; and
  • Using a common set of backends for communicating with printers so that drivers (printer-specific or generic - we've had both since day one) don't need to hardcode support into every driver.

WE ARE NOT SAYING THAT YOU SHOULD BUY A NEW PRINTER.

Rather, we are saying the basically all general-purpose printers sold in the last 10 years already support what we need to provide a more efficient printing system, and that we are moving support for (at least some) driver-based printers to a separate component (a "Printer Application" or its equivalent) which can better address the needs of that "segment" - typically fine-art/photo/label printers with custom USB protocols that don't work well with the current CUPS backend architecture or with complicated configuration/presets that overwhelm print dialogs.

I think it is clear (or it should be given everything I've said and presented over the last 4 months) that enterprise printers (the kinds of printers that last 20+ years and that you can still buy supplies and parts for) will continue to be supported, and that we are still figuring out the best way to do so - PPDs are a natural fit for PostScript printers, but not the best fit for PCL. Windows (SMB) printing is also something we want to continue supporting, but again we don't want to keep doing things the same way because we have tons of issues with the variety of "solutions" in that space.

Raw printing is typically used in two spaces: custom label/report printing and Windows printing. The Windows (SMB) printing case is almost always geared towards general purpose enterprise printing which we can (and will continue to) support from CUPS in some way. Raw label/report printing is primarily done in a closed production environment (SAP, UPS/FedEx/name-your-shipper APIs, etc.), and there I think a dedicated raw spooler is a better, faster, lighter weight solution than keeping raw support in cupsd (where it is a special case we have to check for at every turn). This custom spooler could continue to support printing via CUPS API (where it just appears to be another local IPP queue) but otherwise the jobs would be routed separately where we could pipeline things and optimize the error handling/retry logic.

Finally, we have too many "simple IFs" in CUPS already. Really, look at the code. We're at the point that making changes in one (seemingly) isolated place can cause things to break elsewhere, and we can't add a lot of new functionality without jumping through a lot of hoops and adding lots of "simple IFs".

Printing is not simple - if it were we would not have the number of problems we do, even after 20 years of making it as simple and reliable as possible.

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Jul 26, 2018

@tresf We don't want to leave raw printing users in the cold, but how we support those printers is going to change because raw printing is fundamentally different from general-purpose printing.

A label printer can have a driver (or in the future a Printer Application), and that can be exposed via the standard OS print dialog. That driver (and the Printer Application) can also advertise the printer's native print format (EPL, ZPL, etc.) allowing an application that can produce the format to send printer-ready data. That is the model we want to see, not force users to depend on a specific application to support their printer (at which point the application is on its own).

@lite1979

This comment has been minimized.

Copy link

lite1979 commented Jul 26, 2018

@michaelrsweet

In my experience (I work with @tresf), the drivers for label printers in MacOS and Linux typically do not support raw printing; only raster/2d jobs. Take the ppd for a Zebra printer shared here, for example:

https://gist.github.com/JakeVdub/3cce5604cf82d0172a4c2ee267e6b677

This works well for printing PDFs from MacOS or Linux to my Zebra 2844-Z (an old label printer, but still works like a champion), but if I want to send ZPL to it, I create a separate raw queue that acts as a pass-through to the port/IP at which the printer is connected. Raw printing is much, much faster than 2d printing, since it uses the processing power of the printer, not the PC, and as such is vastly preferred for industries such as shipping/fulfillment, Point of Sales (receipts, kitchen tickets, grocery stores), printing concert/event tickets, and report (think OKIDATA - they still sell for over $300 USD) printing.

Is it the pass-through nature of raw printing and lack of a driver that makes it so different from PostScript/PDF/2d printing in CUPS' perspective?

In my experience, raw queue support is a peripheral communication problem, but this has seemed to be relatively easy to navigate without CUPS, as long as you have a little command-line experience. The trouble is, when polling the printers in the OS, we'd like to see those peripherally connected devices as well as our laserjets and inkjets, even if the OS just sees them as a COM port.

I'd like to offer to test whatever solution you come up to for raw printing via CUPS as soon as possible to make sure we don't have a gap in functionality (alpha, beta, RC, et cetera). I've already noticed some PPDs missing from recent versions of CUPS (like the Zebra one posted above, for example), but with no solution for replacement. I hate telling someone that CUPS doesn't ship with this driver anymore, so download this one from ten years ago.

I want to be able to walk them through the process (or better yet, publish a wiki article that does the same) for both raw and PS-style printing and be certain that the most recent version of CUPS will support it.

"Raw label/report printing is primarily done in a closed production environment"

I think this is changing, too. We get a decent amount of inquiries, particularly from MacOS users, on how to print Amazon labels, which tend to be in ZPL format, from their home offices. We also have clients printing credit/debit cards on location in small environments. Even if they represent a small fraction of print jobs in the grand scheme of things, I don't think these numbers are going down anytime soon.

Again, thank you for the detailed replies.

@tresf

This comment has been minimized.

Copy link

tresf commented Aug 1, 2018

That driver (and the Printer Application) can also advertise the printer's native print format (EPL, ZPL, etc.) allowing an application that can produce the format to send printer-ready data. That is the model we want to see, not force users to depend on a specific application to support their printer (at which point the application is on its own).

Are any steps being taken by Apple/CUPS to warn these hardware manufacturers? Something such as putting word out to Zebra and Epson engineers with a rough timeline? In our experience, Zeba and Epson have a large part of the raw market, such as point-of-sales and shipping and both use raw languages that are commonly cloned by competitors such as Datamax(Honeywell), Intermec, Star, Citizen among many others.

@lite1979 and I can offer some testing resources as well once we have something ready for testing -- either a vendor driver or a "'Printer Application' or its equivalent) which can better address the needs of that 'segment'".

Even if the vendor drivers arrive in time, a lot of them will have little interest in offering drivers for hardware that's one or two models old, so this "custom spooler" will become the new norm as it's a standard setup for all hardware, not to suffer jurisdiction by driver.

And a final wish -- although one I don't expect to get granted -- is that whatever this Custom Spooler ends up looking like, that Apple makes it accessible to users without arbitrary terminal commands and web interfaces. Raw should be a driver option on MacOS like it is on Linux and Windows. :D </rant>

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Aug 7, 2018

@tresf Apple has been in contact with all of the popular printer vendors over the last 10 years to prepare them for this. Some have implemented support for IPP and common file formats, a few have not.

As I've said repeatedly, raw print queues are going away. If there is a dedicated raw printing solution it will be separate from the usual CUPS commands (e.g. "rawprint -d queue filename" instead of "lp -d queue -o raw filename"). But the more likely solution is to allow Clients to send vendor PDL, which would look like this for a ZPL printer:

lp -d queue -o document-format=application/vnd.zebra-zpl filename

Whether we can auto-detect ZPL accurately is another matter - there is a reason we don't have ZPL matching rules in the CUPS mime.types file after 20 years... :/

@RobertKrawitz

This comment has been minimized.

Copy link

RobertKrawitz commented Aug 8, 2018

So what will be the proper workflow for RIP vendors and others (such as people who use printers for T-shirts, chemical deposition, and such) who need to use the full capability of printers? We (Gutenprint) have a fair number of users who do unconventional things like this.

Then again, if we expose DeviceN raster, we may be able to give them the tools they need.

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Aug 8, 2018

@RobertKrawitz A Printer Application can provide support for any RIP workflow. IPP Presets, ICC-based color management, and PWG Raster support (for lower-level DeviceN color raster data) all allow for the same kinds of things we've traditionally done with printer drivers, but better since we are not forcing everything through PPDs.

@tresf

This comment has been minimized.

Copy link

tresf commented Aug 10, 2018

I think a dedicated raw spooler is a better, faster, lighter weight solution than keeping raw support in cupsd (where it is a special case we have to check for at every turn). This custom spooler could continue to support printing via CUPS API (where it just appears to be another local IPP queue) but otherwise the jobs would be routed separately where we could pipeline things and optimize the error handling/retry logic.

I think it is clear (or it should be given everything I've said and presented over the last 4 months) that enterprise printers (the kinds of printers that last 20+ years and that you can still buy supplies and parts for) will continue to be supported

If there is a dedicated raw printing solution

Because most printer drivers exist outside of CUPS this change needs to be discussed publicly.

So from a user's perspective, I'm not sure what else we can offer feedback for on this thread. Is the hope to get feedback from actual manufacturers on this thread?

@tresf Apple has been in contact with all of the popular printer vendors over the last 10 years to prepare them for this.

I'm not talking about the past 10 years, I'm talking about the next 2-ish years that this will be removed. Once CUPS is rolled out without raw support, the various software projects have about 4-6 months to hop on whatever replacement surfaces. If device drivers can keep us alive with CUPS, someone needs to write them. For example... the EPL and ZPL drivers that CUPS ships with don't properly handle raw, so many of us have areas that install a printer twice -- once for the raw stuff, once for everything else. When there's mixed signals as to what the replacement will look like and who will provide this replacement, it leaves little room for open discussion. All we can really do is wait for the replacement to be written.

Whether we can auto-detect ZPL accurately is another matter - there is a reason we don't have ZPL matching rules in the CUPS mime.types file after 20 years... :/

ZPL is a language where nearly every command starts with ^ and the print message starts with ^XA and ends with ^XZ. This seems like an excellent candidate for a matching rule. Are you talking about another language perhaps, like EPL?

@jimthouston

This comment has been minimized.

Copy link

jimthouston commented Nov 6, 2018

I was disappointed to find that the generic text only interface has already disappeared in Ubuntu 18.04. I'm playing with Inkscape and Inkcut to drive a CNC engraving machine. Inkcut uses CUPS as a traditional print spooler to access the printer port used to control plotters, vinyl cutters and engravers. These are expensive long lived machines. It would be nice if you didn't break this traditional interface.

@tresf

This comment has been minimized.

Copy link

tresf commented Nov 6, 2018

I was disappointed to find that the generic text only interface has already disappeared in Ubuntu 18.04

@jimthouston a bit off-topic but I wanted to help... as of 18.04 it's only been removed from the graphical wizard/dialog. It's still available through http://localhost:631. (screenshot).

Note, Apple removed "Raw" from the macOS/OS X System Preferences a long time ago, so anyone using CUPS + Raw on Apple has been used to the localhost:631 workaround for quite some time.

@jimthouston

This comment has been minimized.

Copy link

jimthouston commented Nov 7, 2018

Thanks for the answer. I had already found that the localhost:631 interface let me configure the parallel port printer. I didn't find a generic text only interface but I did find a .ppd file which then referenced a "text-only" filter which no longer exists. I tried texttotext but it added LF characters. Eventually I found that I could add a "o -raw" to the lpr command in inkcut. This got things working. With the "-o raw" option any printer description works because the filter is being bypassed.

My point in posting here was just to let you know that there are other users of the classic print spooler.
My guess is that the whole print spooler thing won't need to exist when all printers are network connected and you just need a "dating app" to match the client that wants to print with the printer. Could this new world be achieve as a new project which doesn't have to mess with the existing name space?

@esternin

This comment has been minimized.

Copy link

esternin commented Nov 9, 2018

I'd like to add something in defense of the RAW. As I understand it, starting with v2.2, the only way to connect a CUPS client to a CUPS server is by specifying a RAW protocol on the client side and letting the server-side driver/whatever do the rendering. At least, this is the only way that worked for me (after days of head-scratching). This is particularly important when Chromebooks come and join the existing hard-wired infrastructure, which is a fairly common use in education.

So CUPS 2.2 made me use RAW for the first time in years, and now future plans to remove it would actually kill all my printing from guest devices. I am confused.

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Nov 9, 2018

@esternin Raw is not the way to do it, "everywhere" is.

@markalston

This comment has been minimized.

Copy link

markalston commented Nov 21, 2018

I'm really stressing out over this change. Perhaps I don't need to and another solution is obvious but I'm missing it.

We run point of sale systems for restaurants and retail. While raw isn't needed for directly attached printers, as we can simply cat out to /dev/usb/lp0 or /dev/lp0, we have a situation where one station will very often need to receive receipts from all the other stations. Think drink and food tickets.

With raw support we can use CUPS to direct the printing files as needed and not have to worry about all the complexities. Add a bar printer? No problem, just add that CUPS queue to the software and tickets will print there via lp or lpr.

We will often have many many terminals trying to print tickets to that printer simultaneously which is why we use CUPS to spool them.

None of the printers we use support "ipp everywhere" as far as I know so once raw is gone it looks like we will have to implement spooling ourselves.

Am I missing some obvious other way of spooling to raw printers or is this something that will hopefully be developed later (before raw is gone?)?

Also, I'm confused about the PPD issue affecting raw support. Doesn't raw printing entirely bypass PPD's and make any issues with them irrelevant?

Thanks for any help understanding this.

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Nov 26, 2018

@markalston No change has been made yet. This bug is here so that everyone is aware it is coming, and what the approximate timeframe is.

First, I know of at least two label/receipt manufacturers that are working on IPP solutions and one that is already shipping products (Brother).

The main issue with raw queues is that they cannot be used by ordinary applications - the application has no way to know how to prepare the print file (media size or document format). This has led to enough bug reports that we hide raw queues on macOS, for example. A secondary issue is that they work differently than regular queues, so cupsd has a bunch of code special-casing raw support. Raw queues also cannot be audited (how many pages/labels/inches have been printed) or secured unless you introduce a filter to scan the print data for malicious (or just bogus) content, which is the opposite of what raw spooling is meant to accomplish. Also, your example of adding a new printer will only work if the printer uses the same language and has the same setup/capabilities, OR you configure the application for the new printer... It's hard to be "plug and play" if you don't know anything about what you are plugging in...

Both PPDs and raw queues suffer from common problems: discovery of current printer capabilities and features, accurate printer accounting, and common semantics for applications.

As for a raw spooling solution, that is something that could be implemented on top of libcups - a simple raw spooler/print command combination program would probably take care of 99% of the needs for raw printing, including your PoS setup. We are not planning on providing this solution, but I'm happy to help anyone that wants to do so...

@nahall

This comment has been minimized.

Copy link

nahall commented Nov 26, 2018

I'm in the same place as @markalston. We print to Zebra label printers using shipping software that sends ZPL. It goes through CUPS rather than direct to the printer because we have logic that routes the print job to a printer based on some conditions (which is better abstracted in the print server than the shipping software).

You have to understand that the industrial world is far different than the residential and office space. ZPL is used because it is very lightweight (one print job may only be a few KB) and the printer has the built-in ability to render very high quality barcodes and such by itself, without some separate software doing the rendering. The shipping software is able to simply send a ZPL text file and have the printer do the rest.

So moving to the IPP model of the residential space where the software needs to graphically format the label, then it is converted by the print server, etc, is unrealistic and it seems very unlikely that Zebra (or other label printer manufacturers) would move this direction.

It would also require changing over software to generate label images instead of ZPL, which would require getting many software vendors on board, and many of us use industrial software that isn't updated for years and years.

Also keep in mind how fast these label printers are and how fast they are expected to be. A standard Zebra model can finish printing a 4"x6" label in 1/3 of a second from the time it is issued a job. A fraction of a second delay would slow things down tremendously.

And these printers are used in huge numbers all over the world.

What I need from a print server is not printer discovery, or a common way of rendering print jobs, or accounting, or any of those things. It is to be able to give the print server a job, have it use a hook to decide which printer to send it to, put it in a spool for that destination printer and then just send the data straight through, without changing it. CUPS currently works great for that.

I'm willing to help write software to this end -- unfortunately I have not looked at CUPS internals at all but your thought about putting something on top of libcups could work. We need something like this or else I think everyone using CUPS in a non-office commercial or industrial setting will be broken on an upgrade, and I think there are a lot of us.

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Nov 26, 2018

@nahall IPP has no requirement for the client to render or otherwise prepare the print data. It supports "raw" printing with the corresponding loss of functionality/accounting/security - that's how CUPS has supported it since v1.0. And to be clear, the goal here is NOT to prevent applications from submitting printer-ready files (in whatever format the printer accepts), but instead to remove the special casing that has been causing us grief from day one (raw queues don't work like regular print queues) and the legacy of PPD files that prevents us from (for example) reporting the real media capabilities of a printer (and this includes label/receipt printers!)

Having written the CUPS ZPL drivers I am quite familiar with its capabilities and speed. But like every other vendor PDL that mixes presentation with device control, it has a few commands that vary based on the printer or installed features - not great if you have a heterogenous fleet of printers...

Also, Zebra (in particular) ALREADY supports clients that generate bitmap images, and ALREADY prints those 4x6" bitmaps in about 1/3 second. I know this because the CUPS ZPL driver was used for many years printing shipping labels for my former business, and that on older, slower printers than the GX420d I have in my office today. With compression (implemented by every printer these days) the size of a 4x6" 203dpi label is about 12k on average - larger than a corresponding ZPL barcode/text/vector file, but without the corresponding processing needed on the printer and still small enough that the transfer time is far smaller than the print time.

What you will see in the coming years are more label printers that support IPP with both the standard raster formats used by AirPrint and IPP Everywhere as well as the traditional PDLs. The first label printers released with this capability have been the smaller desktop and mobile printers, as they are often paired with phones or tablets.

@tresf

This comment has been minimized.

Copy link

tresf commented Nov 26, 2018

Having written the CUPS ZPL drivers I am quite familiar with its capabilities and speed. But like every other vendor PDL that mixes presentation with device control, it has a few commands that vary based on the printer or installed features - not great if you have a heterogenous fleet of printers...

I'm sorry for digging into the ZPL conversation as it's not the intent of this conversation but I wanted to make some items clear...

These commands that "vary based on the printer or installed features" are a generally rare exceptions. ZPL (both natively supported by Zebra hardware as well as emulated Datamax/Honeywell as PL-Z, Intermec as ZSim, etc) work with most standard ZPL commands. For industrial printing, the burdons you speak of are not as severe as they're being portrayed. Those that write and support raw languages learn to avoid these slight differences or made model-specific behavior and ZPL is one very common industrial application of raw. Another largely adopted standard is ESC/POS. It too suffers differences based on features but those features are minor enough for dozens of copycat printers to come to market with good, fast emulation.

ALREADY prints those 4x6" bitmaps in about 1/3 second

If you only take CUPS into consideration this statement is true. In practice though, rendering libraries are generally added into code (e.g. PDF, HTML) which take time to load and interact with the print driver. When multiplied times 500 labels, the benchmarks are clear that raw saves time, and thus money.

Problems that the industry run into with raw (in support of @michaelrsweet's points) are printer status (lack there of), DPI detection (rare, since most industry printers don't change this) and with certain application like in the case of card printers, there are capabilities such as laminators, duplexers that can drastically affect the commands. ZPL and ESC/POS seem to operate quite happily in most industries without the advanced features. In my case -- not first-hand but rather speaking on behalf of hundreds of companies that we service worldwide -- use raw through Windows, Linux and macOS for industry printing and are very happy with it.

P.S. I hope I'm not taking this too far off-topic, but I wanted to specifically clarify the severity of the feature variance point.

@tresf

This comment has been minimized.

Copy link

tresf commented Nov 26, 2018

What you will see in the coming years are more label printers that support IPP with both the standard raster formats used by AirPrint and IPP Everywhere as well as the traditional PDLs. The first label printers released with this capability have been the smaller desktop and mobile printers, as they are often paired with phones or tablets.

My experience on Windows is that both Epson's ESC/POS as well as Zebra's EPL and ZPL drivers have the capability of detecting and sending the raw commands using the vendor supplied driver without any special setup. I think the industry would be happy with similar behavior from CUPS if that's what you're describing.

@michaelrsweet

This comment has been minimized.

Copy link
Collaborator

michaelrsweet commented Nov 26, 2018

@tresf Yes, the intended "destination" of this change is that you'll end up with all queues supporting standard formats in addition to any vendor-specific PDLs that are available.

The intent is to stop directly supporting "legacy" printers that need drivers or lack any kind of driver. Instead those printers will need separate applications/services that can be run on demand and/or for long periods of time that provide an IPP-compatible front-end (the "Printer Application" we keep talking about) that is optimized for the printer in question. Basically, instead of using a PPD file and multiple filter/backend executables, we are using IPP and a single executable.

In the case of label printers, this architecture also will allow the printer application to keep a single connection open to the printer, eliminating the inter-job delays we often get today with CUPS raw queues.

@nahall

This comment has been minimized.

Copy link

nahall commented Nov 26, 2018

IPP has no requirement for the client to render or otherwise prepare the print data. It supports "raw" printing with the corresponding loss of functionality/accounting/security - that's how CUPS has supported it since v1.0. And to be clear, the goal here is NOT to prevent applications from submitting printer-ready files (in whatever format the printer accepts), but instead to remove the special casing that has been causing us grief from day one (raw queues don't work like regular print queues) and the legacy of PPD files that prevents us from (for example) reporting the real media capabilities of a printer (and this includes label/receipt printers!)

I would be completely happy to have Zebra support IPP and use that but unfortunately I have not seen plans by them to do so. I am actually in the market for some new Zebra printers and would like them to have this capability as that would make things easier for the future. Baring Zebra introducing an upgrade in the next couple of weeks I will have to buy what exists in the market, and, being an industrial printer would expect it to last around 10 years. So if CUPS won't print to the printer (currently over a port 9100 socket) then I will need to find some other solution.

If I am understanding you correctly, is the solution to create a separate program that basically simulates an IPP printer in raw mode so that CUPS can "print" to that and then it just forwards the data on to the actual endpoint printer (the Zebra printer in my case)?

We currently use tea4cups to run our script to determine which printer to send the job to. With ending raw queues would it still be possible to create a virtual CUPS printer that uses tea4cups in this manner? I imagine that the virtual printer would then need to look like an IPP printer that accepts raw print data?

Thanks for walking us through this @michaelrsweet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment