Generic plugin API (for output processors and such) #177

jakubroztocil opened this Issue Nov 8, 2013 · 4 comments


None yet
2 participants

jakubroztocil commented Nov 8, 2013

Add a more generic plugin API that would allow adding new output processors.

(There already is an API for auth plugins).

Candidates / use cases

@ghost ghost assigned jakubroztocil Nov 8, 2013

bdejong commented Jul 22, 2014

This would really interest me: like #176 we would like to support protocol buffers in our API. This does mean that a simple "output processor" would not be enough. Protocol buffers need a schema file (a .proto file) in order to unserialise data. I.e. I could see it as something like...

http --output-processor=protobuffer --output-processor-schema=/path/to/xyz.proto

Which could potentially -behind the scened- get the binary output from the message, parse it using the xyz.proto schema definition and print something. I understand this is a very peculiar/particular case, but it might be good to check against the plugin interface to see if such things would be possible.

Plugins for input processors would also be interesting: if the input to a site speaks protocol buffers (or message-pack, or whatever) one would need to translate data again into the particular format before sending it over.

Just brainstorming here...


jakubroztocil commented Jul 22, 2014

I'll look into this later this week, but so far I've been working on an API that allows for plugins like #176 (msgpack). The idea is that each processor plugin would be asked if it supports given (binary) mime, and convert it to a textual representation that would be shown to the user in the terminal, if it does. So the msgpack plugin will look like this:

import json

import msgpack

from httpie.plugins import ConverterPlugin

__version__ = '1.0.2'
__author__ = 'Jakub Roztocil'
__licence__ = 'BSD'

class MessagePackPlugin(ConverterPlugin):

    def supports(cls, mime):
        return 'msgpack' in mime

    def convert(self, content_bytes):
            obj = msgpack.loads(content_bytes)
        except ValueError:
        return 'application/json', json.dumps(obj)
from setuptools import setup
    import multiprocessing
except ImportError:

    description='MessagePack plugin for HTTPie.',
    author='Jakub Roztocil',
        'httpie.plugins.auth.v1': [
            'httpie_msgpack = httpie_msgpack:MessagePackPlugin'
        'Development Status :: 5 - Production/Stable',
        'Programming Language :: Python',
        'Environment :: Plugins',
        'License :: OSI Approved :: BSD License',
        'Topic :: Internet :: WWW/HTTP',
        'Topic :: Utilities'

Another example:

bdejong commented Jul 22, 2014

That looks great as a start. Any kind of way to "configure" your plugin would be great though... That would make it a lot more flexible for hackin' away...

A problem with -for example- protocol buffers is that the recommended mime-type is actually application/octet-stream which is not a lot of help. Access to more of the headers could remedy that possibly...

bdejong commented Aug 5, 2014

I've been working further on our system and I've come to the conclusion that if you would support interpreting (custom) headers in your plugin system it would be enough for us. I.e. for protocol buffers you have to give a "schema" and we are adding the location to the schema in custom headers. Hence: if the plugin can read the headers first we can decode the message.

class ProtoBuffersPlugin(ConverterPlugin):
     def __init__(self):
         self.protobuf_path = None

    def supports(cls, mime):
        return 'x-protobuf' in mime

    def headers(self, headers):
        # check if protocol buffer headers in response
        if "X-Proto-Scheme" in headers:
            self.protobuf_path = headers["X-Proto-Scheme"]

    def convert(self, content_bytes):
        if not self.protobuf_path:

        decoded = pipe_bytes_to_protoc_command_line(self.protobuf_path, content_bytes)

        return 'text/plain', decoded
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment