Generic plugin API (for output processors and such) #177

Open
jakubroztocil opened this Issue Nov 8, 2013 · 4 comments

Comments

Projects
None yet
2 participants
Owner

jakubroztocil commented Nov 8, 2013

Add a more generic plugin API that would allow adding new output processors.

(There already is an API for auth plugins).

Candidates / use cases

@ghost ghost assigned jakubroztocil Nov 8, 2013

bdejong commented Jul 22, 2014

This would really interest me: like #176 we would like to support protocol buffers in our API. This does mean that a simple "output processor" would not be enough. Protocol buffers need a schema file (a .proto file) in order to unserialise data. I.e. I could see it as something like...

http --output-processor=protobuffer --output-processor-schema=/path/to/xyz.proto httpie.org

Which could potentially -behind the scened- get the binary output from the message, parse it using the xyz.proto schema definition and print something. I understand this is a very peculiar/particular case, but it might be good to check against the plugin interface to see if such things would be possible.

Plugins for input processors would also be interesting: if the input to a site speaks protocol buffers (or message-pack, or whatever) one would need to translate data again into the particular format before sending it over.

Just brainstorming here...

Owner

jakubroztocil commented Jul 22, 2014

I'll look into this later this week, but so far I've been working on an API that allows for plugins like #176 (msgpack). The idea is that each processor plugin would be asked if it supports given (binary) mime, and convert it to a textual representation that would be shown to the user in the terminal, if it does. So the msgpack plugin will look like this:

# httpie_msgpack.py
import json

import msgpack

from httpie.plugins import ConverterPlugin


__version__ = '1.0.2'
__author__ = 'Jakub Roztocil'
__licence__ = 'BSD'


class MessagePackPlugin(ConverterPlugin):

    @classmethod
    def supports(cls, mime):
        return 'msgpack' in mime

    def convert(self, content_bytes):
        try:
            obj = msgpack.loads(content_bytes)
        except ValueError:
            raise
        return 'application/json', json.dumps(obj)
# setup.py
from setuptools import setup
try:
    import multiprocessing
except ImportError:
    pass


setup(
    name='httpie-msgpack',
    description='MessagePack plugin for HTTPie.',
    long_description=open('README.rst').read().strip(),
    version='1.0.2',
    author='Jakub Roztocil',
    author_email='jakub@roztocil.name',
    license='BSD',
    url='https://github.com/jkbr/httpie-msgpack',
    download_url='https://github.com/jkbr/httpie-msgpack',
    py_modules=['httpie_msgpack'],
    zip_safe=False,
    entry_points={
        'httpie.plugins.auth.v1': [
            'httpie_msgpack = httpie_msgpack:MessagePackPlugin'
        ]
    },
    install_requires=[
        'httpie>=0.8.0',
        'msgpack-python'
    ],
    classifiers=[
        'Development Status :: 5 - Production/Stable',
        'Programming Language :: Python',
        'Environment :: Plugins',
        'License :: OSI Approved :: BSD License',
        'Topic :: Internet :: WWW/HTTP',
        'Topic :: Utilities'
    ],
)

Another example: https://twitter.com/jakubroztocil/status/462173042626801664

bdejong commented Jul 22, 2014

That looks great as a start. Any kind of way to "configure" your plugin would be great though... That would make it a lot more flexible for hackin' away...

A problem with -for example- protocol buffers is that the recommended mime-type is actually application/octet-stream which is not a lot of help. Access to more of the headers could remedy that possibly...

bdejong commented Aug 5, 2014

I've been working further on our system and I've come to the conclusion that if you would support interpreting (custom) headers in your plugin system it would be enough for us. I.e. for protocol buffers you have to give a "schema" and we are adding the location to the schema in custom headers. Hence: if the plugin can read the headers first we can decode the message.

class ProtoBuffersPlugin(ConverterPlugin):
     def __init__(self):
         self.protobuf_path = None

    @classmethod
    def supports(cls, mime):
        return 'x-protobuf' in mime

    def headers(self, headers):
        # check if protocol buffer headers in response
        if "X-Proto-Scheme" in headers:
            self.protobuf_path = headers["X-Proto-Scheme"]

    def convert(self, content_bytes):
        if not self.protobuf_path:
            raise

        decoded = pipe_bytes_to_protoc_command_line(self.protobuf_path, content_bytes)

        return 'text/plain', decoded
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment