v0.3.0-alpha

Product Updates

Announcement 📣

• VDP (originally, Visual Data Preparation) is officially renamed to Versatile Data Pipeline.

We have realised that as a general ETL infrastructure, VDP is in fact capable of processing all kinds of unstructured data. We should not limit its usage to only visual data but to more general versatile data. In addition, the term, Data Preparation, has been misleading for users often thinking it has to do with only data labelling, cleaning, or wrangling. In our vision, while VDP should involve data preparation in its MLOps practice, it should not be conceptually confined to only data preparation. VDP does more than that and is focused on the overall effectiveness of the unstructured data ETL with a data-centric paradigm. The end form of a VDP infrastructure is nothing but a data pipeline. The term Data Pipeline is more precise to capture the core concept of VDP hence renamed as Versatile Data Pipeline.

Features ✨

• support new task Instance segmentation. Check out the Streamlit example

VDP (0.3.0-alpha)

Features

• support Instance segmentation task (0476f59)
• add console e2e test into vdp (#148) (a779a11)
• add instance segmentation example (#167)(c341e0c)

Bug Fixes

• fix wrong triton environment when deploying HuggingFace models (#150) (b2fda36)
• use COCO RLE format for instance segmentation (4d10e46)
• update model output protocol (e6ea88d)
• fix docker-compose (#174) (e40d607)
• fix: fix typo in makefile (#172) (c861afd)

Pipeline-backend (0.9.3-alpha)

Bug Fixes

• fix pipeline trigger model hanging (instill-ai/pipeline-backend#80) (7ba58e5)

Connector-backend (0.7.2-alpha)

Bug Fixes

• fix connector empty description update (0bc3086)

Model-backend (0.10.0-alpha)

Features

• support instance segmentation task (instill-ai/model-backend#183) (d28cfdc)
• support async deploy and undeploy model instance (instill-ai/model-backend#192) (ed36dc7)
• support semantic segmentation (instill-ai/model-backend#203) (f22262c)

Bug Fixes

• allow updating emtpy description for a model (instill-ai/model-backend#177) (100ec84)
• HuggingFace batching bug in preprocess model (b1582e8)
• model instance state update to unspecified state (instill-ai/model-backend#206) (14c87d5)
• panic error with nil object (instill-ai/model-backend#208) (a342113)

Console

Features

• extend the time span of our user cookie (instill-ai/console#289) (76a6f99)
• finish integration test and make it stable (instill-ai/console#281) (3fd8d21)
• replace prism.js with code-hike (instill-ai/console#292) (cb61708)
• unify the gap between elements in every table (instill-ai/console#291) (e743820)
• update console request URL according to new protobuf (instill-ai/console#287) (fa7ecc3)
• add hg model id field at model_instance page (instill-ai/console#300) (31a6eab)
• cleanup connector after test (instill-ai/console#295) (f9c8e4c)
• disable html report (instill-ai/console#297) (689f50d)
• enhance the warning of the resource id field (instill-ai/console#303) (6c4aa4f)
• make playwright output dot on CI (instill-ai/console#293) (e5c2958)
• support model-backend async long run operation (instill-ai/console#309) (f795ce8)
• update e2e test (instill-ai/console#313) (88bf0cd)
  update how we test model detail page (instill-ai/console#310) (04c83a1)
• wipe out all data after test (instill-ai/console#296) (e4085dd)

Bug Fixes

• fix pipeline e2e not stable (instill-ai/console#285) (a26e599)
• fix set-cookie api route issue due to wrong domain name (instill-ai/console#284) (c3efcdd)

---

This PR was generated with Release Please. See documentation.

v0.2.6-alpha

0.2.6-alpha (2022-09-20)

Product Updates

BREAKING changes 🅱️

• updated VDP protocol

Features ✨

• add data mapping field data_mapping_indices in the data to be sent to the pipeline destination. The field includes unique IDs that can be mapped to CV task output for each processed image
• support all CV tasks when building ASYNC pipelines
• add confidence score for OCR task

Fix bugs 🐛

• fix ASYNC trigger block issue, close https://github.com/instill-ai/pipeline-backend/issues/67
• allow user-defined description for GitHub model

VDP

Bug Fixes

• update examples to be compatible with the latest protocol (36a847b)

Miscellaneous Chores

• release v0.2.6-alpha (b291862)

Pipeline-backend (0.8.0-alpha)

Features

• add data mapping (0db4bfe)

Bug Fixes

• fix async trigger block issue (59f0fb8), closes #67
• fix multipart trigger data_mapping_indices empty (d3160b4)

Connector-backend (0.6.0-alpha)

Features

• add data mapping (5596716)
• support all cv tasks defined in vdp protocol (c38f9e8), closes #33

Bug Fixes

• add http/grpc release stage in connector definitions (f2648c0), closes #30
• fix destination write for multiple models (860f62e)
• fix update not using name field (f8e98c3)
• fix zero-value field dropped (31e297c)

Model-backend (v0.8.1-alpha)

Features

• handle oom (#163) (4db1c45)
• add confidence score for ocr output (#167) (e915452)

Bug Fixes

• update description for GitHub model from user input (#173) (821dab3)

Miscellaneous Chores

• release 0.7.3-alpha (9033c50)

Console (v0.12.0-alpha)

Features

• add buffer into calculation of instance cell more button (#268) (19d1470)
• add destination configuration form (#255) (0164946)
• Implement onboarding integration test (#271) (f9c494e)
• Implement source integration test (#272) (b85a285)
• use env var in pipeline snippet (#275) (3bc4d7a)

Bug Fixes

• fix onboard cookie issue (#252) (a4a93cc)
• fix various typo (#249) (d60f860)
• update the sample github repos (#253) (82592dd)
• fix cookie_token wrongly set issue (#269) (7e25d9d)
• fix Select a existing to Select an existing (#264) (87a8584)
• fix the missing destination deletion (#267) (f96e65c)
• fix various issues related to destination icons and displayed title (#266) (d86c25c)

v0.2.5-alpha

0.2.5-alpha (2022-08-18)

Product Updates

Features ✨

• add new CV Task Optical Character Recognition (OCR)
• adopt new standard CV Task output, check the VDP Protocol
• add code snippets about how to trigger a pipeline in the Console

Fix bugs 🐛

• fix HTTP and gRPC connector state change logic
• make the response consistent across all triggering pipeline operations

VDP

Miscellaneous Chores

• release 0.2.5-alpha (0ebcd04)

Console (v0.10.0-alpha)

Features

• add error boundary (#232) (56f9f97)
• Implement code-hike snippet (#223) (c0b1b21)
• update design system (#228) (410d4b8)
• update logo (#226) (a3b566b)
• update object-detection icon and add ocr icon (#227) (652f114)

Bug Fixes

• fix code hike issue (#237) (d582ada)

Mgmt-backend (v0.2.5-alpha)

Miscellaneous Chores

• release 0.2.5-alpha (f2b418f)

Pipeline-backend (v0.7.1-alpha)

Bug Fixes

• fix multipart trigger resp fmt inconsist (0663542)

Connector-backend (v0.5.2-alpha)

Bug Fixes

• fix http and grpc state change logic (9e7b7eb)
• fix worker container naming (b0ad69e)

Miscellaneous Chores

• update instill connector doc url (34560f6)
• release v0.5.2-alpha (d2daad6)

Model-backend (v0.7.0-alpha)

Features

• add release stage for model definition (#153) (4e13ba5)
• support ocr task (#150) (7766c6f)

v0.2.4-alpha

0.2.4-alpha (2022-08-01)

Product Updates

fix console bugs 🐛

• fix keep redirecting back to onboarding form issue
• fix Airbyte form default value issue
• fix Toggle Field and Upload file Field issue
• adapt new design-system component props structure
• fix the wrong connector state
• remove default scroll behaviour on type=number input

VDP

Miscellaneous Chores

• release 0.2.4-alpha (955cff3)

Console (0.7.0-alpha)

Features

• set default value for Airbyte form (#202) (82d352a)

Bug Fixes

• fix hostname issue (#201) (4a8f196)
• fix model table's instance icon (#196) (e521d97)
• fix toggle field issue (#203) (5959c87)
• fix wrong connector state (#199) (9df6d64)
• remove default scroll behavior of number input field (#200) (376e1bd)

v0.2.3-alpha

0.2.3-alpha (2022-07-29)

Product Updates

features ✨

• annotate destination data name with batch index and timestamp

fix bugs 🐛

• fix ArtiVC model import, destination creation and pipeline display issue
• improve management backend error message
• fix pipeline destination data loss issue

VDP

Miscellaneous Chores

• release v0.2.3-alpha (ca6486f)

Console (0.6.3-alpha)

Bug Fixes

• fix get pipeline bug on model page (#174) (5e8116d)
• fix ArtiVC model creation issue (#178) (d8ef4e0)
• fix can't correctly create Airbyte destination (#185) (732844d)
• fix destination cache not remove and create http & grpc issue (#184) (8c35d95)
• fix onboarding form (#182) (a944e1d)

Mgmt-backend (0.2.4-alpha)

Bug Fixes

• change edition to local-ce:dev (037d738)
• refactor error code (#11) (4114e8e)

Pipeline-backend (0.7.0-alpha)

Features

• add data association with dst conn (9233429)

Connector-backend (0.5.0-alpha)

Features

• add data association with pipeline (c3236f7)

Bug Fixes

• fix dst data duplicate and lost issue (eb7ad97), closes #25

v0.2.2-alpha

0.2.2-alpha (2022-07-20)

Product Updates

fix bugs 🐛🐛🐛

• ⚡️ speed up creating model operation
• better console form handling
• improve output format for models that are not labelled with any task
• handle inference for images with RGBA channels

VDP

Miscellaneous Chores

• release v0.2.2-alpha (2459ab1)

Console (0.6.0-alpha)

Console update

• Update sidebar design
• Refactor lots of types, make our codebase much more consistent
• Fix tracking and collect user usage data issue
• Fix Airbyte nested fields validation and not correct value issue

Features

• make email field required at onboarding form (#169) (1411de7)
• move ga code to PageHead (#160) (3c49122)
• update sidebar design (#167) (d8899f5)
• update TextField props and remove props that have default (#164) (7d2a8b6)

Bug Fixes

• fix action issue (#172) (c02e4ad)
• fix Airbyte nested form validation issue (#171) (1101acc)
• fix description field is duplicated when create pipeline (#165) (7ef7b1a)

Connector-backend (0.4.5-alpha)

Bug Fixes

• fix airbyte emitted_at timestamp (3eca9d6)
• fix wrong state when connector check failed (92367e1)
• remove airbyte namespace (bd7baa4)

Model-backend (0.6.3-alpha)

Features

• speed up creating model operation (#115 ) (6930140)
• standardize the output of the unspecified task(7fcd867)

Bug Fixes

• fix client stream server recv wrong file length interval (#143) (0e06f7c)
• post process for unspecified task output (ad88068)
• trigger image with 4 channel (#141) (7445f5f)

Pipeline-backend (0.6.5-alpha)

Bug fixes

complete trigger async binary file for REST multipart (5003e5c)

v0.2.1-alpha

0.2.1-alpha (2022-07-12)

Product Updates

• 🔥🔥🔥 New YOLOv4 vs. YOLOv7 live demo
• Add optional description field when creating models
• Better console page title
• Support default model card for models without README.md

VDP

Miscellaneous Chores

• release 0.2.1-alpha (d098333)

Console (0.5.0-alpha)

Features

• add additional tracking attribute (#142) (451d830)
• add ga (#147) (2183570)
• add hugging face icon at model table (#152) (5f50328)
• make description field be a universal optional field when create model (#151) (f7718a5)
• make sure readme is correct (#139) (e5622a8)
• set page title (#146) (f5ecca8)
• stop user delete resource on demo website (#149) (dd68df6)

Bug Fixes

• use instill owned icons when interact with instill owned destination (#150) (d466a09)

Model-backend (v0.6.0-alpha)

Miscellaneous Chores

• release v0.6.2-alpha (4365f32)

v0.2.0-alpha

0.2.0-alpha (2022-07-08)

Product Updates

A lot of gooooodies 🍬 🍰 🧁 to share with the community

• 🥳 Add new management server mgmt-backend for user management
• 🎊 Add new connector server connector-backend for pre-built source/destination connector management
  • new HTTP connector
  • new gRPC connector
• 🔥 Now you can import models from GitHub, Hugging Face, or remote storage managed by DVC or ArtiVC
• 🎉 New vision task: besides image classification and object detection, VDP supports keypoint detection
• 🎮 New Console UI to enable youenabling you to use VDP without writing code
• 🚀 Build pipelines for diverse scenarios
  • SYNC pipeline for real-time inference
  • ASYNC pipeline for on-demand workload

VDP

Features

• add console (3151126)

Bug Fixes

• fix connector-backend-worker in docker-compose-dev (c5d7260)
• fix OpenAPI swagger service (#104) (2e3fd90)
• fix temporal dynamicconfig (57edcde)

Mgmt-backend (v0.2.3-alpha)

Features

• add reporter for usage collection (320d9a4)
• add retrieve all user function (da78893)

Bug Fixes

• support CORS (instill-ai/mgmt-backend#3) (5fb1439)
• use cors package to replace naive implementation (ae49c1c)
• add tlsenabled in usage backend configuration (dd8843a)
• init config before logger (455ef7a)
• init config first (4e3dea6)
• put client connection in main (3ff4821)
• refactor usage collection (c6a41f6)
• specify time duration unit (27b77d2)
• check existing user by id (dd862a4)
• change port from 8080 to 8084 (364f155)

Connector-backend (v0.4.3-alpha)

Features

• add delete guardian (355922a)
• add usage collection (c06e98a)
• add write destination connector (5995260)

Bug Fixes

• fix det jsonschema for empty case (cd4f102)
• fix duration configuration bug (4a4111c)
• fix usage collection (c19cf9b)
• fix usage disbale logic (664660d)
• fix usage-backend non-tls dial (457a74d)
• remove worker debug volume (acaa01c)
• return 422 delete error msg (5498494)
• fix error handler middleware (2fc9487)
• skip JSON Schema snake_case convert (c19cc53), closes https://github.com/instill-ai/connector-backend/issues/17

Model-backend (v0.6.0-alpha)

Features

• add credential definition (instill-ai/model-backend#109) (92d3391)
• support artivc (instill-ai/model-backend#102) (b8e21a4)
• support creating a HuggingFace model (instill-ai/model-backend#113) (1577d87)

Bug Fixes

• fix config path (a8cf2c0)
• regexp zap logger with new protobuf package (8b9c463)
• bug usage storage (instill-ai/model-backend#103) (975fdc1)
• fix duration configuration bug (ee4a310)
• init config before logger (9d3fb4a)
• status code when deploy model error (instill-ai/model-backend#111) ([31d3f11](https://github.com/instill-ai/model- backend/commit/31d3f11ba04ee59b12521b8e0dd724849a81b94f))
• update model definitions and tasks in usage collection (instill-ai/model-backend#100) ([c593087](https://github.com/instill-ai/model- backend/commit/c5930870595c5d280d7db005a711c0cc9bff802c))
• wrong logic when checking user account and service account (7058db6)
• model definition in list model and missing zero in output (instill-ai/model-backend#121) (a90072d)

Pipeline-backend (v0.6.3-alpha)

Features

• add cors support (c847912)
• add async pipeline trigger (6d39b0a)
• add filter for list pipeline (ffe8856)
• add usage collection (7e71744)
• support trigger multi model instances (e3d4263)

Bug Fixes

• fix create pipeline recipe resource name (bab3eaa)
• fix duration configuration bug (998eafa)
• fix usage collection (243e7a1)
• fix usage disbale logic (962823b)
• fix usage-backend non-tls dial (b864df3)
• close https://github.com/instill-ai/pipeline-backend/issues/56 ([c627a53](https://github.com/instill-ai/pipeline- backend/commit/c627a539999d65bc96ac6f88e2bd203548c34825))
• fix empty description update (2579f2e)

Console (v0.3.3-alpha)

Features

• add ArtiVC Icon for model definition (#18) (b5250db)
• add delete connector, pipeline service (#9) (526ff70)
• add DISABLE_USAGE_COLLECTION args config (#45) (7f9352d)
• add GitHub action related to docker hub (#11) (5e7569a)
• add mock server and render method of react-query (#12) (80d89d7)
• add mock source ...

v0.1.6-alpha

0.1.6-alpha (2022-04-03)

Miscellaneous Chores

• release 0.1.6-alpha (66e4988)

v0.1.5-alpha

0.1.5-alpha (2022-03-22)

Miscellaneous Chores

• release 0.1.5-alpha (36da44e)