08 Jul 14:52

kmova

257abac

openebs-4.1.0 Latest

Latest

Containerized Attached Storage for Kubernetes

Assets 3

08 Jul 13:24

Abhinandan-Purkait

v4.1.0

7ff4ac4

v4.1.0

OpenEBS Release 4.1

Release Summary

OpenEBS version 4.1 provides replicas to be distributed across different nodes and pools within a cluster, enhancing redundancy and performance. The ability to restore from snapshots across multiple replicas has been added in this release. In addition, this release has fixes related to snapshots, upgrades, availability, stability, and supportability along with solutioning for deploying OpenEBS in various kubernetes platforms.

Change Summary

Local Storage

LocalPV analytics changes

Replicated Storage

Placement of replica volumes across the different nodes/pools
Snapshot across multiple replicas
Restore across multiple replicas
Resizing a volume with snapshots

Upgrade and Backward Incompatibilities

Before deciding to upgrade, review the list below:

Kubernetes 1.23 or higher is recommended to install this release.
Upgrade to OpenEBS 4.1.0 is only supported for the selected engines namely Local PV Hostpath, Local PV LVM, Local PV ZFS, and Mayastor from the earlier editions of OpenEBS (3.10.x or below).

Replicated PV Mayastor Release Notes

Features

Placement of Replica Volumes Across Different Nodes/Pools: Mayastor now uses topology parameters defined in the storage class to determine the placement of volume replicas. This allows replicas can be controlled via labels from the storage class.

Snapshot Across Multiple Replicas: Mayastor has enhanced its snapshot capabilities to ensure file-system consistency across multiple replicas before taking snapshots. This ensures that snapshots are consistent and reliable across multiple replicas

Restore Across Multiple Replicas: The capability to restore from snapshots across multiple replicas has been introduced in recent releases, enhancing data recovery options.

Expansion of volumes with snapshots: The latest release, Mayastor v2.7, includes support for volume expansion even when snapshots are present.

Snapshot Fixes:

openebs/mayastor-extensions#500
openebs/mayastor-control-plane#826
openebs/mayastor-control-plane@53500c0

Upgrade Fixes:

openebs/mayastor-extensions#512

Enhancements:

openebs/mayastor-control-plane@9dfb153
openebs/mayastor@c3816b9
openebs/mayastor@6d6fc65
openebs/mayastor@5575858
openebs/mayastor-control-plane@07ca344
openebs/mayastor@5575858
openebs/mayastor@349a07c
openebs/mayastor@7ad7a33

Solutions:

Mayastor support in GKE with local SSD’s
Mayastor support in Talos
Provisioning RWX PVC’s
Velero DR for LocalPV ZFS

Known behavioural limitations

As with the previous versions, the Mayastor IO engine makes full utilisation of the allocated CPU cores regardless of I/O load. This is the poller operating at full speed, waiting for I/O.
As with the previous versions, a Mayastor DiskPool is limited to a single block device and cannot span across more than one block device.

Known Issues

Mayastor does not support the capacity expansion of DiskPools as of v2.7.0.
Under heavy IO and constant scaling up-down of volume replicas, the io-engine pod has been observed to restart occasionally.

LocalPV Release Notes

What’s Changed

openebs/lvm-localpv#318
openebs/zfs-localpv#508
openebs/dynamic-localpv-provisioner#177
openebs/dynamic-localpv-provisioner#188
openebs/zfs-localpv#548
openebs/zfs-localpv#535
openebs/zfs-localpv#508

Known Issues:

ZFS- localpv/LVM-localpv on a single worker node encounters issues after upgrading to the latest versions. Specifically, the problem is related to the controller manifest being changed to a Deployment type, causing new controller pods to not enter the Running state.The issue appears to be due to the affinity rules set in the old pod, which are not present in the new pods. As a result, since both the old and new pods have relevant labels, the scheduler is unable to place the new pod on the same node, leading to scheduling failures when there's only a single node

Workaround is to delete the old pod so new pod can get scheduled:

Please check issue for more details #3751.

New Contributors

Thanks to our new contributors !!

@pando85 - openebs/zfs-localpv#508
@aep - openebs/zfs-localpv#535
@GreyXor - #3725
@mdonoughe - openebs/lvm-localpv#297

Contributors

mdonoughe, aep, and 2 other contributors

Assets 2

19 Apr 15:39

niladrih

v4.0.1

2717e0c

v4.0.1

Release Summary

This patch fixes an issue with Replicated PV Mayastor on older Linux kernels. Click here to go to the v4.0.0 release notes.

Changes

Read the Replicated PV Mayastor release notes.

Assets 2

19 Apr 15:25

kmova

openebs-4.0.1

2717e0c

openebs-4.0.1

Containerized Attached Storage for Kubernetes

Assets 3

16 Apr 15:01

niladrih

v4.0.0

0ed93f1

v4.0.0

Release Summary

OpenEBS v4.0.0 is a major milestone change in entire project.

Important

This release increases and simplifies the overall user experience, adopter experience, repo/developer experience and CNCF community experience. With the guidance of the Cloud Native Computing Foundation TOC (Technical Oversight Committee), CNCF Exec Leadership, CNCF Storage TAG (TOC Technical Advisory Group).... v4.0.0 has implemented major & significant changes to the OpenEBS project structure and architecture. - These changes are critical and necessary for the long-term health, maintenance and community experience of the project.

Detailed info on this topic is tracked by OpenEBS and CNCF. Please view it here...

In a nutshell, the major project changes can be broadly classified into the following categories:

Level	Type of restructuring effort undertaken
1	Deprecation of older and legacy engines
2	Restructuring of the technical documentation
3	Restructuring of the project repos, code, dependencies and assets
4	A unifying experience during installation

These are in addition to the vast majority of super cool features 😎 and enhancements 🚀 to the Local-PV and Replicated-PV storage Engines, that OpenEBS is well-known for.

With this release, OpenEBS project Maintainers intends to re-enter the CNCF Storage landscape as a CNCF Sandbox project and re-establish itself as a contributing member of the CNCF Storage Landscape... in respect of our responsibility to the 1.2 Million active users (in our community) that have installed OpenEBS 9 million times over the past few years. They have spoken and they want OpenEBS to remain part of the CNCF project Landscape.

OpenEBS is the leading Cloud-native storage solution for stateful persistent Kubernetes workloads.

Deprecation of Legacy Solutions

Prior to the v4.0.0 release, OpenEBS contained the following solutions or engines (note: These are the pre-v4.0.0 Legacy names):

Local Storage Engines

Num	Storage engine name	Mode	Type of provisioner
1.	LocalPV Hostpath	Out-of-tree	non-CSI provisioner
2.	LocalPV Device	Out-of-tree	non-CSI provisioner
3.	LVM LocalPV	Native plugin	CSI provisioner
4.	ZFS LocalPV	Native plugin	CSI provisioner
5.	Device LocalPV	Native plugin	CSI provisioner
6.	Rawfile LocalPV	Native plugin	CSI provisioner

Replicated Engines

Num	Storage engine name	Mode	Type of provisioner
1.	Jiva	Native plugin	CSI provisioner
2.	CStor	Native plugin	CSI provisioner
3.	Mayastor	Native plugin	CSI provisioner

RWM (Read Write Many)

Num	Storage engine name	Mode	Type of provisioner
1.	NFS provisioner

Auxiliary storage services

Num	Storage engine name	Mode	Type of provisioner
1.	Node Device Manager (NDM)
2.	Monitoring
3.	Velero Plugin for Data Protection
4.	Data Populator

Decision to Unify the project

In v4.0.0, the maintainers have decided to trim down the above list, for improved focus and sustained future maintenance.
This was a tough but strategic decision. Several factors like aged of old project code, adoption levels, relevance to K8s trends, available expertise, community involvement, compliance with CNCF project guidelines; and a bit of common sense (😉) have been taken into account in deciding which OpenEBS projects and technologies to retain, which to deprecate, which to migrate out of the parent project and which components to move forward with.

In all decisions, the community is the primary deciding factor

Note

The NEW and re-organized structure of OpenEBS parent (Umbrella) project now comprises a set of very specific repos, projects, tech stacks, assets and Storage engines. Note: Some of the v4.0.0 Storage engines have been slightly renamed to provide a better community experience and more unified consistency across all Storage services.

No.	Local PV Storage engine	Mode	Type of provisioner
1.	Local PV Hostpath	Out-of-tree	non-CSI provisioner
2.	Local PV LVM	Native plugin	CSI provisioner
3.	Local PV ZFS	Native plugin	CSI provisioner
4.	Local PV RawFile*	Native plugin	CSI provisioner

*Currently under review

No.	Replicated PV Storage engine	Mode	Type of provisioner
1.	Replicated PV Mayastor	Native plugin	CSI provisioner

Auxiliary storage services

Num	Storage engine name	Mode	Type of provisioner
1.	Monitoring
2.	Velero Plugin for Data Protection
3.	Data Populator

With the unification of the above Storage engines and auxiliary services, OpenEBS can now serve a large base of 900,000 users. For the users of the legacy engines (the v4.0.0 release formally deprecates the old storage engines, services, tools and tech stack). They will soon be physically migrated out of the OpenEBS CNCF GitHub org as indicated by OpenEBS Issue #3701 and OpenEBS Issue #3709.

We would like to offer our ♥️ whole-hearted ♥️ gratitude 👍 for being the early adopters and supporters of OpenEBS and contributing to the maturity that OpenEBS has reached today. Migration procedures are documented here to migrate data from legacy OpenEBS engines to OpenEBS 4.0.0 storage.

ℹ️ INFO
The code, tools, assets and dependent repositories associated with the deprecated Legacy projects and Storage engines will soon be transferred our of the CNCF GitHub openebs organization to a new CNCF owned GitHub openebs-archive Archive organization as per discussions and guidance provided by CNCF.

📗 Restructuring Tech Docs

Before the v4.0.0 release, the OpenEBS technical documentation was distributed across two main websites:

🔖 https://openebs.io/docs - Hosts tech docs for all engines except Mayastor
🔖 https://mayastor.gitbook.io/ - Hosts tech docs for Mayastor

Release 4.0.0 integrates OpenEBS Local and Replicated Storage into a new modern and updated documentation set. It is published here: https://openebs.io/docs website. The release also updates and clarifies documentation related to the legacy engines that are now deprecated and will be migrated out of the project.

Since the project documentation is versioned, users of deprecated legacy engines can still view all legacy documentation content as follows:

📒 Documentation for non-Mayastor engines (v3.10.x or earlier) will be available at https://openebs.io/docs/3.10.x/.
📒 Documentation for Mayastor (v2.5 or earlier) will be available at https://mayastor.gitbook.io/introduction/v/version-2.5.

⏩ One Installer

OpenEBS v4.0.0 Helm chart is the OFFICAL way to install both OpenEBS Local-PV and Replicated-PV Storage engines.
The OpenEBS 4.0.0 Helm chart bundles sub-charts and are now officially named as the following components:

Num	Helm Installer - Storage engine name
1.	Local PV Hostpath
2.	Local PV LVM
3.	Local PV ZFS
4.	Replicated PV Mayastor
5.	Kubernetes Snapshot CRDs

Important

ALL of the above storage components are now INSTALLED as part of the default Helm installation process. The Helm chart provides an option to exclude Replicated PV Mayastor, if the user is interested in only installing Local PV storage (non-replicated storage).

Note

The method of using the operator.yaml file to control the installation of individual OpenEBS engines/components is now DEPRECATED and NOT supported.
Using operator.yaml is not recommended for independently installing Helm sub-charts. Please do NOT install the products using this method.

Component Versions

OpenEBS is now a Unified project that contains Storage engines and operators to deliver different types of K8s Storage services (Local and Replicated Persistent Volumes (PVs) for Kubernetes stateful workloads). - Kubernetes PVs can be provisioned via Container Storage Interface (CSI) Drivers or using Out-of-tree Provisioners. The status of the various components as of v4.0.0 are as follows:

Type	CSI Driver name	Version	Status
	CSI Drivers
1.	[Replicated PV Mayastor](https:/...

Assets 2

16 Apr 14:52

kmova

openebs-4.0.0

cc080b2

openebs-4.0.0

Containerized Attached Storage for Kubernetes

Assets 3

19 Dec 06:32

niladrih

v3.10.0

bf68639

v3.10.0

Release Summary

🎉 🎉 🎉 OpenEBS 3.10 introduces new Mayastor features, and bug fixes. LocalPV-Provisioner (hostpath), NFS-Provisioner, ZFS-LocalPV and LVM-LocalPV welcome enhancements which improve QoL.

Upgrade and Backward Incompatibilities

Please review this list prior to deciding to upgrade:

Kubernetes 1.21 or higher release is recommended as this release uses features of Kubernetes that will not be compatible with older Kubernetes releases. Kubernetes 1.19.12 or higher is recommended for using Rawfile Local PV.

Component versions

OpenEBS is a collection of data engines and operators to create different types of replicated and local persistent volumes for Kubernetes Stateful workloads. Kubernetes volumes can be provisioned via CSI Drivers or using Out-of-tree Provisioners. The status of the various components as of v3.10.0 are as follows:

CSI Drivers
- Mayastor 2.5.0 (stable)
- cStor 3.6.0 (stable)
- Jiva 3.6.0 (beta)
- Local PV ZFS 2.4.0 (stable)
- Local PV LVM 1.4.0 (stable)
- Local PV Rawfile 0.8.0 (beta)
- Local PV Partitions 0.9.0 (alpha)
Out-of-tree provisioners
- Local PV hostpath 3.5.0 (stable)
- Local PV device 3.5.0 (stable)
- Dynamic NFS Volume 0.11.0 (beta)
Other components
- NDM 2.1.0 (beta)
- CLI 0.5.1 (beta)
- Dashboard 0.4.13 (beta)

Change Summary

A detailed changelog is available under the component repositories listed above. Here is a quick summary of what has changed since the last release.

Mayastor changes: https://github.com/openebs/mayastor/releases/tag/v2.5.0
LocalPV-Provisioner: fix quota calculation for XFS and EXT4 quota options
NFS-Provisioner: add support for disabling garbage collection of the backend volume
ZFS-LocalPV: add support for showing used-capacity info on ZFSNode resource
CI fixes, depedency updates, and other maintenance to cStor, Jiva, and external-provisioner based projects.

Show your Support

Are you using or evaluating OpenEBS? You can help OpenEBS in its journey towards becoming a CNCF Incubation project by sharing your OpenEBS story and join the league of OpenEBS Adopters.

Shoutouts!

MANY THANKS to our existing contributors and for everyone helping OpenEBS Community going. @Abhinandan-Purkait @avishnu @blaisedias @chriswldenyer @niladrih @tiagolobocastro @w3aman @abhilashshetty04 @rohan2794 @dsavitskiy @sinhaashish @datacore-vvarakantham @dsharma-dc @hrudaya21 @pchandra19 @milanhajek @r1jt @AJDatacore @avishnu

A very special thanks to our first-time contributors to code, tests, and docs: @ukd1 @trunet @omric-runai @emiran-orange @cmontemuino @maxs-rose @MingZhang-YBPS @njuptlzf @testwill @pentago

Documentation

https://openebs.io/docs

Install

OpenEBS can be installed via kubectl or helm. Follow the installation instructions here.

Upgrade

The upgrade instructions for various OpenEBS engines are here.

Do not upgrade if you are using legacy cstor or jiva provisioners. You have to first migrate those to the corresponding CSI Drivers. Please reach out to us for support

Known Issues

Check our open issues uncovered through e2e and community testing.

Check out the Mayastor release notes for known Mayastor issues.