Skip to content

Release v3.0.0#37

Merged
mboglesby merged 259 commits intomainfrom
release-v3.0.0
Mar 10, 2026
Merged

Release v3.0.0#37
mboglesby merged 259 commits intomainfrom
release-v3.0.0

Conversation

@mboglesby
Copy link
Copy Markdown
Member

NetApp DataOps Toolkit for Traditional Environments:

  • Add new "Dataset Manager" feature.
  • Add support for Azure NetApp Files.
  • Add "list flexcaches", "get flexcache-origin", and "update flexcache" operations.
  • Add CIFS share operations.
  • Add Qtree operations
  • Code modernization.

sc93787 and others added 30 commits August 26, 2025 20:20
- Add missing notebookInstance-FSXVolume-stack.yaml CloudFormation template
- Remove duplicate create_snap_mirror_relationship from __init__.py
- Remove duplicate sync_snap_mirror_relationship from __init__.py
- Renamed netapp_dataops_traditional/netapp_dataops/traditional/storage/ to ontap/
- Updated all import statements in __init__.py to use .ontap instead of .storage
- Updated comments to reflect new package name
- All functionality remains the same, improved naming to reflect ONTAP-specific operations
- Renamed netapp_dataops_traditional/netapp_dataops/traditional/cloud/ to data_movement/
- Updated all import statements in __init__.py to use .data_movement instead of .cloud
- Updated import comments to reflect new package name
- Updated package documentation in data_movement/__init__.py to reflect broader scope
- All functionality remains the same, improved naming to reflect data movement operations
- Created new gcnv package with volume, snapshot
- Added volume_operations.py with create, clone, delete, and list volume functions
- Added snapshot_operations.py with create, delete, and list snapshot functions
- Updated main __init__.py to import GCNV functions with gcnv_ prefixes
- Organized GCNV functions similar to ontap package structure
- All functions have placeholder implementations ready for Google Cloud API integration
- Add Dataset class for high-level dataset abstraction
- Support dataset creation, cloning, snapshots, and file management
- Integrate with existing ONTAP volume and snapshot operations
- Add DatasetManagerConfig to configuration system
- Add comprehensive error handling with dataset-specific exceptions
- Support hierarchical dataset storage with junction paths
- Enable dataset discovery and management through get_datasets()

Key components:
- datasets/dataset.py: Core Dataset class implementation
- datasets/exceptions.py: Dataset-specific exception classes
- datasets/__init__.py: Package exports
- config/models.py: DatasetManagerConfig integration
- config/manager.py: Interactive dataset manager configuration
- Enhanced snapshot_operations.py to return snapshot names

This implements Phase 1 of the Dataset Manager as specified in the requirements.
- Added test_dataset_features.py demonstrating all dataset functionality
- Includes configuration detection and setup
- Simulates data engineer workflow with customer analytics pipeline
- Uses mock objects for testing without ONTAP connection
- Tests create, clone, snapshot, file operations, and cleanup
…nd testing

- Enhanced ConfigManager with one-time root volume creation and intelligent mounting
- Implemented automatic fstab management with detection and auto-editing capabilities
- Added robust mount operations with CLI and direct NFS fallback strategies
- Enhanced Dataset._validate_root_volume() to automatically detect and fix missing junction paths
- Added Dataset._fix_junction_path() method using ONTAP REST API for programmatic fixes
- Eliminates need for manual CLI commands when junction paths are not set
- Add automatic OS detection (_detect_os method) supporting Ubuntu/Debian, RHEL/CentOS, and SUSE
- Implement automatic NFS client package installation (_install_nfs_client method)
- Enhance _mount_volume method to detect NFS client errors and trigger automatic installation
- Add comprehensive error handling with fallback to manual installation instructions
- Support for multiple package managers: apt, yum, zypper
- Add proactive NFS client check at start of Dataset Manager configuration
- Check for mount.nfs availability before attempting any mount operations
- Enhance user experience with clearer messaging and status indicators
- Extract DatasetManagerConfigurator class to separate module
- Add centralized prompt utilities for consistent user interaction
- Improve NFS client checking timing in configuration flow
- Remove empty constructor from DatasetManagerConfigurator
- Remove redundant NFS installation prompts when user already agreed to mount/fstab
- Automatically install NFS client utilities when needed without extra prompts
- Simplify mount error diagnostics with concise single-line messages
- Clean up verbose print statements for better user experience
- Add NFS troubleshooting guidance to test configuration script
- Enhance mount failure error handling with specific diagnostic hints
- Remove redundant mounting permission prompt in _handle_root_volume_mounting()
- Mount volume automatically when users proceed with Dataset Manager setup
- Simplify user experience by eliminating duplicate confirmation dialogs
- Maintain fstab setup prompt for persistence configuration
- Fix 'argument of type NoneType is not iterable' error in _junction_path_exists()
- Optimize performance by caching volume name lookup and early exit conditions
- Add explicit null checks for NFS Mount Target field
- Reduce unnecessary string operations for volumes without NFS targets
- Improve code readability with clearer conditional flow
- Restructure dataset manager configuration to collect user inputs first
- Create and save configuration before performing operational actions
- Add clear success message after config creation
- Implement separate setup methods for existing and new root volumes
- Improve error handling with config preservation on operation failures
- Follow best practices: input → validate → persist → operate pattern

This ensures user configuration is never lost due to operational failures
and provides better separation of concerns between config and infrastructure operations.
- Remove redundant _handle_fstab_setup method that was no longer used
- Rename _handle_fstab_setup_for_already_mounted to _handle_fstab_setup for cleaner interface
- Improve code maintainability while preserving all functionality
- All methods now serve distinct purposes without duplication
- removing unnecessary function in client
- adding missing function in init
- adding pre-requisites for anf in setup.cfg
- serializing output of listing volumes
- serializing output of listing snapshots
- deleting delete replication functionality
- adding protocol types input arg in replication_management and mcp_server
- Add create_qtree() function with simplified API matching ONTAP REST requirements
- Add list_qtrees() function to retrieve qtree information
- Add delete_qtree() function for qtree removal
- Integrate qtree functions into main traditional module
- Use modular imports from core.config and core.connection
- Functions follow same patterns as existing toolkit operations
sc93787 and others added 28 commits February 16, 2026 12:14
Merge in SIE-BB/netapp-dataops-toolkit from feature/dataset-manager to release-v3.0.0

* commit '5390426af6320c506c4fb09130269ff1cba6aeb0': (31 commits)
  NSOL-4223: Updated delete() readme
  NSOL-4223: Unix permission update
  NSOL-4223: Added read only permission to dataset root volume
  NSOL-4223: Add comprehensive Dataset Manager user documentation
  [NSOL-4223] Improve fstab modification safety and UX
  NSOL-4223: Optimization changes
  NSOL-4223: fstab entry issue
  NSOL-4223: fstab write issue
  NSOL-4223: fixed fstab entry issue
  NSOL-4223: replaced print with logger
  NSOL-4223: Removed config summary display
  NSOL-4223: Removing uncesary display
  "NSOL-4223: Addresed bot comments"
  "NSOL-4223: review bot changes"
  NSOL-4223: Remove test files from repository
  NSOL-4223: Fix SnapshotPolicy error during dataset cloning
  Delete: Unnecessary functions are deleted
  refactor: optimize Dataset Manager configuration code
  refactor: Separate config collection from operations in Dataset Manager
  Cleanup: Extra params are removed from fstab entry
  ...
Merge in SIE-BB/netapp-dataops-toolkit from feature/qtree-functions to release-v3.0.0

* commit '06ef57672bfeea9a30f56c495983a8f133d1cfda': (35 commits)
  NSOL-4594 resolved conflicts
  NSOL-4519 updated README
  Revert "NSOL-4519 added "nas_path" to create qtree"
  NSOL-4519 added "nas_path" to create qtree
  NSOL-4521 removed "path" attribute
  NSOL-4521 updated get qtree metrics
  NSOL-4521 updated get qtree metrics
  Handled excpetion for tabulate library
  replace print statements with logger in qtree_operations.py
  used PerformanceQtreeMetric to make API request to fetch qtree metrics
  Added nas.path to get qtree and list qtree functions
  Resolved Code Review Feedback: Used logger instead of print
  Updated init.py
  Updated readme file
  added qtree functions in init.py
  added mcp tools for qtree functions
  updated README.md
  updated README.md
  updated the help command
  handled an edge case when qtree.id=0
  ...
…ing both CIFS share and Qtree/FlexCache features
Merge in SIE-BB/netapp-dataops-toolkit from feature/cifs-share to release-v3.0.0

* commit 'ff1e68a2e5df44e4daedba011e41e23865951b55': (50 commits)
  NSOL-4936 pulling back changes from 'release-v3.0.0'
  update help text NSOL-4936
  fix help text NSOL-4936
  fix NSOL-4936
  NSOL-4936: hash table with caching to look up all cifs shares
  - resolving issues with mutable default arguments for 'properties' and 'acls' in cifs-share - replacing print statements with loggers - svm serialization
  corrections
  correction
  improved admin shares filtering
  'properties' & 'acls' - parameter type validation & documentation
  removing unnecessary 'pass' statements
  restore cifs operations desc in ontap_readme
  replace volume path with volume name for cifs-share
  restoring missing code from 'release-v3.0.0'
  adding cifs tools in __init__.py
  fix
  serializing cifs share object for get cifs share
  serializing cifs share objects
  testing (2)
  testing
  ...
Merge in SIE-BB/netapp-dataops-toolkit from bug/help-text-cifsshare to release-v3.0.0

* commit 'f0a200cd29d6831e4e75a08f0551e6192379edab':
  bug resolution NSOL-5922
Merge in SIE-BB/netapp-dataops-toolkit from bug/logger-errors-cifsshare to release-v3.0.0

* commit 'cbb501a5a0a5884d7b40754c46233b4ae89e7e97':
  NSOL-5924 catch error via logger
Merge in SIE-BB/netapp-dataops-toolkit from bugfix/release-v3.0.0-bugfixes to release-v3.0.0

* commit '656ea1aebce48c31a47e2dd75c3164ad0ac1c936':
  NSOL-5951: displaying config file path to the user
  NSOL-5951: displaying configuration summary to the user
  NSOL-5942: saving base config before configuring dataset manager
  NSOL-5942: removing config existence checks
  NSOL-5942: fixing dataset manager config when no config file exists
  NSOL-5941:mounting root volume automatically
  NSOL-5943: removing function duplications
  NSOL-5940: including mounting process
Merge in SIE-BB/netapp-dataops-toolkit from bugfix/blackduck to release-v3.0.0

* commit 'da79289ac2b5066580c5da67d85f85201e50c570':
  NSOL-5851 resolved blackduck vulnerability
@mboglesby mboglesby merged commit e11e1d6 into main Mar 10, 2026
1 check passed
mboglesby pushed a commit that referenced this pull request Apr 7, 2026
Merge in SIE-BB/netapp-dataops-toolkit from sdl-r1-r6 to release-v3.1.0

* commit '61ed8e4c1c2cb5e7010116ce692fff7551934b12':
  NSOL-6009 NSOL-6010: kubeconfig 0600 on Linux, Secret/etcd/RBAC guidance
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants