Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NAS-110433 / 12.0 / Restart mdns after installing a plugin #6847

Merged
merged 1 commit into from May 6, 2021

Conversation

sonicaj
Copy link
Member

@sonicaj sonicaj commented May 6, 2021

We should restart mdns after a jail is started following #6587.

We should restart mdns after a jail is started following #6587.
@sonicaj sonicaj self-assigned this May 6, 2021
@bugclerk
Copy link
Contributor

bugclerk commented May 6, 2021

@bugclerk bugclerk changed the title Restart mdns after installing a plugin NAS-110433 / 12.0 / Restart mdns after installing a plugin May 6, 2021
@rick-mesta rick-mesta merged commit 8efa130 into truenas/12.0-stable May 6, 2021
@rick-mesta rick-mesta deleted the NAS-110433 branch May 6, 2021 15:31
rick-mesta pushed a commit that referenced this pull request Aug 24, 2021
* NAS-110174 / 12.0 / WBC_ERR_WINBIND_NOT_AVAILABLE is not an SSH login failure (#6800)

* NAS-110401 / 12.0 / Fix regression in service restart after pool decryption (#6824)

Refactoring a cherry-pick of fix from master introduced a
regression whereby we were sending a list of service names to
to be restarted to the core.bulk rather than a list of lists.

* NAS-109545 / 12.0 / Prevent nested loop queries in iscsi.target (#6657)

* NAS-109033 / 12.0 / Bug fix for correctly retrieving DN value from a cert (#6846)

There are cases where simple decode can fail when trying to read cert attributes. We should rely on openssl module instead to give us normalized values instead of trying to decode ourselves.

(cherry picked from commit b513057)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110243 / 12.0 / Allow endpoints to specify explicit http methods (by sonicaj) (#6819)

* Allow endpoints to specify explicit http methods

(cherry picked from commit 649ee8e)

* Add explicit get method on ui_restart endpoint

(cherry picked from commit 0dc94f2)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110433 / 12.0 / Restart mdns after installing a plugin (#6847)

We should restart mdns after a jail is started following #6587.

* Fix syntax error in SSSD-compatibility idmap config

(cherry picked from commit fe365ee)

* Do not report successful S.M.A.R.T. test starts on SCSI/SAS drives as failures

(cherry picked from commit acd7185)

* NAS-110496 / 12.0 / Explicitly set system dataset to readonly=off (#6869)

* NAS-110576 / 12.0 / R20A enclosure support (#6871)

* NAS-110240 / 12.0 / Do not show disks with temperature collection disable in corresponding Reporting UI section (#6861)

* NAS-110214 / 12.0 / Do not wait for NFS restart when configuring network interfaces. (#6866)

It might hang due to unknown reasons and cause old-style FreeBSD network setup on boot.

* NAS-109910 / 12.0 / Bring back iscsi_target_extent_name unique constraint (#6864)

* NAS-110602 / 12.0 / Fix concatenating enclosures (#6873)

* NAS-105205 / 12.0 / Sort pool.filesystem_choices by name (#6875)

* NAS-110607 / 12.0 / Fix (almost all) unit test failures (#6876)

* NAS-109989 / 12.0 / Convert dscache.refresh into a middleware job (#6872) (#6877)

This commit serializes cache refresh requests into a job queue and
ensures that cache is refreshed before writing to file.

Add docstring to directoryservices.cache_refresh

* NAS-110417 / 12.0 / Have unique serial number for certs (#6883)

When creating a self generated certificate, we should have unique serial numbers. The upper limit for serial is 2^160 but in the changes i am keeping it 30 as otherwise it will just be a really long number which is hard to access and it has a low probability that we have 2 self signed certificates generated by us with same serial number.

(cherry picked from commit 3909aea)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110575 / 12.0 / Stop TrueCommand connection on BACKUP node (#6886)

* NAS-110714 / 12.0 / fix memory allocation/leak issue in multipath.query() (#6897)

* fix memory allocation/leak issue in multipath.query()

* restore Config child class

* remove old debugging code

* storytime!

* run cert check every 24hrs and not on standby node

* NAS-110779 / 12.0 / Consistently retrieve boot time (#6918)

This commit fixes an issue where we were not normalizing the datetime object.

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110769 / 12.0 / remove lxml from fcport plugin and use correct verbiage (#6912)

* NAS-110793 / 12.0 / remove lxml and xml.etree from identify_freebsd (#6921)

* remove lxml and xml.etree from identify_freebsd

* remove unused modules, one of which references lxml

* fix logic for checking matches by normalizing data

* NAS-110618 / 12.0 / Start SSH service on SSH pairing (#6885)

(cherry picked from commit 53ae118)

Co-authored-by: themylogin <themylogin@gmail.com>

* NAS-110719 / 12.0 / remove lxml from vm plugin on CORE (#6926)

* NAS-110783 / 12.0 / only unlock SED disks on active controller (#6924)

* only unlock SED disks on active controller

* use global variable so we dont unlock SED drives more than once

* NAS-110766 / 12.0 / remove lxml from global_freebsd plugin (#6911)

* remove lxml from global_freebsd plugin

* get rid of inner method

* NAS-110765 / 12.0 / remove lxml from fc_hba_not_present alert source (#6910)

* NAS-110701 / 12.0 / Remove copy-paste error from old AD LDAP code. (#6898)

LDAP plugin doesn't have `verbose_logging` key. We were using this
in code related to SASL_EXTERNAL binds, which are almost never used
in typical environments.

(cherry picked from commit e714861)

Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* NAS-110635 / 12.0 / Do not allow comma in cdrom path (#6902)

This commit adds changes to ensure user does not specify a comma in cdrom path as that would result in bhyve not able to parse the argument correctly.

* NAS-110712 / 12.0 / add --trace-malloc arg to middlewared (#6896)

* add --trace-malloc arg to middlewared

* address reviews and make logs more readable

* NAS-110667 / 12.0 / Add testparm output to debug (#6893)

(cherry picked from commit 2875104)

Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* NAS-110773 / 12.0 / Fix kerberos error (#6863)

* NAS-110922 / 12.0 / remove py-lxml build depdendency (#6967)

* NAS-110754 / 12.0 / Properly validate rsync extra arguments (#6968)

(cherry picked from commit 6428776)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110671 / 12.0 / prevent unlimited running cron jobs (#6971)

* NAS-110928 / 12.0 / Do not run camcontrol reprobe on vtbd drives (#6998)

* NAS-110969 / 12.0 / Increase verbosity of range-related idmap verrors (#6992)

Do a bit more work to pinpoint the exact idmap range overlap.

(cherry picked from commit 26c29cf)

Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* Add github action to use flake8

* Show flake8 errors introduced

* Add default value for extra list attribute (#7049)

* NAS-110600 / 12.0 / Correctly retrieve s3 attachment (#7041)

This commit fixes an issue if s3 path did not exist for some reason ( got deleted or was an old path ), retrieving s3 attachments would error out as the path would not exist. Secondly, if the path does not exist but has been configured - we create the path before starting the s3 service.

(cherry picked from commit c40a4d5)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-111124 / 12.0 / cache static info and optimize system.info on CORE (#7065)

* cache dmidecode results to reduce CPU time

* fix typo

* optimize system.info

* address review

* NAS-111046 / 12.0 / update recommended zvol blocksize depending on zpool geometry (#7072)

(cherry picked from commit 415bbea)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111132 / 12.0 / Fix r10 enclosures (by darkfiberiru) (#7092)

* Fix r10 enclosures. Including mapping drives bays to match standard pattern(left to right then top to bottom)instead of inverse. Tested on R10 and Mini-X+ to confirm mini mapping still works

(cherry picked from commit fd1995d)

* Additional enclosure fixes/cleanup

(cherry picked from commit 30123f3)

* document why we add back expansion shelves

(cherry picked from commit f719eb5)

* Fix small typos and type mixup added while fixing r10 enclosure management

(cherry picked from commit 1ce30e2)

* flake8 fixes and use double-quotes

(cherry picked from commit 1124780)

* flake8 fixes for enclosure.py

(cherry picked from commit 6a23820)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>
Co-authored-by: Caleb St. John <30729806+yocalebo@users.noreply.github.com>

* NAS-111223 / 12.0 / Fix R40 to comply with enclosure management nep specified single mapped enclosure to UI. (#7097)

Previous method didn't work along with not being nep compliant. Testing
shows that UI was already compliant and works with this change.

(cherry picked from commit 089528e)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111080 / 12.0 / Fix Enclosure Mapping for early version Mini X 3.0 Systems (by darkfiberiru) (#7098)

* Fix Enclosure Mapping for early version Mini 3.0 Systems

(cherry picked from commit 1b3741c)

* Enclosure map regex string needs to be a raw string to stop collision of string and regex escaping

(cherry picked from commit de17ad5)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111146 / 12.0 / add interfaces marked critical for failover to network.sh (#7088)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111259 / 12.0 / fix r20(a) to comply with enclosure NEP (#7105)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111273 / 12.0 / fix r50 to comply with enclosure NEP (#7107)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-106539 / 12.0 / Copy over profile(4) localbase/etc to reflect with upstream changes (#7110)

This commit adds changes to copy over profile(4) to localbase/etc as bash reads the login initialization changes from there now with recent changes introduced to freebsd upstream last year ( https://forums.freebsd.org/threads/what-happened-to-etc-profile.78406/ ).

* NAS-111283 / 12.0 / Bug fix for recommending zvol blocksize (#7114)

This commit fixes an issue where we are not sending 'K' suffix with the recommended blocksize which we were doing before and this results in change in behavior with existing implementation in UI/middleware breaking.

(cherry picked from commit 4e28818)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* Load if_atlantic module built form ports.

Ticket:	NAS-111219
(cherry picked from commit 80c7922)

* NAS-111343 / 12.0 / Additional Enclosure Fixes (by darkfiberiru) (#7127)

* Mini-E+ enclosure fix

(cherry picked from commit de493c9)

* Make sure that cobra expander works with r20/alternative r20 expander names

(cherry picked from commit a4174cb)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111335 / 12.0 / Fix Syntax error: "}" unexpected (expecting "fi" (#7133)

* NAS-111188 / 12.0 / support.new_ticket should fully consume system.debug file to prevent debug job running (and holding lock) forever (#7134)

* NAS-111099 / 12.0 / Fix replication progress saying "total 11.12 TiB of 11.04 TiB" (#7140)

* NAS-111042 / 12.0 / pool.dataset: optimized create/update operations (#6955)

* NAS-111384 / 12.0 / fix cpu temp crash in snmp-agent on VMs (#7143)

* NAS-111390 / 12.0 / Enclosure Management fix for R20A. (#7145)

System has weird drive mapping top to bottom then left to right naturally
Needs correspondending UI change to map drives Left to Right and top to
bottom as is standard(same as r20)

(cherry picked from commit 58f59d6)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111406 / 12.0 / Fix for mini-3.0-xl+ 2.5 inch drive bays being swapped. (#7155)

Prior to this change the side internal bay showed as slot 1(hot swap
2.5inch bay)
No UI change is needed for this fix mapping only
Confirmed against multiple systems both based opon production builds and
redbook verified builds

(cherry picked from commit d9bdfde)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* Implement NVMe resize in disk_resize for FreeBSD.

(cherry picked from commit 0338e09)

* NAS-110987 / 12.0 / Dataset checksum (#7160)

(cherry picked from commit 3651bfe)

Co-authored-by: themylogin <themylogin@gmail.com>

* Correct NVMe capacity check in disk_resize.

Appears for some device NVM Capacity is not equal to Size multiplied
by sector size, but is a pretty arbitrary value.  Use it directly.

(cherry picked from commit f19ea9a)

* NAS-111422 / 12.0 / fix issues with enclosure.sync_zpool (#7180)

* NAS-111462 / 12.0 / fix license detection on r-series devices (#7182)

* NAS-111487 / 12.0 / Do not crash pool.dataset.kill_processes if a process has terminated on its own (by themylogin) (#7193)

* Fix service.terminate_process return value documentation and prevent it from throwing psutil.NoSuchProcess

(cherry picked from commit 02ca757)

* Do not crash pool.dataset.kill_processes if a process has terminated on its own

(cherry picked from commit 4cee5fe)

Co-authored-by: themylogin <themylogin@gmail.com>

* NAS-111573 / 12.0 / fix race between dd and gpart destroy (#7232)

* Changing the timeout for plex job api test to 20 minutes (#7235)

* NAS-111682 / 12.0 / Improve smartd restart performance (#7263)

* Improve smartd config generation performance

* Improve pool.transform_topology performance

* NAS-111685 / 12.0 / make get_ses_enclosures scale on large systems (#7262)

* NAS-111690 / 12.0 / call enclosure.query once in disk.sync_all (#7265)

* NAS-111693 / 12.0 / fix enclosure.sync_zpool (#7266)

* NAS-111696 / 12.0 / dont call system.info in enclosure plugin (#7269)

* NAS-111720 / 12.0 / fix numerous issues with disk.sync_all (#7282)

* fix numerous issues with disk.sync_all

* address reviews

* NAS-111758 / 12.0 / fix IndexError in network.py (#7291)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111744 / 12.0 / Fix SSHd IPV6 link local ListenAddress (#7293)

When sshd is set to listen on an IPv6 link local address the interface
name must also be appended to the address.

(cherry picked from commit 6229e7a)

Co-authored-by: Benjamin M. Hughes <bmhughes@bmhughes.co.uk>

* NAS-111764 / 12.0 / move disk.sync_all and enclosure.sync_zpool on HA (#7297)

* NAS-111766 / 12.0 / fix slot number calculation for expansion shelves (#7299)

* NAS-111769 / 12.0 / fix disk identification with vmware nvme disks (#7305)

* NAS-111788 / 12.0 / fix typo (#7306)

* NAS-111790 / 12.0 / dont restart services twice on failover (#7310)

* NAS-111819 / 12.0 / Fix taking vmware-aware manual snapshots (#7321)

This was broken by acef89b

* removing OSX ssh code from AFP API testing (#7330)

removed scale related code from AFP API testing
added sleep on Enter your root password in install.exp to avoid
race condition when inputing passowrd

* NAS-111695 / 12.0 / Make sure we freeup unused pages in sqlite (#7338)

This commit adds changes to make sure that we force sqlite3 to give up space which has accumulated over time by deletions of entries. What sqlite3 does is that each data which is removed from the database, sqlite3 does not use the space consumed by that payload and instead moves it to it's list of free pages and whenever more data is added it uses these free pages first and then asks for more space. However with time these free pages can consume lots of space like for a user his db size was 15 mb approx and after vacumming it got to 706K. This change uses vacuum functionality of sqlite3 to make sure that we remove such free unused pages which are in hold by sqlite3.

(cherry picked from commit 6951ca4)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* DOCS-2484 / 12.0  Replaced ssh to OSX and BSD by smbclient command (#7346)

* Replaced ssh to OSX and BSD by smbclient command

removed scale related code
putted wait_on_job in the test that the job id is taken from

* removed OSX and BSD ssh and replaced it with smbclient

removed scale related code

* fixed test_019 set guestok to false

moved test_046 at test_011

* capitalize AD_USER SMB_NAME SMB_PATH

* remove guestok from activedirectory test

* remove raw string from smbclient command for ad and smb API test

* removing SMB1 testing from api2/activedirectory.py test

* NAS-111858 / 12.0 / Give a proper title to SSHLoginFailuresAlertClass (#7354)

* NAS-111676 / 12.0 / Do not allow deleting a target if it's in use by fcport service (#7358)

This commit adds validation to not allow deletion of a target if it's in use by fcport service as the process will faill because of foreign key reference.

* NAS-111276 / 12.0 / VMware VM UUIDs are not unique and we should use FindAllByUuid (#7356)

* NAS-111885 / 12.0 / Update check_user method to allow checking passwords of other users (by sonicaj) (#7371)

* Update check_user method to allow checking passwords of other users

This commit adds a change to check_user method adapting it so that it can be used for other users as well for checking if a specified password is accurate/correct against the specified username.

(cherry picked from commit 5604b7b)

* Add check_password endpoint

(cherry picked from commit 999aa2d)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-111954 / 12.0 / Fix GMail thread safety (#7378)

(cherry picked from commit a639860)

Co-authored-by: themylogin <themylogin@gmail.com>

* NAS-111779 / 12.0 / improve IoThreadPoolExecutor (by yocalebo) (#7381)

* improve IoThreadPoolExecutor

(cherry picked from commit 9efec7e)

* <= 1 for semaphore value check

(cherry picked from commit 407ba6e)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111949 / 12.0 / Gather rsyncmod statistics in usage plugin (#7387)

(cherry picked from commit f171bb5)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* remove TrueNAS SCALE related code (#7383)

removed api2/pool_acltype.py since it is for scale
backport webdav test from master branch and removed api2/webdav_bsd.py
and api2/webdav_osx.py
improved api2/smb_protocol.py
cleaned some code

* NAS-111808 / 12.0 / Add endpoint to retrieve serial port choices with serial port names in freebsd (#7390)

(cherry picked from commit d12d000)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* Fix renamed ports

* Fix bad git dependency

* NAS-108910 / 13.0 / run fenced on single-node HA capable systems (#6956)

* NAS-111594 / 13.0 / improve process for formatting disks (#7240)

* NAS-111667 / 13.0 / fix minor typo in wipe.py (#7254)

Co-authored-by: themylogin <themylogin@gmail.com>
Co-authored-by: Andrew <awalker@ixsystems.com>
Co-authored-by: bugclerk <40872210+bugclerk@users.noreply.github.com>
Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>
Co-authored-by: William Grzybowski <william@grzy.org>
Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>
Co-authored-by: Alexander Motin <mav@FreeBSD.org>
Co-authored-by: Richard Kojedzinszky <rkojedzinszky@users.noreply.github.com>
Co-authored-by: Eric Turgeon <4249848+ericbsd@users.noreply.github.com>
Co-authored-by: Benjamin M. Hughes <bmhughes@bmhughes.co.uk>
sonicaj added a commit that referenced this pull request Dec 7, 2021
* NAS-110174 / 12.0 / WBC_ERR_WINBIND_NOT_AVAILABLE is not an SSH login failure (#6800)

* NAS-110401 / 12.0 / Fix regression in service restart after pool decryption (#6824)

Refactoring a cherry-pick of fix from master introduced a
regression whereby we were sending a list of service names to
to be restarted to the core.bulk rather than a list of lists.

* NAS-109545 / 12.0 / Prevent nested loop queries in iscsi.target (#6657)

* NAS-109033 / 12.0 / Bug fix for correctly retrieving DN value from a cert (#6846)

There are cases where simple decode can fail when trying to read cert attributes. We should rely on openssl module instead to give us normalized values instead of trying to decode ourselves.

(cherry picked from commit b513057)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110243 / 12.0 / Allow endpoints to specify explicit http methods (by sonicaj) (#6819)

* Allow endpoints to specify explicit http methods

(cherry picked from commit 649ee8e)

* Add explicit get method on ui_restart endpoint

(cherry picked from commit 0dc94f2)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110433 / 12.0 / Restart mdns after installing a plugin (#6847)

We should restart mdns after a jail is started following #6587.

* Fix syntax error in SSSD-compatibility idmap config

(cherry picked from commit fe365ee)

* Do not report successful S.M.A.R.T. test starts on SCSI/SAS drives as failures

(cherry picked from commit acd7185)

* NAS-110496 / 12.0 / Explicitly set system dataset to readonly=off (#6869)

* NAS-110576 / 12.0 / R20A enclosure support (#6871)

* NAS-110240 / 12.0 / Do not show disks with temperature collection disable in corresponding Reporting UI section (#6861)

* NAS-110214 / 12.0 / Do not wait for NFS restart when configuring network interfaces. (#6866)

It might hang due to unknown reasons and cause old-style FreeBSD network setup on boot.

* NAS-109910 / 12.0 / Bring back iscsi_target_extent_name unique constraint (#6864)

* NAS-110602 / 12.0 / Fix concatenating enclosures (#6873)

* NAS-105205 / 12.0 / Sort pool.filesystem_choices by name (#6875)

* NAS-110607 / 12.0 / Fix (almost all) unit test failures (#6876)

* NAS-109989 / 12.0 / Convert dscache.refresh into a middleware job (#6872) (#6877)

This commit serializes cache refresh requests into a job queue and
ensures that cache is refreshed before writing to file.

Add docstring to directoryservices.cache_refresh

* NAS-110417 / 12.0 / Have unique serial number for certs (#6883)

When creating a self generated certificate, we should have unique serial numbers. The upper limit for serial is 2^160 but in the changes i am keeping it 30 as otherwise it will just be a really long number which is hard to access and it has a low probability that we have 2 self signed certificates generated by us with same serial number.

(cherry picked from commit 3909aea)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110575 / 12.0 / Stop TrueCommand connection on BACKUP node (#6886)

* NAS-110714 / 12.0 / fix memory allocation/leak issue in multipath.query() (#6897)

* fix memory allocation/leak issue in multipath.query()

* restore Config child class

* remove old debugging code

* storytime!

* run cert check every 24hrs and not on standby node

* NAS-110779 / 12.0 / Consistently retrieve boot time (#6918)

This commit fixes an issue where we were not normalizing the datetime object.

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110769 / 12.0 / remove lxml from fcport plugin and use correct verbiage (#6912)

* NAS-110793 / 12.0 / remove lxml and xml.etree from identify_freebsd (#6921)

* remove lxml and xml.etree from identify_freebsd

* remove unused modules, one of which references lxml

* fix logic for checking matches by normalizing data

* NAS-110618 / 12.0 / Start SSH service on SSH pairing (#6885)

(cherry picked from commit 53ae118)

Co-authored-by: themylogin <themylogin@gmail.com>

* NAS-110719 / 12.0 / remove lxml from vm plugin on CORE (#6926)

* NAS-110783 / 12.0 / only unlock SED disks on active controller (#6924)

* only unlock SED disks on active controller

* use global variable so we dont unlock SED drives more than once

* NAS-110766 / 12.0 / remove lxml from global_freebsd plugin (#6911)

* remove lxml from global_freebsd plugin

* get rid of inner method

* NAS-110765 / 12.0 / remove lxml from fc_hba_not_present alert source (#6910)

* NAS-110701 / 12.0 / Remove copy-paste error from old AD LDAP code. (#6898)

LDAP plugin doesn't have `verbose_logging` key. We were using this
in code related to SASL_EXTERNAL binds, which are almost never used
in typical environments.

(cherry picked from commit e714861)

Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* NAS-110635 / 12.0 / Do not allow comma in cdrom path (#6902)

This commit adds changes to ensure user does not specify a comma in cdrom path as that would result in bhyve not able to parse the argument correctly.

* NAS-110712 / 12.0 / add --trace-malloc arg to middlewared (#6896)

* add --trace-malloc arg to middlewared

* address reviews and make logs more readable

* NAS-110667 / 12.0 / Add testparm output to debug (#6893)

(cherry picked from commit 2875104)

Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* NAS-110773 / 12.0 / Fix kerberos error (#6863)

* NAS-110922 / 12.0 / remove py-lxml build depdendency (#6967)

* NAS-110754 / 12.0 / Properly validate rsync extra arguments (#6968)

(cherry picked from commit 6428776)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110671 / 12.0 / prevent unlimited running cron jobs (#6971)

* NAS-110928 / 12.0 / Do not run camcontrol reprobe on vtbd drives (#6998)

* NAS-110969 / 12.0 / Increase verbosity of range-related idmap verrors (#6992)

Do a bit more work to pinpoint the exact idmap range overlap.

(cherry picked from commit 26c29cf)

Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* Add github action to use flake8

* Show flake8 errors introduced

* Add default value for extra list attribute (#7049)

* NAS-110600 / 12.0 / Correctly retrieve s3 attachment (#7041)

This commit fixes an issue if s3 path did not exist for some reason ( got deleted or was an old path ), retrieving s3 attachments would error out as the path would not exist. Secondly, if the path does not exist but has been configured - we create the path before starting the s3 service.

(cherry picked from commit c40a4d5)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-111124 / 12.0 / cache static info and optimize system.info on CORE (#7065)

* cache dmidecode results to reduce CPU time

* fix typo

* optimize system.info

* address review

* NAS-111046 / 12.0 / update recommended zvol blocksize depending on zpool geometry (#7072)

(cherry picked from commit 415bbea)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111132 / 12.0 / Fix r10 enclosures (by darkfiberiru) (#7092)

* Fix r10 enclosures. Including mapping drives bays to match standard pattern(left to right then top to bottom)instead of inverse. Tested on R10 and Mini-X+ to confirm mini mapping still works

(cherry picked from commit fd1995d)

* Additional enclosure fixes/cleanup

(cherry picked from commit 30123f3)

* document why we add back expansion shelves

(cherry picked from commit f719eb5)

* Fix small typos and type mixup added while fixing r10 enclosure management

(cherry picked from commit 1ce30e2)

* flake8 fixes and use double-quotes

(cherry picked from commit 1124780)

* flake8 fixes for enclosure.py

(cherry picked from commit 6a23820)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>
Co-authored-by: Caleb St. John <30729806+yocalebo@users.noreply.github.com>

* NAS-111223 / 12.0 / Fix R40 to comply with enclosure management nep specified single mapped enclosure to UI. (#7097)

Previous method didn't work along with not being nep compliant. Testing
shows that UI was already compliant and works with this change.

(cherry picked from commit 089528e)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111080 / 12.0 / Fix Enclosure Mapping for early version Mini X 3.0 Systems (by darkfiberiru) (#7098)

* Fix Enclosure Mapping for early version Mini 3.0 Systems

(cherry picked from commit 1b3741c)

* Enclosure map regex string needs to be a raw string to stop collision of string and regex escaping

(cherry picked from commit de17ad5)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111146 / 12.0 / add interfaces marked critical for failover to network.sh (#7088)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111259 / 12.0 / fix r20(a) to comply with enclosure NEP (#7105)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111273 / 12.0 / fix r50 to comply with enclosure NEP (#7107)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-106539 / 12.0 / Copy over profile(4) localbase/etc to reflect with upstream changes (#7110)

This commit adds changes to copy over profile(4) to localbase/etc as bash reads the login initialization changes from there now with recent changes introduced to freebsd upstream last year ( https://forums.freebsd.org/threads/what-happened-to-etc-profile.78406/ ).

* NAS-111283 / 12.0 / Bug fix for recommending zvol blocksize (#7114)

This commit fixes an issue where we are not sending 'K' suffix with the recommended blocksize which we were doing before and this results in change in behavior with existing implementation in UI/middleware breaking.

(cherry picked from commit 4e28818)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* Load if_atlantic module built form ports.

Ticket:	NAS-111219
(cherry picked from commit 80c7922)

* NAS-111343 / 12.0 / Additional Enclosure Fixes (by darkfiberiru) (#7127)

* Mini-E+ enclosure fix

(cherry picked from commit de493c9)

* Make sure that cobra expander works with r20/alternative r20 expander names

(cherry picked from commit a4174cb)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111335 / 12.0 / Fix Syntax error: "}" unexpected (expecting "fi" (#7133)

* NAS-111188 / 12.0 / support.new_ticket should fully consume system.debug file to prevent debug job running (and holding lock) forever (#7134)

* NAS-111099 / 12.0 / Fix replication progress saying "total 11.12 TiB of 11.04 TiB" (#7140)

* NAS-111042 / 12.0 / pool.dataset: optimized create/update operations (#6955)

* NAS-111384 / 12.0 / fix cpu temp crash in snmp-agent on VMs (#7143)

* NAS-111390 / 12.0 / Enclosure Management fix for R20A. (#7145)

System has weird drive mapping top to bottom then left to right naturally
Needs correspondending UI change to map drives Left to Right and top to
bottom as is standard(same as r20)

(cherry picked from commit 58f59d6)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111406 / 12.0 / Fix for mini-3.0-xl+ 2.5 inch drive bays being swapped. (#7155)

Prior to this change the side internal bay showed as slot 1(hot swap
2.5inch bay)
No UI change is needed for this fix mapping only
Confirmed against multiple systems both based opon production builds and
redbook verified builds

(cherry picked from commit d9bdfde)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* Implement NVMe resize in disk_resize for FreeBSD.

(cherry picked from commit 0338e09)

* NAS-110987 / 12.0 / Dataset checksum (#7160)

(cherry picked from commit 3651bfe)

Co-authored-by: themylogin <themylogin@gmail.com>

* Correct NVMe capacity check in disk_resize.

Appears for some device NVM Capacity is not equal to Size multiplied
by sector size, but is a pretty arbitrary value.  Use it directly.

(cherry picked from commit f19ea9a)

* NAS-111422 / 12.0 / fix issues with enclosure.sync_zpool (#7180)

* NAS-111462 / 12.0 / fix license detection on r-series devices (#7182)

* NAS-111487 / 12.0 / Do not crash pool.dataset.kill_processes if a process has terminated on its own (by themylogin) (#7193)

* Fix service.terminate_process return value documentation and prevent it from throwing psutil.NoSuchProcess

(cherry picked from commit 02ca757)

* Do not crash pool.dataset.kill_processes if a process has terminated on its own

(cherry picked from commit 4cee5fe)

Co-authored-by: themylogin <themylogin@gmail.com>

* NAS-111573 / 12.0 / fix race between dd and gpart destroy (#7232)

* Changing the timeout for plex job api test to 20 minutes (#7235)

* NAS-111682 / 12.0 / Improve smartd restart performance (#7263)

* Improve smartd config generation performance

* Improve pool.transform_topology performance

* NAS-111685 / 12.0 / make get_ses_enclosures scale on large systems (#7262)

* NAS-111690 / 12.0 / call enclosure.query once in disk.sync_all (#7265)

* NAS-111693 / 12.0 / fix enclosure.sync_zpool (#7266)

* NAS-111696 / 12.0 / dont call system.info in enclosure plugin (#7269)

* NAS-111720 / 12.0 / fix numerous issues with disk.sync_all (#7282)

* fix numerous issues with disk.sync_all

* address reviews

* NAS-111758 / 12.0 / fix IndexError in network.py (#7291)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111744 / 12.0 / Fix SSHd IPV6 link local ListenAddress (#7293)

When sshd is set to listen on an IPv6 link local address the interface
name must also be appended to the address.

(cherry picked from commit 6229e7a)

Co-authored-by: Benjamin M. Hughes <bmhughes@bmhughes.co.uk>

* NAS-111764 / 12.0 / move disk.sync_all and enclosure.sync_zpool on HA (#7297)

* NAS-111766 / 12.0 / fix slot number calculation for expansion shelves (#7299)

* NAS-111769 / 12.0 / fix disk identification with vmware nvme disks (#7305)

* NAS-111788 / 12.0 / fix typo (#7306)

* NAS-111790 / 12.0 / dont restart services twice on failover (#7310)

* NAS-111819 / 12.0 / Fix taking vmware-aware manual snapshots (#7321)

This was broken by acef89b

* removing OSX ssh code from AFP API testing (#7330)

removed scale related code from AFP API testing
added sleep on Enter your root password in install.exp to avoid
race condition when inputing passowrd

* NAS-111695 / 12.0 / Make sure we freeup unused pages in sqlite (#7338)

This commit adds changes to make sure that we force sqlite3 to give up space which has accumulated over time by deletions of entries. What sqlite3 does is that each data which is removed from the database, sqlite3 does not use the space consumed by that payload and instead moves it to it's list of free pages and whenever more data is added it uses these free pages first and then asks for more space. However with time these free pages can consume lots of space like for a user his db size was 15 mb approx and after vacumming it got to 706K. This change uses vacuum functionality of sqlite3 to make sure that we remove such free unused pages which are in hold by sqlite3.

(cherry picked from commit 6951ca4)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* DOCS-2484 / 12.0  Replaced ssh to OSX and BSD by smbclient command (#7346)

* Replaced ssh to OSX and BSD by smbclient command

removed scale related code
putted wait_on_job in the test that the job id is taken from

* removed OSX and BSD ssh and replaced it with smbclient

removed scale related code

* fixed test_019 set guestok to false

moved test_046 at test_011

* capitalize AD_USER SMB_NAME SMB_PATH

* remove guestok from activedirectory test

* remove raw string from smbclient command for ad and smb API test

* removing SMB1 testing from api2/activedirectory.py test

* NAS-111858 / 12.0 / Give a proper title to SSHLoginFailuresAlertClass (#7354)

* NAS-111676 / 12.0 / Do not allow deleting a target if it's in use by fcport service (#7358)

This commit adds validation to not allow deletion of a target if it's in use by fcport service as the process will faill because of foreign key reference.

* NAS-111276 / 12.0 / VMware VM UUIDs are not unique and we should use FindAllByUuid (#7356)

* NAS-111885 / 12.0 / Update check_user method to allow checking passwords of other users (by sonicaj) (#7371)

* Update check_user method to allow checking passwords of other users

This commit adds a change to check_user method adapting it so that it can be used for other users as well for checking if a specified password is accurate/correct against the specified username.

(cherry picked from commit 5604b7b)

* Add check_password endpoint

(cherry picked from commit 999aa2d)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-111954 / 12.0 / Fix GMail thread safety (#7378)

(cherry picked from commit a639860)

Co-authored-by: themylogin <themylogin@gmail.com>

* NAS-111779 / 12.0 / improve IoThreadPoolExecutor (by yocalebo) (#7381)

* improve IoThreadPoolExecutor

(cherry picked from commit 9efec7e)

* <= 1 for semaphore value check

(cherry picked from commit 407ba6e)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111949 / 12.0 / Gather rsyncmod statistics in usage plugin (#7387)

(cherry picked from commit f171bb5)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* remove TrueNAS SCALE related code (#7383)

removed api2/pool_acltype.py since it is for scale
backport webdav test from master branch and removed api2/webdav_bsd.py
and api2/webdav_osx.py
improved api2/smb_protocol.py
cleaned some code

* NAS-111808 / 12.0 / Add endpoint to retrieve serial port choices with serial port names in freebsd (#7390)

(cherry picked from commit d12d000)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* Fix renamed ports

* Fix bad git dependency

* NAS-108910 / 13.0 / run fenced on single-node HA capable systems (#6956)

* NAS-111594 / 13.0 / improve process for formatting disks (#7240)

* NAS-111667 / 13.0 / fix minor typo in wipe.py (#7254)

Co-authored-by: themylogin <themylogin@gmail.com>
Co-authored-by: Andrew <awalker@ixsystems.com>
Co-authored-by: bugclerk <40872210+bugclerk@users.noreply.github.com>
Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>
Co-authored-by: William Grzybowski <william@grzy.org>
Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>
Co-authored-by: Alexander Motin <mav@FreeBSD.org>
Co-authored-by: Richard Kojedzinszky <rkojedzinszky@users.noreply.github.com>
Co-authored-by: Eric Turgeon <4249848+ericbsd@users.noreply.github.com>
Co-authored-by: Benjamin M. Hughes <bmhughes@bmhughes.co.uk>
sonicaj added a commit that referenced this pull request Dec 7, 2021
* NAS-110174 / 12.0 / WBC_ERR_WINBIND_NOT_AVAILABLE is not an SSH login failure (#6800)

* NAS-110401 / 12.0 / Fix regression in service restart after pool decryption (#6824)

Refactoring a cherry-pick of fix from master introduced a
regression whereby we were sending a list of service names to
to be restarted to the core.bulk rather than a list of lists.

* NAS-109545 / 12.0 / Prevent nested loop queries in iscsi.target (#6657)

* NAS-109033 / 12.0 / Bug fix for correctly retrieving DN value from a cert (#6846)

There are cases where simple decode can fail when trying to read cert attributes. We should rely on openssl module instead to give us normalized values instead of trying to decode ourselves.

(cherry picked from commit b513057)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110243 / 12.0 / Allow endpoints to specify explicit http methods (by sonicaj) (#6819)

* Allow endpoints to specify explicit http methods

(cherry picked from commit 649ee8e)

* Add explicit get method on ui_restart endpoint

(cherry picked from commit 0dc94f2)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110433 / 12.0 / Restart mdns after installing a plugin (#6847)

We should restart mdns after a jail is started following #6587.

* Fix syntax error in SSSD-compatibility idmap config

(cherry picked from commit fe365ee)

* Do not report successful S.M.A.R.T. test starts on SCSI/SAS drives as failures

(cherry picked from commit acd7185)

* NAS-110496 / 12.0 / Explicitly set system dataset to readonly=off (#6869)

* NAS-110576 / 12.0 / R20A enclosure support (#6871)

* NAS-110240 / 12.0 / Do not show disks with temperature collection disable in corresponding Reporting UI section (#6861)

* NAS-110214 / 12.0 / Do not wait for NFS restart when configuring network interfaces. (#6866)

It might hang due to unknown reasons and cause old-style FreeBSD network setup on boot.

* NAS-109910 / 12.0 / Bring back iscsi_target_extent_name unique constraint (#6864)

* NAS-110602 / 12.0 / Fix concatenating enclosures (#6873)

* NAS-105205 / 12.0 / Sort pool.filesystem_choices by name (#6875)

* NAS-110607 / 12.0 / Fix (almost all) unit test failures (#6876)

* NAS-109989 / 12.0 / Convert dscache.refresh into a middleware job (#6872) (#6877)

This commit serializes cache refresh requests into a job queue and
ensures that cache is refreshed before writing to file.

Add docstring to directoryservices.cache_refresh

* NAS-110417 / 12.0 / Have unique serial number for certs (#6883)

When creating a self generated certificate, we should have unique serial numbers. The upper limit for serial is 2^160 but in the changes i am keeping it 30 as otherwise it will just be a really long number which is hard to access and it has a low probability that we have 2 self signed certificates generated by us with same serial number.

(cherry picked from commit 3909aea)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110575 / 12.0 / Stop TrueCommand connection on BACKUP node (#6886)

* NAS-110714 / 12.0 / fix memory allocation/leak issue in multipath.query() (#6897)

* fix memory allocation/leak issue in multipath.query()

* restore Config child class

* remove old debugging code

* storytime!

* run cert check every 24hrs and not on standby node

* NAS-110779 / 12.0 / Consistently retrieve boot time (#6918)

This commit fixes an issue where we were not normalizing the datetime object.

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110769 / 12.0 / remove lxml from fcport plugin and use correct verbiage (#6912)

* NAS-110793 / 12.0 / remove lxml and xml.etree from identify_freebsd (#6921)

* remove lxml and xml.etree from identify_freebsd

* remove unused modules, one of which references lxml

* fix logic for checking matches by normalizing data

* NAS-110618 / 12.0 / Start SSH service on SSH pairing (#6885)

(cherry picked from commit 53ae118)

Co-authored-by: themylogin <themylogin@gmail.com>

* NAS-110719 / 12.0 / remove lxml from vm plugin on CORE (#6926)

* NAS-110783 / 12.0 / only unlock SED disks on active controller (#6924)

* only unlock SED disks on active controller

* use global variable so we dont unlock SED drives more than once

* NAS-110766 / 12.0 / remove lxml from global_freebsd plugin (#6911)

* remove lxml from global_freebsd plugin

* get rid of inner method

* NAS-110765 / 12.0 / remove lxml from fc_hba_not_present alert source (#6910)

* NAS-110701 / 12.0 / Remove copy-paste error from old AD LDAP code. (#6898)

LDAP plugin doesn't have `verbose_logging` key. We were using this
in code related to SASL_EXTERNAL binds, which are almost never used
in typical environments.

(cherry picked from commit e714861)

Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* NAS-110635 / 12.0 / Do not allow comma in cdrom path (#6902)

This commit adds changes to ensure user does not specify a comma in cdrom path as that would result in bhyve not able to parse the argument correctly.

* NAS-110712 / 12.0 / add --trace-malloc arg to middlewared (#6896)

* add --trace-malloc arg to middlewared

* address reviews and make logs more readable

* NAS-110667 / 12.0 / Add testparm output to debug (#6893)

(cherry picked from commit 2875104)

Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* NAS-110773 / 12.0 / Fix kerberos error (#6863)

* NAS-110922 / 12.0 / remove py-lxml build depdendency (#6967)

* NAS-110754 / 12.0 / Properly validate rsync extra arguments (#6968)

(cherry picked from commit 6428776)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110671 / 12.0 / prevent unlimited running cron jobs (#6971)

* NAS-110928 / 12.0 / Do not run camcontrol reprobe on vtbd drives (#6998)

* NAS-110969 / 12.0 / Increase verbosity of range-related idmap verrors (#6992)

Do a bit more work to pinpoint the exact idmap range overlap.

(cherry picked from commit 26c29cf)

Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* Add github action to use flake8

* Show flake8 errors introduced

* Add default value for extra list attribute (#7049)

* NAS-110600 / 12.0 / Correctly retrieve s3 attachment (#7041)

This commit fixes an issue if s3 path did not exist for some reason ( got deleted or was an old path ), retrieving s3 attachments would error out as the path would not exist. Secondly, if the path does not exist but has been configured - we create the path before starting the s3 service.

(cherry picked from commit c40a4d5)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-111124 / 12.0 / cache static info and optimize system.info on CORE (#7065)

* cache dmidecode results to reduce CPU time

* fix typo

* optimize system.info

* address review

* NAS-111046 / 12.0 / update recommended zvol blocksize depending on zpool geometry (#7072)

(cherry picked from commit 415bbea)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111132 / 12.0 / Fix r10 enclosures (by darkfiberiru) (#7092)

* Fix r10 enclosures. Including mapping drives bays to match standard pattern(left to right then top to bottom)instead of inverse. Tested on R10 and Mini-X+ to confirm mini mapping still works

(cherry picked from commit fd1995d)

* Additional enclosure fixes/cleanup

(cherry picked from commit 30123f3)

* document why we add back expansion shelves

(cherry picked from commit f719eb5)

* Fix small typos and type mixup added while fixing r10 enclosure management

(cherry picked from commit 1ce30e2)

* flake8 fixes and use double-quotes

(cherry picked from commit 1124780)

* flake8 fixes for enclosure.py

(cherry picked from commit 6a23820)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>
Co-authored-by: Caleb St. John <30729806+yocalebo@users.noreply.github.com>

* NAS-111223 / 12.0 / Fix R40 to comply with enclosure management nep specified single mapped enclosure to UI. (#7097)

Previous method didn't work along with not being nep compliant. Testing
shows that UI was already compliant and works with this change.

(cherry picked from commit 089528e)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111080 / 12.0 / Fix Enclosure Mapping for early version Mini X 3.0 Systems (by darkfiberiru) (#7098)

* Fix Enclosure Mapping for early version Mini 3.0 Systems

(cherry picked from commit 1b3741c)

* Enclosure map regex string needs to be a raw string to stop collision of string and regex escaping

(cherry picked from commit de17ad5)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111146 / 12.0 / add interfaces marked critical for failover to network.sh (#7088)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111259 / 12.0 / fix r20(a) to comply with enclosure NEP (#7105)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111273 / 12.0 / fix r50 to comply with enclosure NEP (#7107)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-106539 / 12.0 / Copy over profile(4) localbase/etc to reflect with upstream changes (#7110)

This commit adds changes to copy over profile(4) to localbase/etc as bash reads the login initialization changes from there now with recent changes introduced to freebsd upstream last year ( https://forums.freebsd.org/threads/what-happened-to-etc-profile.78406/ ).

* NAS-111283 / 12.0 / Bug fix for recommending zvol blocksize (#7114)

This commit fixes an issue where we are not sending 'K' suffix with the recommended blocksize which we were doing before and this results in change in behavior with existing implementation in UI/middleware breaking.

(cherry picked from commit 4e28818)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* Load if_atlantic module built form ports.

Ticket:	NAS-111219
(cherry picked from commit 80c7922)

* NAS-111343 / 12.0 / Additional Enclosure Fixes (by darkfiberiru) (#7127)

* Mini-E+ enclosure fix

(cherry picked from commit de493c9)

* Make sure that cobra expander works with r20/alternative r20 expander names

(cherry picked from commit a4174cb)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111335 / 12.0 / Fix Syntax error: "}" unexpected (expecting "fi" (#7133)

* NAS-111188 / 12.0 / support.new_ticket should fully consume system.debug file to prevent debug job running (and holding lock) forever (#7134)

* NAS-111099 / 12.0 / Fix replication progress saying "total 11.12 TiB of 11.04 TiB" (#7140)

* NAS-111042 / 12.0 / pool.dataset: optimized create/update operations (#6955)

* NAS-111384 / 12.0 / fix cpu temp crash in snmp-agent on VMs (#7143)

* NAS-111390 / 12.0 / Enclosure Management fix for R20A. (#7145)

System has weird drive mapping top to bottom then left to right naturally
Needs correspondending UI change to map drives Left to Right and top to
bottom as is standard(same as r20)

(cherry picked from commit 58f59d6)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111406 / 12.0 / Fix for mini-3.0-xl+ 2.5 inch drive bays being swapped. (#7155)

Prior to this change the side internal bay showed as slot 1(hot swap
2.5inch bay)
No UI change is needed for this fix mapping only
Confirmed against multiple systems both based opon production builds and
redbook verified builds

(cherry picked from commit d9bdfde)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* Implement NVMe resize in disk_resize for FreeBSD.

(cherry picked from commit 0338e09)

* NAS-110987 / 12.0 / Dataset checksum (#7160)

(cherry picked from commit 3651bfe)

Co-authored-by: themylogin <themylogin@gmail.com>

* Correct NVMe capacity check in disk_resize.

Appears for some device NVM Capacity is not equal to Size multiplied
by sector size, but is a pretty arbitrary value.  Use it directly.

(cherry picked from commit f19ea9a)

* NAS-111422 / 12.0 / fix issues with enclosure.sync_zpool (#7180)

* NAS-111462 / 12.0 / fix license detection on r-series devices (#7182)

* NAS-111487 / 12.0 / Do not crash pool.dataset.kill_processes if a process has terminated on its own (by themylogin) (#7193)

* Fix service.terminate_process return value documentation and prevent it from throwing psutil.NoSuchProcess

(cherry picked from commit 02ca757)

* Do not crash pool.dataset.kill_processes if a process has terminated on its own

(cherry picked from commit 4cee5fe)

Co-authored-by: themylogin <themylogin@gmail.com>

* NAS-111573 / 12.0 / fix race between dd and gpart destroy (#7232)

* Changing the timeout for plex job api test to 20 minutes (#7235)

* NAS-111682 / 12.0 / Improve smartd restart performance (#7263)

* Improve smartd config generation performance

* Improve pool.transform_topology performance

* NAS-111685 / 12.0 / make get_ses_enclosures scale on large systems (#7262)

* NAS-111690 / 12.0 / call enclosure.query once in disk.sync_all (#7265)

* NAS-111693 / 12.0 / fix enclosure.sync_zpool (#7266)

* NAS-111696 / 12.0 / dont call system.info in enclosure plugin (#7269)

* NAS-111720 / 12.0 / fix numerous issues with disk.sync_all (#7282)

* fix numerous issues with disk.sync_all

* address reviews

* NAS-111758 / 12.0 / fix IndexError in network.py (#7291)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111744 / 12.0 / Fix SSHd IPV6 link local ListenAddress (#7293)

When sshd is set to listen on an IPv6 link local address the interface
name must also be appended to the address.

(cherry picked from commit 6229e7a)

Co-authored-by: Benjamin M. Hughes <bmhughes@bmhughes.co.uk>

* NAS-111764 / 12.0 / move disk.sync_all and enclosure.sync_zpool on HA (#7297)

* NAS-111766 / 12.0 / fix slot number calculation for expansion shelves (#7299)

* NAS-111769 / 12.0 / fix disk identification with vmware nvme disks (#7305)

* NAS-111788 / 12.0 / fix typo (#7306)

* NAS-111790 / 12.0 / dont restart services twice on failover (#7310)

* NAS-111819 / 12.0 / Fix taking vmware-aware manual snapshots (#7321)

This was broken by acef89b

* removing OSX ssh code from AFP API testing (#7330)

removed scale related code from AFP API testing
added sleep on Enter your root password in install.exp to avoid
race condition when inputing passowrd

* NAS-111695 / 12.0 / Make sure we freeup unused pages in sqlite (#7338)

This commit adds changes to make sure that we force sqlite3 to give up space which has accumulated over time by deletions of entries. What sqlite3 does is that each data which is removed from the database, sqlite3 does not use the space consumed by that payload and instead moves it to it's list of free pages and whenever more data is added it uses these free pages first and then asks for more space. However with time these free pages can consume lots of space like for a user his db size was 15 mb approx and after vacumming it got to 706K. This change uses vacuum functionality of sqlite3 to make sure that we remove such free unused pages which are in hold by sqlite3.

(cherry picked from commit 6951ca4)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* DOCS-2484 / 12.0  Replaced ssh to OSX and BSD by smbclient command (#7346)

* Replaced ssh to OSX and BSD by smbclient command

removed scale related code
putted wait_on_job in the test that the job id is taken from

* removed OSX and BSD ssh and replaced it with smbclient

removed scale related code

* fixed test_019 set guestok to false

moved test_046 at test_011

* capitalize AD_USER SMB_NAME SMB_PATH

* remove guestok from activedirectory test

* remove raw string from smbclient command for ad and smb API test

* removing SMB1 testing from api2/activedirectory.py test

* NAS-111858 / 12.0 / Give a proper title to SSHLoginFailuresAlertClass (#7354)

* NAS-111676 / 12.0 / Do not allow deleting a target if it's in use by fcport service (#7358)

This commit adds validation to not allow deletion of a target if it's in use by fcport service as the process will faill because of foreign key reference.

* NAS-111276 / 12.0 / VMware VM UUIDs are not unique and we should use FindAllByUuid (#7356)

* NAS-111885 / 12.0 / Update check_user method to allow checking passwords of other users (by sonicaj) (#7371)

* Update check_user method to allow checking passwords of other users

This commit adds a change to check_user method adapting it so that it can be used for other users as well for checking if a specified password is accurate/correct against the specified username.

(cherry picked from commit 5604b7b)

* Add check_password endpoint

(cherry picked from commit 999aa2d)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-111954 / 12.0 / Fix GMail thread safety (#7378)

(cherry picked from commit a639860)

Co-authored-by: themylogin <themylogin@gmail.com>

* NAS-111779 / 12.0 / improve IoThreadPoolExecutor (by yocalebo) (#7381)

* improve IoThreadPoolExecutor

(cherry picked from commit 9efec7e)

* <= 1 for semaphore value check

(cherry picked from commit 407ba6e)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111949 / 12.0 / Gather rsyncmod statistics in usage plugin (#7387)

(cherry picked from commit f171bb5)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* remove TrueNAS SCALE related code (#7383)

removed api2/pool_acltype.py since it is for scale
backport webdav test from master branch and removed api2/webdav_bsd.py
and api2/webdav_osx.py
improved api2/smb_protocol.py
cleaned some code

* NAS-111808 / 12.0 / Add endpoint to retrieve serial port choices with serial port names in freebsd (#7390)

(cherry picked from commit d12d000)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* Fix renamed ports

* Fix bad git dependency

* NAS-108910 / 13.0 / run fenced on single-node HA capable systems (#6956)

* NAS-111594 / 13.0 / improve process for formatting disks (#7240)

* NAS-111667 / 13.0 / fix minor typo in wipe.py (#7254)

Co-authored-by: themylogin <themylogin@gmail.com>
Co-authored-by: Andrew <awalker@ixsystems.com>
Co-authored-by: bugclerk <40872210+bugclerk@users.noreply.github.com>
Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>
Co-authored-by: William Grzybowski <william@grzy.org>
Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>
Co-authored-by: Alexander Motin <mav@FreeBSD.org>
Co-authored-by: Richard Kojedzinszky <rkojedzinszky@users.noreply.github.com>
Co-authored-by: Eric Turgeon <4249848+ericbsd@users.noreply.github.com>
Co-authored-by: Benjamin M. Hughes <bmhughes@bmhughes.co.uk>
rick-mesta pushed a commit that referenced this pull request Dec 9, 2021
* NAS-110174 / 12.0 / WBC_ERR_WINBIND_NOT_AVAILABLE is not an SSH login failure (#6800)

* NAS-110401 / 12.0 / Fix regression in service restart after pool decryption (#6824)

Refactoring a cherry-pick of fix from master introduced a
regression whereby we were sending a list of service names to
to be restarted to the core.bulk rather than a list of lists.

* NAS-109545 / 12.0 / Prevent nested loop queries in iscsi.target (#6657)

* NAS-109033 / 12.0 / Bug fix for correctly retrieving DN value from a cert (#6846)

There are cases where simple decode can fail when trying to read cert attributes. We should rely on openssl module instead to give us normalized values instead of trying to decode ourselves.

(cherry picked from commit b513057)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110243 / 12.0 / Allow endpoints to specify explicit http methods (by sonicaj) (#6819)

* Allow endpoints to specify explicit http methods

(cherry picked from commit 649ee8e)

* Add explicit get method on ui_restart endpoint

(cherry picked from commit 0dc94f2)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110433 / 12.0 / Restart mdns after installing a plugin (#6847)

We should restart mdns after a jail is started following #6587.

* Fix syntax error in SSSD-compatibility idmap config

(cherry picked from commit fe365ee)

* Do not report successful S.M.A.R.T. test starts on SCSI/SAS drives as failures

(cherry picked from commit acd7185)

* NAS-110496 / 12.0 / Explicitly set system dataset to readonly=off (#6869)

* NAS-110576 / 12.0 / R20A enclosure support (#6871)

* NAS-110240 / 12.0 / Do not show disks with temperature collection disable in corresponding Reporting UI section (#6861)

* NAS-110214 / 12.0 / Do not wait for NFS restart when configuring network interfaces. (#6866)

It might hang due to unknown reasons and cause old-style FreeBSD network setup on boot.

* NAS-109910 / 12.0 / Bring back iscsi_target_extent_name unique constraint (#6864)

* NAS-110602 / 12.0 / Fix concatenating enclosures (#6873)

* NAS-105205 / 12.0 / Sort pool.filesystem_choices by name (#6875)

* NAS-110607 / 12.0 / Fix (almost all) unit test failures (#6876)

* NAS-109989 / 12.0 / Convert dscache.refresh into a middleware job (#6872) (#6877)

This commit serializes cache refresh requests into a job queue and
ensures that cache is refreshed before writing to file.

Add docstring to directoryservices.cache_refresh

* NAS-110417 / 12.0 / Have unique serial number for certs (#6883)

When creating a self generated certificate, we should have unique serial numbers. The upper limit for serial is 2^160 but in the changes i am keeping it 30 as otherwise it will just be a really long number which is hard to access and it has a low probability that we have 2 self signed certificates generated by us with same serial number.

(cherry picked from commit 3909aea)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110575 / 12.0 / Stop TrueCommand connection on BACKUP node (#6886)

* NAS-110714 / 12.0 / fix memory allocation/leak issue in multipath.query() (#6897)

* fix memory allocation/leak issue in multipath.query()

* restore Config child class

* remove old debugging code

* storytime!

* run cert check every 24hrs and not on standby node

* NAS-110779 / 12.0 / Consistently retrieve boot time (#6918)

This commit fixes an issue where we were not normalizing the datetime object.

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110769 / 12.0 / remove lxml from fcport plugin and use correct verbiage (#6912)

* NAS-110793 / 12.0 / remove lxml and xml.etree from identify_freebsd (#6921)

* remove lxml and xml.etree from identify_freebsd

* remove unused modules, one of which references lxml

* fix logic for checking matches by normalizing data

* NAS-110618 / 12.0 / Start SSH service on SSH pairing (#6885)

(cherry picked from commit 53ae118)

Co-authored-by: themylogin <themylogin@gmail.com>

* NAS-110719 / 12.0 / remove lxml from vm plugin on CORE (#6926)

* NAS-110783 / 12.0 / only unlock SED disks on active controller (#6924)

* only unlock SED disks on active controller

* use global variable so we dont unlock SED drives more than once

* NAS-110766 / 12.0 / remove lxml from global_freebsd plugin (#6911)

* remove lxml from global_freebsd plugin

* get rid of inner method

* NAS-110765 / 12.0 / remove lxml from fc_hba_not_present alert source (#6910)

* NAS-110701 / 12.0 / Remove copy-paste error from old AD LDAP code. (#6898)

LDAP plugin doesn't have `verbose_logging` key. We were using this
in code related to SASL_EXTERNAL binds, which are almost never used
in typical environments.

(cherry picked from commit e714861)

Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* NAS-110635 / 12.0 / Do not allow comma in cdrom path (#6902)

This commit adds changes to ensure user does not specify a comma in cdrom path as that would result in bhyve not able to parse the argument correctly.

* NAS-110712 / 12.0 / add --trace-malloc arg to middlewared (#6896)

* add --trace-malloc arg to middlewared

* address reviews and make logs more readable

* NAS-110667 / 12.0 / Add testparm output to debug (#6893)

(cherry picked from commit 2875104)

Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* NAS-110773 / 12.0 / Fix kerberos error (#6863)

* NAS-110922 / 12.0 / remove py-lxml build depdendency (#6967)

* NAS-110754 / 12.0 / Properly validate rsync extra arguments (#6968)

(cherry picked from commit 6428776)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-110671 / 12.0 / prevent unlimited running cron jobs (#6971)

* NAS-110928 / 12.0 / Do not run camcontrol reprobe on vtbd drives (#6998)

* NAS-110969 / 12.0 / Increase verbosity of range-related idmap verrors (#6992)

Do a bit more work to pinpoint the exact idmap range overlap.

(cherry picked from commit 26c29cf)

Co-authored-by: Andrew Walker <awalker@ixsystems.com>

* Add github action to use flake8

* Show flake8 errors introduced

* Add default value for extra list attribute (#7049)

* NAS-110600 / 12.0 / Correctly retrieve s3 attachment (#7041)

This commit fixes an issue if s3 path did not exist for some reason ( got deleted or was an old path ), retrieving s3 attachments would error out as the path would not exist. Secondly, if the path does not exist but has been configured - we create the path before starting the s3 service.

(cherry picked from commit c40a4d5)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-111124 / 12.0 / cache static info and optimize system.info on CORE (#7065)

* cache dmidecode results to reduce CPU time

* fix typo

* optimize system.info

* address review

* NAS-111046 / 12.0 / update recommended zvol blocksize depending on zpool geometry (#7072)

(cherry picked from commit 415bbea)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111132 / 12.0 / Fix r10 enclosures (by darkfiberiru) (#7092)

* Fix r10 enclosures. Including mapping drives bays to match standard pattern(left to right then top to bottom)instead of inverse. Tested on R10 and Mini-X+ to confirm mini mapping still works

(cherry picked from commit fd1995d)

* Additional enclosure fixes/cleanup

(cherry picked from commit 30123f3)

* document why we add back expansion shelves

(cherry picked from commit f719eb5)

* Fix small typos and type mixup added while fixing r10 enclosure management

(cherry picked from commit 1ce30e2)

* flake8 fixes and use double-quotes

(cherry picked from commit 1124780)

* flake8 fixes for enclosure.py

(cherry picked from commit 6a23820)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>
Co-authored-by: Caleb St. John <30729806+yocalebo@users.noreply.github.com>

* NAS-111223 / 12.0 / Fix R40 to comply with enclosure management nep specified single mapped enclosure to UI. (#7097)

Previous method didn't work along with not being nep compliant. Testing
shows that UI was already compliant and works with this change.

(cherry picked from commit 089528e)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111080 / 12.0 / Fix Enclosure Mapping for early version Mini X 3.0 Systems (by darkfiberiru) (#7098)

* Fix Enclosure Mapping for early version Mini 3.0 Systems

(cherry picked from commit 1b3741c)

* Enclosure map regex string needs to be a raw string to stop collision of string and regex escaping

(cherry picked from commit de17ad5)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111146 / 12.0 / add interfaces marked critical for failover to network.sh (#7088)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111259 / 12.0 / fix r20(a) to comply with enclosure NEP (#7105)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111273 / 12.0 / fix r50 to comply with enclosure NEP (#7107)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-106539 / 12.0 / Copy over profile(4) localbase/etc to reflect with upstream changes (#7110)

This commit adds changes to copy over profile(4) to localbase/etc as bash reads the login initialization changes from there now with recent changes introduced to freebsd upstream last year ( https://forums.freebsd.org/threads/what-happened-to-etc-profile.78406/ ).

* NAS-111283 / 12.0 / Bug fix for recommending zvol blocksize (#7114)

This commit fixes an issue where we are not sending 'K' suffix with the recommended blocksize which we were doing before and this results in change in behavior with existing implementation in UI/middleware breaking.

(cherry picked from commit 4e28818)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* Load if_atlantic module built form ports.

Ticket:	NAS-111219
(cherry picked from commit 80c7922)

* NAS-111343 / 12.0 / Additional Enclosure Fixes (by darkfiberiru) (#7127)

* Mini-E+ enclosure fix

(cherry picked from commit de493c9)

* Make sure that cobra expander works with r20/alternative r20 expander names

(cherry picked from commit a4174cb)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111335 / 12.0 / Fix Syntax error: "}" unexpected (expecting "fi" (#7133)

* NAS-111188 / 12.0 / support.new_ticket should fully consume system.debug file to prevent debug job running (and holding lock) forever (#7134)

* NAS-111099 / 12.0 / Fix replication progress saying "total 11.12 TiB of 11.04 TiB" (#7140)

* NAS-111042 / 12.0 / pool.dataset: optimized create/update operations (#6955)

* NAS-111384 / 12.0 / fix cpu temp crash in snmp-agent on VMs (#7143)

* NAS-111390 / 12.0 / Enclosure Management fix for R20A. (#7145)

System has weird drive mapping top to bottom then left to right naturally
Needs correspondending UI change to map drives Left to Right and top to
bottom as is standard(same as r20)

(cherry picked from commit 58f59d6)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* NAS-111406 / 12.0 / Fix for mini-3.0-xl+ 2.5 inch drive bays being swapped. (#7155)

Prior to this change the side internal bay showed as slot 1(hot swap
2.5inch bay)
No UI change is needed for this fix mapping only
Confirmed against multiple systems both based opon production builds and
redbook verified builds

(cherry picked from commit d9bdfde)

Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>

* Implement NVMe resize in disk_resize for FreeBSD.

(cherry picked from commit 0338e09)

* NAS-110987 / 12.0 / Dataset checksum (#7160)

(cherry picked from commit 3651bfe)

Co-authored-by: themylogin <themylogin@gmail.com>

* Correct NVMe capacity check in disk_resize.

Appears for some device NVM Capacity is not equal to Size multiplied
by sector size, but is a pretty arbitrary value.  Use it directly.

(cherry picked from commit f19ea9a)

* NAS-111422 / 12.0 / fix issues with enclosure.sync_zpool (#7180)

* NAS-111462 / 12.0 / fix license detection on r-series devices (#7182)

* NAS-111487 / 12.0 / Do not crash pool.dataset.kill_processes if a process has terminated on its own (by themylogin) (#7193)

* Fix service.terminate_process return value documentation and prevent it from throwing psutil.NoSuchProcess

(cherry picked from commit 02ca757)

* Do not crash pool.dataset.kill_processes if a process has terminated on its own

(cherry picked from commit 4cee5fe)

Co-authored-by: themylogin <themylogin@gmail.com>

* NAS-111573 / 12.0 / fix race between dd and gpart destroy (#7232)

* Changing the timeout for plex job api test to 20 minutes (#7235)

* NAS-111682 / 12.0 / Improve smartd restart performance (#7263)

* Improve smartd config generation performance

* Improve pool.transform_topology performance

* NAS-111685 / 12.0 / make get_ses_enclosures scale on large systems (#7262)

* NAS-111690 / 12.0 / call enclosure.query once in disk.sync_all (#7265)

* NAS-111693 / 12.0 / fix enclosure.sync_zpool (#7266)

* NAS-111696 / 12.0 / dont call system.info in enclosure plugin (#7269)

* NAS-111720 / 12.0 / fix numerous issues with disk.sync_all (#7282)

* fix numerous issues with disk.sync_all

* address reviews

* NAS-111758 / 12.0 / fix IndexError in network.py (#7291)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111744 / 12.0 / Fix SSHd IPV6 link local ListenAddress (#7293)

When sshd is set to listen on an IPv6 link local address the interface
name must also be appended to the address.

(cherry picked from commit 6229e7a)

Co-authored-by: Benjamin M. Hughes <bmhughes@bmhughes.co.uk>

* NAS-111764 / 12.0 / move disk.sync_all and enclosure.sync_zpool on HA (#7297)

* NAS-111766 / 12.0 / fix slot number calculation for expansion shelves (#7299)

* NAS-111769 / 12.0 / fix disk identification with vmware nvme disks (#7305)

* NAS-111788 / 12.0 / fix typo (#7306)

* NAS-111790 / 12.0 / dont restart services twice on failover (#7310)

* NAS-111819 / 12.0 / Fix taking vmware-aware manual snapshots (#7321)

This was broken by acef89b

* removing OSX ssh code from AFP API testing (#7330)

removed scale related code from AFP API testing
added sleep on Enter your root password in install.exp to avoid
race condition when inputing passowrd

* NAS-111695 / 12.0 / Make sure we freeup unused pages in sqlite (#7338)

This commit adds changes to make sure that we force sqlite3 to give up space which has accumulated over time by deletions of entries. What sqlite3 does is that each data which is removed from the database, sqlite3 does not use the space consumed by that payload and instead moves it to it's list of free pages and whenever more data is added it uses these free pages first and then asks for more space. However with time these free pages can consume lots of space like for a user his db size was 15 mb approx and after vacumming it got to 706K. This change uses vacuum functionality of sqlite3 to make sure that we remove such free unused pages which are in hold by sqlite3.

(cherry picked from commit 6951ca4)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* DOCS-2484 / 12.0  Replaced ssh to OSX and BSD by smbclient command (#7346)

* Replaced ssh to OSX and BSD by smbclient command

removed scale related code
putted wait_on_job in the test that the job id is taken from

* removed OSX and BSD ssh and replaced it with smbclient

removed scale related code

* fixed test_019 set guestok to false

moved test_046 at test_011

* capitalize AD_USER SMB_NAME SMB_PATH

* remove guestok from activedirectory test

* remove raw string from smbclient command for ad and smb API test

* removing SMB1 testing from api2/activedirectory.py test

* NAS-111858 / 12.0 / Give a proper title to SSHLoginFailuresAlertClass (#7354)

* NAS-111676 / 12.0 / Do not allow deleting a target if it's in use by fcport service (#7358)

This commit adds validation to not allow deletion of a target if it's in use by fcport service as the process will faill because of foreign key reference.

* NAS-111276 / 12.0 / VMware VM UUIDs are not unique and we should use FindAllByUuid (#7356)

* NAS-111885 / 12.0 / Update check_user method to allow checking passwords of other users (by sonicaj) (#7371)

* Update check_user method to allow checking passwords of other users

This commit adds a change to check_user method adapting it so that it can be used for other users as well for checking if a specified password is accurate/correct against the specified username.

(cherry picked from commit 5604b7b)

* Add check_password endpoint

(cherry picked from commit 999aa2d)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* NAS-111954 / 12.0 / Fix GMail thread safety (#7378)

(cherry picked from commit a639860)

Co-authored-by: themylogin <themylogin@gmail.com>

* NAS-111779 / 12.0 / improve IoThreadPoolExecutor (by yocalebo) (#7381)

* improve IoThreadPoolExecutor

(cherry picked from commit 9efec7e)

* <= 1 for semaphore value check

(cherry picked from commit 407ba6e)

Co-authored-by: caleb <yocalebo@gmail.com>

* NAS-111949 / 12.0 / Gather rsyncmod statistics in usage plugin (#7387)

(cherry picked from commit f171bb5)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* remove TrueNAS SCALE related code (#7383)

removed api2/pool_acltype.py since it is for scale
backport webdav test from master branch and removed api2/webdav_bsd.py
and api2/webdav_osx.py
improved api2/smb_protocol.py
cleaned some code

* NAS-111808 / 12.0 / Add endpoint to retrieve serial port choices with serial port names in freebsd (#7390)

(cherry picked from commit d12d000)

Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>

* Fix renamed ports

* Fix bad git dependency

* NAS-108910 / 13.0 / run fenced on single-node HA capable systems (#6956)

* NAS-111594 / 13.0 / improve process for formatting disks (#7240)

* NAS-111667 / 13.0 / fix minor typo in wipe.py (#7254)

Co-authored-by: themylogin <themylogin@gmail.com>
Co-authored-by: Andrew <awalker@ixsystems.com>
Co-authored-by: bugclerk <40872210+bugclerk@users.noreply.github.com>
Co-authored-by: Waqar Ahmed <waqarahmedjoyia@live.com>
Co-authored-by: William Grzybowski <william@grzy.org>
Co-authored-by: Nick 'darkfiberiru' Wolff <darkfiberiru@gmail.com>
Co-authored-by: Alexander Motin <mav@FreeBSD.org>
Co-authored-by: Richard Kojedzinszky <rkojedzinszky@users.noreply.github.com>
Co-authored-by: Eric Turgeon <4249848+ericbsd@users.noreply.github.com>
Co-authored-by: Benjamin M. Hughes <bmhughes@bmhughes.co.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants