Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Socket overhaul #128

Open
wants to merge 76 commits into
base: master
Choose a base branch
from
Open

Socket overhaul #128

wants to merge 76 commits into from

Commits on Jan 31, 2020

  1. Changed socket type to Dealer, and added outline for creating message… 

    … map
    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    213caa3 View commit details
    Browse the repository at this point in the history

  2. implement create function for message request map

    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    d4dfb92 View commit details
    Browse the repository at this point in the history

  3. Add operation to message map

    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    56371f8 View commit details
    Browse the repository at this point in the history

  4. get an operation from the map

    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    7bde826 View commit details
    Browse the repository at this point in the history

  5. Remove an op from the message map

    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    c0f9dc5 View commit details
    Browse the repository at this point in the history

  6. Update the add_map_op function to update the map if it already has an… 

    … op and return the old one
    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    9957a99 View commit details
    Browse the repository at this point in the history

  7. Get a specific disk hashmap

    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    82bd8f3 View commit details
    Browse the repository at this point in the history

  8. Added channels and edited disk-manager's add_disk and listen function… 

    …s with threadPool for threaded worker add_disk operations
    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    5336b74 View commit details
    Browse the repository at this point in the history

  9. Changed disk-manager handler for all functions to use threadpool/DEAL… 

    …ER sockets
    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    50a46a3 View commit details
    Browse the repository at this point in the history

  10. Swapped DEALER with STREAM in disk-manager

    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    72f3456 View commit details
    Browse the repository at this point in the history

  11. Added handling of sending messages from Bynar to disk-manager

    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    9d4b09d View commit details
    Browse the repository at this point in the history

  12. Added macros, handling add_disk

    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    6fd3fb8 View commit details
    Browse the repository at this point in the history

  13. Fix a little client stuff

    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    4767549 View commit details
    Browse the repository at this point in the history

  14. Some client fixing + macros

    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    c3a59d6 View commit details
    Browse the repository at this point in the history

  15. Updated Zmq version

    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    02a9ac4 View commit details
    Browse the repository at this point in the history

  16. fix notify?

    @mzhong1
    mzhong1 committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    770147f View commit details
    Browse the repository at this point in the history

Commits on Feb 3, 2020

  1. Filter states in returned state machines and clippy fixes

    @mzhong1
    mzhong1 committed Feb 3, 2020
    Configuration menu
    Copy the full SHA
    bf586c5 View commit details
    Browse the repository at this point in the history

  2. Filter disks for WaitingForReplacement and not in-progress

    @mzhong1
    mzhong1 committed Feb 3, 2020
    Configuration menu
    Copy the full SHA
    f444936 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2020

  1. Switched to STREAM/STREAM system

    @mzhong1
    mzhong1 committed Feb 5, 2020
    Configuration menu
    Copy the full SHA
    479ade6 View commit details
    Browse the repository at this point in the history

  2. Fix bynar-client to use new socket system

    @mzhong1
    mzhong1 committed Feb 5, 2020
    Configuration menu
    Copy the full SHA
    5e50318 View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2020

  1. parse_from_bytes grabs messages from the end of the byte vector given… 

    …, added function to parse message properly and drain message bytes properly
    @mzhong1
    mzhong1 committed Feb 6, 2020
    Configuration menu
    Copy the full SHA
    51e6041 View commit details
    Browse the repository at this point in the history

  2. change remove/safeto-remove to be stricter with config/non-lvm bluest… 

    …ore removals
    @mzhong1
    mzhong1 committed Feb 6, 2020
    Configuration menu
    Copy the full SHA
    08bce43 View commit details
    Browse the repository at this point in the history

  3. Fix macro use for parsing opresults

    @mzhong1
    mzhong1 committed Feb 6, 2020
    Configuration menu
    Copy the full SHA
    6e7b3a5 View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2020

  1. filter state machines for the disks that need replacement + add in th… 

    …e partitions on the disks to be replaced to the list
    @mzhong1
    mzhong1 committed Feb 7, 2020
    Configuration menu
    Copy the full SHA
    3528e8c View commit details
    Browse the repository at this point in the history

  2. Push SafeToRemove + Remove to queue

    @mzhong1
    mzhong1 committed Feb 7, 2020
    Configuration menu
    Copy the full SHA
    91ec3b8 View commit details
    Browse the repository at this point in the history

  3. Error's cannot be cloned, so change check_for_failed_disk to error ou… 

    …t immediately if check_all_disks failed
    @mzhong1
    mzhong1 committed Feb 7, 2020
    Configuration menu
    Copy the full SHA
    c9936c2 View commit details
    Browse the repository at this point in the history

  4. handle add and remove return values

    @mzhong1
    mzhong1 committed Feb 7, 2020
    Configuration menu
    Copy the full SHA
    397e153 View commit details
    Browse the repository at this point in the history

Commits on Feb 10, 2020

  1. Add new message handler to main function

    @mzhong1
    mzhong1 committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    04f690f View commit details
    Browse the repository at this point in the history

  2. Add functions to create a req_map for disk-manager + check if ops are… 

    … already running
    @mzhong1
    mzhong1 committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    85d3d61 View commit details
    Browse the repository at this point in the history

  3. Added request map to disk-manager, skip repeat requests

    @mzhong1
    mzhong1 committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    96ce4b0 View commit details
    Browse the repository at this point in the history

  4. Add behavior to output correct message if skipping an operation becau… 

    …se the disk-manager is already running an op on specified disk
    @mzhong1
    mzhong1 committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    eacbe7b View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2020

  1. fix the return values to not error incorrectly

    @mzhong1
    mzhong1 committed Feb 12, 2020
    Configuration menu
    Copy the full SHA
    6737dde View commit details
    Browse the repository at this point in the history

  2. Fix Error handling for bynar-client'

    @mzhong1
    mzhong1 committed Feb 12, 2020
    Configuration menu
    Copy the full SHA
    7c5f52a View commit details
    Browse the repository at this point in the history

  3. added checks before removing partitions/unmounting partitions

    @mzhong1
    mzhong1 committed Feb 12, 2020
    Configuration menu
    Copy the full SHA
    b2a9a96 View commit details
    Browse the repository at this point in the history

  4. SafeToRemove returning error is expected, disk path might not be remo… 

    …vable
    @mzhong1
    mzhong1 committed Feb 12, 2020
    Configuration menu
    Copy the full SHA
    1d332c5 View commit details
    Browse the repository at this point in the history

Commits on Feb 17, 2020

  1. Handle looping and fix error out, Change handle all_finished to check… 

    … for a safely completed remove operation. If none do usual, otherwise use the

safe one, if all safe-to-remove finished (as unsuccessful) return, figure out how to only run safe-to-remove/remove once, handle errors properly
    @mzhong1
    mzhong1 committed Feb 17, 2020
    Configuration menu
    Copy the full SHA
    7f6efe2 View commit details
    Browse the repository at this point in the history

  2. Add the Successful call but SafeToRemove is false to all_finished check

    @mzhong1
    mzhong1 committed Feb 17, 2020
    Configuration menu
    Copy the full SHA
    b345ff6 View commit details
    Browse the repository at this point in the history

Commits on Feb 18, 2020

  1. Test without ticket erroring out

    @mzhong1
    mzhong1 committed Feb 18, 2020
    Configuration menu
    Copy the full SHA
    2587893 View commit details
    Browse the repository at this point in the history

  2. error out on ticket creation failure

    @mzhong1
    mzhong1 committed Feb 18, 2020
    Configuration menu
    Copy the full SHA
    cb6459f View commit details
    Browse the repository at this point in the history

Commits on Feb 19, 2020

  1. Working on Unit tests, added condition to state machine so itonly ski… 

    …ps disks that aren't mounted
    @mzhong1
    mzhong1 committed Feb 19, 2020
    Configuration menu
    Copy the full SHA
    780fbdc View commit details
    Browse the repository at this point in the history

  2. Added unit testing for filters and creating the request map

    @mzhong1
    mzhong1 committed Feb 19, 2020
    Configuration menu
    Copy the full SHA
    6b71682 View commit details
    Browse the repository at this point in the history

  3. Unit tests for create_msg_map, creation without partitions, with part… 

    …itions, with DB devices
    @mzhong1
    mzhong1 committed Feb 19, 2020
    Configuration menu
    Copy the full SHA
    b1109b0 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2020

  1. Rearranged the code in add_or_update_op for smaller function + easier… 

    … to read and added tests for it
    @mzhong1
    mzhong1 committed Feb 20, 2020
    Configuration menu
    Copy the full SHA
    dfc2111 View commit details
    Browse the repository at this point in the history

  2. Moved repeat code into seperate key request function, squished down r… 

    …eq-map operations
    @mzhong1
    mzhong1 committed Feb 20, 2020
    Configuration menu
    Copy the full SHA
    e030a1c View commit details
    Browse the repository at this point in the history

  3. Add unit tests for the other req map functions

    @mzhong1
    mzhong1 committed Feb 20, 2020
    Configuration menu
    Copy the full SHA
    2b1621d View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2020

  1. check filtering waiting for replacement disks from state machine output

    @mzhong1
    mzhong1 committed Feb 21, 2020
    Configuration menu
    Copy the full SHA
    6947fda View commit details
    Browse the repository at this point in the history

  2. Add unit tests for getting the right partitions/disks associated with… 

    … the paths to be removed
    @mzhong1
    mzhong1 committed Feb 21, 2020
    Configuration menu
    Copy the full SHA
    0dad151 View commit details
    Browse the repository at this point in the history

Commits on Feb 24, 2020

  1. added test for removing duplicates/sorting and reformatted files to d… 

    …ecrease filesize
    @mzhong1
    mzhong1 committed Feb 24, 2020
    Configuration menu
    Copy the full SHA
    07240d8 View commit details
    Browse the repository at this point in the history

  2. squish tests down

    @mzhong1
    mzhong1 committed Feb 24, 2020
    Configuration menu
    Copy the full SHA
    6036e1a View commit details
    Browse the repository at this point in the history

Commits on Feb 25, 2020

  1. Fix all finish check for SafeToRemove handling and add unit testing

    @mzhong1
    mzhong1 committed Feb 25, 2020
    Configuration menu
    Copy the full SHA
    0d2c817 View commit details
    Browse the repository at this point in the history

  2. use new function get_request_keys instead

    @mzhong1
    mzhong1 committed Feb 25, 2020
    Configuration menu
    Copy the full SHA
    a7e03d3 View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2020

  1. Clippy and code squishing

    @mzhong1
    mzhong1 committed Feb 26, 2020
    Configuration menu
    Copy the full SHA
    55d2107 View commit details
    Browse the repository at this point in the history

  2. pull out repeat behavior into smaller functions for reduced complexity

    @mzhong1
    mzhong1 committed Feb 26, 2020
    Configuration menu
    Copy the full SHA
    2facc6f View commit details
    Browse the repository at this point in the history

  3. Removed repeat code and replaced with function call

    @mzhong1
    mzhong1 committed Feb 26, 2020
    Configuration menu
    Copy the full SHA
    53df069 View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2020

  1. pulled out repeat behavior into a macro

    @mzhong1
    mzhong1 committed Feb 28, 2020
    Configuration menu
    Copy the full SHA
    c80adb2 View commit details
    Browse the repository at this point in the history

  2. clean up some redundant code

    @mzhong1
    mzhong1 committed Feb 28, 2020
    Configuration menu
    Copy the full SHA
    cc798ff View commit details
    Browse the repository at this point in the history

  3. Fixed up returning an error instead of ok, and revamped the listen fu… 

    …nction and reorganized it for easier comprehension
    @mzhong1
    mzhong1 committed Feb 28, 2020
    Configuration menu
    Copy the full SHA
    3e384cb View commit details
    Browse the repository at this point in the history

Commits on Mar 2, 2020

  1. Fix issue with is_all_finished not correctly determining when all ope… 

    …rations on a disk have completed (either as successful or failed)
    @mzhong1
    mzhong1 committed Mar 2, 2020
    Configuration menu
    Copy the full SHA
    f54c709 View commit details
    Browse the repository at this point in the history

Commits on Mar 3, 2020

  1. Update toml's GPT crate

    @mzhong1
    mzhong1 committed Mar 3, 2020
    Configuration menu
    Copy the full SHA
    5070465 View commit details
    Browse the repository at this point in the history

Commits on Mar 10, 2020

  1. remove unused disk-manager code and update backend

    @mzhong1
    mzhong1 committed Mar 10, 2020
    Configuration menu
    Copy the full SHA
    4702c9a View commit details
    Browse the repository at this point in the history

Commits on Mar 31, 2020

  1. add comment, small loop sleep, and CentOS package dependencies

    @mzhong1
    mzhong1 committed Mar 31, 2020
    Configuration menu
    Copy the full SHA
    bbd763f View commit details
    Browse the repository at this point in the history

  2. move the smartctl enable as a soft error, since smartctl -H runs rega… 

    …rdless
    @mzhong1
    mzhong1 committed Mar 31, 2020
    Configuration menu
    Copy the full SHA
    d383c9b View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2020

  1. Fix checking if disk operation already done to account for bluestore … 

    …non-lvm OSDs and fix database ticket update function to use correct column name
    @mzhong1
    mzhong1 committed Apr 1, 2020
    Configuration menu
    Copy the full SHA
    103e7bf View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2020

  1. Add check in osd metadata for non-lvm bluestore osds, remove SkipRepe… 

    …at operations once finished running
    @mzhong1
    mzhong1 committed Apr 2, 2020
    Configuration menu
    Copy the full SHA
    50ed446 View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2020

  1. update TOML to use correct block-utils version and vault crates

    @mzhong1
    mzhong1 committed Apr 13, 2020
    Configuration menu
    Copy the full SHA
    e13a72c View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2020

  1. update goji and arrange function to prevent opening Client without use

    @mzhong1
    mzhong1 committed Apr 20, 2020
    Configuration menu
    Copy the full SHA
    eae5e98 View commit details
    Browse the repository at this point in the history

  2. Sleep the poll_events so it doesn't spin

    @mzhong1
    mzhong1 committed Apr 20, 2020
    Configuration menu
    Copy the full SHA
    4d8be60 View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2020

  1. fix the skip signal handling issue and add sleep to loop

    @mzhong1
    mzhong1 committed Apr 22, 2020
    Configuration menu
    Copy the full SHA
    9bfc05a View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2020

  1. remove extraneous sleep

    @mzhong1
    mzhong1 committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    8413563 View commit details
    Browse the repository at this point in the history

  2. add explicit sleep to macro for polling events

    @mzhong1
    mzhong1 committed Apr 23, 2020
    Configuration menu
    Copy the full SHA
    0d6dc3f View commit details
    Browse the repository at this point in the history

Commits on May 4, 2020

  1. Fix the fd leak

    @mzhong1
    mzhong1 committed May 4, 2020
    Configuration menu
    Copy the full SHA
    a43e444 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2020

  1. Update libatasmart

    @mzhong1
    mzhong1 committed May 7, 2020
    Configuration menu
    Copy the full SHA
    af492ac View commit details
    Browse the repository at this point in the history

  2. update ceph crates

    @mzhong1
    mzhong1 committed May 7, 2020
    Configuration menu
    Copy the full SHA
    73b2ea6 View commit details
    Browse the repository at this point in the history

Commits on May 14, 2020

  1. Fix soft error printing so common errors are changed to debug to prev… 

    …ent exploding log sizes
    @mzhong1
    mzhong1 committed May 14, 2020
    Configuration menu
    Copy the full SHA
    fa685d2 View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2020

  1. fix JIRA crate version

    @mzhong1
    mzhong1 committed Jun 12, 2020
    Configuration menu
    Copy the full SHA
    82e5276 View commit details
    Browse the repository at this point in the history

  2. fix remove operation on successful Add so it removes all operations o… 

    …n the disk instead of the partition only
    @mzhong1
    mzhong1 committed Jun 12, 2020
    Configuration menu
    Copy the full SHA
    7477053 View commit details
    Browse the repository at this point in the history