IDT‐Userspace Code Explained

IDT-Userspace consists of several Python scripts:

driver.py

driver.py serves as the main driver of the IDT-Userspace to run training and inferences.

• run_infer: Handles inference. It resets the infer class instance, runs the inference operation for some iterations, writes the experience (saved states, rewards, actions), and finally resets the experience.
• run_train: Handles training. It runs the training and returns the saved checkpoint file.
• driver: This is the main driver that sets the demotion path, initializes Action and State, creates instances of the Infer and Train classes, and then executes the infer and train operations repeatedly for a specified number of iterations.

experience.py

experience.py defines Experience class, which maintains an experience buffer to store state, reward, and action tuples indexed by state.

• save: Saves state, reward, and action tuple to the experience buffer.
• lookup: Retrieves the tuple corresponding to a given state from the experience buffer.
• write: Saves the experience buffer to a file.
• read: Reads the experience buffer from a file.
• reset: Clears the experience buffer.
• get_state, get_reward, get_action: Returns the state, reward, and action, respectively, from a given experience buffer entry.

idt_action.py

Action class manages the actions to change the demotion policy value. The action represents age_thres and sets as Q2, Q3, Q4 of the age statistics distribution.

• action_apply_wait: The atomic wait time between getting the reward after an action is applied. If the reward value is not obtainable although waiting this interval, wait another action_apply_Wait.

• max_action_apply_wait: Skip getting reward if the total wait time is over this value.

• action_num: Total number of possible actions.

• idt_action: Current action value.

• read: Reads the /proc/idt_action file to obtain the current action for each node.

• write: Writes the action for a specific node to the /proc/idt_action file.

• nn_to_action: Converts the output of the policy's neural network to an action.

• apply_action: Applies an action by updating the age_thres value.

• random_action: Selects a random action.

idt_config.py

Configuration for IDT-Userspace.

• num_nodes: Number of memory nodes IDT supports.
• demote_wmark: Watermark when demotion starts.
• apply_wait: Time to wait after action is applied. If the reward is un-obtainable, wait another apply_wait.
• wait_max: Stop waiting for the reward after wait_max.

idt_reward.py

Reward class calculates the reward after an action has been applied. The reward is calculated after waiting for IDT_Config.apply_wait to 'fully' apply the action.

• wait_for_reward: Waits until applied action is "effective". The waiting mechanism are as following:
  1. Wait for action_apply_wait.
  2. Check if there are demoted or promoted pages.
     1. If there is, stop waiting.
     2. If not, Go back to step 1.
  3. If total waiting time is max_apply_wait, stop waiting.
• get_reward: Gets the reward when an action is applied.
  • Reward is log(demoted_pages/promoted_pages).