User Experience
- Introduction
- Users make hiring choices
- Primer to machine learning and the use of algorithms in hiring
- Users build training data for their own hiring algorithm
- Comparison of human and machine choices
- Conclusion
Survival of the Best Fit discusses issues concerning artificial intelligence through an example of machine learning algorithms used during the hiring process. Broadly, the piece is divided into two parts. The first part exposes job hiring as an act of balanced, and often difficult, assessment and decision-making. The second part focuses on machine analogues of human decision-making and the implications of deferring human decisions to algorithms.
Our goal in the introductory part is to underscore the impact hiring decisions have on our lives, and reveal the complexity of reasoning required to make hiring choices that commit to fairness, openness, and equal opportunity for all.
The experience begins by asking users to express their professional interest in a dropdown form. Upon completing the form, a paragraph of text would be revealed, tailored to the users' field of interest. Later, users are asked to evaluate sample CV's that reflect their professional interest.
Figure 1: Professional interest form
At the beginning of the experience, users are asked indicate their field of professional interest in a dropdown form
Figure 2: Introductory text paragraph
Whereas many professionals have only took part in interviews as interviewees, we want to expose the perspective of those who sit at the other side of the table, that is, the interviewers. To be able to make informed assessments of what it means to delegate human choices to algorithms, we want to make users experience the process of choosing the right candidate for hire. To that end, Survival of the Best Fit puts users in a scenario in which they have to accept or reject virtual applicants' CV's based on their own notions about what constitutes a successful candidate.
Figure 3: Hiring simulation
Users are tasked with choosing candidates that get invited to a dummy interview. The simulation is set in an office environment, with a pile of CV's lying around on the office desk. Individual CV's come into view after the user clicks on one from the desk pile. The CV in focus can then be accepted, rejected, or the decision is held off until later time. To accept an applicant, one has to drag the selected CV to a box on the desk on the left side of the screen. To reject an applicant, the CV has to be dragged to the right side of the screen and effectively thrown in the trash bin placed next to the desk. Having to drag CV's around the screen surface between an acceptance box and a trash bin gives rise to an embodied interaction designed to underscore the significance of hiring choices and their underlying reasoning.
After users conduct their own hiring process, the experience shifts focus to hiring choices performed by machine learning algorithms. To establish a shared terminology and conceptual understanding of the issues at stake, an introduction to machine learning and A.I. in lay terms will be included in the experience, with the help of visual aids. In addition, we will motivate the use of hiring algorithms and use it as an example to discuss issues pertaining to machine learning more broadly.
Once a conceptual framework for evaluating A.I. applications is established, users will be asked to build their own training data for a custom hiring algorithm that will later be used to pick successful candidates, just like the users did previously. The training phase will consist of statements about characteristics of successful job applicants. By indicating their dis/agreement with the statements, users will prime the hiring algorithm according to their own values and preferences.
Upon completing the training phase, users will see a loading animation that will represent the deployment and processing of the hiring algorithm. An opaque representation of work such as a loading animation is chosen deliberately to shed light to the so-called black box problem in A.I. applications, wherein the actual algorithmic decision making is inaccessible, often even to developers themselves.
Figure 4: Choosing training data for the hiring algorithm
Figure 5: Deployment of the hiring algorithm
The main issue tackled by Survival of the Best Fit is the risk that machine learning algorithms either propel existing biases of its creators, or distort the intentions of developers even if the latter are genuinely committed to fair outcomes of the applications they are building. We want to highlight this risk by putting in comparison the hiring decisions made by users and the decisions made by the machine learning algorithm. After the hiring algorithm completes the processing of CV's, users can see the overall agreement in decision-making between themselves and the algorithm. Moreover, the experience will also allow for a more granular view of the decisions made by users and the hiring algorithm on same CV's. To remove the usual cloak of secrecy surrounding the inner workings of machine learning algorithms, we will allow users to inspect individual algorithm choices. By clicking on CV's evaluated by the hiring algorithm, a list of parameters taken into account by the algorithm in its decision-making will come into view. Although we deliberately 'hide' the algorithm behind a loading animation earlier in the experience to introduce users with the 'black-box' problem, we ultimately believe that making the behind-the-scenes of A.I. applications more visible and accessible to a lay audience will increase users' engagement with the topic and their willingness to take actions with respect to issues related to A.I. applications in the future.
Figure 6: Comparison of hiring decisions
After comparing their choices to those of the hiring algorithm, users will be given a summary of the issues they explored first-hand throughout the experience. In addition, the concluding part will feature a list of useful resources for further engagement with the topic at hand.