tenure-tech-selfeval.tex

\chapter{Technological self-evaluation of students}
\label{sec:tech-self-eval}
\lfoot{Technological self-evaluation of students}
%\rfoot{Technological self-evaluation of students}
\normalsize

The following tables report technological self-evaluations of students conducted in the form of online polls. The same questionnaire was voluntarily filled by students twice: once during the first lecture and once during the final lecture. My goal is two-fold (1) to assess the entry level of students related to key technologies covered in a course and (2) to obtain a technical feedback about the technological progress of students during the course. 

\begin{table}[htb]
\footnotesize
\begin{tabular}{@{}p{10cm}p{1cm}p{1cm}p{1cm}p{1cm}p{1cm}p{1cm}@{}}
\toprule
\textbf{ECSE 321 Questions} & 
\textbf{W18} \newline \textbf{Begin} & 
\textbf{W18} \newline \textbf{End} & 
\textbf{W18} \newline \textbf{Incr} & 
\textbf{W19} \newline \textbf{Begin} & 
\textbf{W19} \newline \textbf{End} &
\textbf{W19} \newline \textbf{Incr} \\ \toprule
Participants (\#) & 64 & 43 &  & 80 & 32 &  \\ \midrule
I am familiar with Java as a programming language. & 1.54 & 1.36 & \textbf{0.18} & 1.45 & 1.24 & \textbf{0.21} \\ \midrule

I am familiar with JavaScript as a programming language. & 3.75 & 2.66 & \textbf{1.09} & 3.90 & 2.61 & \textbf{1.29} \\ \midrule

I am familiar with Java Spring or RESTful services. & 4.60 & 2.05 & \textbf{2.55} & 4.68 & 1.83 & \textbf{2.85}  \\ \midrule

I am familiar with Android applications. & 4.39 & 2.54 & \textbf{1.85} & 4.52 & 4.19 & \textbf{0.33}   \\ \midrule

I am familiar with a modern web frontend technology (e.g. Vue.js, React, Angular or similar). & 4.61 & 2.92 & \textbf{1.69} & 4.36 & 2.23 & \textbf{2.13}  \\ \midrule

I am familiar with database technologies (e.g. Hibernate, MySQL or similar). & N/A & N/A & & 
4.18 & 2.41 & \textbf{1.77}  \\ \midrule

I am familiar with UML (the standard modeling language). & 3.80 & 1.75 & \textbf{2.05} & 
2.90 & 1.76 & \textbf{1.14}  \\ \midrule

I am familiar with coding conventions (or other techniques for developing clean source code). & 2.32 & 1.71 & \textbf{0.61} & 2.35 & 1.93 & \textbf{0.42}   \\ \midrule

I am familiar with testing principles or technologies.
& 3.53 & 1.78 & \textbf{1.75} & 3.89 & 2.03 & \textbf{1.86}  \\ \midrule

I am familiar with Git as a version control system.
& 2.51 & 1.29 & \textbf{1.22} & 2.23 & 1.20 & \textbf{2.13}  \\ \midrule

I am familiar with modern automated build or continuous integration technologies (Gradle, Maven, Travis, Jenkins).
& 4.48 & 2.61 & \textbf{1.87} & 4.29 & 2.13 & \textbf{2.16}  \\ %\midrule

\bottomrule
\end{tabular}
\caption{\small Averages of students' technological self-evaluation in ECSE 321 course at the beginning and at the end of course on a Likert scale (1: strongly agree, 2: somewhat agree, 3: neutral, 4: somewhat disagree, 5: strongly disagree ) }
\label{tab:tech-eval-ecse321}
\end{table}

\begin{table}[htb]
\footnotesize
\begin{tabular}{@{}p{14cm}p{1cm}p{1cm}p{1cm}@{}}
\toprule
\textbf{ECSE 429 Questions} & 
\textbf{F18} \newline \textbf{Begin} & 
\textbf{F18} \newline \textbf{End} & 
\textbf{F18} \newline \textbf{Incr} \\ \toprule
Participants (\#) & 85 & 39  \\ \midrule
How familiar are you with JUnit (or other unit testing technology)? & 1.97 & 1.86 & \textbf{0.11} \\ \midrule

How familiar are you with mocking technologies like Mockito (or similar)? & 3.46 & 2.06 & \textbf{1.40} \\ \midrule

How familiar are you with advanced static analysis tools like SonarQube or Infer? & 3.68 & 2.14 & \textbf{1.54} \\ \midrule

How familiar are you with advanced code review tools like Gerrit or ReviewBoard? & 2.76 & 2.22 & \textbf{0.54} \\ \midrule

How familiar are you end-to-end brower testing technologies? & 3.28 & 2.97 & \textbf{0.31}  \\ \midrule

How familiar are you with behavior driven development technologies like Cucumber? &
3.53 & 2.64 & \textbf{0.89}  \\ \midrule

%How familiar are you with model-based testing technologies like GraphWalker? & 3.92 & 3.46 & \textbf{0.46} \\ \midrule

How familiar are you with code-based test generation technologies (like EvoSuite)?
& 3.92 & 2.28 & \textbf{1.64} \\ %\midrule
\bottomrule
\end{tabular}

\caption{\small Averages of students' technological self-evaluation in ECSE 429 course at the beginning and at the end of course on a 1-to-4 Likert scale (1: I used it in industrial project during my work or an internship, 2: I used it in a software engineering project in one or more courses, 3: I learned about it in a tutorial, but have not used it on my own, 4:
I do not know about it)}
\label{tab:tech-eval-ecse429}
\end{table}

\noindent
The results can be interpreted as follows:
\begin{itemize}[leftmargin=0.5cm]
\item \textbf{ECSE 321:} In both editions, at the beginning of the semester, students were only familiar with Java and 
somwhat familiar with coding conventions and UML. By the end of the semester the technological level of students was 
lower than 3.0 for all technologies (except for Android in Winter 2019 which was turned into an optional project deliverable 
due to scheduling constraints) and below 2.0 for 5 out of 10 technologies (i.e. better than at least somewhat familiarity). 
Students had an improvement over 1.0 (out of a 5 point Likert scale) for 8 out of 10/11 technologies. It is very important to 
emphasize that students were totally unfamiliar with 5 out of 10 technologies in the beginning (with a score over 4.0), and 
they could simultaneously make technological progress in most of the technologies. 

\item \textbf{ECSE 429:} Again, major technological improvement has been made by students in most of the key 
technologies covered in the tutorials of the course, especially, in the case, when students had to use a given technology in 
the context of a complex software quality assurance project. Technological progress exceeded 0.5 (on a 1 to 4 scale) for 5 
out of 7 technologies, and students only had difficulties in acquiring end-to-end browser testing technologies.
\end{itemize}