tenure-teaching-statement.tex

\chapter{Teaching Portfolio}
\label{sec:teaching-portfolio}
\lfoot{Teaching Portfolio} 
%\rfoot{Teaching Portfolio} 

%\begin{tcolorbox}[title=Executive Summary of Teaching Activities]
%As a \emph{course instructor}, a main philosophy of mine is that a software engineering related course should \emph{simultaneously teach conceptual foundations and cutting edge software technology}. Without the former, a course becomes empty - without the latter, a course becomes irrelevant. I am dedicated to bring \emph{realistic engineering challenges} frequently with real customers to a course that \emph{facilitate collaboration} within and between student teams while they acquire the new concepts and technologies. I regularly use various  means of \emph{active learning} to make my lectures lively. Furthermore, I always wish to maintain an \emph{inclusive atmosphere} even for a \emph{diverse and international audience}. 
%\vspace{3pt}
%
%As a \emph{supervisor} who actively collaborates with his graduate students with in-depth discussions, my goal is to continuously \emph{strive for excellence} both in engineering tasks as well as in publications while my graduate students are gradually becoming more and more \emph{independent researchers with critical thinking}.  My students typically undertake deep research tasks which go well beyond the state-of-the-art. As a further characteristic, many of our past \emph{results are simultaneously precise and scalable}, and it has been actively \emph{used in an industrial setting}. Active investment in research prototype tools help my students become \emph{excellent software engineers and architects}.
%\end{tcolorbox}


\section{Statement on Teaching and Supervision Philosophy}

\subsection{My philosophy as a course instructor}

My passion for teaching dates back to my first experience as a teaching assistant in 1999 at BME. Since then, it has been 
extremely energizing for me to guide talented and hard-working students on their path towards excellence to succeed as 
software engineering professionals. For me, teaching is more about sharing my experience and expertise rather than the 
actual act of lecturing. For this purpose, I typically combine lectures with games, interactive sessions, individual 
assessment and group work. 

\paragraph{From software engineering students to professional software engineers}
As a course instructor, my main objective is to guide students to become professional software engineers. This is very 
challenging, since by the time a graduated student becomes a software engineering professional, he or she needs to solve 
vaguely defined problems independently, work effectively in teams, in a rapidly changing technical and socio-economical 
context - in addition to the deep knowledge of software engineering concepts and technologies. I find it very important that 
students face these challenges already in some of their university courses instead of dealing only with well-defined 
problems and idealized solutions. In a software engineering course, I frequently position myself as a mentor or a product 
manager: I am ready to help students find solutions to their problems, but they should make serious attempts to find 
solutions first on their own by learning how to use the vast amount of information available on the Internet (e.g. tutorials, 
technical forums, mailing lists). 

\paragraph{Learning by example}
There are typically multiple solutions to any given software engineering problem, and students need to be able to evaluate 
the advantages and disadvantages of a particular solution. For that purpose, they need to be exposed to many examples 
ranging from toy examples demonstrating one aspect of the problem to complex case studies where many problems need to 
be tackled at the same time. Students are exposed to different kinds of hands-on activities such as tutorials, tools, 
assignments, projects, etc. to ensure a balanced development of their software engineering skills. Also in my lectures, I 
postpone abstract definitions during my lectures to the point when the main concepts are sufficiently demonstrated by small 
examples first.

%\paragraph{Interactive classroom for active learning}
%Maintaining the attention of students during lectures can be a challenging task.  %Moreover, 
%I regularly intertwine lecturing with different sort of activities to make the classroom more interactive. I frequently use \emph{best practices of active learning} during my lectures (e.g. bug hunts on ill-formed software, various games and quizzes, small challenges discussed in small groups). I often demonstrate the practical relevance of concepts using real-world examples and industrial case studies sometimes presented by industrial collaborators within invited lectures. 

\paragraph{An all-inclusive classroom}
I am dedicated to make my courses inclusive to students of any diversity group, social or technical background. Even in case of a lively discussion, nobody should ever be humiliated or personally attacked. I actively push students to interact with fellow students with a very different background. I also regularly give advice to problematic teams to carry out groupwork more effectively. To reduce stress caused by high course workload, I do not impose first-time penalties on late submission, and I may adjust the requirements of deliverables when needed. 


\paragraph{Use of modern software technologies}
I strongly believe in \emph{teaching software engineering concepts in the context of modern software technologies} and 
project management frameworks. Although software technologies are evolving rapidly, job advertisements dominantly seek 
for students based on their technological background. When a modern software engineering course needs to use software 
technologies widely used in the industry, motivating students becomes much easier - as they know that what they learn can 
directly appear in their CV. Most of \emph{my courses are complemented with tutorials} on key technologies (e.g. on multi-
tier web applications or software quality assurance techniques) actively co-developed by my graduate students which also 
provides a first teaching experience for them. Online tutorials frequently serve as a reference document for students while 
tackling their complex group projects.

\paragraph{Project-based courses}
Most of my courses are project-based. Students need to invest significant amount of work in completing a team 
project in groups of 3-5 persons. I assign a \emph{complex software engineering challenge ideally proposed by a 
real customer}. As a result, students have to apply the concepts and technologies taught in the course in a realistic setting, 
which results in a much deeper learning experience. Furthermore, students can use their projects as reference work after 
the course in job interviews. 

\subsection{Teaching practices for students' active learning}

There are several techniques I repeatedly use in my lectures to facilitate a positive learning experience for students and 
to make the classroom more inclusive. I provide a brief summary of the key techniques I used in multiple courses. 

%I frequently use \emph{best practices of active learning} during my lectures (e.g. bug hunts on ill-formed software, various games and quizzes, small challenges discussed in small groups). 

\paragraph{Questions by online polling:}
In general, I limit the presentation slots of new material in my lectures to 15 minutes at most, which is followed by some 
other activities. As part of that, I regularly ask questions during my lectures to keep the students involved and make the 
lectures more interactive and lively. However, I also realized that this is not very effective with for students who have a 
rather shy or introverted personality. Therefore, I regularly ask questions in the form of online polls where all students with 
a laptop or a mobile phone can participate and share their views or test the correctness of their answers. 

\begin{wrapfigure}[11]{l}{0.52\textwidth}
  \begin{center}
\includegraphics[width=.5\textwidth]{figures/FlipQuiz}
  \end{center}
%\caption{Jeopardy game in ECSE 321}
%: Introduction to Software Engineering course}
%\label{fig:jeopardy-ecse321}
\end{wrapfigure}

\paragraph{Games as midterm/final exam review:}
I find it very important to regularly review the material covered in the class even (if there are no related quizzes or midterm exams). To make such occasions more interactive, I regularly use games where teams can compete against each other or they need to collaboratively solve particular tasks. A sample screenshot from the Jeopardy game I repeatedly used in the ECSE 321 course is given on the left side. After the game, all questions are made available online for the students as additional review preparation material.

\paragraph{Technological exercises:}
While technologies are traditionally covered only at tutorials and lectures are primarily focusing on teaching novel concepts, concepts and technologies are not always separable in case of modern software. Therefore, I regularly include short 5-10 minute technological exercises during my lectures which is completed by students on their own laptops. For example, teaching modern version control systems (such as Git) or automated builds (like Gradle) is more effective that way in my view. 


%\begin{figure}[htb]
%\centering
%\includegraphics[width=.75\textwidth]{figures/FlipQuiz}
%\caption{Jeopardy game in ECSE 321: Introduction to Software Engineering course}
%\label{fig:jeopardy-ecse321}
%\end{figure}


\subsection{My philosophy as graduate supervisor}

As a supervisor of graduate students, I feel an increased level of responsibility as by the end of several years with collaborative research, I have established very strong ties with my graduate students. When I accept somebody as a PhD student, I normally clarify his or her planned career path after the completion of the PhD (e.g. taking an academic or an industrial path) to ensure a personalized treatment. 

I am very proud to be the main supervisor of 7 successful PhD defenses and the co-supervisor of 4 successful PhD defenses. Three of my former students have (part-time) academic positions at BME while four of them has been co-founders of the successful Hungarian start-up company, IncQuery Labs. Ltd. Others are in leading technological positions in large industrial companies (e.g. ThyssenKruppPresta).

\paragraph{Selection of students}
I normally prefer guiding graduate students from the beginning of their Master's studies to their end of their PhD. I regard their Master's thesis as a probation period (1) to judge if the student is capable of doing research work independently, and (2) to ensure that there is no personal misfit between the graduate student and myself, and (3) to confirm that they are strong team players to collaborate with others. Having strong mathematical and abstraction background and strong software engineering skills is also required prior to starting PhD studies. Having strong writing skills is a plus, but I am quite effective in helping students improve their scientific writing skills. 

\paragraph{Solve challenging problems vs. publications}
I regard publications as bi-products of outstanding research carried out to solve a complex, and highly uncharted software engineering problem. My graduate students do not only publish papers for the sake of their CV, but they use publications as a means to report about their deep scientific results in software engineering. For me, publishing one paper that reports how a problem is actually solved is more valuable than writing three papers which gradually progress towards a solution. I feel strong at motivating students to start exploring highly uncharted research areas, and the lack of significant previous work in a certain area frequently boosts the motivation of talented students. 
When publishing results, I help identify typical errors made by students due to their lack of expertise in scientific writing which helps them achieve early successes. Then I gradually let them act more and more on their own to gradually gain independence in writing by the time they defend their PhD. 

\paragraph{Industrial relevance}
Normally, I prefer that my graduate students find research problems that either directly address industrial needs, or in case of highly uncharted areas, there would be a strong industrial need for the missing solution. Typically, I help them select their first 1-2 challenges, and after that, they can start exploring new research lines on their own. 
My students have regularly participated in long-term industrial collaborations in research intensive tasks. 

\paragraph{Open source projects}
Several PhD projects have been grouped around complex open source projects and demonstrators which have been successfully maintained and continued well beyond the lifetime of a single PhD project. I strongly believe that such open source projects facilitate team work between students and provide a greater visibility of their research for the general public. In fact, our open source projects have been repeatedly used by several research groups as a baseline for comparing their results. 

\paragraph{Public benchmarks for open science}
Since 2005, we have been developing several public benchmarks to allow an in-depth comparison of different software tools to a given research challenge. This helps graduate students learn what others did in a particular field by trying out and experimenting with their tools. Moreover, it also helps them find gaps in the state of the art, and gain more confidence in the novelty of their own algorithms. Finally, it also promotes the open science initiative to ensure that research results can be reproduced outside their original context.

%guiding 




%\section{Teaching Responsibilities}
%Starting as a teaching assistant in 1999, I have been involved in teaching at various levels as a full-time staff member at the Budapest University of Technology and Economics (BME, lecturing dominantly in Hungarian) between 2003 and 2016 and since then at McGill University (Canada, lecturing in English). The list of all courses I taught are listed in my CV (page \ref{xxx}).



\section{Teaching Responsibilities}
\label{sec:teaching-responsibilities}

\subsection{Courses taught at McGill}
Since 2016, I taught five editions of three different courses at McGill University. This includes two undergraduate courses: ECSE 321: Introduction to Software Engineering (taught 3x), ECSE 429: Software Validation (1x), and one graduate-level course ECSE 681: Critical Systems (1x, under temporary code). In 2018, I got teaching relief to prepare for the software engineering co-op program, thus I only taught my graduate course once. 

As the most important result for the undergraduate courses I taught (ECSE 321 and ECSE 429), I successfully integrated various cutting-edge software technologies and involved real customers in team projects. As a result, \emph{several students succeeded in getting summer industrial software engineering internships based solely on course material}. 
%The graduate course, which teaches concepts and standards of model-based systems engineering, has a similar success story. 

\paragraph{ECSE 321: Introduction to Software Engineering (Undergraduate, taught three times)}
I took over this undergraduate course from Prof. Mussbacher and I taught it in alternation with Prof. McIntosh. They successfully modernized many concepts taught in the course. The course covers the entire lifecycle of a modern software project including requirements engineering, domain modeling, test-driven development, continuous integration and software configuration management in the context of a group project and individual assignments. The 75-100 students of the course must have taken 1-2 previous programming courses prior to enrolling to this course. 

\paragraph{ECSE 429: Software Validation (Undergraduate, taught once)}

I gave this undergraduate course in Fall 2018 for over 120 students - most of who were taking this course just before their graduation. The course covers various software quality assurance techniques including testing, static analysis and formal verification. The students need to complete a complex software testing project (in teams) as well as individual testing assignments and three quizzes.

\paragraph{ECSE xxx/681: Critical Systems (Graduate, taught once)}

I proposed and offered the first edition of this course (under a temporary course code) in Winter 2018 for 18 graduate students. As a research component for this first edition, students had to assemble a survey presentation of a research topic by categorizing over 15 papers. The official approval of the course is expected to be completed in Fall 2019. 

\paragraph{Teaching at international graduate schools}
Since joining McGill, I have repeatedly been invited to give a total of 3 lectures and 2 tutorials at the international Summer School on Domain Specific Modelling Theory and Practice (\href{http://msdl.cs.mcgill.ca/conferences/dsm-tp-2017}{DSM-TP}). The summer school provides foundations and best practices of domain modeling, domain-specific languages, model transformation, model verification, and industrial use cases of model-based systems engineering.
In my entire career, I gave a total 10 lectures at 6 different international summer schools dominantly attended by graduate students.
 %In the 2016 edition (in Geneva, Switzerland), I gave a lecture on \emph{Incremental Queries and Transformations in Smart Cyber-Physical Systems}, while in the 2017 edition (in Montreal), I gave a lecture and a related tutorial on \emph{Incremental Queries and Reactive Transformations} and also a pair of lecture and tutorial on \emph{Formal Verification and Validation of Domain-specific Languages}. Together with these lectures, I gave a total 10 lectures at 6 different international summer schools dominantly attended by graduate students in my entire career.

\subsection{Summary of past teaching activities at BME}

Before joining McGill University in 2016, I already had a long track record of teaching and course development since my early PhD studies. Altogether, I developed or co-developed 7 courses (3 undergraduate, 3 MSc-level, 1 PhD-level) and taught  over over 25 editions of those courses. Below, I provide a brief executive summary only about my most significant activities at BME. 

\paragraph{Undergraduate-level teaching}
I contributed to the development of a course on \emph{Formal methods} (with 300+ students each year) since the beginning of the course in 2001 and I gave  guest lectures between 2004-2006. I also contributed three chapters in the textbook of the course (in Hungarian).  
%Moreover, I was a key contributor and strategic founder of an \emph{undergraduate curriculum on Systems Engineering}.  
I was major promoter and initiator of the development of an \emph{autograding software tool} (implemented by my graduate students) and subsequently used in the \emph{Systems Modeling} undergraduate course for over 5 years by now with 600+ students where each student could receive an individual technical assignment. 
In a project manager role, I participated in introducing virtual labs over educational cloud (using Apache VCL) at our faculty for the first time in Hungary which received a \href{https://inf.mit.bme.hu/en/news/2014/02/tempus-stem-call-apache-vcl-based-labs-won-prize}{TEMPUS STEM Award} in 2014. 

\paragraph{Graduate-level teaching}
At BME, I developed a course on \emph{UML-based modeling and analysis} which ran between 2003 and 2009 for 70-80 graduate (MSc) students. \emph{Several teaching assistants of the course were former students} of the course who voluntarily came back from industry to help the course, which was a highly unusual in Hungary. Between 2009-2016, I ran a course on “Model-driven software/systems development” (twice in English) with 15-20 students each year. I also actively contributed to start the first ever Hungarian university course on \emph{Eclipse based technologies} in 2005. I was also involved in several industrial trainings on these topics. I have been running a PhD seminar on the \emph{Foundations of Model-Driven Engineering} between 2004 and 2014 (with 5-8 students each year). In 2009, I was \emph{invited as a panelist at the Educators' Symposium} of the MoDELS conference, the main scientific venue of my field. 


\subsection{Course development activities at McGill}

%I provide an overview of major course development activities carried out in the respective courses. 

\subsubsection{ECSE 321: Introduction to Software Engineering}

Since I only offered the ECSE 321 course more than once at McGill prior to submitting my tenure dossier, my primary focus will be on that course where the impact of my course development activities have been most apparent - and where I had chances to reflect to students' feedback I received in course evaluations.  

%Below I highlight major developments of the course where my own initiative was predominant to improve the industrial relevance of the course, which received very positive feedback from McGill undergraduates. 


\paragraph{Modernization of software technologies and team project:}
It was apparent from student feedback I received in the Winter 2017 edition of the course that the software technologies are outdated, and the students do not feel that they learn modern material despite the modern concepts. Moreover, students had to develop a smaller system to three different platforms, thus they were not imposed to face integration problems which are characteristic to most modern software.
Therefore, since Winter 2018, I took the initiative to modernize the entire underlying software stack to include (1) modern web technologies (like HTML5, JavaScript or Vue.js), (2) a modern backend technology (JavaSpring, RESTful API), (3) a persistance layer with database technology. I also developed lectures to teach related concepts (e.g. the standard object-relational mapping, asynchronous method calls).

\paragraph{Redevelopment of tutorials: contents and technology:}
As a consequence of the above initiative, the \emph{sample project as well as the technical contents of the tutorials of the course had to be redeveloped from scratch} which I carried out together with Márton Búr, my PhD student (with initial help from my graduate students at BME, G. Szárnyas and O. Semeráth). This was a major investment since the total length of the PDF version of the material was 75-100 pages. These course development activities have been supported by two successful applications to the McGill EUSF fund. 
To improve the maintainability of tutorial materials, I promoted to host those tutorial materials on GitHub and continuously compile changes into a new version of the document available both in HTML and PDF formats (see e.g. 
\href{https://mcgill-ecse321-winter2019.github.io/EventRegistration-Tutorials/}{ECSE321-W2019}, 
\href{https://mcgill-ecse321-winter2018.github.io/EventRegistration-Tutorials/}{ECSE321-W2018}, 
\href{https://ecse321-winter2017-mcgill.github.io/EventRegistration-Tutorials/}{ECSE321-W2017}). As such, even the tutorials of the course actually used the same software technologies (Gradle, Github, TravisCI) taught by the course. 

%\paragraph{Basics of system architecture}

%While Montreal is a major aerospace hub, the existing courses of the software engineering curriculum failed to offer specific insights in engineering critical systems. Therefore, I developed lectures on \emph{platform-based design in systems architecture} as well as the \emph{foundations of fault-tolerant systems}.

%used within the course including the following specific major improvements:
%\begin{enumerate}
%\item The team project should \emph{develop an integrated multi-layer web application} with separate backend (i.e. services and database) and different graphical user interface frontends (mobile, web). 
%\item I changed the Java-dominated technological stack to include \emph{heterogeneous technologies} including a modern web technologies (like HTML5, JavaScript or Vue.js), a modern backend technology (JavaSpring, RESTful API). I also included respective programming concepts such as asynchronous method calls or promises. 
%\item I introduced a proper underlying relational database technology on top of an object persistence layer using the standard Java Persistence API. I developed respective lecture material to cover the basic concepts of relational databases (as students take the Database course later) and the standard \emph{object-relational mapping}.  
%\item
% While Montreal is a major aerospace hub, the existing courses of the software engineering curriculum failed to offer specific insights in engineering critical systems. Therefore, I developed lectures on \emph{platform-based design in systems architecture} as well as the \emph{foundations of fault-tolerant systems}.
%\end{enumerate}


\paragraph{Involving real customers in team projects:}

To better motivate students, I put extra emphasis on providing real challenges for the complex software engineering project carried out by teams. Therefore, I selected project themes where McGill employees could serve as real customers (including Prof. Andrew Gonzalez, Prof. Kevin Manaugh and Prof. David Wachsmuth in Winter 2018 and Ms. Lorraine Donald in Winter 2019). Those customers participated in requirements engineering sessions where students could ask questions to clarify the scope of the project. Moreover, these customers also attended (and graded) the final student presentations and the final software artifact.

%\begin{itemize}[leftmargin=0.5cm]
%\item In Winter 2018, student teams had to develop a software application to manage and supervise the trees in Montreal and calculate various forecasts for evaluating sustainability attributes. Many details of the project were proposed by Prof. Andrew Gonzalez, Prof. Kevin Manaugh and Prof. David Wachsmuth who offered their continuous involvement.
%\item In Winter 2019, student teams had to develop an application to help manage co-op programs (i.e. curricula with compulsory industrial internship) at McGill University. In this case, Ms. Lorraine Donald (Industrial Liason Associate and future administrative coordinator of the upcoming software engineering co-op program) offered continuous help during the project. Moreover, final presentations of student teams were also attended by various employees at the Faculty of Engineering.
%\end{itemize}



\paragraph{Auto-grader software for individual software technology assignment:}
To assure that all students of the course successfully learned both the concepts and the software technologies, a programming assignment has already been an integral part of the course where students had to develop a small-scale web application using the same technologies as in the large-scale group project. However, the evaluation of these individual assignments has been very laborious and somewhat subjective and error-prone. 
%This assignment had to be carried out by the students either in pairs (in 2017 and 2018) or individually (in 2019). 

%I successfully led the development of an auto-grader software framework which can help automatically evaluate software-related assignments of students by using test cases and exploiting modern cloud-based software technologies.

Thus, I took the initiative to develop an auto-grader software to enable the automated evaluation of student submissions by running test cases and exploiting modern cloud-based software technologies (like GitHub for version control, and TravisCI for continuous integration). Each student receives a unique combination of software features that (s)he needs to develop. Certain test cases are made available to students as a reference while developing their solution while others are only included for grading. 

%80\% of the test cases are made publicly available to students so that they can use them as a reference while developing their solution. The remaining 20\% of the test cases were only included in the grading process. 

%The key concepts of the auto-grader framework are the following:

%\begin{itemize}[leftmargin=0.5cm]
%\item Each student needs to develop a unique combination of software features by extending a baseline web application made available to them as a starting point. All development needs to be done in an individual (private) GitHub repository of the student (identified by their McGill IDs). The individual specification of the assignment is also generated automatically for each student. 
%\item The auto-grader evaluates student submission by running a dedicated test suite. The Instructors need to develop the superset of all test cases from which the auto-grader selects the ones required for the features addressed by the specific student. 
%\item The 80\% of the test cases are made publicly available to students so that they can use them as a reference while developing their solution. The remaining 20\% of the test cases were only included in the official grading process. 
%\item The actual grading is carried out as part of a continuous integration process, which automatically builds the student project, then executes the relevant set of test cases, and finally reports the evaluation in a separate GitHub branch.
%\end{itemize}

\paragraph{Undergraduate students as project mentors:}
I started to systematically involved excellent undergraduate students in the next edition of the ECSE 321 course using the Tomlinson Engagement Award for Mentoring (TEAM). A total of six TEAM mentors helped teams (in 2018 and 2019) to overcome the difficulties of integrating various software technologies. Moreover, they also helped develop the auto-grader.  %While the TEAM award is on a nomination basis, several undergraduates have indicated to volunteer for the next year, which clearly demonstrates the positive impact of this change.

\paragraph{Impact:}
Since the theme of the software engineering team project was different in every year, students could decide if they wish to open their GitHub repositories that stored the source code of their projects after the completion of the course. Many teams opted for an open repository and \emph{students started to use their ECSE 321 project as a reference work during interviews of summer internship}. This is clear indication of the industrial usefulness and practical effectiveness of the course as a result of the changes above. 
The auto-grader has been successfully used in Winter 2019, but several colleagues at my department have expressed there interest in potentially adapting the framework to their course. 


\subsubsection{ECSE 429: Software Validation}

%I gave this undergraduate course in Fall 2018 for over 120 students - most of who were taking this course just before their graduation. The course covers various techniques of software quality assurance including testing, static analysis and formal verification. The students of the course need to complete a complex software testing project (in teams) as well as individual testing assignments and three quizzes.

Since the lecture material of the course has been recently redeveloped (by Prof. Mussbacher), I built upon and reused significant amount of existing course material. However, the software technologies used within the course were outdated, thus my primary goal was to increase the industrial impact of the course. % by the following initiatives:

Together with teaching assistants, Márton Búr and Mathieu Boucher, we \emph{redeveloped most of the tutorials of the course from scratch}. I adapted the use of GitHub for hosting the \href{https://mcgill-ecse429-fall2018.github.io/Tutorial-Lecture-Notes/}{material of the tutorials}. I \emph{developed course material on code reviews and static analysis techniques} for lectures and introduced an extra deliverable for the team project. To further increase practical relevance, the introductory materials  have been aligned with contents of the foundation level certification offered by International Software Testing Qualifications Board (ISTQB). Furthermore, \emph{student teams had to test a real open source Android application} in the group project.  Finally, I \emph{changed the former closed-book in-class quizzes into open-book, online quizzes} evaluated semi-automatically using existing support in MyCourses.

%\begin{enumerate}[leftmargin=0.5cm]
%\item The \emph{tutorials of the course have been redesigned almost from scratch} (with contributions from the two teaching assistants, Márton Búr and Mathieu Boucher). New material included the introduction of code reviews, continuous integration, modern static analysis tools (SonarQube, Infer), automated test case generators (EvoSuite), various frameworks for integration testing as well as end-to-end testing Android applications, model-based testing tools. I adapted the use of GitHub for hosting the \href{https://mcgill-ecse429-fall2018.github.io/Tutorial-Lecture-Notes/}{material of the tutorials}, which allows that corrections can be immediately published to the live material (without extra manual document upload). 
%
%\item As being of utmost importance in modern software quality assurance, I \emph{developed course material related to code reviews and and static analysis techniques} as part of the lectures and introduced an extra deliverable for the team project. To further increase practical relevance, the introductory materials of software quality assurance and testing has been aligned with contents and recommendations of the International Software Testing Qualifications Board (ISTQB) for foundation level certification.
%
%\item Within the complex team project of the course, \emph{student teams had to test a real open source Android application} (Blokish game). Although the suitability of this very project was criticized by students, they were simultaneously enforced to understand and handle the imperfections of real software projects, which is frequently the case in real software projects. 
%
%\item To reduce the laborious work of grading the three individual quizzes with an increased number of enrolled students, I \emph{changed the former closed-book in-class quizzes into an open-book, online quizzes} evaluated semi-automatically using existing support in MyCourses. Students had to complete each quiz in 75 minutes within a 6-hour time-frame, which gave them extra flexibility to avoid clashes of midterm exams with other courses, and in general, it reduced the perceived stress level. 
%\end{enumerate}

\subsubsection{ECSE xxx/681: Critical Systems (Graduate)}

I proposed and \emph{developed a new graduate course on Critical Systems} to overview the main challenges in the design and assurance of critical software-intensive cyber-physical systems (CPSs). The first part of the course is dedicated to the core concepts, languages and techniques of model-based systems engineering used for designing such systems by covering concepts like safety cases, traceability, viewpoints of system architecture, design space exploration, etc. In this first part, students carry out a complex design project using standard modeling languages (like SysML, Capella) and simulation tools in the context of a cyber-physical system. This first part is complemented with in-class tutorials and modeling workshops to help students effectively use complex industrial systems engineering tools.

The second part of the course is dedicated to the assurance of critical CPSs to guarantee safe behavior of such systems by discussing a wide range of design-time and run-time assurance techniques. In this part, students familiarize themselves with research results used for the assurance of such systems including various approaches of verification and validation (V\&V). The end of the course is dedicated to recent research challenges for the safety assurance of autonomous systems driven by machine learning and other AI techniques. These research-related contents are close to my own research area, thus student presentations of various research topics generate lively and deep discussions. 




%\paragraph{Undergraduate-level courses.}

%At \emph{McGill University}, I was the instructor of a course on \emph{Introduction to Software Engineering} for three times with 75-95 students in 2017-2019 and for a very diverse and international audience. In all three editions, I received an average score between 4.2-4.5 (out of 5.0) for all course evaluation questions related to me as instructor, which was significantly over the department average (by an average of 0.6). In 2018, I significantly modernized the technological background of the course – which now teaches modern software engineering principles and architectures in the context of modern software technologies. Despite the heavy workload, the vast majority of students loved the course and gave very positive feedback, such as “\emph{It has been the most interesting and educational course I have taken. So much effort is put in this course from the detailed \href{https://mcgill-ecse321-winter2019.github.io/EventRegistration-Tutorials/}{Hands-On tutorial} document, to the project requirements supported by ‘real’ clients, and finally, the \href{https://flipquiz.com/flashcards/82085/}{interesting games}!}”, “\emph{Professor Varro is by far one of the most helpful, dedicated professors at McGill}”. At McGill, I voluntarily took several trainings offered by Teaching and Learning Services at McGill on how to increase student participation by \emph{active learning} (e.g. frequent online polls during lectures). In Fall 2018, I also taught a course on \emph{Software validation} for over 120 students, but future editions of this course will need significant further modernization.

%\paragraph{Graduate-level courses.}
%At McGill University, I offer a graduate course on 
%\emph{model-based engineering for cyber-physical systems} 
%\emph{critical systems} ()
%which teaches SysML and related standards in the context of Internet-of-Things and safety-critical CPS.


\section{Talent Care and Student Supervision}



\subsection{Supervision activities as McGill professor}

\paragraph{Supervising PhD students}
In 2015, I simultaneously applied to the professor position at McGill University in Canada and to the prestigious 
\href{https://mta.hu/lendulet}{Lendület research chair position} funded by the Hungarian Academy of Sciences. 
As I succeeded in both cases, I had the unique opportunity to lead a Trans-Atlantic research group and supervise 5 PhD 
students and co-supervise another 3 PhD students.  In addition, I also served as a scientific mentor of two young assistant 
professors at BME who were involved in the Lendület project.

Given that those PhD students were at different stages in their career and McGill's policy disallows a PhD student to start 
later, Márton Búr ended up restarting as a PhD student at McGill in Fall 2017. Other PhD students (namely, Gábor 
Szárnyas, Oszkár Semeráth, Csaba Debreceni, Ákos Hajdu, Vince Molnár, Kristóf Marussy) became graduate research 
trainees at McGill University for several months while continuing their PhD studies in Hungary. 
%(András Nagy gave up his PhD studies in Winter 2017 due to family reasons). 
All of these PhD students have been working along a research program 
related to smart \& safe CPSs initiated and led by me.
% (and co-funded both in Canada and Hungary). 

I have continuously been involved in the supervision of those students where I acted as the main supervisor (namely, 
Márton, Gábor, Oszkár and Csaba) on a weekly or sometimes on a daily basis when writing important publications. While 
Márton has been around me throughout the entire academic year when I had teaching duties, typically two PhD students 
visited me at a time (on a round-robin basis) as graduate research trainees between September and June. I supervised all the others who were not actually at McGill via 
teleconferencing. Those students I co-supervised (Ákos, Vince and Kristóf) visited me at McGill University as graduate 
research trainees once. During summer-time, I spent more time in Hungary to have in-person discussions with PhD students of the Lendület project.
% (and did teleconferencing with Márton in the meanwhile). 

%Detailed evidence of my successful supervision will be provided below, but most importantly, Oszkár and Gábor defended their PhD thesis in June 2019, Csaba's thesis is already under review and his public defense is expected in Fall 2019, while Ákos and Vince are PhD candidates (to defend within a year). Moreover, since Fall 2017, Márton Búr has been making substantial progress in his PhD studies at McGill University thus his research seminar is scheduled in September 2019.


\paragraph{Supervising MEng students}
Starting from Winter 2018 (and after severe delays caused by visa problems), I have been supervising MEng students at McGill University. By now, I have a very international and diverse group with of five MEng students, namely, Maruthi Rangappa (from India, started in W2018), Jasvir Kaur Dhaliwal (from India, started in W2018), Faizan Khan (from Pakistan, started in F2018), Aren Babikian (from Quebec, started in W2019), Sebastian Pilarski (from Canada, started in W2019). I collaborate with them by weekly meetings and extra teleconferences when my students have some roadblocks. With Aren and Sebastian, I have started collaboration while they were still undergraduate students at McGill, and they have already committed to continue for the PhD studies.  

\paragraph{Supervising undergraduate students}
%I have also been active in supervising undergraduate students at McGill University in two different ways.  
Since 2017, I have been supervising a total of six projects as part of Summer Undergraduate Research in Engineering (SURE) program involving a total of 9 talented undergraduate students at McGill. One of those projects ended up with a joint publication \cite{icse2019-tool} where undergraduate students were co-authors. Furthermore, I have been supervising a total of 9 Design Projects involving a total of 29 undergraduate students. 

\subsection{Summary of past supervision activities at BME}


\paragraph{Supervising PhD and MSc theses.}
%Starting from my early postdoctoral years, 
I have continuously been very active in talent care by supervising MSc and PhD students. I have been the \emph{main scientific advisor of 4 PhD completed students} (István Ráth, Ákos Horváth, Gábor Bergmann and Ábel Ujhelyi) and co-supervised 3 more completed PhD students. I also supervised 20+ MSc theses since 2001. Four papers written with my PhD students as first authors received a best paper award at major international conferences of my field. Moreover, one PhD student was successfully competing at the STAF 2015 Doctoral Symposium winning the 1st prize. 


\paragraph{Supervising early research work.} 
23 scientific reports written by MSc students under my tutoring participated successfully in a national two-phase (faculty and national-level) \href{http://www.otdt.hu/hu/cms/otdk/orszagos-tudomanyos-diakkori-konferencia/}{Scientific Students’ Associations Conference} organized for students who carry out early research work in Hungary. Early research results achieved by MSc students provided a basis for over 25 scientific publications published at international conferences. In 2009, I was selected as a \href{http://www.otdt.hu/page/kituntetesek/mak2009.php}{Distinguished Tutor} by the National Scientific Students Council (OTDT), a national prize requiring 10 years of successful tutoring, awarded bi-annually to only 3 tutors in the field of computer science. This way, I started to focus on talent care very early, in fact, as a 1st year PhD student. I was the \emph{first ever recipient} of the \href{https://otdk2017.mik.uni-pannon.hu/index.php/eredmenyek}{Csanád Imreh Award}.
%, commemorating the Hungarian researcher who died %tragically at a young age. 

\paragraph{Careers of former students.} Together with 4 of my former PhD students (István Ráth, Ákos Horváth, Gábor Bergmann and Ábel Hegedüs), we \emph{founded a start-up company, IncQuery Labs Ltd.} in 2013 to provide industrial exploitation of our research results and our expertise in model-driven engineering. András Balogh (whom I co-supervised) is now Chief Technology Officer at ThyssenKrupp Presta Hungary. Four of my former MSc students worked for Google, several of them went for \emph{large international companies} Nokia, Ericsson, Lufthansa Systems, National Instruments or small \emph{innovative startups}.


\section{Evidence of Teaching Effectiveness at McGill}
\label{sec:evid-teach-effect}

\subsection{Course evaluation results}
\label{sec:course-eval-results}

\autoref{tab:course-eval} shows the course evaluation results for all the 14 regular questions for the five course offerings I have taught at McGill since Fall 2016. The four main questions (Q1 to Q4) are highlighted in bold, whereas additional instructor-specific questions (Q3-Q8) are printed in italics. The department means (DM) are shown in parentheses in the table to enable certain comparison. 

\begin{table}[htb]
\footnotesize
\begin{tabular}{@{}p{9cm}p{1.1cm}p{1.1cm}p{1.1cm}p{1.1cm}p{1.1cm}@{}}
\toprule
\textbf{Question} & \textbf{ECSE321} \newline \textbf{W17} \newline \textbf{(DM)} & 
\textbf{ECSE321} \newline \textbf{W18} \newline \textbf{(DM)} & 
\textbf{ECSE321} \newline \textbf{W19} \newline \textbf{(DM)} & 
\textbf{ECSE429} \newline \textbf{F18} \newline \textbf{(DM)} & 
\textbf{ECSE681} \newline \textbf{W18} \newline \textbf{(DM)} \\ \toprule
Response rate (\%) & 44 & 47 & 44 & 34 & 67 \\ \midrule
\textbf{Q1: Overall, this is an excellent course.} & \textbf{3.8} \newline (3.8) & \textbf{4.5} \newline (3.8) & \textbf{4.4} \newline (3.8) & \textbf{3.6} \newline (3.8) & \textbf{3.9} \newline (3.8)  \\ \midrule

\textbf{Q2: Overall, I learned a great deal from this course.} & \textbf{4.2} \newline (4.0) & \textbf{4.5} \newline (3.9) & \textbf{4.4} \newline (4.0) & \textbf{3.9} \newline (4.0) & \textbf{3.7} \newline (3.9)  \\ \midrule

\textbf{Q3: Overall, this instructor is an excellent teacher.} & \textbf{4.3} \newline (3.7) & \textbf{4.4} \newline (3.7)  & 
\textbf{4.5} \newline (3.8)  & \textbf{4.0} \newline (3.9) & \textbf{4.6} \newline (3.7)  \\ \midrule

\textbf{Q4: Overall, I learned a great deal from this instructor.} & \textbf{4.2} \newline (3.7) & \textbf{4.3} \newline (3.7)  & 
\textbf{4.3} \newline (3.8) & \textbf{4.0} \newline (3.9) & \textbf{4.3} \newline (3.7) \\ \midrule

\emph{Q5. The instructor was well organised in class and presented the material clearly.} & 4.1 \newline (3.7)  & 4.2 \newline (3.7)  & 4.4 \newline (3.9)  & 4.0 \newline (4.0) & 3.5 \newline (3.7) \\ \midrule

\emph{Q6. The instructor used effective teaching methods.} & 4.2 \newline (3.6)  & 4.2 \newline (3.7) & 
4.2 \newline (3.7)  & 3.7 \newline (3.7)  & 3.8 \newline (3.6) \\ \midrule

\emph{Q7. The instructor was responsive to students’ questions and concerns, given the class size.} & 4.5 \newline (4.2)  & 4.7 \newline (4.1) & 4.5 \newline (4.2)  & 4.3 \newline (4.2) & 4.9 \newline (4.1)  \\ \midrule

\emph{Q8. The instructor fostered an environment of mutual respect and engagement in learning.} & 4.7 \newline (4.2)  & 4.8 \newline (4.1) & 4.7 \newline (4.1)  & 4.3 \newline (4.2) & 5.0 \newline (4.1)  \\ \midrule

Q9. The course materials contributed to learning the subject matter. & 4.2 \newline (4.0)  & 4.5 \newline (4.0) & 
4.2 \newline (4.0)   & 4.0 \newline (4.1) & 3.3 \newline (4.0)  \\ \midrule

Q10. The course activities (inside and outside the classroom) engaged me actively in my learning process. & 4.4 \newline (3.8)  & 4.6 \newline (3.8) & 4.3 \newline (3.8)  & 3.8 \newline (3.9) & 4.2 \newline (3.8)  \\ \midrule

Q11. The evaluation methods used in this course were fair and appropriate. & 4.2 \newline (3.7)  & 4.3 \newline (3.8) & 
4.2 \newline (3.8)  & 3.8 \newline (3.8)  & 4.2 \newline (3.8) \\ \midrule

Q12. I was provided with useful feedback on my progress in the course. & 3.8 \newline (3.7)  & 4.4 \newline (3.6)  & 
4.4 \newline (3.8)  & 3.5 \newline (3.7)  & 4.1 \newline (3.6) \\ \midrule

Q13. The course workload was appropriate, given the credit weight and the scheduled activity hours. & 3.5 \newline (3.7)  & 3.5 \newline (3.8)  & 3.5 \newline (3.8)  & 3.3 \newline (3.7)  & 4.2 \newline (3.7) \\ \midrule

Q14. As a result of this course, I have a greater appreciation for the relevance of this topic to my chosen profession. & 4.3 \newline (3.9)  & 4.4 \newline (3.8)  & 4.4 \newline (3.8)  & 3.6 \newline (3.9)  & 3.9 \newline (3.9)  \\ \midrule

\bottomrule
\end{tabular}
\caption{Course evaluations}
\label{tab:course-eval}
\end{table}

\paragraph{General feedback}
The course evaluations of all courses I taught at McGill are consistently very high. Over 82\% of the results (58 out of 70 cases) exceed the department mean (DM), while in 44\% of the cases (31 out of 70 cases), the results are significantly higher than the DM (by at least 0.5 points). 
The results for the four core questions (Q1-Q4) are over 4.0 in 75\% of the cases (15 out of 20). Morever, all of the results 
were over 4.3 for repeated editions of the same course (ECSE 321), which shows that I succeeded in making necessary 
adaptations to subsequent editions of the course once I grasp the general background knowledge of students after teaching 
the course for the first time. 

My instructor-specific evaluations (Q3-Q8) are excellent: almost all cases (93.3\%, 28 out of 30) exceed the DM; while, 
63.3\% (19 out of 30 cases) exceeds the DM by a large margin (over 0.5 points).  In general, students seem to be very 
supportive of my teaching style as shown by the fact that 90\% (27 out of 30) of my instructor scores are over 4.0.

\subsection{Technological self-evaluation of students}
%\label{sec:tech-self-eval}
As most software engineering jobs refer to technologies rather than concepts, the technologies learned by students in software engineering courses is a very important aspect despite the rapidly changing technological environment.

Therefore, I have voluntarily collected technological self-evaluations of students since Winter 2018 in the form of online polls. The same questionnaire was filled by students twice: once during the first lecture and once during the final lecture. My goal is two-fold: to assess the entry level of students related to key technologies covered in a course and to obtain a technical feedback about the technological progress of students during the course. 
The results of the technological self-evaluation of students for the two undergraduate courses are presented in \autoref{tab:tech-eval-ecse321} and \autoref{tab:tech-eval-ecse429} in \autoref{sec:tech-self-eval}.

%The measurement of technological progress of students in software engineering courses is very important from a practical perspective: the vast majority of technical descriptions in software engineering jobs refer to technologies rather than concepts. As such, students must simultaneously acquire concepts and technologies as part of their undergraduate training in a rapidly changing technological environment. 

%\paragraph{Evaluation of students' technological progress}

As a summary, the students have made a remarkable technological progress related to the majority of technologies in both 
courses, which is a substantial pedagogical achievement for all instructors and teaching assistants involved in the course. 
This success can be attributed to (1) the extensively redeveloped tutorials and (2) the fact that they had to apply 
technologies in a complex software engineering team project (which naturally caused an increased student workload).  As 
such, it faithfully reflects the effectiveness of my course development activities. 

\subsection{Interpretation of results and action plans for future}

\paragraph{Course-specific feedback: ECSE 321}
Based on the textual feedback I received in Winter 2017, I decided to modernize the software technologies used in the complex team project. As result, all scores for all questions in the Winter 2018 and Winter 2019 editions of the course reached at least 4.2 - and improved with respect to the Winter 2017 edition in all cases (except one). 
A complete set of textual comments received from students of the ECSE 321 course (in Winter 2018 after the modernization) is reproduced in \autoref{sec:course-eval-results}. A few representative excerpts are given below:

\begin{itemize}[leftmargin=0.5cm]
\item Feedback on technological modernization: \emph{ "Love the changes to this course, you actually learn about technologies that are being used in the industry currently which makes this course so valuable." "I am really happy that Varro updated the project of this course to reflect newer frameworks and marketable skills. It gives us, the students, a taste of many of the qualifications asked for in internship applications, such as popular frameworks, testing, automation, and development techniques. "}
%\item Q2: \emph{The most I've ever learned from a single course! }
\item My passion for teaching: \emph{The professors seems to be very passionate about the material which makes his classes very engaging. He encourages asking questions and makes sure we absorbed all the challenging material. He puts a lot of effort ensuring that we got all the help we needed for the project and the assignment.}
\item Active learning practices: \emph{"He encouraged participation during the class, which made us understand and absorb the material better. He used revision games to make the material entertaining. He offered bonus marks for outstanding performance, which was very motivational!" "Use of clickers helps keep students engaged, even if they don't know the answers."}
\item My willingness to help: \emph{"The professor is very supportive and answers all our questions. He stays answering the students after each lecture, during the office hours, through emails and on myCourses discussion board where there's more than 200 posts. Most of the students are form other majors (taking a minor in Software), and they took at most 2 programming courses. So, they asked a lot of questions and required a lot of help. The professor and the TAs were always there for us. And the result: by the end of the year, all groups have created a complicated well designed web page and Android app!!" }
%\item Q8: \emph{ }
\item Re-developed tutorials: \emph{"The first tutorial (Hands-On tutorial) was the most sufficient resource that we used throughout the final project. It was very detailed with screenshots and steps, as well as sources of error. It was very easy to follow especially with the buttons of the sections on the right. "}
%\item Q10: Industrial relevance: \emph{ }
\end{itemize}

The ECSE 321 course is regarded by undergraduate students as the flagship course of the software engineering curriculum which helps students find internships at high-tech software companies thanks to my initiative on technological modernization. Moreover, the complex team software projects proposed by real customers serve as a first reference work for many students.
%My initiative included to introduce new software technologies, modernize tutorials and bring real customers of the team project,  

%Altogether, my initiative evolve a very good course into an excellent one, which is frequently 


%Altogether, I was a key initiator to evolve a very good course into an excellent one, which is frequently regarded by undergraduate students as the flagship course of the software engineering curriculum which helps student find internships at high-tech software companies. After introducing new software technologies, modernizing the tutorials and bringing real customers of the team project, the complex team software projects serve as a first reference work for many students. 

\paragraph{Course-specific feedback: ECSE 429}
The course evaluations of ECSE 429 are lower than that of ECSE 321, but the results are generally close to the department 
mean, so they are reasonable when teaching a course for the first time.  A specific challenge of the ECSE 429 course is that 
many students take it during their final semester at McGill, and as such, frequently after an industrial 
internship. Therefore, they might already be familiar with a significant part of the course content. 
Unfortunately, it is impossible to force these students to take this course earlier due to scheduling constraints. Nevertheless, 
the course was still very effective in teaching modern software technologies related to quality assurance (see \autoref{tab:tech-eval-ecse429}).

%There are three negative outliers, namely, Q12, Q13 and Q14 which need further action in the future. 

%\begin{itemize}
%\item Q13: The very high workload was dominantly caused by (1) selecting a real open source Android project as the subject of the team software quality project, which turned out to be too real as students complained about the low quality of the source code. Moreover, (2) students had to complete three assignments and three on-line quizzes during the semester. Since a real quality assurance project is very important part of the course in my view, in the future, I plan to turn the three extra assignments to be bonus exercises. As such, the exercises are still there for preparing of the final exam, but the amount of compulsory work is reduced. 
%
%\item Q12: The lack of timely feedback was caused by the increased workload of graders. By removing the three compulsory assignments, the workload of the graders will also be reduced. 
%
%\item Q14: Many students take the ECSE 429 course during their final semester at McGill, and as such, frequently after an industrial internship/work experience. Therefore, they might already be familiar with a significant part of the course content. Unfortunately, it is impossible to force these students to take this course earlier due to scheduling constraints. 
%
%\end{itemize}


\paragraph{Course-specific feedback: ECSE 6xx: Critical systems}
%While I have over 15 years of experience in teaching various masters-level graduate courses in Hungary, I had some very unexpected experience during 

The first edition of my graduate course on Critical Systems I gave at McGill University had some rather unexpected mutual mismatch of expectations. My expectation for a graduate student is to be able to self-explore software technologies and tools and to be open to independent problem solving even when facing vaguely defined challenges. Unfortunately, the majority of graduate students who took my course required were not such independent thinkers and they required much more hand-holding than I ever gave during a graduate course. While I used some of the teaching material in the past for industrial trainings on the standard SysML modeling language, I received lower score on course material, which is somewhat surprising.

\emph{Action plan:} Based on my experience and students' feedback, there are two ways to proceed: either (1) the course should be more research-intensive (when the course enrollment may be significantly lower), or (2) it should be turned into a lower-level course with both graduate and (higher-year) undergraduate students (with higher enrollment, but less research content). 
%\emph{Action plan:} The slides of some lectures need further improvement, and 
I also plan to include tutorials of the modeling tools used in the course and further improve the slides of certain lectures. 
%Furthermore, I plan to make the quizzes closed-book and thus, the questions can be directly related to the material covered in the slides of the course.  

\paragraph{High course workload}
%There are only two cases where I received a score below 3.5:. 
%\begin{itemize}[leftmargin=0.5cm]
%\item 
Students have repeatedly noted (in Q13) that the workload in my courses is high (thus I received scores below 3.5). 
%There is a single question (Q13) about the appropriateness of the course workload where my scores are repeatedly below the DM. 
This is dominantly attributed to the fact that all of my courses are project-based, which contributes a lot to the professional development of students, but it  also requires more work from students.  Moreover, certain efforts invested by students in those courses are actually voluntary work (i.e. students aimed for bonuses). Altogether, students normally work a lot in my courses, but they also learn a lot from them (see also \autoref{sec:tech-self-eval}). 

\emph{Action plan:} In case of ECSE 429, I plan to reduce workload by turning three assignments to bonus exercises. Thus those exercises will still be available for students when preparing for the final exam, but the amount of compulsory work is reduced. In case of ECSE 321, no related actions need to be taken as the students love the course despite the high workload.

%\item Students of the ECSE 6xx graduate course noted (in Q9) that the course material was not very helpful when preparing for the quizzes. This feedback is mostly dedicated to a mismatch between students' expectations and my expectations for a graduate-level class, especially, for methods of assessment. Students had to write two open book quizzes where I normally ask questions where students need to relate different parts of the material, but they cannot directly find answers in the slides. 


%\item Q13 in ECSE429: The students noted that the workload was very high (which The very high workload was dominantly caused by (1) selecting a real open source Android project as the subject of the team software quality project, which turned out to be too real as students complained about the low quality of the source code. Moreover, (2) students had to complete three assignments and three on-line quizzes during the semester. Since a real quality assurance project is very important part of the course in my view, in the future, I plan to turn the three extra assignments to be bonus exercises. As such, the exercises are still there for preparing of the final exam, but the amount of compulsory work is reduced. 
%\end{itemize}

%When teaching the course for the first time in Fall 2018, I decided to significantly upgrade and modernize the underlying software quality assurance technologies and tutorials used in the course, and changed the scope of the complex team project to test Android applications, while I reused significant amount of existing course material, thus interpreting the results needs some extra caution. 



%\subsection{Student feedback}

%\subsection{Summary of teaching}




%\subsubsection{Final reflections}

%\paragraph{Course-specific reflections: ECSE 321}

%\paragraph{Course-specific reflections: ECSE 429}





\subsection{Evidence of effective student supervision}
\label{sec:evid-student-super}

The effectiveness of my graduate supervision can be highlighted by numerous factors, such as publications at leading scientific venues of my field and international awards received by our papers or by my students. 

%Detailed evidence of my successful supervision will be provided below, but most importantly, Oszkár and Gábor defended their PhD thesis in June 2019, Csaba's thesis is already under review and his public defense is expected in Fall 2019, while Ákos and Vince are PhD candidates (to defend within a year). Moreover, since Fall 2017, Márton Búr has been making substantial progress in his PhD studies at McGill University thus his research seminar is scheduled in September 2019.

\paragraph{Successful PhD defenses}
Since joining McGill University in 2016, three of my PhD students successfully completed their PhD defenses and a fourth student is a PhD candidate. 

\emph{Gábor Szárnyas} (my PhD student at BME, graduate research trainee (GRT) at McGill) defended his thesis on "Query, Analysis, and Benchmarking Techniques for Evolving Property Graphs of Software Systems" in June 2019. In the same month, \emph{Oszkár Semeráth} (my PhD student at BME, GRT at McGill) also defended his PhD thesis on "Formal Validation and Model Generation for Domain-Specific Languages". Furthermore, \emph{Csaba Debreceni} has also submitted his PhD thesis on "Advanced Techniques and Tools for Secure Collaborative Modeling", which will be defended in Fall 2019. 
While Gábor, Oszkár and Csaba did not restart as McGill PhD students (as they were starting their 2nd/3rd year of PhD studies), they regularly visited me as graduate research trainees. Since most of their supervision was carried out since I joined McGill University, I regard them as PhD students I graduated as a McGill professor. 

\emph{Zoltán Ujhelyi} (BME) defended his PhD thesis on "Program Analysis Techniques for Model Queries and Transformations" in 2017, but in his case, most of his scientific progress towards the PhD was made before I joined McGill. Nevertheless, I still supervised him while writing his thesis as a McGill professor.  

Finally, \emph{Márton Búr} has made substantial progress in his PhD studies at McGill University. Hewill have his Research Seminar (ECSE703) in September 2019, i.e. at the beginning of his 3rd year of PhD at McGill. 
%, thus his progress towards his PhD is also very promising.

\paragraph{Papers co-authored with my graduate students}
Since joining McGill University in August 2016, I published four peer-reviewed journal papers \cite{act2017,sosym2017-mondo,sosym2017-tb,ieeesw2018} (including 2 in Software and Systems Modeling and 1 in IEEE Software) co-authored with my PhD students,  and two more journal papers are accepted for publication \cite{sttt-2019-cps,sttt-2019-div}. Five of these papers had my PhD students as first authors. 
In addition, we published 18 regular and tool demonstration papers and 2 more papers are accepted at leading scientific conferences of my research area (including 8 at MODELS, 2 at ICSE, 2 FASE, 1 at ESEC/FSE, 1 at ICMT) together with my graduate students. Out of these 20 papers,  14 papers had a student first author who I supervised or co-supervised. We published 3 more peer-reviewed papers at international workshops.% of major conferences. 

\paragraph{Best/Distinguished paper awards}
Three papers having my PhD student as co-authors received paper awards at prestigious international conferences. 

The paper co-authored by Márton Búr (my PhD student at McGill), Gábor Szilágyi, András Vörös and myself on "Distributed Graph Queries for Runtime Monitoring of Cyber-Physical Systems" \cite{fase2018-cps} received the \emph{ETAPS EASST Best Paper Award} from the European Association of Software Science and Technology (\href{http://easst.aulp.co.uk/}{EASST}) at the 21st European Joint Conferences on Theory and Practice of Software (ETAPS), which is the primary European forum for academic and industrial researchers working on topics relating to Software Science. Our paper was selected by the EASST committee from more than 140 papers (accepted at the six main conferences of ETAPS). As a follow-up of the award, we were invited to submit an extended version of our work to Software Tools on Technology Transfer journal \cite{sttt-2019-cps} (in press). 

The paper co-authored by Gábor Bergmann (formerly supervised at BME), Csaba Debreceni (my PhD student at BME, graduate research trainee at McGill), István Ráth and myself on "Query-based access control for secure collaborative modeling using bidirectional transformations" \cite{MODELS2016-access} received an \emph{ACM Distinguished Paper Award} at the IEEE / ACM 19th Int. Conf. on Model Driven Engineering Languages and Systems (MODELS 2016). In this case, three papers were selected out of the total of 42 accepted regular papers. Again, we were invited to submit an extended version to a special issue in the Software and Systems Modeling journal (Springer)
%, the top scientific journal of my direct research field, which was recently 
published in \cite{sosym2017-mondo}.

Finally, in collaboration with researchers and software engineers of IncQuery Labs, our paper \cite{models2018-tool} co-authored by Csaba Debreceni got the \emph{Best Tool Paper Award} at the MODELS 2018 conference.

\paragraph{International awards won by my graduate students}

My graduate students were also very successful at research contests organized for graduate students at major international conferences.  \emph{Gábor Szárnyas} (my PhD student at BME,  and GRT at McGill) won the 1st prize at the ACM Student Research Competition (SRC) at the MODELS 2016 conference, at the main forum of model-driven engineering. Furthermore, he also won 2nd prize at the ACM SRC contest organized at the SIGMOD 2018 (International Conference on Management of Data), which is a top conference in the field of databases. As a further success story, Gábor received an invitation to the program committee of the International Conference on Extending Database Technology (EDBT) right before his PhD defense, which shows the great level of research scientific independence he achieved (as databases are not my direct research area!). 
In addition, \emph{Csaba Debreceni} (my PhD student at BME,  and GRT at McGill) was also successful at the ACM Student Research Competition (SRC) at the MODELS 2017 conference winning the 1st prize. 
%He also published follow-up papers in the \emph{IEEE Software} and \emph{Software and Systems Modeling} journals. 


\paragraph{Success in supervising of early research work}

I have successfully continued involving talented undergraduate students in research and persuading them to pursue 
graduate studies afterwards. Aren Babikian and Sebastian Pilarski, two of my current MEng students at McGill University 
started their research in the Summer Undergraduate Research in Engineering (SURE) program, and they became co-authors 
of a tool demonstration paper \cite{icse2019-tool} published at ICSE 2019 thanks to their contributions. 


%All of my \emph{seven best paper awards had one of my PhD students} as the first author. 

%prestigious international Student Research Contests and Doctoral Symposiums (2x 1st prize at MODELS, 1x 2nd prize at SIGMOD, 1x 1st prize at STAF conferences).

\section{Educational Leadership and Teaching Development}



\subsection{Educational leadership experience at McGill University}
I serve as the \emph{program director of the software engineering co-op program} (expected to start in Fall 2020), but this activity is reported as part of my service portfolio in \autoref{sec:university-service}.
 
%At McGill University, I served on \emph{various senior committees on the department level} including faculty search, graduate student ranking for scholarships, strategic advisory board of the department chair, tenure and reappointment committee. On the university-level, I have been elected to serve as one of the two the \emph{representatives of Faculty of Engineering at the Council of Graduate and Postdoctoral Studies}. 

\subsection{Educational leadership experience at BME}
%(on research group, department, faculty, national level) 
At BME, I was the \emph{operative leader} of the \href{http://inf.mit.bme.hu/en/}{Fault Tolerant Systems Research Group} (consisting of over 20-25 members) between 2012 and 2016. Within this role, I was in charge of coordinating many aspects of the everyday life and duties of the group including the strategic supervision of educational activities. 

Since 2012, I have been a member of the \emph{strategic executive board of the department} (and I am still a member in my absence). In 2013, I was nominated as a \emph{department representative at faculty-level coordination meetings} aiming for the development of a new curriculum at the MSc level, which officially started in February 2015. I have continuously been serving in a \emph{cross-departmental committee for talent care} since 2006. 

On the faculty-level, I am member of the Informatics Doctoral School at the Faculty of Electric Engineering. Between 2014-16, I served on the committee of the Doctoral School on Software Engineering (Informatics) to shape the education program of PhD students. At the national level, I am a \emph{member of the Informatics Scientific Committee of the Hungarian Academy of Sciences} (elected as the youngest member first in 2011 and re-elected in 2014 and 2017). I was an external member of the Informatics Doctoral Committee of University of Szeged between 2014 and 2016.

\subsection{Teaching and Learning Training Received}
Since joining McGill University, I have been actively attending various training sessions related to teaching and supervision as summarized in the following list:

\begin{itemize}
\item \emph{Supervisory Alliance Training} organized by GPS (Graduate and Postdoctoral Studies)  2016 (2 hours)
\item \emph{Clarifying Expectations in Graduate Supervision} offered by GPS,  2016 (2 hours)
\item \emph{Mandatory Graduate Student Supervision Training} offered by the Faculty of Engineering,  2017 (2 hours)
\item \emph{How to write recommendation letters?} organized by GPS, 2017 (2 hours)
\item \emph{Equity Training} offered by the Faculty of Engineering, 2017 (2 hours)
\item \emph{Postdoctoral Fellows Financing} offered by GPS, 2017 (2 hours)
\item \emph{Course design workshop} offered by Teaching and Learning Services (TLS), 2018 (2 days)
\end{itemize}

%\section{Teaching and graduate supervision philosophy}
%
%Most of my courses have been organized along a common scheme. Students need to invest significant amount of work in completing a \emph{team project} of 3-5 persons. I assign a \emph{complex software/systems engineering challenge}, and ideally with a real customer, where students need to \emph{use modern software technologies} (e.g. IDEs, version control systems, etc.) and project management frameworks (like Basecamp, Trello) while learning also foundational concepts. 
%
%Maintaining the attention of students during lectures can be a challenging task. I try to postpone abstract mathematical definitions to the point when the main concepts are sufficiently demonstrated by small examples first. I frequently use various \emph{best practices of active learning} during my lectures (e.g. bug hunts on ill-formed software, various games and quizzes, small challenges discussed in small groups). I often \emph{demonstrate the practical relevance of concepts using real-world examples} and industrial case studies sometimes presented by industrial collaborators within invited lectures. Usually, \emph{my lectures are complemented with tutorials} on key technologies (e.g. Eclipse technologies or multi-tier web applications) actively co-developed by my graduate students which also provides a first teaching experience for them. 



%\begin{figure}
%\centering
%\includegraphics[width=.8\textwidth]{figures/teaching-eval}
%\caption{Teaching evaluation at McGill Univeristy}
%\label{fig:teaching-eval}
%\end{figure}



%\end{document}