2a1, 2a2, and 2a3. Description of the proposed research
2a1. Overall aim and key objectives
In the proposed research, the aim is to design a solution to improve the homework experience for students, which in turn will increase their performance. The research question asked is "To what extent does implementing an e-learning solution with data-driven engagement features for homework and practice alongside the classroom increase student grades in secondary education?" and the subquestion is "What is an optimal design for such an e-learning system?" For this, a web and app-based platform will be developed and studied to compare results with not having the platform, and each sub-question of this research aims to achieve a proof-of-concept feature in said platform.
It has been shown that homework plays a critical, long-term role in the development of student's achievement motivation and gives them time and experience to develop positive beliefs about achievement. It also teaches them strategies for coping with mistakes, difficulties, and setbacks (Bempechat, 2010). Research conducted in the United States since 1987 also shows that there is generally consistent evidence for a positive influence of homework on a student’s achievement (Cooper et al., 2006). However, the problem of homework completion has been evidenced by existing grade and homework reports and teacher and student surveys (Renee at al., 1997). The problem aimed to solve that students are not motivated to go home and revise what is being taught in the classroom. This can result in test fatigue when preparing large amounts of content in a short period of time, and also adds pressure and stress in students’ lives.
The problem can be solved by implementing an e-learning solution which can be motivating and engaging for students, so that students practice what they learned in class every day for higher retention and better grades. Many studies have also shown that that effective use of e-learning could help increase student motivation engagement, and attendance. It should also increase student class participation, and improved behavior and performance on core subjects. Students have been proved to be more receptive and willing to solve more complex homework in an e-learning platform than the traditional paper-book (Benta at al., 2014; Brady et al., 2010, Liaw, 2008; Stracke, 2009; Cidral at al., 2018). Furthermore, students’ success rates are also significantly higher for multimedia-based e-learning homework than traditional paper-based homework (Kodippili et al., 2007).
A 2017 study also expects improvements in communication technology (such as higher internet speed at accessible prices) to likely increase the use of multimedia in education. There are already numerous studies which show that students are highly motivated to use new technology. For example, students in Ethiopia were given tablets through the One Laptop Per Child (OLPC) scheme, but without any instructors or teachers. They taught themselves how to use the tablets, and even after several months, the students were still heavily engaged in using and recharging them (Talbot, 2012). This shows the curiosity that age group naturally possesses for interacting with new technology, which can be further extended to inclination towards digital means of education.
For this reason, several companies like Byju's and not-for-profit organizations like Khan Academy are building e-learning platforms which can serve as complete alternatives to classroom learning. However, such a system has its own challenges. For one, it cannot teach students social skills, so there will always be a place where working adults send their kids to be socialized. Furthermore, it cannot offer personalized feedback to students, or even real-time feedback. Therefore, a good e-learning solution with not aim to replace the traditional classroom, but rather complement it with its set of additional strengths (Laurillard, 2008).
Although it has been established that students are comfortable with a digital means of education, there might be a gap of awareness between some students and their teachers about technology in general. A 2017 study asks the question “are online instructors ready to meet the 21st century challenges brought by this generation’s demand for the use of technology in education?”. According to the Organisation for Economic Co-operation and Development (OECD), the average age of a primary school teacher in the Netherlands is 43, which is much younger than that in Italy (50), but would still require an effort from the teacher's end to implement such a solution. A 2017 study from the British University in Egypt also found that e-learning success in depends largely on the effectiveness of delivery and adequate training of instructors. Inadequately trained staff can become an obstacle in a finely-balanced learning process and can lead to problems in application use and in the perception of students.
That being said, tools such as analytics of data generated by student usage will not only help teachers make better in-classroom decisions, but also make them more acquainted with homework process and results. Using a digital e-learning platform is also a preferred way for class management and avoids potential errors (Benta at al., 2014).
Although the aforementioned studies show that e-learning has potential benefits for students and teachers, there is still a gap in the knowledge:
- These studies are hardly about integration with homework and tend to focus on e-learning as an alternate to the traditional classroom; out of 12 papers, only 1 focused on e-learning as an addition to the classroom and the rest focused on distance learning –
- These studies are often conducted in a laboratory environment and have small test groups. We do not know well enough whether an integrated e-learning platform with all relevant features will in fact positively impact learning process and outcomes for real students in longer term practical setting. In the proposed research, the focus will be to realize the theoretical potential of these studies by conducting them in real classrooms and aggregating them through multiple schools in different locations and with diverse backgrounds (for example, both international private schools in Amsterdam and state-sponsored schools in Deurne will be considered)
- There is no one-stop e-learning solution which integrates the findings of all of these studies and truly helps with learning
In addition to these, there are also the pain points of teachers which — despite being addressed previously — have not been solved yet. In essence, the literature research and the features it establishes showcase an opportunity for a unique product, which is the motivation for this proposal.
Therefore, this research proposal aims to:
- Integrate what is already known about e-learning in general with a specific, homework use case
- Show that the platform contributes to learning in real practice by comparing standard test scores of a control group (students without access to the e-learning platform) and two treatment groups (both groups of students will have access to the platform; students in the first group will be forced to use the platform once per day, and students in the second other can choose to use or the platform at their will), and to show that the both treatments group scores higher than the control group
- Show how each of the features in the platform individually contribute to this effect by testing features separately and in combination
- Take into consideration the pain points of teachers when implementing an e-learning solution, and investigate how this learning curve can be reduced; the platform designed in this research proposal aims to bridge the gap by empowering teachers with actionable analytics, along with an easy-to-use interface
In the initial stage of the the proposed project, literature research along with surveys and interviews will be conducted. Based on this, lo-fi prototypes for each feature will be developed and tested. Then, the final e-learning platform will be developed, which will be followed by an empirical experiment on a diverse set of students from different schools around the Netherlands. To show that the platform truly contributes to better learning, a controlled experiment will be conducted at the end of the study with students who previously scored approximately equal grades.
2a2. Research plan
The study will be conducted over four years and will go through the following stages:
- Year 1: Literature research, surveys, and interviews
- Year 2: Platform prototyping and lo-fi testing
- Year 3: Lo-fi testing and platform development
- Year 4. Factorial study and end study
In the first six months, a thorough literature research will be conducted which will help establish a list of features the platform will have. In this step, papers across technology, psychology, and education will be studied, from journals including Social and Behavioral Sciences, Computers in Human Behavior, and Telematics and Informatics. This interdisciplinary perspective will help compile a list of prospective features for the platform based on the current research in e-learning.
For curation, databases such as Scopus, ScienceDirect, and Google Scholar will be used with queries such as the following, which finds documents relating to e-learning in schools:
TITLE-ABS-KEY ( ( "e-learning" OR "elearning" ) AND "school" AND NOT "higher education" AND NOT "university" )
Both, papers relating to "mixed" e-learning and distance learning will be assessed, and combined with interviews and surveys with students, teachers, and industry experts. For example, interviews will be conducted with students to identify the distraction and motivational aspects of revising at home, and the results will be used to build a solution implemented in the platform (e.g., engagement through notifications).
Once the list of features for the platform has been completed, prototypes will be developed for each feature and tested with students. This will be a Build-Measure-Test process, and incremental changes to the prototypes will be part of the testing process, which will bring each feature closer to its final form. Once this process is completed, the platform will be developed. Technologies used for the platform will include a front-end development framework, either React or Vue.js, and a cross-platform native app development framework like Flutter. For the backend, RESTful APIs and services will be developed in Node.js and hosted on Amazon Web Services's Elastic Compute Cloud (AWS EC2).
Using the platform, a factorial study will be conducted over one year evaluating each feature individually and finding which combination works the best for students. The study will be divided into discrete levels (each feature of the platform) and the experimental units take on all possible combinations of these levels, which will measure student grades in each factorial unit. This will allow us to study the effect of each feature in the platform on the response variable (i.e., student performance), as well as the effects of interactions between features.
The following example is a 3x3 factorial study with the following factors: Engagement through notifications, interactive content (videos and games), and gamification of platform. In the proposed study, the factors will be determined using literature research.
|⭐ Interactive content (videos and games)|
|⭐ Interactive content (videos and games)||⭐ Only interactive content|
Therefore, there are 3! + 1 = 7 possibilities (3 individual factors + 3 combinations + 1 with all factors), all of which will be assessed for a student's performance as measured by their grade over a period of one academic year.
There will be three sets of students:
- Students who have no access to the e-learning platform
- Students who have access to the e-learning platform and are required to use the platform once per day
- Students who have access to the e-learning platform but have no usage requirements, and can choose to use the platform at their will
Each group of students will be from the same age group and will study the same material. At the end of this process, the best option (combination of features) will be concluded based on the highest grades of participating students. Parental consent will be prerequisite as part of the study and a thorough ethical evaluation will be conducted by third-party researchers from the University of Twente before starting the study.
The study will be conducted in multiple schools at the same time to get a more aggregated and general result, for which several partnerships with municipalities and educational instituted will be established. Therefore, the primary partners of this study will be the schools who will be associated with the study for a long period of time. Several schools spread around the Netherlands will be approached during the first two years of the study and between 2 and 5 schools will be chosen during the third year to conduct the study.
For the development of the platform, the primary collaborator is Oswald Labs, an accessibility technology company. Oswald Labs has a past track record with high-availability, scalable products which are used by thousands of users every month. They also specialize in research and development of disabled-friendly products, including school students with disabilities. Their experience in technology development and bringing products to the market will be a valuable resource when building and implementing the platform. Furthermore other partners for this study may be the Ministerie van Onderwijs, Cultuur en Wetenschappen, and provinces whose students participate in the study who are also interested in learning about the results and implementing the platform on a larger scale. Other sources for research and potential partners include Inspectie van het Onderwijs (Onderwijsinspectie) and the Organisation for Economic Co-operation and Development (OECD).
2a3. Motivation for choice of host institute
The host institute for the proposed project is University of Twente in Enschede, the Netherlands. As a research-driven university that is highly specialized in engineering, University of Twente (UT) amalgamates technology development with psychology and education. This interdisciplinary approach is necessary for the proposed project since it combines pedagogy with product.
Furthermore, the UT has significant experience in building long-term research collaborations with schools, parents, and students. For example, the Human-Media Interaction department organizes the DesignLab School Trip Research Days which encourages students from around the country visit the university and learn about research and development. The UT also has programs such as Pre-U, a range of unique educational experiences for students to become familiar with scientific education and research, and the UT's Faculty of Behavioural, Management and Social Sciences has several groups working on education and particularly focusing on the intersection of technology and school students.
2b. Knowledge utilisation
Once the study has proved the utility of the platform and increase in learning, it can be implemented in real schools in the Netherlands and around the world. During the research, a diverse set of several schools in the Netherlands will participate and may see improvements in students' learning. After that, a trial can be conducted in all schools in a single province, and then this can be extended to all schools in the Netherlands. This means that the study will not just add value to the understanding of what makes learning better for students, but truly achieve this purpose. Another study can be conducted to measure any long-term effects of the implementation as well.
The underlying technology of the platform can be protected using patents and intellectual property law, which can be used to set up a startup. The company can provide a service to help schools in the Netherlands (and scale globally) a central platform to help students perform better. Furthermore, the proposed study will also add value as scientific knowledge. Once concluded and peer reviewed, a large dataset will be compiled and released of student performance corresponding to each set of e-learning features. This means that other statistics or machine learning applications can also come out of this research. Finally, the publication of these findings will also be useful since it will be the first large-scale and comprehensive study which will evaluate several different aspects of e-learning, and also explore the benefits of homework via e-learning. Not just a journal article, but also a book can be born out of this research. Other researchers can build of top of this study and compare the utility of other features in e-learning which are not considered in this research.
2c. Number of words used
Section 2a: 2,351 (max. 2,000 words)
Section 2b: 287 (max. 750 words)
2d. Literature references
- Bempechat, 2010
- Cooper et al., 2006
- Renee at al., 1997
- Benta at al., 2014
- Kodippili et al., 2007
- Talbot, 2012
- Hong-Ren Chen, Hsiao-Fen Tseng, Factors that influence acceptance of web-based e-learning systems for the in-service education of junior high school teachers in Taiwan, Evaluation and Program Planning, Volume 35, Issue 3, 2012, Pages 398-406, ISSN 0149-7189, https://doi.org/10.1016/j.evalprogplan.2011.11.007
- Tzu-Hua Wang, Developing Web-based assessment strategies for facilitating junior high school students to perform self-regulated learning in an e-Learning environment, Computers & Education, Volume 57, Issue 2, 2011, Pages 1801-1812, ISSN 0360-1315, https://doi.org/10.1016/j.compedu.2011.01.003
- S. Hubalovsky, M. Hubalovska, M. Musilek, Assessment of the influence of adaptive E-learning on learning effectiveness of primary school pupils, Computers in Human Behavior, Volume 92, 2019, Pages 691-705, ISSN 0747-5632, https://doi.org/10.1016/j.chb.2018.05.033.
- M. Bruri Triyono, The Indicators of Instructional Design for E- learning in Indonesian Vocational High Schools, Procedia - Social and Behavioral Sciences, Volume 204, 2015, Pages 54-61, ISSN 1877-0428, https://doi.org/10.1016/j.sbspro.2015.08.109.
- Saeed Mazloumiyan, Seyed Mohamad Shobeiri, Mehran Farajollahi, Mehdi Mohamadi, Blended e-Learning: A New Approach to Environmental Education of Iran High Schools, Procedia - Social and Behavioral Sciences, Volume 47, 2012, Pages 1216-1220, ISSN 1877-0428, https://doi.org/10.1016/j.sbspro.2012.06.803.
- Bhavana Srivastava, Md. Tanwir Uddin Haider, Personalized assessment model for alphabets learning with learning objects in e-learning environment for dyslexia, Journal of King Saud University - Computer and Information Sciences, 2017, ISSN 1319-1578, https://doi.org/10.1016/j.jksuci.2017.11.005.
- Antonio Garrido, Lluvia Morales, Ivan Serina, On the use of case-based planning for e-learning personalization, Expert Systems with Applications, Volume 60, 2016, Pages 1-15, ISSN 0957-4174, https://doi.org/10.1016/j.eswa.2016.04.030.
- Bikram Acharya, Jongsu Lee, Users’ perspective on the adoption of e-learning in developing countries: The case of Nepal with a conjoint-based discrete choice approach, Telematics and Informatics, Volume 35, Issue 6, 2018, Pages 1733-1743, ISSN 0736-5853, https://doi.org/10.1016/j.tele.2018.05.002.
- Nilcan Ciftci Ozuorcun, Feride Tabak, Is M-learning Versus E-learning or are They Supporting Each Other?, Procedia - Social and Behavioral Sciences, Volume 46, 2012, Pages 299-305, ISSN 1877-0428, https://doi.org/10.1016/j.sbspro.2012.05.110.
- Nina Hrtoňová, Jiří Kohout, Lucie Rohlíková, Jiří Zounek, Factors influencing acceptance of e-learning by teachers in the Czech Republic, Computers in Human Behavior, Volume 51, Part B, 2015, Pages 873-879, ISSN 0747-5632, https://doi.org/10.1016/j.chb.2014.11.018.
- Kevin Brady, Lori Holcomb, and Bethany Smith, The Use of Alternative Social Networking Sites in Higher Educational Settings: A Case Study of the E-Learning Benefits of Ning in Education, Journal of Interactive Online Learning, Volume 9, Number 2, Summer 2010, Pages 151-170, ISSN 1541-4914.
- Shu-Sheng Liaw, Investigating students’ perceived satisfaction, behavioral intention, and effectiveness of e-learning: A case study of the Blackboard system, Computers & Education, Volume 51, Issue 2, 2008, Pages 864-873, ISSN 0360-1315, https://doi.org/10.1016/j.compedu.2007.09.005.
- Christian Stracke, Quality Development and Standards in e-Learning: Benefits and Guidelines for Implementations, ASEM Lifelong Learning Conference: e-Learning and Workplace Learning, 2009.
- Wilmar Audye Cidral, Tiago Oliveira, Massimo Di Felice, Manuela Aparicio, E-learning success determinants: Brazilian empirical study, Computers & Education, Volume 122, 2018, Pages 273-290, ISSN 0360-1315, https://doi.org/10.1016/j.compedu.2017.12.001.
- Diana Laurillard, The teacher as action researcher: using technology to capture pedagogic form, Studies in Higher Education Volume 33, Issue 2, 2008, Pages 139-154. https://doi.org/10.1080/03075070801915908.
2e. Data Management
- Will data be collected or generated that are suitable for reuse? Yes
- Where will the data be stored during the research? During the research, real-time data will be collection from platform usage and stored in a NoSQL database in the cloud. All information will be encrypted. A data processing agreement will be signed between the researchers and a third party (such as Amazon Web Services or Google Cloud) and the data processors will be clearly indicated.
- After the project has been completed, how will the data be stored for the long-term and made available for the use by third parties? To whom will the data be accessible? Once the research is concluded, all collected data will be anonymized and any personally-identifiable information will be redacted. After this process is completed, all data will be released to the public domain under the Twente License, to ensure privacy.
- Which facilities (ICT, (secure) archive, refrigerators or legal expertise) do you expect will be needed for the storage of data during the research and after the research? Are these available? The researcher is well-versed in cloud storage and data protection and part of the budge will be allocated for cloud storage. Apart from this, no additional facilities are required.
Insert cost analysis table here
3b. Contributions ‘in kind’
|Oswald Labs Research||Cloud credits for data storage and hosting||€5,000|
3c. Contributions ‘in cash’
|Educatieve Agenda Limburg||Funding to build and test platform||€12,500|
|Grand total||€0 = 3a|
|Budget requested from NWO||€0 = 3a – (€17,500)|
3e. Intended starting date
September 1, 2019
3f. Additional grants
Have you applied for any additional grants for this project either from NWO or from any other institution, and/or has the same idea been submitted elsewhere? Yes, a prototype of the proposed platform was presented at a hackthon (Brighthack) in November 2018. As the winning prize, Educatieve Agenda Limburg committed to contributing €12,500 for building the platform. More information about the prize. However, this grant it specifically for the further development and implementation of the platform in Limburg's schools, and not the initial research requirements.