By Dr Nuddy Pillay | Professional Teaching Fellow, University of Auckland
Source: Feedbackfruits.com
As the use of educational technology (EdTech) accelerates, particularly with the integration of AI, learning analytics, and scalable peer-assessment tools, there is a growing need to ensure that such tools genuinely support learning, rather than merely delivering novelty. Pilot phases and structured user testing enable us to observe how tools perform under the unpredictable conditions of real learning environments, with diverse learners, competing demands, and varied contexts. Such early testing helps identify usability issues, pedagogical misalignments, accessibility challenges, or opportunities for improvement that purely lab-based development may miss.
Piloting with Purpose: Why Early Testing Matters
Implementing digital tools in real learning environments reveals challenges and opportunities that are often not visible in lab-based development. A recent four-year study on the systematic implementation of EdTech found that teacher perspectives and structured feedback loops are crucial for the sustained and effective integration of digital learning materials and analytics dashboards (Nordmark, 2024).
Design‑based research (DBR), a methodology gaining traction in educational innovation, supports this model. DBR emphasises iterative cycles of implementation, evaluation, and redesign, grounded in real teaching contexts. In short: user testing and pilots aren’t nice extras, instead, they’re essential for ensuring EdTech supports actual pedagogical needs.
In my own practice at the University of Auckland, I used FeedbackFruits to support team‑based learning (TBL) and peer feedback across foundation and undergraduate Business courses. Rather than simply adopting the platform, I engaged in a pilot phase: configuring peer review workflows, surveying student experience, collecting qualitative feedback, and making iterative adjustments. That process revealed important insights: what kinds of prompts supported better feedback; how asynchronous and synchronous modes affected participation; and how peer assessment could be scaffolded to support learners unfamiliar with such practices.
This collaborative design approach reflects what EdTech‑testbed frameworks describe: real partnerships where educators, developers, researchers and students work together to define problems, design solutions, test them, and refine them.
Educators as Essential Co‑Designers
When educators are involved from the start, not just as users, but as active partners, EdTech tools are more likely to be pedagogically aligned, usable, widely adopted, and inclusive. As highlighted in a literature review of human‑centred learning analytics and AI in education, involving teachers (and learners) throughout the design and deployment phases helps balance automation with human agency, builds trust, and ensures usability, equity, and safety.
In the case of FeedbackFruits, this collaborative use produced measurable gains: peer‑review participation, improved collaboration, and deeper engagement. A number of institutional case studies report positive outcomes, increased student interaction, higher quality group work, and smoother peer feedback workflows, especially when educators had the freedom to adapt and contextualise the use.
Diversity, Equity, Inclusion and Scaffolded Design
Robust pilots help ensure that digital tools benefit all learners, including those from underrepresented backgrounds or those facing barriers (e.g., first-in-family, international, non‑traditional, or culturally diverse students). As studies on technology-supported collaborative learning and peer assessment suggest, digital scaffolding can help narrow achievement gaps by providing repeated opportunities for feedback, reflection, and support. This aligns with my commitment to inclusive pedagogy, particularly for Māori, Pasifika, international and first‑in‑family learners. By piloting with diverse student groups and collecting data on participation, engagement, and learning outcomes, educators can identify and address equity issues early, ensuring EdTech supports, rather than unintentionally disadvantaging, specific cohorts.
Collaboration: An Invitation to EdTech Providers & Educators
To Edtech developers and providers: I urge you to establish sustainable, ongoing partnerships with educators, treating us not only as test-users, but also as co-researchers and collaborators. Continue to build feedback loops and support iterative re-designs as a part of your product roadmap. Empower educators to contribute to and inform design decisions.
To educators: consider engaging with emerging tools not just as consumers, but as active collaborators. Share your classroom data, feedback, and experiences. Your insights can guide the development of tools that are meaningful, inclusive, and effective for real learners.
When developers, educators, and learners collaborate from the start, we move from “using technology” to building tools that truly enhance learning.
References
Amiel, T. & Reeves, T. C. (2008). Design‑Based Research and Educational Technology: Rethinking Technology and the Research Agenda. Educational Technology & Society. ResearchGate+1
FeedbackFruits. (2024). What is Collaborative Learning? FeedbackFruits Blog. feedbackfruits.com+1
Hall, T. & Kollegen. (2021). The Emerging Potential of Design‑Based Research in Educational Innovation. ERP Journal. erpjournal.net
Northmark, S. et al. (2024). Piloting Systematic Implementation of Educational Technology: Insights from Schools. Implementation Science in Education. SpringerLink
Wood, J. (2022). Making Peer Feedback Work: The Contribution of Technology‑Mediated Peer Feedback Practices. Assessment & Learning in Higher Education. Taylor & Francis Online
Zou, D. et al. (2023). Effects of Technology‑Enhanced Peer, Teacher and Self Feedback on Collaborative Writing in Higher Education. Journal of Learning & Technology Research. SpringerLink