-
I am Shota Aoyama, an 11th grade student at Gunma International Academy High School. I would like to present my research on “Kineto,” which enables classroom experiences that go beyond time. I would like to express my gratitude to Mr. Arase for mentoring me in the first stage and Mr. Kataoka in the second stage.
-
This is my profile.
-
I am interested in the concept of a timeline in virtual reality (VR), and in this research, I focused on the timeline of classroom experiences.
-
Until now, I have been attending face-to-face classes in elementary, middle, and high school, or real-time classes like Zoom, and I didn’t feel any inconvenience. However, when I started watching educational videos on YouTube for test preparation, I suddenly felt frustrated because I couldn’t perform simple operations like pause, fast-forward, or rewind that are readily available on YouTube during real-time classes. The ability to go back a little to review something I didn’t understand or pause the video to take notes calmly can only be done with video-based lessons. Of course, it is not feasible to have all classes on YouTube, as face-to-face or Zoom classes are essential for student interaction and communication.
-
To achieve a balance between the benefits of real-time communication among students and the advantages of time control in on-demand learning, we developed a software called “Kineto.”
-
To achieve this balance, we separated the timeline that was previously consolidated in traditional real-time classes. By allowing independent operations on each timeline, we were able to achieve this balance.
-
Specifically, here are some of the things that can be done:
- Teachers can upload their class videos to Kineto or stream them live.
- Students can share their thoughts and emotions with each other on the Kineto board while attending the class.
- Various interactions such as sticky notes, handwriting pens, and voting functions are available, allowing students to communicate with their classmates on Kineto.
- Students can freely pause, fast-forward, rewind, and perform other time-related operations while attending the class.
- When rewinding the video, the sticky notes and written memos posted by other students also rewind together.
-
I will now explain how Kineto achieves the balance between student communication and time control that I mentioned earlier.
-
First, when we categorize the existing classroom environment based on synchronicity, we can classify classes into two types:
- The first type includes face-to-face classes and Zoom classes, where each student experiences the same content synchronously. The second type includes platforms like YouTube and NicoNico Douga.
- The advantages of face-to-face and Zoom classes lie in the rich communication among students, while the disadvantage is the inability to control time asynchronously.
- On the other hand, YouTube and NicoNico Douga allow time control, but the communication among students becomes limited.
- By comparing these two types, we can see that there is a trade-off relationship between the desire for rich student communication and the desire for asynchronous time control.
-
The extent to which these two aspects are demanded depends on the form of the class. Let’s take a closer look at the specific situations where these demands arise.- First, regarding communication between students, it is not necessary to have a classroom environment where students can communicate with each other if the class consists of students only listening to the teacher without any student participation. However, it is often beneficial to have some level of shared emotions such as laughter and surprise. Recently, there has been a trend towards more student-centered classes where students actively participate and engage in discussions. In such classes, rich communication among students is desired.
-
As for how well traditional classroom environments cover these aspects, in reality and Zoom classes, it is possible to have all forms of communication. However, in Youtube videos, it is difficult to have anything other than one-sided communication where students only listen to the teacher. Using NicoNico Douga for classes allows for pseudo-synchronous communication through the comment system, enabling more student interaction than Youtube. However, it is limited to sharing emotions like laughter and surprise, as well as simple sharing of opinions.
-
Next, this diagram illustrates the extent to which time control is desired in various forms of classes.
-
In classes where students simply passively listen without much engagement, time control is not necessary. However, if students are actively participating in the class, they may want to rewind a few seconds when they miss something, take short breaks, or pause the class to take notes.
-
In terms of how well existing classroom environments cover this, it is obvious that Zoom and real-life classes do not allow for such flexibility. On the other hand, Youtube and NicoNico Douga enable free control of playback, pause, and fast-forward.
-
By combining these two axes, as shown in the graph, we can see that the current classroom environment covers the shaded area when focusing on synchronicity. Advanced schools successfully incorporate these elements into their classes.
-
However, this also means that classes can only exist within this shaded area, and classes that require student discussions while performing actions such as playback, pause, and rewind cannot be realized.
-
Filling this empty space, the uncharted territory, and expanding the way classes are conducted is what “Kineto” aims to achieve.
-
As you can see from the first video, Kineto meets the need for time control. In reality and Zoom, you cannot control the pace of time according to your own preferences, but with Kineto, as shown on the screen, you can certainly do so.
-
Furthermore, with Kineto, you can maintain time control while facilitating communication between students. I will now explain the three key features of Kineto that enable this: a sense of shared time, synchronized communication within the context, and bidirectional dialogue facilitated by elastic synchronization.
-
First, the sense of shared time is not achievable in traditional platforms like Youtube. This sense is crucial as it directly affects students’ motivation to participate in the class, and it is considered a major drawback of on-demand classes. With Kineto, by seeing comments written by others, students can feel the presence of other students who are not actually in sync with them, thus creating a sense of shared time. Additionally, the high resolution of time direction and the pace at which content changes are important factors in creating this sense. In Kineto, both handwritten and sticky note comments are shared in real-time, as shown here.- Second, one of the disadvantages of asynchronous classes is that they lack “synchronized contextual communication”. What this means is that in a YouTube class, for example, in order to exchange opinions with other students, you need to use the YouTube comment section or another shared document. However, if you do this, people who are at different points in watching the video will directly communicate, which may result in spoilers from others. This may not be a problem in meetings, but if it happens during the learning process of a class, it can lead to a loss of opportunity for individual thinking.
-
In a class experience using Kineto, you can know what others thought or felt at that moment on the timeline. This allows you to avoid spoilers during information exchange and communication.
-
And the biggest feature of Kineto is this third point, bidirectional dialogue through elastic synchronization. In YouTube videos or Nico Nico Douga classes, it is not possible to have bidirectional dialogue with other students during the class. With Kineto, partial bidirectional dialogue is possible through the mechanism of elastic synchronization.
-
Elastic synchronization is a system that synchronizes users who are watching asynchronously through automatic changes in playback speed.
- In this graph, the x-axis represents real-time, the y-axis represents the timeline of the video being watched, and the slope represents the playback speed.
- In traditional classes, you could only choose either constant synchronization or constant asynchrony (right figure).
- This is…
-
I have explained the concept so far, but I think it may be difficult to imagine, so let me introduce a class example.
-
In this way, we have tried various positions in this graph, but this time I will introduce the most Kineto-like class in the upper right corner.
-
In a moral class taught by Mr. Takamatsu at my school, there were seven classmates participating.
-
This class discussed the trolley problem, and first…
-
In the interview after this class, we received the following opinions:
- Read aloud
-
Finally, I would like to discuss the future development that came to mind during the development of Kineto.
Finally, I would like to express my gratitude to the people who cooperated with us.
Thank you very much.
- We have tried many things, so let me introduce one example from them.
- Mr. Kiryu, Professor Inami, and others
- Filmed the usage
- We tested it in various environments such as the lower left and lower right to demonstrate that what we have been talking about actually works.
- We will introduce a class example in the upper right corner, which is the most impressive.
- We collaborated with Mr. Takamatsu from Gunma International Academy Middle and High School to create a moral class where students could share their opinions on the trolley problem. We also created a clicker that allows students to vote for option A or B for this class.
- Feedback
- Show a sense of synchronization
- Show that being active leads to more participation
- In parallel with this software development, we are also conducting research on lecture video analysis using computer vision (CV).
- As there is not much progress at the moment, I will explain the previous research and future directions.
- Our goal is to evaluate the importance of each frame in lecture videos.
- In the elastic synchronization I explained earlier,
- Frames with low importance
- So, even if the students fast forward the video, it is not a problem. This is what we mean by low importance here.
- For this purpose, we believe that we can use saliency.
- Saliency is when…
- An example of previous research that uses this in videos is called UNISAL.- Originally, I had doubts about whether this could handle special videos like classroom footage, so I first wanted to create a saliency evaluation model to make it possible.
- However, when I actually tried UNISAL, the evaluation was done more accurately than I expected.
- The saliency of the content written on the blackboard in recent times is high, so the saliency of the parts that students would actually pay attention to is also high.
- Therefore, as a future direction, I would like to work on research to evaluate the importance of frames based on this.