日時 5月２２日（土）16:00-17:30 オンライン会議 内容 情報科学の達人プログラムにて行ってきた研究 今後行なっていきたい研究 報告 20分、質疑10分
-
やっべ、何も準備してない
-
てか映像処理の進捗が無だ…
- まあ無い進捗は無い
-
①Kineto
-
ぐんま国際アカデミー高等部12年 青山柊太朗です。今回取り組んだ研究「時間」を超えてつながる授業体験、そしてそれを実現するKineto（ゆっくり）について発表させていただきます。よろしくお願いします。メンターは第一段階では荒瀬先生、第二段階では片岡先生にご指導頂きました。
-
こういったプロフィールとなっています。
-
時間軸のVRとありますが、「没入」できる軸としての時間軸について興味を持っていて、今回は授業の時間軸について取り組んだ研究について発表させて頂きます。
-
今まで小中高の学校生活ではこのような対面授業、もしくはコロナ禍ではZoomなどの授業を受けてきて、何も不自由を感じていませんでした。しかし、テスト勉強などのためにYoutube上の授業動画を見出すようになると、Youtubeでは当たり前にできる、再生停止早送り十秒戻しなどの操作が普段の授業で出来ないことを急にもどかしく感じるようになりました。わからないことがある時にちょっと前にもどって確認したり、落ち着いてノートを取るために映像を止めたり、という操作は映像授業でないとできません。ただなら全部の授業がYoutubeで良いかというとそういう訳はなく、対面やZoom授業でないと学生感のコミュニケーションが失われてしまいます。
-
学生間のコミュニケーションと時間のコントロール、この二つの両立はドラえもんの秘密道具のようなものが無いとできない、、ように思えますが、今回それをKinetoで実現しました。
-
どの様に二つの両立を実現したかというと、従来のリアルタイム授業では授業の時間は一本にまとまっていました。この時間軸をこのように分離し、それぞれについて自由な操作をできるようにすることによって両立を実現しました。
-
具体的に何ができるかというと、
- 教師はKinetoに授業映像をアップローどもしくはライブ配信できます
- 生徒は授業を受けながら、Kinetoのボード上でお互いに考えや感情を共有することができます。
- ふせん・手書きペン・投票機能など、さまざまなインタラクションの場が設けられており、生徒はkineto上でクラスメイトとコミュニケーションを取ることができます。
- 生徒は授業を受けながら、自由に一時停止や早送り、巻き戻しなどの時間の操作を行うことができます。
- 映像を巻き戻すと、他の生徒が貼った付箋や書き込んだメモも一緒に巻き戻っていきます
-
この「時間軸をもった黒板」がどのように先程の学生間のコミュニケーションと時間のコントロールを両立させているのかについて、これから説明していきます。
-
まず、既存の授業の環境を同期性に着目して整理すると、授業をこの二つに分類することができます。
- 対面授業やZoom授業などの各生徒が同じ内容を同期的に体験するものが一つ目の種類で、Youtubeやニコニコ動画などが二つ目にあたります。
- 対面授業やZoom授業などのメリットは、生徒同士のコミュニケーションが豊富であるという点であり、欠点が非同期的な時間のコントロールができないという点になります。
- Youtube動画やニコニコ動画はその逆で、時間のコントロールが可能であるものの、生徒間のコミュニケーションが乏しくなってしまいます。
- こうして比べてみると、豊かな⭐（押す）️生徒間のコミュニケーションがしたいという需要と、と非同期的な⭐️（押す）時間のコントロールがしたいという需要が、トレードオフの関係になっていることが分かります。
Date: May 22 (Sat) 16:00-17:30 Online Meeting
Agenda:
- Presentation on the research conducted in the Information Science Master program
- Discussion on future research
- Presentation: 20 minutes
- Q&A: 10 minutes
-
Oh no, I haven’t prepared anything.
-
And there’s no progress on the image processing…
- Well, no progress means no progress.
-
①Kineto
-
I am Shota Aoyama, a 12th-year student at Gunma International Academy High School. I will be presenting my research on “Kineto,” which connects classroom experiences beyond time. I would like to express my gratitude to Professor Arase for mentoring me during the first stage, and to Professor Kataoka during the second stage.
-
This is my profile.
-
The term “time axis in VR” caught my interest, and this research focuses on the time axis of classroom experiences. I will be presenting my findings.
-
Up until now, in my school life from elementary to high school, I have been attending face-to-face classes or online classes like Zoom, and I didn’t feel any inconvenience. However, when I started watching lecture videos on YouTube for test preparation, I suddenly felt frustrated because I couldn’t do simple operations like pause, fast forward, or rewind during regular classes, which are easily done on YouTube. Being able to go back a bit to confirm something when I don’t understand or pause the video to take notes calmly can only be done in video classes. Of course, not all classes can be done on YouTube, as face-to-face or Zoom classes are necessary to maintain student interaction.
-
The combination of student interaction and time control seems impossible without a secret tool like Doraemon’s gadgets, but we achieved it with Kineto.
-
How did we achieve this combination? In traditional real-time classes, the class time was consolidated into one timeline. We separated this timeline and allowed free manipulation for each part, thus achieving the combination.
-
Specifically, here’s what we can do:
- Teachers can upload class videos to Kineto or livestream them.
- Students can share their thoughts and emotions with each other on Kineto’s board while attending class.
- Various interactive features such as sticky notes, handwriting pens, and voting functions are available, allowing students to communicate with their classmates on Kineto.
- Students can freely pause, fast forward, rewind, and perform other time-related operations while attending class.
- When rewinding the video, the sticky notes and written notes created by other students also rewind together.
-
I will now explain how this “blackboard with a time axis” achieves the combination of student interaction and time control.
-
First, let’s categorize the existing class environments based on their synchronicity. There are two types of classes:
- The first type includes face-to-face classes and Zoom classes, where students experience the same content synchronously. The second type includes platforms like YouTube and Nico Nico Douga.
- The advantages of face-to-face and Zoom classes are the rich student-to-student communication, while the disadvantage is the inability to control time asynchronously.
- On the other hand, YouTube and Nico Nico Douga allow time control but lack student-to-student communication.
- When comparing these, we can see that there is a trade-off relationship between the desire for rich student-to-student communication and the desire for asynchronous time control.- The extent to which these two aspects are required depends on the format of the class. Let’s take a closer look at when they are needed.
-
First, when it comes to communication between students, it is not necessary to have a classroom environment where students can communicate with each other if the class consists of students just listening to the teacher without any input. However, it is rare for this to be acceptable, and it is often beneficial for students to be able to share emotions such as laughter and surprise. Recently, there has been an increase in classes where students are actively participating, and in these classes, rich communication among students is desired, such as discussing questions.
- As for how traditional classroom environments cover these aspects, in real-life and Zoom classes, all forms of communication are possible. On the other hand, in YouTube videos, it is difficult to do anything other than passively listening to the teacher. With NicoNico Douga videos, it is possible to have more communication among students than YouTube, thanks to the pseudo-synchronous comment system. However, it is limited to sharing emotions like laughter and surprise, as well as simple sharing of opinions.
-
Next, the graph represents the extent to which time control is required in various types of classes.
- In classes where students are just passively listening, time control is not necessary. However, if students are actively engaged in the class, they may want to rewind a few seconds if they missed something, take a quick break, or pause the class to take notes.
- As for how existing classroom environments cover this aspect, it is obvious that Zoom and real-life classes do not allow for any time control. On the other hand, YouTube and NicoNico Douga videos allow for free playback, pause, and fast-forward.
-
By combining these two aspects, we can see that the current classroom environment covers the shaded area when we focus on synchronicity. There are specific examples of this, and I believe that innovative schools are effectively combining these aspects.
-
However, it also means that classes can only exist within this shaded area, and classes that require more than this cannot be realized. For example, it is not possible to have discussions among students while performing operations like playback, pause, or rewinding by ten seconds.
-
Kineto aims to fill this empty space, this uncharted territory, and expand the way classes are conducted.
-
As you may have seen in the first video, Kineto meets the need for time control. While it is not possible to control time according to one’s own pace in real-life or Zoom, it can be done in Kineto, as shown on the screen.
-
In order to provide students with a synchronous experience similar to Zoom, Kineto intentionally does not adopt a seek bar UI like YouTube, but instead implements a time control UI centered around the current position.
-
When we apply this to the previous graph, it is evident that Kineto can cover all aspects.
-
Furthermore, in Kineto, students can maintain control over time while also engaging in communication with each other. I will now explain the three features of Kineto that enable this: a shared sense of time, synchronized contextual communication, and bidirectional dialogue through elastic synchronization.- First, let’s talk about the sense of sharing time. In traditional online platforms like YouTube, you don’t get the feeling of attending a class together with others. This sense of shared time is important for motivating students to participate in the class, and it is considered a major drawback of on-demand classes. However, with Kineto, you can see the presence of other students through the comments they write, so you can still feel like you are sharing this time with others who are not actually synchronized with you. In addition, the resolution of time and the pace at which the content changes are important for this sense of shared time. In Kineto, the process of writing, whether it’s handwritten or on sticky notes, is shared in real-time.
-
Secondly, let’s talk about synchronized contextual communication. One of the drawbacks of asynchronous classes is the lack of “synchronized contextual communication.” This means that in platforms like YouTube, if you want to exchange opinions with other students, you need to use the comment section or other shared documents. However, if you do this, people will be at different points in the video, and there is a possibility of getting spoilers from others. This may not be a problem in meetings, but in the learning process of a class, it can deprive students of the opportunity to think for themselves. With Kineto, in the class experience, you can know what others thought and felt at that moment on the timeline. This allows for information exchange and communication without spoilers.
-
And finally, the third major feature of Kineto is the bidirectional dialogue enabled by elastic synchronization. In platforms like YouTube or NicoNico, you cannot have a bidirectional conversation with other students during the class. With Kineto, partial bidirectional dialogue is possible through the mechanism of elastic synchronization.
-
Elastic synchronization is a system that synchronizes users who are watching asynchronously through automatic changes in playback speed.
- In this graph, the horizontal axis represents real-time, the vertical axis represents the timeline of the video being watched, and the slope represents the playback speed.
- In traditional classes, you could only choose between always being synchronized or always being asynchronous (right figure).
- This is where elastic synchronization comes in.
-
I have explained the concept so far, but I understand that it may be difficult to imagine, so let me introduce a case study.
-
We have tried various types of classes in different positions on this graph, but this time, I would like to introduce a class in the top right corner, which is the most characteristic of Kineto.
-
It was a moral education class taught by Mr. Takamatsu from my school, with seven classmates participating.
-
This class was a discussion about the trolley problem, and it went like this…
-
After conducting this class, we received the following feedback in an interview:
- Read the feedback.
-
③Mastering Information Science: Image Processing Research
- Progress report for Information Science Master 202103 manuscript
- In addition to software development, I am also working on research on video analysis in education using computer vision.
- Regarding this, there hasn’t been much progress yet as it is still in progress, so let me explain the previous research and future directions.
- The goal is to evaluate the importance of each frame in the class video.
- In elastic synchronization as I explained earlier,
- Frames with low importance can be fast-forwarded by students without any problem, which means that these frames are considered less important.
- For this purpose, I believe that saliency can be used.
- Saliency refers to the distinctiveness of an object.
- An example of previous research that uses this in videos is UNISAL.
- Originally, I thought that it might not be able to handle special videos like class videos, so I first wanted to create a saliency evaluation model that can make this possible.- However, when I actually tried UNISAL, the evaluation was conducted more accurately than I had expected.
-
The salience of the recently written content on the blackboard has increased, making the salience of the parts that students would actually pay attention to higher.
-
Therefore, as a future direction, I would like to conduct research on evaluating the importance of frames based on this.
-
I presented this in March, but I haven’t been able to work on it due to school assignments and exams…
-
In addition, related to the previous content, I have recently been thinking about experiences involving a virtual timeline.
-
At the beginning of the presentation, Kineto mentioned that while phones and Zoom have eliminated the physical space constraint of needing to be in the same location to communicate, they are also considering if there is a way to eliminate the inconvenient constraints of physical time.
-
I am still at the stage of loosely considering it, but I would like to research the way timelines work in communication, the relationship between timelines and spatial axes in experiences, and interfaces that allow for flexible manipulation of time.
-
- Benefits of being an expert
-
It was great to receive support for research in areas such as image processing.
-
I learned about various fields through video lectures.
-
I think it was great to be able to learn about different approaches to research in the field of information science.
-
I originally decided on my research theme based on the development of Kineto, but during the development process, I thought about the challenges and things I wanted to do, and considered that they might be related to or connected to research in the field of XX.
-
That’s about it, I think I can get through with the past slides.