• Nice to meet you, hello! ­čÖî

  • image

  • Bluemo @blu3mo / Shutaro Aoyama

    • I was under the care of Professor Inami during the ÔÇťŠť¬ŔŞĆ2020ÔÇŁ program.
    • I am a sophomore at Columbia University, studying Computer Science and Philosophy.
    • I had my coming-of-age ceremony the day before yesterday.
      • image
  • What will I talk about?

    • Introducing specific development and research projects.
    • Connecting them with abstract stories.
  • Why?

    • I want to receive feedback.
      • ItÔÇÖs okay to interrupt me during the discussion.
    • I would be happy if the members of Inami Lab remember me.
  • It is closer to a casual format than a research presentation, but I would be delighted if you find it interesting.

  • 2020 IPAŠť¬ŔŞĆ (Inami PM)

    • Online lecture environment balancing real-time and on-demand advantages.
      • image
      • Advantages of ÔÇťreal-timeÔÇŁ: student interaction.
      • Advantages of ÔÇťon-demandÔÇŁ: optimizing speed through time control.
    • Key point: ÔÇťElastic SynchronizationÔÇŁ that distorts time imperceptibly.
      • image
  • Columbia CGUI Lab (Prof. Steven Feiner)

    • An interface where moving objects in a VR space indirectly control factory robots.
    • Paper to be presented at IEEE VR 2024.
    • Video: https://drive.google.com/file/d/1GTalVXEei-yV65Anmlx_149NEIdSOjbw/view?usp=sharing
    • Key point: Asynchronous Human-Robot Interaction.
      • The robot executes actions in VR with a delay after the user in the VR world.
      • Misalignment between ÔÇťVR presentÔÇŁ and ÔÇťreal-world present.ÔÇŁ
      • Two proposed interfaces for control speed, preferences, etc.
  • Hobby Development

    • image
    • Side story: I created this project as an escape just before the final exam of a relativity class.
      • image
      • Please donÔÇÖt ask about the exam results.
    • GitHub: https://github.com/Godlysk/SRVR
  • image

    • ÔÇťMonitoring your past self as if they were someone else.ÔÇŁ
    • ÔÇťBeing monitored by your future self as if they were someone else.ÔÇŁ
    • Q: How far apart in time makes someone else?
    • Demo: https://time-mirror.vercel.app/
  • ÔćĹÔćĹÔćĹ Specific development and research discussions

  • ÔćôÔćôÔćô Abstract discussions

  • Something I have been thinking about for the past two years: By considering ÔÇťvirtual time,ÔÇŁ Removing the constraints of physical time

    • Western modern time concept
      • Train schedules, school timetables, factory schedules.
      • ÔÇťAppointments,ÔÇŁ ÔÇťDeadlines.ÔÇŁ
      • Synchronous economic transactions
        • Synchronized to microseconds.
      • ÔÇťThis socially constructed artificial time has come to encompass everything, and as a result, it seems as if it is time itself, as if there were no other time.ÔÇŁ

        • Time is thought to be ÔÇťone-dimensional,ÔÇŁ ÔÇťobjectively shared globally.ÔÇŁ
    • In the reality shared through computers (Computer-mediated reality), it is possible to remove the assumption that ÔÇťeveryone shares the same structured time.ÔÇŁ
    • Without that assumption, various structures of time are conceivable.
      • Circular time, branching and merging time, etc.
      • Sharing time with others in the past => Kineto (online classes).
      • Sharing time with your past self => Task delivery to your future self.
      • Structures of time and space with discrepancies => Relativity theory VR.
      • Sharing reality asynchronously between humans and robots => Factory robot control research.
    • => Removing the constraints of physical time- Recent Insight: This discussion can be expanded beyond the ÔÇťstructure of time.ÔÇŁ
    • The timeline is just one dimension for understanding reality.
    • In general, it is possible to consider a mechanism where subjective elements of various realities such as time, space, language, personality, sensory information, etc., are not shared by everyone, creating an ÔÇťobjective reality.ÔÇŁ
  • Questioning the dichotomy of Subjective <----------------------------------> Objective

    • Far left (Subjective Reality): Individual VR games,
      • Everyone shares different realities.
      • The red pill from The Matrix (individually optimized simulated world).
    • Far right (Objective Reality): Traditional reality, Cluster, The blue pill from The Matrix
      • Everyone shares the same reality.
      • The blue pill from The Matrix (objective reality).
    • Can we consider something between these two?
  • For example, a VR restaurant

    • You see a Chinese restaurant, while your companion sees a French one.
    • If you can hear each other and see avatars, the experience of ÔÇťdining togetherÔÇŁ can be shared.
  • For example, space

    • For instance, considering a scenario where multiple people collaborate in a VR space.
      • The room contains building blocks and monitors.
      • The top image represents the original reality.
        • However, the red/blue/green individuals see slightly different, individually customized realities.
        • The world seen by the red person (bottom left):
          • Avatars and monitors of others are placed in different positions from the original.
          • Customization intent: wanted to rearrange for better monitor visibility.
        • The world seen by the blue person (bottom center):
          • Objects on the table are larger than in the original.
          • Customization intent: wanted to see details of the objects.
        • The world seen by the green person (bottom right):
          • Avatars and monitors of others are transparent.
          • Customization intent: wanted to focus on the objects on the table.
      • Thus, even in collaborative spaces, in VR, one can experience different realities customized individually.
      • However, such individual customization of reality can lead to communication discrepancies due to ÔÇťmisalignmentÔÇŁ in reality.
        • Issues like ÔÇťan object I see is not visible to the other personÔÇŁ or ÔÇťan object IÔÇÖm pointing at appears in a different position for the other personÔÇŁ can arise.
      • To resolve this: adjust other parts of reality to absorb the ÔÇťmisalignmentÔÇŁ effectively.
        • If the object you are pointing at appears in a different position for the other person, display the pointing finger at a different position for them.
    • Demo of a 2D space: https://asym2d.vercel.app/
  • Implementation:

    • Sharing a Set of Various SituationsÔÇÖ Necessities Instead of Sharing a Lump of Objective Facts in Asymmetric Reality
      • Traditional objective VR space
        • The server has a shared database of spatial positions (x, y, z) of various items.
          • Table: (0,1,0), apple: (0,2,0), User AÔÇÖs avatar: (1,1,1), User BÔÇÖs avatar: (-1,1,1), and so on.
          • Data represents the one correct reality.
          • Data is complete = contains all information about the world.
        • Each userÔÇÖs client constructs the same reality based on this shared database.
      • Asymmetric Reality that shares a ÔÇťset of necessities of various situationsÔÇŁ
        • The server has a shared database of constraints on spatial positions (x, y, z) of various items.
          • Constraints vary based on the situation or context.
          • Data is incomplete = multiple possible worlds that satisfy the constraints exist.
        • Example: When shaking hands: Shared Constraints | Constraint | Importance | | ÔÇö | ÔÇö | | Relative Angle of User A and BÔÇÖs Gaze | 90% | | Relative Position of Avatars of User A and B | 70% | | Relative Position of Avatars and Room Interior | 30% | | Size and Height of the Room | 10% | Individual Local Constraints | User | Constraint | | ÔÇö | ÔÇö | | A | Due to the small room, wants to display compactly | | B | Due to the spacious room, wants to utilize space to display information widely |
  • Each userÔÇÖs client constructs a different reality based on shared constraints and individual local constraints.

  • Why does ÔÇťmisalignmentÔÇŁ occur?

    • Balancing individual optimization and shared reality
      • Examples of common structures in previous discussions
      • Examples of individual optimization: restaurant meal preferences, playback position and speed of classes, VR content display
      • Examples of shared reality: eating together, synchronous communication in classes, collaborative VR work
    • Alternatively
      • When multiple people share reality, there are things they ÔÇťtruly want to shareÔÇŁ and things they may not
      • By creating a certain ÔÇťmisalignment,ÔÇŁ it can absorb another ÔÇťmisalignmentÔÇŁ
      • Example: Real-time simultaneous interpretation between Japanese and English speakers. There may be a ÔÇťmisalignmentÔÇŁ in the auditory information, but the essential shared content can still be communicated
  • Research

    • Q: How can we differentiate what to share and what not to share?
      • Creating ÔÇťmisalignmentsÔÇŁ based on context, situation, and local constraints
    • Q: How to share what we want to share while maintaining beneficial ÔÇťmisalignmentsÔÇŁ?
  • To generalize this discussion a bit more:

I thought we should aim for ÔÇťversatility of constraintsÔÇŁ rather than ÔÇťfreedom from constraints.ÔÇŁ People might struggle if there are infinite choices without constraints, so having the ability to control constraints on choices from a meta perspective would be valuable (Seems applicable to sports rules, smartphone screen time, VR gravity, etc.) @(blu3mo) October 22, 2023