Recordings of 3D environments are the focus of this document. The concept of “4D UI” is introduced, referencing the paper titled “Who Put That There? Temporal Navigation of Spatial Recordings by Direct Manipulation.” The paper explores manipulating the arrangement of objects within a search cube to serve as a search query, specifically for similar objects.
The document highlights the interesting aspect of analyzing the results of these searches, including studying the characteristics of the user interface and examining what objects were interacted with.
Two other papers are mentioned: “Challenges and Design Considerations for Multimodal Asynchronous Collaboration in VR” and “Causality-preserving Asynchronous Reality.”
The document emphasizes the difference between searching and navigating timelines of recordings, stating that while both tasks involve replaying design recordings to understand rationale or learn new workflows, they are distinct tasks.
The paper “Spacetime” is referenced, which proposes using WiMs (Wearable Interactive Mirrors) as containers for collaborative editing and navigation of 3D scenes. However, this technique does not allow for searching within or across containers. In contrast, the document proposes using WiMs as a centralized input canvas for users to express their search input via stage-to-search, enabling the retrieval of interesting moments in spatial design recordings. Once a moment is retrieved as a spatial clip, it can be played back using existing WiM-based playback techniques.
The document concludes by mentioning the unique challenges of querying spatial design recordings.