Gliss'N'Chill is an interactive and immersive experiment developed by Ahmed Kabbej and me, with design contributions from Nathan Limbos, Baptiste Gaudron, and Sofiane Hedaji.
Gliss'N'Chill was developed during a workshop titled "Movement and Sounds." The project aimed to create an immersive experience that responds to movement and sound.
Gliss'N'Chill transports you to a serene forest environment surrounded by seven wisteria flowers. By moving your body, you can interact with these flowers to create music. Moving your right hand triggers different notes, with each flower representing a note in a Pentatonic scale, allowing you to compose unique melodies through movement.
Demonstration
To create this project, we needed to track the user's hand position in real-time. We achieved this using the user's webcam video feed combined with the Google MediaPipe API. This part of the code captures the position of the user's right hand in every frame.
Another crucial aspect of the development was the physics simulation of the flowers. We used the Matter.js library to create and simulate complex objects with constraints and realistic physical behavior. This part of the code retrieves the user's hand position from the tracker, moves the circle, and calculates interactions with the flowers, considering speed and direction to create a sensation of gently touching the flowers.
We then needed to detect collisions between the circle and the flowers. When a flower is touched, a corresponding note is emitted, allowing the user to create melodies.
To fully immerse the user, we created a complex sound environment. We mixed forest sounds, including wind and nature noises. Additionally, we composed a calm and relaxing musical background in the same scale as the flower notes to complement the user's melody.
Our project is available online ! You can try it on desktop right now by clicking here