Eduardo Castelló Ferrer

DJESTHESIA uses tangible interaction to craft real-time audiovisual multimedia, blending sound, visuals, and gestures into a unified live performance. The project supports four interaction modes: I) Knob changes music, where standard DJing is performed. II) Music changes visuals, where changes in the audio parameters done through the mixer have a direct impact in the visualizations representing the music (e.g., color palette). III) Gesture changes visuals, where gestures and body movements give the possibility to interact physically with the visual representation of the music (e.g., grab, release, throw). IV) Gesture changes music, where, gestures can convey information to an audio composition software to alter aspects of the music being played (e.g., EQs). The aim of DJESTHESIA is to transform the DJ into both a performer and a performance.

Tangible User Interfaces (TUI) for human--computer interaction (HCI) provide the user with physical representations of digital information with the aim to overcome the limitations of screen-based interfaces. Although many compelling demonstrations of TUIs exist in the literature, there is a lack of research on TUIs intended for daily two-handed tasks and processes, such as cooking. In response to this gap, we propose SPICE (Smart Projection Interface for Cooking Enhancement). SPICE investigates TUIs in a kitchen setting, aiming to transform the recipe following experience from simply text-based to tangibly interactive. SPICE uses a tracking system, an agent-based simulation software, and vision large language models to create and interpret a kitchen environment where recipe information is projected directly onto the cooking surface. We conducted comparative usability and a validation studies of SPICE, with 30 participants. The results show that participants using SPICE completed the recipe with far less stops and in a substantially shorter time. Despite this, participants self-reported negligible change in feelings of difficulty, which is a direction for future research. Overall, the SPICE project demonstrates the potential of using TUIs to improve everyday activities, paving the way for future research in HCI and new computing interfaces.

Urban mobility can often be categorized as a complex system—e.g., a nonlinear system composed of many interacting components with interdependent relationships. The growing trend towards shared, lightweight, and autonomous vehicles requires planning solutions that are less centralized and can manage the increasing complexities of new mobility. This research investigates planning strategies for shared micro-mobility systems, focusing on shared autonomous bicycles. Vehicle rebalancing within such systems poses a critical technical challenge and has substantial environmental and economic implications. To tackle this challenge, we propose a fully decentralized approach that allows autonomous bicycles to rebalance in a self-organizing manner via stigmergy, a bio-inspired mechanism for indirect communication. While the bicycles autonomously navigate their urban environment, they locally update RFID tags at intersections, leaving virtual pheromone trails that collectively guide each other toward high-demand areas. The efficacy of our approach is assessed through a realistic agent-based model of Cambridge, MA (USA). Results highlight the capacity of autonomous bicycles to rebalance in a self-organized manner, using strictly decentralized local communication, while significantly reducing the average user wait time compared to no rebalancing and random rebalancing. These findings emphasize the feasibility and potential of decentralized planning strategies in handling complexity within new mobility systems.

The physical autonomy of robots is well understood both theoretically and practically. By contrast, there is almost no research exploring their potential economic autonomy. In this paper, we present the first economically autonomous robot-a robot able to produce marketable goods while having full control over the use of its generated income. Gaka-chu (“painter” in Japanese) is a 6-axis robot arm that creates paintings of Japanese characters from an autoselected keyword. By using a blockchain-based smart contract, Gaka-chu can autonomously list a painting it made for sale in an online auction. In this transaction, the robot interacts with the human bidders as a peer not as a tool. Using the blockchain-based smart contract, Gaka-chu can then use its income from selling paintings to replenish its resources by autonomously ordering materials from an online art shop. We built the Gaka-chu prototype with an Ethereum-based smart contract and ran a 6-month long experiment, during which the robot created and sold four paintings, simultaneously using its income to purchase supplies and repay initial investors. In this work, we present the results of the experiments conducted and discuss the implications of economically autonomous robots.

Robotics systems of all types are revolutionizing a wide variety of industries—transportation, manufacturing, and even healthcare—and yet, many essential ingredients for robotics systems in the real world are not technologically ready for deployment. Currently, robots lack the protocols and standards required to be safe and secure outside factories. In an attempt to close this gap, recent research has demonstrated the security benefits of combining robotics systems with blockchain-based and related technologies (e.g., smart contracts, zero-knowledge proofs). In this article, I argue that blockchain-based robotics is starting to provide innovative solutions to urgent problems of robot security. I list the most important takeaways so far from this field that I helped establish together with a growing community. I close the article by discussing the implications of the security challenges that the robotics research community is facing, and possible ways for us to move forward.

As autonomous robots are becoming more widespread, more attention is being paid to the security of robotic operations. Autonomous robots can be seen as cyber–physical systems: they can operate in virtual, physical, and human realms. Therefore, securing the operations of autonomous robots requires not only securing their data (e.g., sensor inputs and mission instructions) but securing their interactions with their environment. There is currently a deficiency of methods that would allow robots to securely ensure their sensors and actuators are operating correctly without external feedback.