Qooqle is a mobile based system that frees us from the confines of search boxes by integrating digital information directly into our everyday activities. In combining mobile, cloud computing, and social search, Qooqle aims to make information more accessible and devices less visible.
Demo upon request at the MIT Media Lab.
MatchMaker is an automated collaborative filtering system that recommends friends to people on Facebook by analyzing and matching people's online profile with the profiles of TV characters. The goal of MatchMaker is to produce friend recommendations with rich contextual information through collaborative filtering in the existing social network. Using relationships in TV programs as a parallel comparison matrix, MatchMaker projects these relationships into reality to help people find friends whose personality and characteristics have been voted to suit them well by their social network. MatchMaker also encourages more TV content viewing by using the social network context and connections to provoke people's curiosity of TV characters whom they have been matched with in their social network.
In today's world, the digital information retrieval experience is inherently a sparse device-centric activity. Users rely on the ability of the currently used device to supply the requested information, in some disconnection from past activities on other devices. There is a growing need to develop new methods of connecting cross-context information retrieval sessions. PalimPost is a converged system for storing, searching, and sharing digital and physical world information using sticky notes and mobile devices. PalimPost extracts contextual cues from a user's physical environment and activities, and connects them to the user's digital world research. Subsequently, the system presents systematically categorized information that is relevant to the moment of interaction in a just-in-time manner.
PalimPost uses physical sticky notes with embedded QR codes, as well as virtual sticky notes on mobile devices. The system incorporates Automatic Speech Recognition (ASR), Optical Character Recognition (OCR), and Natural Language Processing (NLP) techniques for understanding and categorizing the content.
The above describes the second prototype of PalimPost, video coming soon. Meanwhile, you can view the first prototype here.
Second World is a platform that allows children to expand their imagination, create their own virtual world, and compose stories with hand gestures which mimic the shapes of various animals. In children's world, a block can be a mountain; a pencil can be a tree. Various living creatures can be created through different hand gestures, in the same way that we used to play puppy games with hand shadows on the wall.
The Second World system includes a shapematching algorithm which, upon receiving input images from camera live stream, identifies the skeleton of the images and compares the skeleton against the structures inside a 3D image database.
PlanTodo is a way to store and orchestrate important activities in your life. You pick up a meaningful leaf or flower that will represent the task you want to remember and hold the piece in front of PlanTodo while saying the task out-loud. The task is then recorded and you can retrieve it anytime by holding the leaf or flower in front of PlantTodo for playing back. PlantTodo can also send you reminder when the task has a deadline. Build your work day by building your plant. PlantTodo is the place to leave tasks and let ideas blossom.
PlanTodo was implemented in Processing with Minim, TUIO and Arduino libraries. It has a camera for recognizing fiducial markers on the leaves and flowers and a microphone for recording tasks. All the leaf and flower pieces have embedded magnet so that they can be easily attached onto the plant. A special “eraser” piece is used together with a task piece in order to delete an old task from the leaf or flower.
OpenGest is an initiative by me that aims to enable people to communicate with one another using natural gestures through an interface based upon computer vision and gesture recognition techniques. With the OpenGest interface, anyone having access to the internet and with a webcam can input what he/she believes are the natural gestures and their corresponding meanings into the OpenGest library; at the same time, everyone can use the interface to compose emails, send SMS, control software applications with the collective data stored in OpenGest library.
Current stage of OpenGest: I have designed and implemented the first offline version prototype of OpenGest. I plan to publish the online version of OpenGest in the near future, after a couple more offline prototyping and testing.
Video filmed during CIID projects exhibition.
In accordance with the 2009 Climate Change Conference at Copenhagen, my teammates and I designed and implemented SharedSkin, a shoulder-worn suit with solar cells attached on its surface, which enables individual humans to collect and store solar energy during their daily routine. The solar energy can then be used for individual purposes such as charging one’s electronic devices. More importantly, the energy can be shared with and transferred to friends through the friendly gesture of hugging and patting on one another’s back. When more than two people hold hands together, the energy can also be aggregated to power larger structures or machines.
SharedSkin was implemented with Arduino, along with embedded rechargeable batteries, RGB LED lights, etc.
The project was presented in the Climate Change Conference. More photo documentation.
Texk is an application that makes writing a letter more playful and personal, and creates a new way to express/communicate one’s feelings with text. It allows users to focus on writing, does not contain disturbing menus, has minimal yet sufficient formatting tools, reflects users’ writing style and is playful through its simultaneous and constant response. Texk gives a personal touch to writing a letter in the digital world. Instead of replacing the traditional writing style with pen and paper, Texk cultivates a playful and creative environment for a new way of letter writing by utilizing the power of computers.
Project Presentation Slides and Video.
Unburdened Sleep Pillow is a trustworthy pillow that records and stores a user’s bed time thoughts as the user is falling asleep, thereby unloading the user’s mind at sleeping time, and organizes the thoughts and reminds user of the messages the next morning.
The design of the interaction starts with people’s familiarity and intimacy with common bedroom objects such as a regular pillow, and tries to enhance the affordance of the pillow by making it more interactive and to allow the pillow to function as a confidant to the user at bed time. The design also focuses on using metaphors to make intangible information more tangible and completing a user’s journey from night to morning with the switch of location of the user’s mind burden.
Messenger Dog is an informal messaging system specifically designed for disaster zones such as Haiti right after the earthquake, when all network infrastructures are down. Messenger dogs can go through the destroyed paths around the city to collect and send messages among displaced family members, and deliver information to rescue camps where the information will be downloaded for further utilization by the rescue experts. A messenger dog wears a suit with embedded iPhone, which changes from sleep mode to app mode upon changing its orientation whenever the dog sits down waiting for a person to interact with it. The application allows the person to speak or type the message recipient’s name and to record a video message. It also gathers the geo-location information of displaced people, through the messenger dog’s journey, to help rescue experts restore order in the city.
Project presentation slides,video, and press release.
BlindGuardian is a scarf designed for helping blind people walk around the street just like normal people. It enables blind people to get rid of their wooden stick and still be aware of the surroundings through tactile contact with their clothing.
The scarf has an embedded ultrasonic sensor for measuring the distance and speed of approaching obstacles and a small servo motor which vibrates at increasing speed in order to alert the user of the approaching obstacles.
If you are a frequent visitor to Chinese news websites, you probably know about Sina.com or Sohu.com. While these popular sites provide readers large amount of information, the layout and interaction design of the websites can certainly throw readers off if they are not frequent visitors or if they cannot read Chinese ten lines at a glance.
With that in mind, I created a news website, Sinsina.com, which displays news items through only pictures. The philosophy behind it is that pictures are a more direct tool for conveying messages than Chinese text, which is a visual language in abstract form. With pictures, readers can actually “read Chinese ten lines at a glance.” The website gets RSS feeds from various sources and thus is constantly updating.
If interested, please visit www.sinsina.com.
Despite their popularity, traditional methods for teaching a second language contain many unsatisfactory features such as “flash card” approaches to vocabulary acquisition, complex grammar rule memorization, and a little direct application to the learner’s life. To remedy these issues, we present Touchstone, an adaptive language environment that integrates into the user’s home and helps them learn a second language as they go about their daily life. Through the use of activity recognition and common sense reasoning, Touchstone avoids the use of native language and provides knowledge in an immersive and natural manner.
This project was created by Ned Burns and Henry Lieberman of MIT Media Lab; I have been part of the research team so far in this ongoing project.
Never before has there been a time when humans are so thirsty for information that mobile computers are thriving prosperously and big display screens are crowding the street. But as Mark Weiser has put it, “The truly profound technologies are those that disappear,” I made my first humble attempt of designing news fountains around MIT campus. Having noticed the extraordinary amount of water fountains inside Stata Center, the convenience of wireless connection, and the good habit of students’ drinking water constantly, I designed a simple system that required only a computer to retrieve RSS news feeds and a projector to project news headlines onto the flowing water.
Just as water is for the body, information is for the mind. Just as water quenches our thirst, information satisfies our curiosity. At the same time, both water and information has the properties of scarcity, circulation, and taking the shape of their container or medium. These compelling common properties have led me design the News Fountains to show how we can merge the physical world and digital world in a natural way.
Project presentation slides.
Inspired by sunflowers in nature, I designed and built a flower that could track the sun or the direction with greatest light intensity in any environment. This was the final project I did in MIT’s Microcontroller Programming class to demonstrate the ability of individual design and fast prototyping skills, after learning Intel 8051 programming within short time. The major components of my Sun Flower include a light sensing unit with four photo resistors, a rotating unit with two step motors that enable the flower to move toward any direction on a sphere, and ADC chips that help my software program communicate between the analog inputs and digital outputs.
Beyond the project model, I have been thinking of applications in solar panels. As we enter the energy conscious age and as the auto industry competes to make energy efficient cars—such as the next generation Toyota Prius that would rely solely on solar power—it is important to increase the efficiency of capturing solar power. The Sun Flower project has helped me move one step closer toward designing future projects and applications that focus on using green energy efficiently.
Video, in which I built the flower again with Arduino at CIID.
The vast improvement in classical methods of information access and information retrieval has resulted from the invention of keyword-based search mechanism. Keywords serve as filters for desired information when users search in structured knowledge bases or with search engines such as Yahoo! or Google. TaPuMa is a Tangible Public Map, which allows people to use their own belongings, the objects they usually carry with them to access relevant and just-in-time information and locations of places from a public map. The project also explores and analyzes the advantages and challenges of this novel interaction mechanism, where real life objects serve as the interface for information acquirement. The broad concept behind the project TaPuMa is ‘Object Amelioration’, where the functions of everyday objects can be expanded by using their affordances or functionalities in a variety of different contexts.
Tangram is an ancient Chinese moving piece puzzle. In the old days, people used to play it with paper cuts. Nowadays numerous software tools are available to the public for playing this game online.
Tangible Tangram aims to bring back the puzzle’s tangibility and increase its interactivity with people. With Tangible Tangram, people will be able to play in the same way that they used to play with paper cuts. Assessment of correct shape being formed is enabled through Computer Vision techniques. When the correct shape is formed, the whole Tangram will give visual and audio feedback with embedded LED lights and inbuilt audio systems.
Overall, Tangible Tangram represents my effort of bringing the ancient, interesting art crafts into the modern world by adding a little bit more intelligence to them and interactivity between them and people.
GambleFlower enables people to play a simple guessing game in Arduino, with one person controlling the button on one side of the "grass" and the other person guesses the result by touching the flower. Correct answer(s) are indicated with the flower lighting up.
The research was done in the Gynaecology and Maternity Department of Hillerød Hospital, Denmark. The design concept, Baby Book, is a guide to - and a record of - the pregnancy journey, pulling together many of the different experiences of both pregnancy and delivery into one coherent process. The mother can gather all documents, both medical and personal, from the pregnancy period in one place, whilst staying informed about specific stages within the process. At the delivery stage, this information also gives the midwife an overview of the mother’s medical and emotional journey to date. This personal guide serves as a memoir of the experience for mother, child and family, both in immediate reflection and many years down the line.
Motivated by Google’s 10th Year Anniversary Ideas Competition, I designed an environment friendly trashcan system that rewards people with money for throwing both recyclable and non-recyclable trash into the trashcan.
Trash can... is specifically designed for developing countries such as India, based upon my six-month experience of living and traveling in both rural and urban areas of India. I was astonished by the pollution along the streets, witnessed people throwing big water bottles out of car window when driving in the highway, and was shocked by hundreds of beggars wandering around unemployed, begging for only 1 or 2 Rupees with hungry stomach and without clothes. While improvement in education definitely can help solving all these problems in the long run, I believe that an instantly rewarding system is more practical and suitable for the extremely money conscious society in a dire condition of environment pollution and unemployment.
Trash can... functions similarly to recycling machines in the US technically, except that it also gives people money for ordinary trash. With recyclable trash, Trash can... gives Rs.2 at a time; with ordinary trash, it gives Rs.1 at a time (Rs. 40 = $ 1). With Rs.10, a beggar can easily buy 2 or 3 fried egg sandwich along the street. With the amount of trash along the street currently, a beggar can easily get Rs.50 in a single day. The total money needed to realize such a system, however, is much smaller than that the Indian government has spent in hiring people with at least Rs.2000 monthly salary per worker.
User-Centered Design approach toward product development has gathered increasing amount of attention in the IT industry in recent years. Having a set of pure technical skills is no longer sufficient for the successful design and implementation of any useful and usable application. With the importance of User-Centered Design in mind, in my senior year of college at MIT, I did a research project titled The Effect of an MIT Engineering Education on User-Centered Design Skills. The aim of this project was to find out how well an education in MIT Electrical Engineering and Computer Science Undergraduate Department prepares its students for the necessary design and implementation skills that the industry demands upon students’ graduation. The project also aimed to help MIT EECS undergraduate students recognize the importance of User-Centered Design by revealing to them the disconnection between what they are designing and what the real world needs and wants.
The project was voted the No.1 winning final project in our management class.
You can view the project presentation slides here