A recently uncovered Apple patent filing shows the company has looked into using more advanced physics metaphors in a mulitouch environment to replace the traditional virtual desktop and user can make an intuitive, physical gesture with a first device, which can be detected by one or more onboard motion sensors. The plan to connect devices also brings in the motion sensors inside the iPad and iPhone, showing an iPhone being held over an iPad to pour icons from one to the other……….
Apple has been working on a concepts envisioned appear to be far more creative as they implement gesturing with Physics Metaphors. In one example, as noted in our cover graphic, you’ll be able to pour out files from one device to another in a manner that mimics that of pouring water. Apple lists a host of other examples that include sound effects. One such example includes an iPhone sucking up files from an iPad with the sound effect of a vacuum cleaner. While this newly proposed feature will certainly apply to all of Apple’s hardware, it will also eventually apply to other devices such as televisions, whiteboards, projectors and more. Whether this ends up being a spin-off of Apple‘s forthcoming AirDrop feature or not remains to be seen. For now, it’s Apple adding some badly needed personality to the boring process of file sharing. When an individual performs an action in a real world, physical environment, the individual experiences various physical phenomenon that indicates that the task is being performed or has been completed. For example, if an individual pours objects from a first container into a second container, the individual could observe the objects reacting to the forces of friction and gravity. If the objects having different shapes and masses, then the individual would observe different reactions to the forces.
Conventional personal computers include operating systems that often provide a virtual desktop metaphor where users could manipulate and organize various objects. This metaphor is easily understood by users because it is intuitive, and like the pouring act described above, relates to their real world, physical environment. Modern computing devices, such as smart phones, often provide a large variety of applications. Some of these applications, however, provide interfaces that lack an equivalent of the desktop metaphor and as a result are more difficult to comprehend by the average user. Apple‘s solution begins by stating that a user could make an intuitive, physical gesture with a first device, which could then be detected by one or more onboard motion sensors. The detected motion triggers an animation having a physics metaphor, where the object appears to react to forces in a real world, physical environment. The first device detects the presence of a second device and a communication link is established allowing a transfer of data represented by the object to the second device. During the transfer, the first device could animate the object to simulate the object leaving the first device and the second device could animate the object to simulate the object entering the second device.
In some implementations, in response to an intuitive, gesture made on a touch sensitive surface of a first device or by physically moving the device, an object could be transferred or broadcast to other devices or a network resource based on a direction, velocity or speed of the gesture. Particular embodiments of the subject matter described in this specification could be implemented to realize one or more of the following advantages:
- Users could transfer files and other data between devices using intuitive gestures combined with animation based on physics metaphors.
- Users could transfer files to a network using intuitive physical gestures
- Users could broadcast files and other data to other devices using intuitive interface or physical gestures.
Apple’s patent FIGS. 1A-1C note below illustrate an exemplary intuitive, gesture-based communication between two devices using animation based on a physics metaphor. The first example illustrates an iPhone transferring files to an iPad and of course the reverse could be true as well. The iPhone’s Interface is shown displaying a collection of graphical objects (114a-114f) which are in this case file icons representing stored files. In some implementations, the user could select one or more files for transfer to one or more devices by placing the files into a transfer state. In the example shown, the user has selected four files for transfer by touching their respective objects 114a-114d for a predetermined amount of time and/or using a predetermined amount of pressure during the touch. The user could also select a group of files for transfer by drawing a circle around the icons with a finger or stylus, then using a touch, gesture or other input to select the group of files for transfer. In some implementations, the user could drag and drop individual files onto a container object displayed on interface 112 and then use a touch, gesture or other input to select the container of files for transfer. Other means for selecting individual files or groups of files for transfer are also possible, including but not limited to selecting files through menus or other conventional user interface elements.
Apple’s patent FIG. 1B illustrates an iPhone (device 110) in motion relative to the iPad (device 120). Here, the user has rotated their iPhone above the iPad’s interface (122) in a manner similar to tipping a glass of water. This angular motion can be detected by one or more onboard motion sensors. As shown in FIG. 1B, detached objects 114a-114d could be animated to simulate the effect of gravity by sliding toward the lowermost portion of interface 112 as device 110 is rotated. The animation of the objects creates the appearance that the objects have mass and are reacting to forces of a real world, physical environment. In FIG. 1C, we see that the iPad could interpret the rotation of the iPhone as a pouring motion as an indication of the user’s intent to transfer the files represented by selected objects 114a-114d. Upon determining that the user of the iPhone intends to transfer the files, the iPhone (device 110) determines if the iPad (device 120) is present and available to receive the files. After a link is established between the two devices by Bluetooth or RFID and authenticated, the iPhone could request that the iPad accept a file transfer.
In some implementations, a file could be detached from the interface so that it appears to float on in the interface. In Patent FIG. 1C we see that the files are edging to a corner of the iPhone. The patent states that there could be an effect that the files are actually going down a drain or sand flowing through an hourglass or a Genie being pulled out of a bottle. It could even appear like the files are dissolving like a tablet in water which could be accompanied by sound effects like liquid pouring, a tablet fizzing, gas traveling through a valve and believe it or not, they even point to the sound of a teleporter playing. On the flipside, you don’t want all of your files falling out of your system and so Apple has dreamed up resistance fields when two or more devices come together. Apple‘s patent FIG. 2 shown below illustrates initiation of a communications session with a device in response to an interface gesture. For example, the user could touch graphical object 240 that states 5PM and then make a swipe or flick gesture on an iPad’s interface (250) with one or more fingers in the direction of other iOS devices in a conference room or class. The iPad senses the interface gesture input interacting with graphical object 240 and interprets the gesture as a request to transmit data represented by graphical object 240 to another device.
Before receiving the data transfer request, the iPad could scan the entire conference room or class for the presence of other available iOS devices. In this example, devices 220 and 230 are detected. If communication has not been established, the iPad could establish communication with other devices. The system, if all else fails, could be set up manually for transferring files. Apple’s patent FIG. 3 illustrates initiation of a data broadcast from a device to multiple devices in response to a physical gesture. Devices 310, 320, 325, 330 are located in proximity to each other. Devices 310, 320, 325, 330 could be a computer enabled display device, such as an electronic tablet, computer monitor, projection screen, electronic whiteboard, teleconferencing screen, television, or other type of device that could display information.
The user has selected graphical object 340 (a file icon) to indicate an intention to perform a data transfer action. Device 310 is also shown in a rotational or sweeping motion due to a user performing a clockwise (or counterclockwise) rotational or sweeping gesture that emulates a toss of a Frisbee. Motion sensors onboard device 310 senses this physical gesture and interprets it to indicate the user’s intent to broadcast data represented by graphical object 340 to devices 320, 320, 325, 330. Apple‘s patent FIG. 4 illustrates a data transfer between two devices in response to intuitive, physical gestures. In the example shown, the user intends to request that data corresponding to icon 436 be transferred from an iPad (device 420) to an iPhone (device 410). The user indicates an intent to transfer data by placing the iPhone in position and orientation (440a) relative to the iPad (device 420) and then moving the iPhone across display of the iPad. In some implementations, the gesture just described could be performed with a metaphor for the user holding and using the iPhone as a vacuum sucking up the files.
[ttjad keyword=”iphone”]
