Remember Project Tango? Google’s 3D-sensing smartphone project started development three years ago and first became public in 2014. The goal was to equip a smartphone with a depth sensor and other computer vision abilities. Why? To see what developers could come up with. It’s a bit like strapping an Xbox Kinect to the back of a smartphone.
Now, Tango is finally out in a consumer device, but this is not the triumphant launch of a new flagship product. After years of buildup, Tango is quietly debuting on a mid-range Lenovo phone, the Phab 2 Pro. The bloated, ugly hardware Lenovo created for the Phab 2 feels more like a barely designed developer kit than a slick consumer product.
Even Google doesn’t seem interested in the product. Contrary to most Google initiatives, Tango didn’t get a Google-led launch event or press outreach. The most the company could muster was a blog post.
Ars still cares about Project Tango, though. So let’s see what fun we can have with a 3D-sensing smartphone. While the hardware is a non-starter, the technology shows promise.
Let’s not waste much time on the actual smartphone parts. Just know that the Phab 2 Pro is a disaster. First of all, it’s gigantic—the Phab 2 Pro is the Canyonero of smartphones. The combination of a 6.4-inch screen and huge bezels means this “phone” is 7 inches tall. It weighs over half a pound—double the weight of an iPhone 7—and it’s so thick Lenovo doesn’t list an official depth on its website.
The as-basic-as-possible rectangular design with minimally rounded corners contributes to the “ridiculously big” vibe the phone gives off. Compared to other really big phones like the Motorola Nexus 6 and the Xiaomi Mi Mix, the Phab 2 Pro feels like Lenovo didn’t even try to make it look nice or compact. The Mi Mix is a great comparison since both phones have a 6.4-inch screen. Xiaomi tried to slim down and made a 158.8mm (6.25 inches) tall device, while Lenovo’s design is a whopping 21mm (0.83 inches) taller.
Much of the Phab’s girth is presumably to make room for the hefty 4050 mAh battery. Even that heifer can only run a Tango app for about an hour before draining to zero. Don’t let the size of the battery fool you: the Phab 2 Pro is grossly underpowered. For some reason, the heavy 3D processing of Tango has been paired with a mid-range Snapdragon 652 processor. One of the coolest Tango apps, a 3D scanner, quickly bogs down to a few FPS thanks to the anemic processor. Tango, especially the scanner app, is a product that easily pushes the limits of top-of-the-line processors. The Snapdragon 652 doesn’t stand a chance.
The Phab 2 Pro also feels old for an end-of-2016 device. I don’t just mean the weight and massive bezels—the phone still has a Micro USB port on the bottom. The entire smartphone industry moved to USB Type C this year. Even low-end, sub-£180 devices are using the reversible plug. We’re sure a decent amount of money went into the extra Tango sensors, but for £500 we expect an overall better package. The P2P’s body is aluminium, at least.
Lenovo mostly stayed away from the software. Sure, there is a custom (and easily replaceable) home screen and custom app icons, but recent apps and the settings are intact. The one thing Lenovo touched was the notification panel (and by extension, the lock screen notifications). And wow, did Lenovo ruin it.
Notifications that pull from Lenovo’s custom UI don’t have the usual solid white background. Instead, they’re directly stuck onto a frosty transparent panel. This makes app notifications basically go crazy, with some picking white text, some picking black text, and some picking gray text. This chromatic free-for-all leads to all sorts of readability issues.
Things are further complicated by how apps don’t use the built-in notification style. Instead, they roll their own. That AccuWeather notification (which came with the phone) would look right at home on a device that follows Material Design, but it’s a hot mess next to the rest of the notifications. I don’t think this transparent design is against the Android compatibility docs, but it needs to be.
How Tango works
On the back of the device you’ll find a tower of sensors. Some of them are just the usual smartphone parts, and some are the special sauce that makes Tango work. Let’s start at the bottom: just above the fingerprint reader is a huge motion sensor camera that gives off a HAL 9000 vibe, and above that is a rear pinhole microphone. At the bottom of the top sensor cluster is a standard dual-tone LED flash, followed by the depth detection setup: an infrared emitter and a time-of-flight depth camera. At the very top is the regular 16MP RGB camera.
With the extra sensors, Tango can do three things for developers: motion tracking, area learning, and depth perception.
- Motion tracking allows the device to understand its motion as it moves through an area. This happens in full “3D” with six degrees of freedom. Not only can you tilt and spin, but you can walk around or kneel down and you’ll still be tracked. Most developers use this mode to make the phone a “portal to another world” that works like a VR headset without any of the stereoscopic 3D or immersion. Motion tracking compares the images from the big fish-eye camera against movement data from the “inertial measurement unit” (a fancy combo of gyroscopes and accelerometers). The camera is used for “feature tracking”—visually identifying features like edges and corners as they move from frame to frame. Google’s Tango stack can do this 100 times a second, if the hardware can keep up.
- Depth perception allows Tango to see in 3D. Tango OEMs have a few options for doing this, but the Phab 2 Pro uses a Time of Flight (ToF) camera and an IR projector. The process works like LIDAR—tiny points of light are sent out of the IR projector and reflected into the ToF camera, which calculates the distance via the travel time. Depth perception apps can make a 3D point cloud of an area, which is neat in and of itself but also good for augmented reality apps that want to place something in 3D space.
- Area learning combines motion tracking and depth perception to create a map of the area and identify your location within that map. Once a baseline scan has been completed, area learning works as a super-precise indoor GPS. You can stick a virtual object on a shelf, walk away from it, and it will stay there. Or—if someone ever creates an indoor map of something—area learning could work as an indoor navigation system. Tango saves these area scans in an Area Description File (ADF), which can be shared across apps.
That’s basically it. Google provides this extra baseline of sensors and leaves figuring out what to do with them to the apps. Now that your phone can “see in 3D,” what can you do with it?
Matterport Scene, a 3D scanner app
Matterport Scene is easily the coolest Tango app, and it gives us an idea of how Tango sees the world. It makes a 3D point cloud of whatever you point it at and saves that image in a viewable, explorable 3D scene. An app like Matterport Scene is impossible on any other smartphone.
All the Tango sensors go full blast with Matterport Scene. The time-of-flight camera and IR projector constantly take 3D measurements of the world in front of them. They blast out points of light and measure how long they take to bounce back. This system doesn’t see color, only depth, so the RGB camera is active, too. It applies some color to the 3D point cloud. You’ll want to scan an object from multiple angles, so your movements are tracked by the fish-eye camera, which allows every new depth pass to be correctly applied to the growing point cloud.
Once you’re done scanning, Matterport Scene saves your 3D scans to an internal gallery that’s easy to navigate. A one-finger drag rotates the view, pinch-zoom zooms, and a two-finger drag pans. The app also has nifty presets like “3/4 overhead,” “First person,” and “Blueprint.” You can measure your scans with a built-in ruler, there’s a 3D crop tool, and the “share” button can send the full 3D data to another Matterport user. You can also snap a picture to send to everyone else.
Listing image by Ron Amadeo