Since its debut a few short weeks ago, Microsoft Corp.’s new Xbox One gaming console has jumped off store shelves at the rate of more than 100,000 per day. The pace of adoption has led to shortages in all of the 13 launch markets.
And now, less than two weeks before Christmas, there’s one more market that may be trying to get its hands on the new Xbox — the homeland security market.
While purchases by federal, state and local security agencies won’t be having a real impact on the commercial availability of the new Xbox One anytime soon, officials acknowledged in interviews with FedScoop unnamed federal agencies have been leveraging the Kinect 3-D motion-sensing, facial and voice recognition technologies that power the Xbox One for security purposes.
“They’re using it to look for threats,” said Greg Myers, vice president of Microsoft Federal. Although he mentioned improvised explosive devices, Myers declined to provide additional details or other specific examples, citing security and customer confidentiality concerns.
The Microsoft Technology Center in Reston, Va., offers a fully functional Xbox & Kinect technology demonstration space called “Home Land.” (Photo: Dan Verton-FedScoop)“It’s a gaming system, but it’s a development platform that we can do all kinds of things on,” said Joan Barrow, director of the Microsoft Technology Center in Reston, Va., one of 11 such centers around the country. Barrow gave FedScoop a tour of the new center, which includes a large space setup like a family room that features a new Xbox One system. Barrow also alluded to a wide range of non-gaming applications where the popular Xbox is quickly gaining traction.
“It’s great for gaming, but think about it in an operating room where a doctor needs to look at X-rays, but he’s already sterilized and he can’t touch it,” Barrow said. “That doctor can manipulate the image with his hands. In physical therapy, it can record movement progress and can tell me whether or not I’m making progress. And in war-gaming, we’ve used it with some of our DOD systems integrators to leverage this technology in simulations.”
Microsoft’s Kinect is the sensing technology that allows gamers to use their bodies and body motions as the game controller. The company re-engineered Kinect for the Xbox One to be more responsive and precise.
Information about the use of the Xbox’s Kinect motion-sensing and optical technologies is scarce in the U.S. But in Singapore, the government is promoting multiple proof-of-concept studies that show just how powerful the Xbox Kinect system is in a public safety and homeland security context.
One area in which the advanced sensors and infrared systems of Kinect have shown great promise is in detecting illegal border crossings, said Joan Liu, information systems officer for the Singapore Ministry of Home Affairs.
In a presentation posted on the Microsoft public safety and national security website, Liu offers a detailed look at the tests her government conducted using Kinect. In one example, Liu shows how a standard optical camera detects a black blob on the back of a motorcycle that looks like nothing more than a backpack in the final image. Using the Kinect depth sensors, however, evaluators were able to easily discern a human figure trying to blend in with the driver’s form in order to avoid detection.
In another example, Liu explains the challenge of seeing through heavily tinted glass on automobiles. Using a standard camera, no human figures can be seen. But the Kinect sensors can easily detect human figures behind the tinted glass.
One of the more surprising examples offered by Liu involved the ability of the Kinect sensors to detect the difference between a real face and a fake face printed on a sheet of paper.
“Traditional 2-D facial recognition systems can be easily spoofed using fake faces printed on paper,” Liu said. But testers in Singapore were able to prove Kinect’s depth sensor as a viable additional layer of security for customs and border protection. “Kinect detects very accurately whether a face is fake or not,” she added.
The enhanced fidelity and depth perception of the new Kinect sensor will allow developers tocreate apps that see a person’s form better, track objects with greater detail, and understand
voice commands in noisier settings. (Photo: Microsoft)
Just a few years ago, Microsoft shied away from granting third-party developers access to the Xbox and its technologies. The company didn’t want the technology used on anything other than an Xbox. Officials eventually relented, realizing the multitude of enterprise-level applications the technologies could benefit, from health care to search and rescue, and homeland security. Version 1.8 of the software development kit is now available to third-party developers.
The main hardware components of the Kinect system are a video camera, a monochrome sensor, an infrared projector/sensor and a multiarray microphone. Add to this some sophisticated software behind the scenes and the Kinect is able to determine an object’s depth (distance from camera to subject) to within 1 centimeter and its height and width to within 3 millimeters.
The new active infrared capabilities allow the new sensor to work in nearly any lighting condition. “This will offer developers better built-in recognition capabilities in different real-world settings—independent of the lighting conditions—including the sensor’s ability to recognize facial features, hand position and more,” wrote Bob Heddle, director of Kinect for Windows, in a blog posting in May.
“We find that problems with 2-D optical cameras can be mitigated using Kinect’s 3-D sensor,” Liu said. “This is just the beginning of what is possible.”
And there are, in fact, other areas of security where research and development into the potential uses of Kinect are likely, Liu said. One such area is perimeter protection using Kinect technology to produce virtual fences and leveraging its ability to obtain high-contrast images in low light conditions. Another is being able to detect dangerous actions, such as a person pointing a weapon.
Liu also sees a future for security personnel to be able to conduct “Minority Report” style manipulation of multiple video feeds, referring to the Hollywood blockbuster in which the title character, played by Tom Cruise, is able to control the programs on a large screen display using hand gestures.
The Xbox Kinect system has even found its way into the disaster response field. In 2011, student researchers from the Warwick University in the U.K. designed a rescue robot that leveraged the Xbox Kinect sensing technologies capable of producing 3-D images of an earthquake disaster site and scan for survivors.
The use of Xbox Kinect in the homeland security arena may come as a surprise to many, but not to Barrow. “We integrate many of our consumer products today with our more enterprise technologies,” he said. “And everything is on the same Windows 8 platform. If you think about how Xbox works, there’s just a lot of ways to use that technology that are not just fun and games.”
