Facebook announces Ego 4D first-person video dataset for AI training

Facebook Inc. Mark Zuckerberg, Chief Executive Officer and Founder of the House of Representatives, spoke at a House Energy and Commerce Commission hearing in Washington, DC, USA on Wednesday, April 11, 2018.

Andrew Haller | Bloomberg | Getty Images

Facebook announced Thursday a research project to collect 2,200 hours of first-person footage from around the world to train the next generation of artificial intelligence models.

This project is called Ego4D and may prove important to Facebook’s Reality Labs division. The department is working on a number of projects that can benefit from AI models trained using video footage shot from a human perspective. This includes smart glasses like Ray-Ban Stories, released last month on Facebook, and virtual reality, which Facebook has invested heavily since acquiring Oculus for $ 2 billion in 2014. It will be.

This video can teach artificial intelligence to understand or identify something in the real or virtual world that can be seen from a first-person perspective through glasses or an Oculus headset.

Facebook announced in November that it would release its Ego4D dataset to researchers.

“This release is an open dataset, a research agenda, and facilitates progress both internally and externally in academia. [allow] Facebook’s senior researcher, Kristen Grauman, told CNBC.

According to Grauman, datasets can be deployed in AI models used to train technologies such as robots to understand the world more quickly.

“Traditionally, robots learn by doing something in the world, or by showing how to do things literally on hand,” Glauman said. “From our own experience, there is an opening for them to learn from the video.”

Facebook and a consortium of 13 university partners relied on more than 700 participants from nine countries to capture first-person footage. According to Facebook, Ego4D has more than 20 times as long video as other datasets of this type.

Facebook’s university partners include Carnegie Mellon University in the United States, Bristol University in the United Kingdom, National University of Singapore, the University of Tokyo in Japan, and the Institute for International Information Technology in India.

The footage was shot in the United States, United Kingdom, Italy, India, Japan, Singapore and Saudi Arabia. Facebook said it hopes to expand the project to more countries, including Colombia and Rwanda.

“The important design decisions of this project are, in the first place, the leaders in this field, interested in these issues, willing to pursue them, and with geographical diversity. We wanted a partner to be with, “Grauman said.

Facebook Rayvan Story Glass

Sal Rodriguez | CNBC

The announcement of Ego4D will take place at an interesting time for Facebook.

The company has steadily stepped up its hardware efforts. Last month, we released our first smartglass, the $ 299 Ray-Ban Stories. And in July, Facebook announced the formation of a product team specializing in “Metaverse,” a concept that involves creating a digital world where multiple people can live at the same time.

But over the past month, Facebook has been hit by a barrage of news articles from a pile of internal investigations leaked by Franceshausen, a former Facebook product manager who became a whistleblower. Among the studies published was a slide showing that Instagram is harmful to the mental health of teenagers.

The footage was captured using off-the-shelf devices such as GoPro cameras and Vuzix smart glasses.

Facebook said it has instructed participants not to capture personally identifiable traits when collecting footage indoors to protect their privacy. This includes people’s faces, conversations, tattoos and jewelry. Facebook said it removed personally identifiable information from the video, blurring the faces of bystanders and vehicle license plate numbers. Audio has also been removed from many videos, the company said.

“Step 1 of all the university partners who did this video collection was a very focused and important process in developing the right collection policy,” Grauman said.

Facebook announces Ego 4D first-person video dataset for AI training

Source link Facebook announces Ego 4D first-person video dataset for AI training

Back to top button