Man Re-Creates Body in VR with Oculus Rift, Three Kinect Sensors

Man Re-Creates Body in VR with Oculus Rift, Three Kinect Sensors

Telepresence, realized.

Oculus might be busy fending off a possible lawsuit, but that's not stopping its users from being some incredibly creative geeks.

Oliver Kreylos has constructed a telepresence system using an Oculus Rift, three Microsoft Kinect sensors, and a fair amount of coding know-how.

Video captured by the Kinect trio is fused together to create a mushy, but stable, 3D video of Kreylos.

"One of the things we've noticed since we started working with 3D video to create "holographic" avatars many years ago was that, even with low-res and low-quality 3D video, the resulting avatars just feel real," said Kreylos on his blog. "I believe it's related to the uncanny valley principle, in that fuzzy 3D video that moves in a very lifelike fashion is more believable to the brain than high-quality avatars that don't quite move right."

All the work done in the above video was achieved with Xbox 360-era Kinect sensors. If and when Kreylos upgrades to the latest Xbox One sensors, the 3D video quality should see a serious bump in quality.

The hardware used by Kreylos on the project? "It's run by a single Linux computer (Intel Core i7 @ 3.5 GHz, 8 GB RAM, Nvidia Geforce GTX 770), which receives raw depth and color image streams from three Kinects-for-Xbox, is connected to an external tracking server for head position and wand position and orientation, and drives an Oculus Rift and a secondary monoscopic view from the same viewpoint (exactly the view shown in the video) on the desktop display."

[Additional Source: Engadget]


That. Is. Really. Cool! O:

Though.. I'm not sure if the implications of what we could do with this tech is awe inspiring or disturbing.

If only they could render the person without the Oculus headset, this would be perfect. Still amazing what you can do with a Kinect or three.

Very, very cool.

The Kinect has fairly good potential for some fields, but it's such a shame that the one application it was made for - gaming - is where it just doesn't work well and people hate it as a result (rightfully so).

Well, to be fair - this application has some problems as well, such as the various seams created around his 3D model, as well as the low resolution it has when recreating the edges of a structure (his model's hands are especially messed up). But it's still an awesome application for an interesting device.

I'd like to see how Kinect 2.0 fares with this, but as far as I know, they didn't release an SDK for that yet. And considering the latest news, I'm not too sure how much support that thing will get.

The fact that I'm currently reading Otherland makes all those VR news that much more awesome.

I'm really looking forward to VR

You know, when Microsoft revealed the original Kinect, the very first thing I said to my friends and colleagues was, "I'll bet my next paycheck that Microsoft doesn't do a damn thing useful with that thing. It'll only show it's real potential when the home-brew/home-hack community gets a hold of it. Until then it'll be a useless, buggy, worthless gimmick

And what'ya know? Here we have yet another example that validates my claim.

Anyway, this is a pretty fantastic bit of home-brew modding. Certainly not the first time I've seen this kind of setup, but to see one so clear and low-latency, with what amounts to a pittance worth of hardware (all from off-the-shelf components), is damned impressive.


This is the reason that Facebook would be willing to pay $2B on the Rift. With better sensors and maybe something to render your face instead of the headset(maybe inward facing cameras?), we could be looking at a possible next step in social networking(or at least chat rooms). Imagine this with a better rendered model and an overlay to give you your own customized avatar(doesn't have to look like your real self) that responds 1:1 with your own movements.

As an added bonus, can you imagine what this could do for the point & click adventure genre?

Once again the kinect does good when it isn't hindered by an Xbox running shovel ware. Considering how game companies treat new interface hardware, I fear VR will be the same as kinect for a few years. A neat gimmick who's full pontenital will be unlocked in other fields.

I can see this making "pen n paper" roleplaying session quite interesting :P

I can see this ending very, very poorly.

Finally am impressed by this.

Ok, ramp it up by 3 (+2 on top) more cameras, double the resolution. See what can be done with the computer. Hm, Problem with the Occulus rift though. Maybe we can try and get the "beam the pictures on your retina directly" working. Could lower the weight and "looking stupid"-factor down to a google glass-level. Hm... or the contact lenses with see-through OLED's? (Which i can't wear because screw my sensitive eyes)

But this is going somewhere. Maybe as soon as 5 years and i might have something like this in my gaming room. Soon we will be living in the future... *g*

This looks really really good! Congrats on the dev.

With some better sensors and some more hands free technology (the cables would be terrifying) this would be a VERY interesting video conferencing software.


Reply to Thread

Posting on this forum is disabled.