Kinect. It's all Happening!

In my new Year predictions which I am expecting to get totally wrong I actually got something right already!

Someone has managed to hook up a Kinect to their PC and use it in Second Life to control their avatar movements and gestures.

Those same guys from the University of Southern California, Institute for Creative Technologies, have pulled it off again so it's sure to surface in an Opensim world before long. I understand there is a two second delay though which might make it a bit laggy for combat. Still,those who need exercise might soon have great way to work out while chating to and dancing with their friends in their favourite virtual world.

Two video clips to follow; the first Second Life but in the next video clip a little more magic takes the form of 3D rendering and animation using 3ds Max software. A Chinese hacker has used the OpenNI drivers to get Kinect to translate the motions of the user. This works by detecting the position of body and hands to control the animated avatar on screen. Anyone with 3ds Max will be able, at low cost, to easily generate lifelike gestures to use with their avatar in an Opensim world.

The future is defiantly on the march!

FAAST. Open source control software for Kinect

WOW! No I don't mean World of Warcraft I really mean WOW!

Following Pathfinder's lead, which he gave in my Predictions for 2011 comments I came upon a hack for the Microsoft Kinect sensor from  researcher, Even Suma of the University of South California. Working with his team at the school's Institute of Creative Technologies they have managed to put a middleware program together based on the open source framework tool, OpenNI that exploits Kinect's sensor to recognize the player's skeleton and translate body gestures, movements and voice into keyboard commands. This lets World of Warcraft players plug Kinect into their PC USB and directly interact with the game. The software package is called FAAST (Flexible Action and Articulated Skeleton Toolkit) and released as open source so anyone, they say, can downloaded it and use with WoW or modify to work with other applications.

Given that FAAST is open source anyone working with virtual worlds like Opensim should be able to play with the functions to get an avatar to sit, walk, run, turn about and activate a selection of gestures. I am sure a lot more might be possible too such as bringing up the inventory menu and clicking to wear an item of clothing while at the same time the avatar is seen, apparently, dressing as the result of an appropriate gesture getting activated (I have always wished for this since clothes simply appearing seems to me very un-life like).

Currently, the number of gestures available to WoW players is limited but the researchers aim to add more. In the video the team released (seen here), Suma uses his body to walk, his left hand to move the camera and his right hand to select attack spells. I can imagine in a virtual world a gun or sword can be taken in hand and used in combat too by those users more inclined to action adventure but the possible application for interactive social, medical and education use would benefit tremulously. As Skip Rizzo, said in the video, the system could help get young people out of their chairs doing more exercise. In deed, rehabilitation exercises and the fight against childhood obesity and diabetes could all be helped with this system.

I make no secret that want to see a free and open metaverse of hypergrid-connected virtual worlds come about where we, our avatars, can live some precious moments as extra-sentient beings playing, socializing, working and making money and, above all, being able to marvel at the creativity of others and enjoy the fantastic worlds we create. I think Kinect could open up new and exciting possibilities for virtual worlds and facilitate interaction in ever more life-like ways. I imagine too that a few more years ahead Haptics will come into play too and we will be meeting and shaking hands no matter where on Earth we live.

Imagine being separated by thousands of miles from work colleagues or those you love and yet being able to walk towards their hologram while they walk towards yours. You meet and shake hands or just hold the hand of your wife or husband, and by the miracle of technology you actually feel the touch. How amazing would that be?

Predicting the Metaverse in 2011

Everyone who blogs about virtual worlds seems to make predictions for the year ahead at this time so, not to be out done, I though I too would have a stab at it if only to look back next January and see how much I got wrong.

Gaga arrives at the Museum on Heritage Key grid

First off I want to mention that Justin Clark-Casey, a core Opensim dev, predicted last year that Open Simulator would not come out of Alpha in 2010. That's right it didn't happen but I have to say, despite it's continuing quirkiness it certainly seems a lot better and more stable than a year ago. It's frustrating when you consider how long Opensim has been in Alpha and I think the Chicken & Egg policy for submitting patches may have held it back more than anything else. I say that because the Imprudence viewer devs have decided to team up with Aurora now (a separate development of Opensim) when they had decided to work closer with Opensim in response to the Linden Labs TPV policy. They switched to Aurora and will develop the   Kokua viewer after being frustrated with core Opensim policy so I read.

So I would now guess that Opensim will finally go to Beta in the 3rd or 4th quarter of 2011, maybe (I'm playing it safe!).

Gaga at Stonehenge on Heritage Key grid

My 2nd prediction is that Hypergrid will see more growth than proprietary grids over all especially if HG 2 arrives with a means for content creators to block their creations leaving the grid in which it is sold. I have though about this a lot and it seems to me that if sales are tracked on the sellers own server they could provide outlets on trusted grids for purchasers to receive what they paid for free of charge on their home grid and, since HG 1.5, avatars  appearance is called from their home grid and not downloaded to the grid one visits this to my mind would be the kind of content protection that might just work. Certainly, objects like furniture wont be travelling to outside worlds anyway if the seller dose not allow it.

I have read that many HG-enabled grids claim up to 50% of their traffic is arriving via HG teleports so I predict substantial growth in Hypergrid travel in 2011 generally.

My 3rd prediction is that more developers will pull out of Second Life TPVs and turn their attention to Opensim, either getting involved with the official core developers (especially if they change their 'Chicken & Egg' six month rule of not looking at the code), or new teams setting their own goals and branching off. I think some of the laid-off Linden employees might end up contributing too. The  Aurora team is an example and they have set an impressive list of objectives which I am happy to say includes improving the Opensim physics which, according to one core developer, they never considered it their place to do any better with -  not that I want to knock them, I really don't. I think the whole Open Metaverse community owes a great deal of gratitude to them for their hard unpaid work over the years, without which we might not have Opensim today.

Gaga gets to see the Terracotter army on Heritage Key grid

My 4th prediction is that Second Life will continue to loose market share in the early part of 2011 due in part to the doubling of tier charges to none-profits and education but this will be offset by opening the main grid to Teens - a timely move. However, I suspect Linden Labs will alienate more of their older users by this move especially if the Lab starts bullying them about adult content still lingering in the Mature sector. I suspect many sim owners will find themselves between the devil and the deep blue sea when it comes to handling the visitors to their Sims. There will be fear about age and constant worry about nudity and public cybersex (actually, all sex is public really since one can cam in and listen through walls so LL has kind of made a nonsense of what Mature means). Probably many sim owners will either go to full Adult or shut down and be done with it and, possibly, move to Opensim where they are not subjected to looney Linden's policy.

My 5th prediction is that Second Life will recover considerably in the mid year onwards as Mesh comes to the grid and Viewer 2 and V2 TPVs get wider adoption but I don't think any of this will save Second Life from a very slow long-term decline. Simply because Opensim and other platforms will be advancing and seriously giving Linden Labs a run for their money.

My 6th prediction concerns browser based portals into virtual worlds. I am confident that some serious breakthroughs are on the cards. I have looked at a number of these and visited some of them. KataSpace from  Katalabs  which is built on WebGL and HTML 5. This can run on Firefox without a plugin. It works but still rather primitive. 3Di  was, however, the first to port Opensim through a web page back in 2008 and the company is currently re-designing it to take advantage of Unity3d. Canvas   is the latest to begin testing and based on technology licensed from IBM. Next up is  Rezzable's  Unity3d based portal and the one that impressed me the most (see pictures). I went to the page and opened the portal window. I found it smooth, and though it was merely a waltz down a corridor introducing me to Heritage Key's  Opensim-based virtual museum, I was tempted to register and see the world. I loved it!

Then there was Friends Hangout  yet another Unity3d based portal on a web page which I got to from Evolver  where I made the animated avatar image seen on these pages. Apart from these efforts Blue Mars and Linden Labs are experimenting with Cloud to stream the virtual world to the web page but this is the more expensive route so the end-user is likely to have to pay for it. My prediction then is that anything based on Unity3d are the ones to watch and I think it might be safe to predict that by the second quarter of 2011 there will be a usable Unity3d web-based portal available for Opensim from the open source community.

My 7th prediction concerns Blue Mars from Avatar Reality. This impressive virtual world, with it's bigger regions and Mesh-based objects and avatars really shows what a future virtual environment should look like but I can't say I am convinced it will get mass adoption, not in the short term anyway. The viewer is a heavy download and good graphics card and high-end computer are a must. Then you have to download each city in turn before you can visit. If the owner makes changes then you have to download again. The developers are well aware of the download problem though and, as mentioned above, they are working on a cloud streaming to deliver the experience on a web page. I think BM has a long term future if their capital holds out. I predict major improvements through 2011 if it survives.

My 8th prediction is that Kinect will come to virtual worlds during 2011 although I hesitate to put any firm date on it. What happened was someone created drivers  for a PC that allows devices to interface with the Kinect for Xbox 360. This lead to an Austrian company that makes hardcore porn videos showing how the controller can be used in interactive sex games. There demo shows a female avatar being grouped on screen. It's kind of spooky but it shows the shape of things to come and were we are heading.

Next up will be Haptics! What, you don't know what Haptics means? Think of the Holodeck in Star trek. Yes, using ultra sound to touch and feel virtual holograms. The mind boggles!