Paparmali 4 – AR SuperHeroe Outfit using Kinect 2 Body Tracking
This project can be used only from within the Unity3D Editor (Play mode). Exporting as an exe file it has some pink shader issues.
Originally the project was built using:
- Unity3D 2018.2.x version
- Kinect 2 plugin (from Unity Asset Store) – Body tracking information was used for outfit augmentations.
- Execution file provided to test out the project on PC. Make sure you have installed Kinect 2 drivers:
- * Kinect for Windows SDK v2.0 (Windows-only) can be found here.
- * MS Speech Platform Runtime v11 can be downloaded here. Please install both x86 and x64 versions, to be on the safe side.
- * Kinect for Windows SDK 2.0 language packs can be downloaded here.
Support is TOP priority for this project. If you find any bug/weird behavior of the app – let us know and we will fix it as soon as possible. If Unity3D versions evolved and for some reason the project doesn’t work properly anymore – let us know and we will provide the fix to work on the newest Unity3D version you’re working on. At the moment this project should work from Unity3D 2018.2.x version and above.
For any questions please write to: email@example.com
AR solution purpose
This project is a perfect AR Body Tracking solution for a “Virtual Dressing Room” to augment different types of outfits. Speech recognition can be used to trigger different functions in the same manner as triggering UI element with red symbol on the tracked users’ hand. You can trigger next/previous outfits with a smooth effect transition, turn on a cast effect on your hand or whole body (at random). Also, it is possible to change weight/size of the outfit. It is a perfect fit for the events to attract lots of people to you and your brand. This solution works for one user that should stand in the middle of the screen.
AR Solution consists of 13 different outfits that may appear on top of users body.
Once the solution is started user has 2 options:
Option 1. User can trigger functions with hands by pointing the red symbol to a specific button:
- “Weight” button – increases or decreases outfit size.
- “Snapshot” button – takes a screenshot after seconds once the button is triggered.
- “Cast Effect” button – lightning/glowing effect appears on hand (only 1 type of effect).
- “Body Effect” button – a random effect appears on body (1 from 11 possible different effects).
- “Next”/”Previous” buttons – changes outfit with a smooth transition effect.
All interactions are visible once the buttons are in a touch zone – it increases and a loader appears. Checkmark indicates that it is selected/applied or absence of checkmark that it was deselected/removed.
Option 2. User can trigger each functions with speech recognition + additional feature to remove or add different outfit parts. Voice commands that are recognized:
- “Next” / “Previous” – the same as in point a).
- “Effect” – turns on or off body effect as described in point a).
- “Weapon” – turns on or off cast effect as described in point a).
- “Weight” – the same as in point a).
- “Snapshot” – the same as in point a).
- “All on” or “Reset” – turns on all outfit parts.
- “All off” – removes all outfit parts (outfit is removed), but effects may still be applied.
- “Arms on” / “Arms off” – turns on/off all the assets (gloves, gountlets etc.) from shoulder to fingertips on both arms.
- “Legs on” / “Legs off” – turns on/off all the assets from the stomach to toes on both legs.
- “Helmet on” / “Helmet off” – turns on/off helmets, hats, masks etc depending on the outfit style user has chosen
- “Chest on” / “Chest off” – turns on/off chest armor.
- “Body on” / “Body off” – turns on/off mostly clothing(s) underneath the armor of the whole body outfit.
Note: speech recognition can be expanded in “SpeechGrammar.grxml” file. How the logic recognized speech is recognized is described below in the scripts section.