MAD_KinectMasker tutorial

In this tutorial, will use the newly released MAD_Lab applications
to plug a Kinect to your Mac and do so realtime masking… !

So first, launch MAD_KinectMasker application.
This app. uses the v002 OpenKinect plugin (bundled into the app) kindly provided by v002 team and the Syphon plugin.

You should see a grayscale depth image on the left, and a masked blue image on the right.
Adjust the tilt position of your Kinect, in order to see the whole scene.
Ideally, have someone stand in front of the unit to help set up everything.

Now that there’s someone in the frame, press the REC button. This wil turn the updating off, allowing your guinea-pig friend to take a break from standing in front of the camera, and also help you fiddle the settings.

Then, play with the min/max parameters, to actually select which depth of the image you want to extract…

Voila ! You’ve selected an interesting part, now launch another app to provide the texture you want under your mask. Here I’ve used MAD_GraphicPatterns from the MAD_Lab, but I could have launched Modul8 for instance, or any other Syphon-enabled application.

You’ll want to actually combine the output of both applications into one unique Syphon texture for use with MadMapper.
Here comes MAD_Doubler, which will just do this. So launch it !

Select your Kinect output on the left side, and the graphic patterns on the other side.

Now everythin is ready to be piped into MadMapper.
Launch it and select MAD_Doubler in the media list (under the Syphon section):

Make a bunch of Quads with an interesting texture. Here I’ve selected some diagonal lines moving around, applied on a Quad that I duplicated (cmd-D).
I like when it goes wild with a lot of surfaces. I’ve also set the blending mode to additive, and modified a few of the quad’s flipping option, to give some slight variations:

On top of all these quads, we’ll add the Kinect generated mask.
So make a new Quad and select the left part of your input. It is all black, but that’s normal, as your selecting a mask. It will be used to reveal the underlying quads.

Position the Kinect quad on the output, over all the previous textured quads.

Now you can ask you model friend to come back in front of the Kinect, switch back to MAD_KinectMasker, hit REC to enable the live feed again.

Come back to MadMapper and adjust the Kinect quad’s corner to fit the outline of your model.

Bravo ! You’re done.

Here’s a snap of the final output:

24 Responses to “MAD_KinectMasker tutorial”

  1. Awesome!

    Is it possible to save/record the mask to use again at a later date??

  2. [...] And the tutorial on how to kinect mask [...]

  3. How about the position of both: projector and Kinect?

  4. Hey People,
    i don’t know why but everytime i open the app my kinect freezes, meaby i could have installed ant QC plugins that could affect it??? i really want to test this, but it is imposible… it looks like the communication Mac/Kinect doesn’t exist, tilt, blur and all the parameters are working but the image is frozen, any tip???

    • Im in the same boat. My kinect works but not with the kinectmasker, it just freezes up. Been looking all over for someone to help, but its so new, there isn’t much info around. Really wanna see it work.

    • ok, it was something in my laptop… but now, the Kinect mask app doesn’t send the signal… i can’t see it in any syphom soft…

      • Yeh I got mine working as well. Reinstalled libfreenect and needed to do sudo make install for it to work with this. Its 6 am. I doubt I can get any sleep now.

      • Francois Wunschel Says:

        please try to re-download KinectMasker.
        There was a problem with the very early build, which was fixed very quickly. Maybe you’re one of the unlucky early-downloader.
        Sorry about it.

  5. [...] をっとと、そういえばMadMapper1.02と同時に出た小粋なアプリ群、MADLABに入っているMAD_KINECTMASKERを試してみよう。 をををっ!面白い。 あっさりと動画でマスクが作れてる。 ここいら辺のどうなっているかは、また後日にキャプチャーも交えて書くとしようっと。 Tweet You can leave a response, or trackback from your own site. [...]

  6. Where can I watch the video?

  7. [...] / spare wall spaces etc. The Madmapper folk have been releasing an inspiring set of very detailed tutorials too, as well as pretty useful [...]

  8. Technomorph Says:

    Wonderful….

    can i make few suggestions?

    1) Would love to have a “preview” button in the program so I
    can actually SEE the alpha, for aligning stuff in MadMapper.
    We work in the visual world and even though it’s there and
    working, would be great to temporarily VISUALIZE the Alpha.
    Then click and shut it off (something like the blue preview)

    2) Edge Blurring, rather than the straight up Blur. Some of the edges from the Kinect depths are a big jaggey, while the blur
    helps, when cranked to where I get the edge blur I like it seems
    to also blur out the alpha too.

    I found the Kinect Depth Select Patch here:
    http://kineme.net/forum/Applications/Compositions/KinectDepthSelectopenCL
    Does some great edge blurring

    3) As an alternative to the whole edge blurring and setting Min
    and Max depth levels (which you still always end up with unwanted material being captured such as the floor, ceiling, furniture etc)
    Have you considered tapping into the Kinect NITE ability to capture and DISPLAY USER function? It view the frame and once a human enters it it captures and masks and creates a full human in color you assign. Really clean and fast and no skeleton user initializtion has to be done. Up to 4 users can be detected and displayed.
    Many of the folks in Open Frameworks have been utilizing it.
    I tried to create my own app using it, integrating Syphon but have gotten stuck on integrating the two. Got the user app done, got Syphon App frame grab done. But can’t merge the two (my programming is not so hot).

    Here’s video demo:
    http://vimeo.com/20323281

    Here’s some Open Framework forum threads:
    http://forum.openframeworks.cc/index.php/topic,5510.0.html

    4) Would be awesome to be able to INVERT the alpha as with
    some of the work I do with some dancers I prefer to not project
    onto them, only around them.

    I figured out a work around by using some Quartz Filters.
    It took me a while to figure it out, but finally got something to work.
    Using 1st an INVERT patch and then a COLOR MATRIX patch.
    Setting all of the Alpha Parameters to -1 and then chaning the
    Bias Vector from 0 to 1.

    for any interested, you can download the QC file here:
    https://files.me.com/technomorph/8r1rzi

    It will then be a Syphon Server as “MADKinect INVERT”

    • Francois Wunschel Says:

      1- in MadMapper you can change the background color (to blue for instance) so you should be able to see the alpha.
      2- openCL is too buggy at the moment. I might find another blurring method. Will look into it.
      3- NITE is overkill for the moment.
      4- you’re right. I should do this. (using my own method tho’)

  9. Fun with KinectMasker on the street. Kinect worked OK through a double window, as long as I blocked as much reflection as possible.

    http://vimeo.com/30431615

  10. Hi there! Thanks for the tutorial. I suppose the only way to get the MAD_KinectMasker application is by purchasing Mad Mapper?
    Is it accessible via the demo version to try out? I have Modul8 but Eur 299 is a lot of wedge to fork out…
    Looking for simpler ways to use the Kinect for visuals/mapping. Cheers for your huge efforts!

  11. Hello again!

    Is it possible to test the MAD_KinectMasker application via the Mad Mapper demo version please? Wanted to see how it works with moving objects/people dancing. Have heard that there are some latency issues although perhaps these are connected to the Kinect camera itself.

    Thanks for any feedback/advice…

  12. it’s not seeing the figure til they get really close (waist and above) is there any way to get the whole figure in? what about using a stronger ir lamp?

  13. ah so i read the infra red it sees is quite a narrow band…

  14. tornadoo Says:

    Is it possible to mask the black area with another video? i.e. the moving bodies mask video A, while the area outside the bodies masks video B.

  15. Guys, is it normal that my Kinect max view distance is 1.3m? Anything above this distance is invisible for the Masker. I wanna use it on an outdoor installation, but I can not ask people to come so close to Kinect. Any suggestions?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 183 other followers