MAD_KinectMasker tutorial

EDIT: this tutorial is now depreciated, as MadMapper supports QuartzComposer files natively.
We offer the original QC patch for free to anyone, get it from
MadMapper forum

In this tutorial, will use the newly released MAD_Lab applications
to plug a Kinect to your Mac and do so realtime masking… !

So first, launch MAD_KinectMasker application.
This app. uses the v002 OpenKinect plugin (bundled into the app) kindly provided by v002 team and the Syphon plugin.

You should see a grayscale depth image on the left, and a masked blue image on the right.
Adjust the tilt position of your Kinect, in order to see the whole scene.
Ideally, have someone stand in front of the unit to help set up everything.

Now that there’s someone in the frame, press the REC button. This wil turn the updating off, allowing your guinea-pig friend to take a break from standing in front of the camera, and also help you fiddle the settings.

Then, play with the min/max parameters, to actually select which depth of the image you want to extract…

Voila ! You’ve selected an interesting part, now launch another app to provide the texture you want under your mask. Here I’ve used MAD_GraphicPatterns from the MAD_Lab, but I could have launched Modul8 for instance, or any other Syphon-enabled application.

You’ll want to actually combine the output of both applications into one unique Syphon texture for use with MadMapper.
Here comes MAD_Doubler, which will just do this. So launch it !

Select your Kinect output on the left side, and the graphic patterns on the other side.

Now everythin is ready to be piped into MadMapper.
Launch it and select MAD_Doubler in the media list (under the Syphon section):

Make a bunch of Quads with an interesting texture. Here I’ve selected some diagonal lines moving around, applied on a Quad that I duplicated (cmd-D).
I like when it goes wild with a lot of surfaces. I’ve also set the blending mode to additive, and modified a few of the quad’s flipping option, to give some slight variations:

On top of all these quads, we’ll add the Kinect generated mask.
So make a new Quad and select the left part of your input. It is all black, but that’s normal, as your selecting a mask. It will be used to reveal the underlying quads.

Position the Kinect quad on the output, over all the previous textured quads.

Now you can ask you model friend to come back in front of the Kinect, switch back to MAD_KinectMasker, hit REC to enable the live feed again.

Come back to MadMapper and adjust the Kinect quad’s corner to fit the outline of your model.

Bravo ! You’re done.

Here’s a snap of the final output:

54 Responses to “MAD_KinectMasker tutorial”

  1. Awesome!

    Is it possible to save/record the mask to use again at a later date??

  2. [...] And the tutorial on how to kinect mask [...]

  3. How about the position of both: projector and Kinect?

  4. Hey People,
    i don’t know why but everytime i open the app my kinect freezes, meaby i could have installed ant QC plugins that could affect it??? i really want to test this, but it is imposible… it looks like the communication Mac/Kinect doesn’t exist, tilt, blur and all the parameters are working but the image is frozen, any tip???

    • Im in the same boat. My kinect works but not with the kinectmasker, it just freezes up. Been looking all over for someone to help, but its so new, there isn’t much info around. Really wanna see it work.

    • ok, it was something in my laptop… but now, the Kinect mask app doesn’t send the signal… i can’t see it in any syphom soft…

      • Yeh I got mine working as well. Reinstalled libfreenect and needed to do sudo make install for it to work with this. Its 6 am. I doubt I can get any sleep now.

      • Francois Wunschel Says:

        please try to re-download KinectMasker.
        There was a problem with the very early build, which was fixed very quickly. Maybe you’re one of the unlucky early-downloader.
        Sorry about it.

  5. [...] をっとと、そういえばMadMapper1.02と同時に出た小粋なアプリ群、MADLABに入っているMAD_KINECTMASKERを試してみよう。 をををっ!面白い。 あっさりと動画でマスクが作れてる。 ここいら辺のどうなっているかは、また後日にキャプチャーも交えて書くとしようっと。 Tweet You can leave a response, or trackback from your own site. [...]

  6. Where can I watch the video?

  7. [...] / spare wall spaces etc. The Madmapper folk have been releasing an inspiring set of very detailed tutorials too, as well as pretty useful [...]

  8. Technomorph Says:

    Wonderful….

    can i make few suggestions?

    1) Would love to have a “preview” button in the program so I
    can actually SEE the alpha, for aligning stuff in MadMapper.
    We work in the visual world and even though it’s there and
    working, would be great to temporarily VISUALIZE the Alpha.
    Then click and shut it off (something like the blue preview)

    2) Edge Blurring, rather than the straight up Blur. Some of the edges from the Kinect depths are a big jaggey, while the blur
    helps, when cranked to where I get the edge blur I like it seems
    to also blur out the alpha too.

    I found the Kinect Depth Select Patch here:

    http://kineme.net/forum/Applications/Compositions/KinectDepthSelectopenCL

    Does some great edge blurring

    3) As an alternative to the whole edge blurring and setting Min
    and Max depth levels (which you still always end up with unwanted material being captured such as the floor, ceiling, furniture etc)
    Have you considered tapping into the Kinect NITE ability to capture and DISPLAY USER function? It view the frame and once a human enters it it captures and masks and creates a full human in color you assign. Really clean and fast and no skeleton user initializtion has to be done. Up to 4 users can be detected and displayed.
    Many of the folks in Open Frameworks have been utilizing it.
    I tried to create my own app using it, integrating Syphon but have gotten stuck on integrating the two. Got the user app done, got Syphon App frame grab done. But can’t merge the two (my programming is not so hot).

    Here’s video demo:

    Here’s some Open Framework forum threads:

    http://forum.openframeworks.cc/index.php/topic,5510.0.html

    4) Would be awesome to be able to INVERT the alpha as with
    some of the work I do with some dancers I prefer to not project
    onto them, only around them.

    I figured out a work around by using some Quartz Filters.
    It took me a while to figure it out, but finally got something to work.
    Using 1st an INVERT patch and then a COLOR MATRIX patch.
    Setting all of the Alpha Parameters to -1 and then chaning the
    Bias Vector from 0 to 1.

    for any interested, you can download the QC file here:

    https://files.me.com/technomorph/8r1rzi

    It will then be a Syphon Server as “MADKinect INVERT”

    • Francois Wunschel Says:

      1- in MadMapper you can change the background color (to blue for instance) so you should be able to see the alpha.
      2- openCL is too buggy at the moment. I might find another blurring method. Will look into it.
      3- NITE is overkill for the moment.
      4- you’re right. I should do this. (using my own method tho’)

    • Colossus Says:

      Hey Technomorph, the link below doesn’t work anymore. Is there a different way to do the inversion now or something?

      https://files.me.com/technomorph/8r1rzi

  9. Fun with KinectMasker on the street. Kinect worked OK through a double window, as long as I blocked as much reflection as possible.

  10. Hi there! Thanks for the tutorial. I suppose the only way to get the MAD_KinectMasker application is by purchasing Mad Mapper?
    Is it accessible via the demo version to try out? I have Modul8 but Eur 299 is a lot of wedge to fork out…
    Looking for simpler ways to use the Kinect for visuals/mapping. Cheers for your huge efforts!

  11. Hello again!

    Is it possible to test the MAD_KinectMasker application via the Mad Mapper demo version please? Wanted to see how it works with moving objects/people dancing. Have heard that there are some latency issues although perhaps these are connected to the Kinect camera itself.

    Thanks for any feedback/advice…

  12. it’s not seeing the figure til they get really close (waist and above) is there any way to get the whole figure in? what about using a stronger ir lamp?

  13. ah so i read the infra red it sees is quite a narrow band…

  14. tornadoo Says:

    Is it possible to mask the black area with another video? i.e. the moving bodies mask video A, while the area outside the bodies masks video B.

  15. Guys, is it normal that my Kinect max view distance is 1.3m? Anything above this distance is invisible for the Masker. I wanna use it on an outdoor installation, but I can not ask people to come so close to Kinect. Any suggestions?

  16. Hello! I want to know if this will work with any kind of Kinect. Because i´m buying one and i don´t know whether to buy the Kinect for Xbox or the Kinect for Windows. I will be using it with a Mac. Thanks for your suggestions! Tian.-

    • Francois Wunschel Says:

      tested on a Kinect for Xbox.
      Perhaps borrow a unit from a friend and see how it goes
      (as users have reported problems when running off an intel GPU)

  17. please, please, please could you some ware uploud MAD_KinectMasker.app Guys from CarageCube Team removed MAD_Kinect from the downloads od mapmapper site. or email me THX guys

    • Francois Wunschel Says:

      hi Pavel,
      MAD_KinectMAsker was removed b/c it apparently didn’t work on every computer/kinect. It became too much of a burden for us to maintain, too many request for support, while it was just a little demo app, an experimental tool to add some extra power to MadMapper.
      But in order to keep the feature around, I’ve given away the QC patch, for people to tinker by their own. You can get it from here: http://tinyurl.com/aczzncm or directly from the MM forum: http://forum.garagecube.com/viewtopic.php?f=20&t=8532 .
      Once you get the patch, you should be able to drag an drop it into MadMapper – provided you installed the v002 kinect plugin – . Hope this helps, kind regards, frZ

  18. Decent MAD_KinectMasker tutorial 1024 Architecture Blog / MadMapper, Video Mapping, Quartz Composer plugins, whatever submit!
    Now i’m really captured through your proficiencies in composing. That brought to mind an article regarding pk auto I just read couple of hours previously. In any case, very nice piece… thanks for posting.

  19. Do I have to install any drivers or plugins (other than Syphon) before I use the MAD Kinect Masker on my Mac or I can just plug in the Kinect and start working?

    • Francois Wunschel Says:

      no install should be needed, however, depending on some computer configuration, MAD_KinectMasker might NOT work properly.
      Please download the QC patch instead, and see for yourself.

      • carolina Says:

        Thank you, it’s working. Is there a way to combine the Kinect Masker input with the Video Delay on Madmapper? Because I think the video delay only works with iSight.

      • Francois Wunschel Says:

        mmm… MAD_Doubler or your custom QC solution…

  20. carolina Says:

    One more thing, how do I apply the Kinect Mask quad? I can’t figure it out.

    • carolina Says:

      I created random quads and then selected the left side (which is black) and placed it over the quads but no body outline was shown. Mad Kinect Masker is working fine (the preview on Mad Doubler is also black), but I can’t get the mask to work on Madmapper. Am I doing something wrong? All I get are the quads but not inside the outline.

    • Francois Wunschel Says:

      put it over your content, using “over” blend mode … ?

      • carolina Says:

        Thank you, it’s working. Apparently the Kinect wasn’t always connected (it would stop suddenly) but I got another adaptor.

  21. Thomas Ludwin Says:

    Where i can find this Application?

    • Francois Wunschel Says:

      we removed it from yourspace, download the QC patch instead and load it into MadMapper directly…

      • jeremie Says:

        i’m sorry, i downloaded the QC patch and loaded it into MM, then I plugged the kinect, and nothing happens, I must be missing something…

  22. Rodrigo Franco Says:

    Does anybody have a custom QC patch that does the same as MAD_Doubler? I really need this!

    • technomorph Says:

      yes… but there are a few tweaks that it could use..
      as I don’t completely understand the “sizing” that goes on with QC… I was also looking for something that did a top/bottom doubling too…

  23. Hello there! I know this is kinda off topic but I was wondering if you knew
    where I could locate a captcha plugin for my comment form?
    I’m using the same blog platform as yours and I’m having
    difficulty finding one? Thanks a lot!

  24. I absolutely love your website.. Excellent colors & theme.
    Did you make this site yourself? Please reply back as I’m trying to create my own personal blog and want to find out where you got this from or what the theme is named. Thank you!

  25. I seldom leave responses, however i did a few searching and wound up here MAD_KinectMasker tutorial | 1024 Architecture Blog / MadMapper, Video Mapping, Quartz Composer plugins, whatever .
    … And I actually do have a few questions for you if it’s allright. Is it only me or does it look as if like some of the comments come across as if they are coming from brain dead visitors? :-P And, if you are posting at other sites, I’d like to follow everything fresh
    you have to post. Could you list of the complete urls of
    all your community sites like your Facebook page, twitter feed, or linkedin
    profile?

  26. thanks!
    Great!
    good!

  27. Hi! Someone in my Facebook group shared this site with us
    so I came to take a look. I’m definitely loving the information.

    I’m bookmarking and will be tweeting this to my followers!

    Outstanding blog and outstanding design.

  28. Hey would you mind letting me know which web host you’re using?
    I’ve loaded your blog in 3 different browsers and I must say this
    blog loads a lot quicker then most. Can you recommend a good web hosting provider at a
    fair price? Cheers, I appreciate it!

  29. Hey! Would you mind if I share your blog with my zynga group?

    There’s a lot of folks that I think would really enjoy your content.

    Please let me know. Many thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 693 other followers