MAD_KinectMasker tutorial
EDIT: this tutorial is now depreciated, as MadMapper supports QuartzComposer files natively.
We offer the original QC patch for free to anyone, get it from
MadMapper forum
In this tutorial, will use the newly released MAD_Lab applications
to plug a Kinect to your Mac and do so realtime masking… !
So first, launch MAD_KinectMasker application.
This app. uses the v002 OpenKinect plugin (bundled into the app) kindly provided by v002 team and the Syphon plugin.
You should see a grayscale depth image on the left, and a masked blue image on the right.
Adjust the tilt position of your Kinect, in order to see the whole scene.
Ideally, have someone stand in front of the unit to help set up everything.
Now that there’s someone in the frame, press the REC button. This wil turn the updating off, allowing your guinea-pig friend to take a break from standing in front of the camera, and also help you fiddle the settings.
Then, play with the min/max parameters, to actually select which depth of the image you want to extract…
Voila ! You’ve selected an interesting part, now launch another app to provide the texture you want under your mask. Here I’ve used MAD_GraphicPatterns from the MAD_Lab, but I could have launched Modul8 for instance, or any other Syphon-enabled application.
You’ll want to actually combine the output of both applications into one unique Syphon texture for use with MadMapper.
Here comes MAD_Doubler, which will just do this. So launch it !
Select your Kinect output on the left side, and the graphic patterns on the other side.
Now everythin is ready to be piped into MadMapper.
Launch it and select MAD_Doubler in the media list (under the Syphon section):
Make a bunch of Quads with an interesting texture. Here I’ve selected some diagonal lines moving around, applied on a Quad that I duplicated (cmd-D).
I like when it goes wild with a lot of surfaces. I’ve also set the blending mode to additive, and modified a few of the quad’s flipping option, to give some slight variations:
On top of all these quads, we’ll add the Kinect generated mask.
So make a new Quad and select the left part of your input. It is all black, but that’s normal, as your selecting a mask. It will be used to reveal the underlying quads.
Position the Kinect quad on the output, over all the previous textured quads.
Now you can ask you model friend to come back in front of the Kinect, switch back to MAD_KinectMasker, hit REC to enable the live feed again.
Come back to MadMapper and adjust the Kinect quad’s corner to fit the outline of your model.
Bravo ! You’re done.
Here’s a snap of the final output:
August 9, 2011 at 15:34
Awesome!
Is it possible to save/record the mask to use again at a later date??
August 9, 2011 at 16:22
You would have to use the SyphonRecorder.
Check the Syphon website.
August 9, 2011 at 19:30
[…] And the tutorial on how to kinect mask […]
August 10, 2011 at 23:13
How about the position of both: projector and Kinect?
August 22, 2011 at 19:10
Kinect on the ground, projector at 1m. high.
Both is the same axis if possible.
October 1, 2012 at 01:10
is there any difference if I use a short-thorw projector? Im using an Acer u5200.
ps.: hello François! :)
October 1, 2012 at 12:02
hello anna,
there’s shouldn’t be any major difference with a short throw projector
August 11, 2011 at 17:26
Hey People,
i don’t know why but everytime i open the app my kinect freezes, meaby i could have installed ant QC plugins that could affect it??? i really want to test this, but it is imposible… it looks like the communication Mac/Kinect doesn’t exist, tilt, blur and all the parameters are working but the image is frozen, any tip???
August 13, 2011 at 06:43
Im in the same boat. My kinect works but not with the kinectmasker, it just freezes up. Been looking all over for someone to help, but its so new, there isn’t much info around. Really wanna see it work.
August 13, 2011 at 21:50
ok, it was something in my laptop… but now, the Kinect mask app doesn’t send the signal… i can’t see it in any syphom soft…
August 14, 2011 at 11:25
Yeh I got mine working as well. Reinstalled libfreenect and needed to do sudo make install for it to work with this. Its 6 am. I doubt I can get any sleep now.
August 22, 2011 at 19:08
please try to re-download KinectMasker.
There was a problem with the very early build, which was fixed very quickly. Maybe you’re one of the unlucky early-downloader.
Sorry about it.
August 16, 2011 at 10:21
[…] をっとと、そういえばMadMapper1.02と同時に出た小粋なアプリ群、MADLABに入っているMAD_KINECTMASKERを試してみよう。 をををっ!面白い。 あっさりと動画でマスクが作れてる。 ここいら辺のどうなっているかは、また後日にキャプチャーも交えて書くとしようっと。 Tweet You can leave a response, or trackback from your own site. […]
August 18, 2011 at 15:55
Where can I watch the video?
August 31, 2011 at 15:19
[…] / spare wall spaces etc. The Madmapper folk have been releasing an inspiring set of very detailed tutorials too, as well as pretty useful […]
September 23, 2011 at 07:39
Wonderful….
can i make few suggestions?
1) Would love to have a “preview” button in the program so I
can actually SEE the alpha, for aligning stuff in MadMapper.
We work in the visual world and even though it’s there and
working, would be great to temporarily VISUALIZE the Alpha.
Then click and shut it off (something like the blue preview)
2) Edge Blurring, rather than the straight up Blur. Some of the edges from the Kinect depths are a big jaggey, while the blur
helps, when cranked to where I get the edge blur I like it seems
to also blur out the alpha too.
I found the Kinect Depth Select Patch here:
http://kineme.net/forum/Applications/Compositions/KinectDepthSelectopenCL
Does some great edge blurring
3) As an alternative to the whole edge blurring and setting Min
and Max depth levels (which you still always end up with unwanted material being captured such as the floor, ceiling, furniture etc)
Have you considered tapping into the Kinect NITE ability to capture and DISPLAY USER function? It view the frame and once a human enters it it captures and masks and creates a full human in color you assign. Really clean and fast and no skeleton user initializtion has to be done. Up to 4 users can be detected and displayed.
Many of the folks in Open Frameworks have been utilizing it.
I tried to create my own app using it, integrating Syphon but have gotten stuck on integrating the two. Got the user app done, got Syphon App frame grab done. But can’t merge the two (my programming is not so hot).
Here’s video demo:
Here’s some Open Framework forum threads:
http://forum.openframeworks.cc/index.php/topic,5510.0.html
4) Would be awesome to be able to INVERT the alpha as with
some of the work I do with some dancers I prefer to not project
onto them, only around them.
I figured out a work around by using some Quartz Filters.
It took me a while to figure it out, but finally got something to work.
Using 1st an INVERT patch and then a COLOR MATRIX patch.
Setting all of the Alpha Parameters to -1 and then chaning the
Bias Vector from 0 to 1.
for any interested, you can download the QC file here:
https://files.me.com/technomorph/8r1rzi
It will then be a Syphon Server as “MADKinect INVERT”
September 23, 2011 at 09:02
1- in MadMapper you can change the background color (to blue for instance) so you should be able to see the alpha.
2- openCL is too buggy at the moment. I might find another blurring method. Will look into it.
3- NITE is overkill for the moment.
4- you’re right. I should do this. (using my own method tho’)
December 1, 2012 at 04:40
Hey Technomorph, the link below doesn’t work anymore. Is there a different way to do the inversion now or something?
https://files.me.com/technomorph/8r1rzi
October 13, 2011 at 08:37
Fun with KinectMasker on the street. Kinect worked OK through a double window, as long as I blocked as much reflection as possible.
December 30, 2011 at 02:33
Hi there! Thanks for the tutorial. I suppose the only way to get the MAD_KinectMasker application is by purchasing Mad Mapper?
Is it accessible via the demo version to try out? I have Modul8 but Eur 299 is a lot of wedge to fork out…
Looking for simpler ways to use the Kinect for visuals/mapping. Cheers for your huge efforts!
January 13, 2012 at 02:38
Hello again!
Is it possible to test the MAD_KinectMasker application via the Mad Mapper demo version please? Wanted to see how it works with moving objects/people dancing. Have heard that there are some latency issues although perhaps these are connected to the Kinect camera itself.
Thanks for any feedback/advice…
January 13, 2012 at 10:06
there a +-65ms latency for the kinect alone.
there a +96ms latency overall, approx.
https://1024d.wordpress.com/2011/03/10/hardware-latency/
January 13, 2012 at 17:49
it’s not seeing the figure til they get really close (waist and above) is there any way to get the whole figure in? what about using a stronger ir lamp?
January 14, 2012 at 02:14
ah so i read the infra red it sees is quite a narrow band…
January 20, 2012 at 05:02
Is it possible to mask the black area with another video? i.e. the moving bodies mask video A, while the area outside the bodies masks video B.
January 21, 2012 at 09:15
see my post Sept.23/2011
March 6, 2012 at 22:41
Guys, is it normal that my Kinect max view distance is 1.3m? Anything above this distance is invisible for the Masker. I wanna use it on an outdoor installation, but I can not ask people to come so close to Kinect. Any suggestions?
September 17, 2012 at 16:04
Hello! I want to know if this will work with any kind of Kinect. Because i´m buying one and i don´t know whether to buy the Kinect for Xbox or the Kinect for Windows. I will be using it with a Mac. Thanks for your suggestions! Tian.-
September 17, 2012 at 16:27
tested on a Kinect for Xbox.
Perhaps borrow a unit from a friend and see how it goes
(as users have reported problems when running off an intel GPU)
January 17, 2013 at 18:10
please, please, please could you some ware uploud MAD_KinectMasker.app Guys from CarageCube Team removed MAD_Kinect from the downloads od mapmapper site. or email me THX guys
January 18, 2013 at 11:16
hi Pavel,
MAD_KinectMAsker was removed b/c it apparently didn’t work on every computer/kinect. It became too much of a burden for us to maintain, too many request for support, while it was just a little demo app, an experimental tool to add some extra power to MadMapper.
But in order to keep the feature around, I’ve given away the QC patch, for people to tinker by their own. You can get it from here: http://tinyurl.com/aczzncm or directly from the MM forum: http://forum.garagecube.com/viewtopic.php?f=20&t=8532 .
Once you get the patch, you should be able to drag an drop it into MadMapper – provided you installed the v002 kinect plugin – . Hope this helps, kind regards, frZ
February 12, 2013 at 15:42
Decent MAD_KinectMasker tutorial 1024 Architecture Blog / MadMapper, Video Mapping, Quartz Composer plugins, whatever submit!
Now i’m really captured through your proficiencies in composing. That brought to mind an article regarding pk auto I just read couple of hours previously. In any case, very nice piece… thanks for posting.
February 22, 2013 at 11:33
Do I have to install any drivers or plugins (other than Syphon) before I use the MAD Kinect Masker on my Mac or I can just plug in the Kinect and start working?
February 27, 2013 at 15:14
no install should be needed, however, depending on some computer configuration, MAD_KinectMasker might NOT work properly.
Please download the QC patch instead, and see for yourself.
March 5, 2013 at 20:08
Thank you, it’s working. Is there a way to combine the Kinect Masker input with the Video Delay on Madmapper? Because I think the video delay only works with iSight.
March 7, 2013 at 18:56
mmm… MAD_Doubler or your custom QC solution…
March 6, 2013 at 21:13
One more thing, how do I apply the Kinect Mask quad? I can’t figure it out.
March 6, 2013 at 23:21
I created random quads and then selected the left side (which is black) and placed it over the quads but no body outline was shown. Mad Kinect Masker is working fine (the preview on Mad Doubler is also black), but I can’t get the mask to work on Madmapper. Am I doing something wrong? All I get are the quads but not inside the outline.
March 7, 2013 at 18:56
put it over your content, using “over” blend mode … ?
March 8, 2013 at 10:14
Thank you, it’s working. Apparently the Kinect wasn’t always connected (it would stop suddenly) but I got another adaptor.
March 9, 2013 at 20:42
Where i can find this Application?
March 19, 2013 at 15:36
we removed it from yourspace, download the QC patch instead and load it into MadMapper directly…
May 31, 2013 at 03:25
i’m sorry, i downloaded the QC patch and loaded it into MM, then I plugged the kinect, and nothing happens, I must be missing something…
April 20, 2013 at 04:36
Does anybody have a custom QC patch that does the same as MAD_Doubler? I really need this!
April 24, 2013 at 02:11
yes… but there are a few tweaks that it could use..
as I don’t completely understand the “sizing” that goes on with QC… I was also looking for something that did a top/bottom doubling too…
April 28, 2013 at 20:04
Hello there! I know this is kinda off topic but I was wondering if you knew
where I could locate a captcha plugin for my comment form?
I’m using the same blog platform as yours and I’m having
difficulty finding one? Thanks a lot!
May 28, 2013 at 21:40
I absolutely love your website.. Excellent colors & theme.
Did you make this site yourself? Please reply back as I’m trying to create my own personal blog and want to find out where you got this from or what the theme is named. Thank you!
June 15, 2013 at 12:24
custom CSS
July 28, 2013 at 19:01
I seldom leave responses, however i did a few searching and wound up here MAD_KinectMasker tutorial | 1024 Architecture Blog / MadMapper, Video Mapping, Quartz Composer plugins, whatever .
… And I actually do have a few questions for you if it’s allright. Is it only me or does it look as if like some of the comments come across as if they are coming from brain dead visitors? :-P And, if you are posting at other sites, I’d like to follow everything fresh
you have to post. Could you list of the complete urls of
all your community sites like your Facebook page, twitter feed, or linkedin
profile?
August 16, 2013 at 13:47
thanks!
Great!
good!
October 27, 2013 at 23:19
Hi! Someone in my Facebook group shared this site with us
so I came to take a look. I’m definitely loving the information.
I’m bookmarking and will be tweeting this to my followers!
Outstanding blog and outstanding design.
March 28, 2014 at 18:54
Hey would you mind letting me know which web host you’re using?
I’ve loaded your blog in 3 different browsers and I must say this
blog loads a lot quicker then most. Can you recommend a good web hosting provider at a
fair price? Cheers, I appreciate it!
May 13, 2014 at 12:43
Hey! Would you mind if I share your blog with my zynga group?
There’s a lot of folks that I think would really enjoy your content.
Please let me know. Many thanks
May 21, 2014 at 05:30
np