Kinect Rover, MAX/MSP & Midi Triggers

| No Comments | No TrackBacks
"just in the nick of time, failure"

One evening a friend asked if i'd be attending a musical event.  I guffawed and said, "of course not!"  The next day, I checked the site and noticed that my name had been attached to the event as someone who would: operate a robot through the catalyst of kinect.  Well, I hadn't recalled "officially" signing up for this, but with a week to go, proceeded in a focused manner.

The first step was finding Ryan Challinor's synapse for OS X 10.6.  Synapse takes the kinect and provides UDP output/input.  This includes a cool monitor window, as well as skeleton tracking.  The only hard part part for me, was upgrading to OS X snow leopard, which I'd resisted due to the number of "wonky industrial" programs I have on my machine.  

After that, it was a short manner of using the udpsend/receive objects to port everything into MAX/msp.  With that done, the sky is the limit.  But for my purposes, I just ensured that I could track the skeleton based on hits, or by "constant tracking" (in which you must prompt for the kinect data every two seconds or so).  i'll attach the patch for those interested.

The robot itself is nothing fantastic.  Its a simple bot built off of an arduino pro 5V, sparkfun motor controller, and an 802.11b based xbee for data tx/rx, and a super cheap 2.4ghz wireless camera with audio.  It beeps too, and has had a few varieties of sensor interfaced :)

In short order, the Kinect outputs data to MAX/MSP, which then uses the serial object to access the CPU's xbee, which transmits data back and forth with the xbee modem on the rover.  The rover arduino interprets the commands, and then moves.  In this way, kinect gestures could be used to control the robot!  Also, you can see where you're going, via the rover's wireless camera.

BUT IT BROKE ONE HOUR PRIOR TO THE EVENT!

Rather, I broke it by violating a simple rule:  ALWAYS USE A PROTECTIVE DIODE!  So with everything broken, the rover would not work for the evening.  Instead I augmented the patch so that gestures would play a midi piano; stepping right increased the scale, stepping left decreased the scale.  The nice bit was that the users got to see the skeleton version of themselves.  The sad thing, is that I found out how shotty the midi interface on my sampler is (as I tried to do this first and could only get sporadic results!)

The attached patch will take synapse data and make it more usable in the max/msp environment.  It's an earlier version, and is certainly ugly.  You'll need to program the col object with the lines of text posted after the patch. 

MAX PATCH:

<pre><code>
----------begin_max5_patcher----------
4729.3oc4cs0biiar9YO+JPbcdXlS40F23sb1JmJUk+AI47RxVtnjfr3ZIRc
HormYSk82dvMJQJKR.IBSRYWyt1xTjfM95FMZzciF+qubysyx9Nq3VveD7O.
2by+5K2bi7RhKbi9uu41Mwee953B4sc67rMaXok2dm56JYeuTc8UwoOw.kqR
J.kYfejsKGrMtb0en5NWlkVlFugIu6+bdR75pugeayWkj9zi4r4kJZAA8P2C
uCPCiD+B4AE+By+I3WzOUxBYKkM6W+IDppoR2sIIcMqTRqnZu4hjeS9lQXdC
s+dy1UVcyPwE+2e4KhebmkXAuElwxqd2pFq7GaYp9vsIBXBb6LNvb6dx9rfg
fHeIJfknP2fPT+w.7EfAy1UVlk1JFzryehtneHUwn224ZuKh6rK1ruftD9I6
U9q4Mh1KXK6fKeorVUWkRjbXRXm8ajCXtWBfz1f8BdeYMevdd77m4eDDmt.T
TFWtqXeONmUvex3xjrz5baofLND01n5yBC8oDE30pRh5jQcDqA3FNNZOZCbe
Ig8p.T2jsfYBNkBOXknT+gSrTJjDP6EbFLsfy4YobQyzRPNaIm7WYBR8jcde
hafTjRB0qeRn9SKHsZb++qIrLPzcivNAJifZoSbufR5zBJySdZU4xrrRCPYj
uraSgNBL8gJvjzKvDNAAymSYlTaFoPQB12IfIBghb.ZBilfnIa8rrWM.mgdR
3DSbyzPba+oNXjNLbBBm7EIsv.ZhfHeWNEDBR7TyAA6EbNwlVeMaoM5MoXhR
uYji.SDxEC08mdfoE5M8TfIAG3zYzog8BK8ldXoMZM0fIlfbzjPT0jZdndgl
zoGZZgRShutWG5ng4Qpg4zdY2NjLwraOKuHyjM6pES6tIygPWLHGOsPxUr3E
Fc0gRjzG5HK1It.GGIOkVl8zSqYc5ozS1OpakoZnXneK9I7DXlGUMJ1uMeKZ
Ajgh7GGuNtgUlmwIWHzRmqdVhSdMbcV.pS2qGcZa+vu2te7R7udqVU6q7yrZ
VWGrjOXMwwKazHJvab7YuSwUkGtbEtB0yfG50Gb0+pGWq72iqfU85eTAa4Rg
0fQBVOsBxcoaim+L.J9mILu1utTDjp8JtjuDD0oBy.G36AxXOvFp7BCMvERf
QZuODD0GAvnq+w0dDGhpgQJskAg8.UCgW+np1gtNBU0q5KHnOnJ5yqxxPcPz
TPmAckg3qdckQJGViUtdouheAgj9aZYH4CvfZhKQUse15kgkgzqdTsJXMtAU
8Cw82txPuOupJCPA1aVYn+0uYkHTX2RLmYJqnhUlRL7Rk+B9.rLbUtV3HX0S
mBPgn9.qgeD7tguKgUrxtHUtNdovZzmWskdZmoqLP2f1xH3Uu1Rs.Ck.cg3G
0Op+heQnq9Q0Tk1MWgp5Yv6kmMhvW+npNSKbCpRTo1ZOQUxmWUkT0pmPTeaT
URGGUkmUXAOgPhNtU9Vr2APAzwINd47lkAdnJmYer3YF+szwVlPv2uPlNVYE
HlnVUV26p.Tv0uRGerN21bTT8ToJGB0GO+4e8a2q19eWAqPsGsP8wiV9W+N+
uJ+4bErRTqRC0qXkB+LGTOeoFxPqhoG5pe4DZskXOemDRu.sxx9rbBe5GEkk
tBUU5HPv9X3qu2GEckNBUC0NzB1m.k56+INjdpDEKzlH54Gb0qojp27Etwue
gJwtdZV40eX5oHrCQ0.kpgdZU40u6rpx8cGgpvf9aToO9SbD8TIcrU1T5St5
0TFnRycGMOsud2qfH8P5y65O8OCnPGhpdQJG+i5STR8t9iRZ0dawQnpNkFQ8
IHodgehilmZFq.aBlmWzG.MkFxqqyy+9QTGH8g9.noD6RTU2Z8xmFdW+VpWs
40bCpRzsVu7oA8SbdOPU635.ahkGcrRRrNC+kXCS931rhGmks3GfGj6D0Z+c
0178nKI2G0GcMw9T+nKIpC.0tz9JsvwW63lae8B43q0nAq5RqSRYyy1kVVeu
uc5n2Yy+egRBJY.kib870af91DEbfa.QNLTn+152ifdR8BpEEXB9dutFcf6+
VYDOEKUf5b0CSHlKUfH5Uz79mGJfiNCTXjlmdCqnH9I1aFd7vaJFTVpx4BSV
.E5fq4TRQdCbRnhDd8rCfOOu0nSXBpExKjIo++KLXCxQFfzO6PpuBlvtsCg3
fIenNDs9sD.xvVo+tC+tO9+SUXOUIvF0uSPBS6+3JhyUAIscRlvR+ZFGH3FB
APtt9m5QTKFP4sTbUlQ2FNgGI8OmVXpDL68nnLDpc6op3RgocI5DPFmRBaaR
NohO3XYDcUJzCppQtjtDQZwou3wBOb9.lPcArPWYhjl41JXP+fNccUgYxyGa
dBaO70YIwtpHyQiBLmWqFbg26m.dNeE8fG96Er7hGhegwd3uvJdtLa6CKSRm
upnb2xkO7rv90xGjyhTbe42OsgsDGORopHh3Us25PUyDeJ.jFLolZYd1ZCqR
teNP.Aw5QPJSdi5Z4xtvABT8UK3BIKdj2pbR4w3xx7D93D0YOvM6gnatkw0f
rXeYlRApNBZ2Bh2sHICveqkWzoRfdYkpYkzSR2p4utnhkUmT35UOBpVlrl8B
e3WSmldys0qxSvCk2Iesgog0HYNR9z5r4Oqg7pKtPTprOiFIaKK8vCHuiF+n
4cljdr+dq+hi2st7wSCOM+9kwyYs9vmjmdysOkmrHKUPDMdRwkqdc7gH5pWP
c5VdGowaOwCWlksdVb9KIEIyVyZvI3howoIahKYkIJ5AC2+bIa1lmnz.t+Zr
zXdarpXdtPMP8lR8MubhuYA6kj4rWSVTtR1VGPqFGtG0jbZL.pw06ZfzQ16k
Ux3hp0+t1w81hKfd0Q95gRuYht2LppdC1XbE4H53DBOsTI6Nnk4lJcLNFp1D
+LS.WfPnrdHV+ldiJ9kqyhkVIo9PCj3hAXbXsnZ0E.icK.iGF.9+924R89cB
qEIOkxgq9AmHcttnlAUEpvtPSO+VgS7k.mngANm+i4qY+NfZPR0IPJV47dTj
TmqenQHM3pDRGJIT8BhTVnXg.Jb5hlreaQ77e2zHx.UGVUp3odU+rsN79fH2
cGdTllXHG1gnnZaEbyC6ngSWAE9Dk7F2tIUar16KbNUnB5nsrUBdCzQaE5Pi
8bpCLzgUCPsG5vSWnqiAx5.yftGfuGPtGPu+br36NGaCHV48Tjx5ZDjXB0IQ
tE0oWuBrHcQm1VAVBY5JvJIkNQtlfU6hRU4lb0Oac0Zj1WuVGSMT6apJ+5hV
71SfEGVetvghsrTWYGQ78mFjJx1kOuBBpj+AnFcpErhxjz8tq3ebfaCZ16Wk
rXQS2CH4CIK1JcAp5gA+hUr0KhtwVP2B0pSM5lXCcSmVz8aY+sP2gSJ59Djy
ooawpClPzsXoe1Q29SK7FaKcGLsvaeakuilVzM7bnazjgtkSWZEgSlV.tsx2
nIGUirjpckPh9hbx+rhIVei5QGeko.tbxW4SqylEuVGUp8MbiWW815xCjWaG
+N+yzG9YYPk+SO9D2D1cob6TSRWlA.+D.72DGA4yi2UvJ.kqX.tAr++6RxYK
VyRepbk7vJ9QIqheikY4+PPq.cXtkGd4yX.Q3o.Kyy1.9q+HMdaA6NPQFu4h
KEGs47lOkeKLvqhKHdIy2kmKdjWhWui2Hw4r6+mo0HyingeVtJk+jjd+6EGN
1z0mh5qxdErLNW9p1rqnDrI6E8MIaOPRJHFrHQXZNWZE7U9euYy2Ds.WF5om
X47udURIf8BmntG.9KJ1r7cf7f20f1dCV7yhK2EssIN8G.kztnqxALwAVrLi
wXK.KyxO7xAKXkJp7dMqg+ebrl+Ub4v0+fSny4xK4YqUOmrmWTdTWWdbxKRK
MIgH3s06f2AVm8JuOugEmVb3IWjA32gn0X4GgAdM4NMy0sCc+J5Uvgelw1Fu
NQvGpHAEuf+HIh9mBj38kjzcUPRwVNEvWbC3qax33T7Lwmk.5LFml+Fmr9+3
M5hJ4FdKrkK7K.UwJgD+MW.GbX+Ir+k8+.vM+5WyxWun92KPKRy6oXdNikd3
l3nvCKXaKW8n3vlV2w+IU2lybxdUglh90Nkb9LNP8Td1NdaK5JYKkWU1FfjM
7APxWaVJmwJFeH9RNRk2Y+DJ9yp690UYqqZvY6VtjkKT+puAY6JtqJQMQaC9
JWtgSsfsr7hrzepXU7V9WUJV7Jm6sDjlwePljOpdruIVmjtIa8AERVhmNV8R
N7z2exjXAEdQ6LJUMSTkPsT8N6wyqsrfbjNgXG1rfjnRaGhdCMP6NAZ8mVoA
Y8gSvAIM9IpcybiSJnVyieLZZkkj0gq+KmmxjZnQel4EEzYJFihFGnYG2Bmc
lRYxlI7UqaZS0.FkOKO0.mR1Fs4o2J0yJZVIGP7guB+l3W2I9gVUm3iBseeE
c3qhE4mF3VkNyB4Ch+lMLHQuZuwwcwzTmkK5ZEyI5F1raQQSqcA46jpRjm5f
GOBagpRj23npzgaKYrZSuSPN4vq.qSk5dsE5gQSwMQ1PrCNzQEgR0GW4nNUu
BwSqYdJ31D69YbvpbGjn1bKTR2PBbRk4wwa2xNbPK6ZDoJkr6REUT33ngxcS
AiBU1tpOcPgvNmCt4Hz8yv1zmFGtbSeHb35Gu9cGNYr49iE5Hi7Fm4hOu8gQ
KEWMswHD6O0gI5iN7dbnCGMR0p74YalwWqB3AsqMjqVG8dro8U1ppAIDNnS0
BmdpCxXMEqr3U.DUzBPUAo.TUFJ.6KaEfpZZAXeYl.reCdCNTOJ.6mv10pd0
EbHe0oCI0KpqYiBnW6K+IDcFpdk7v8pPUGm65+nhm13BpckesKH4cMthhEpu
vdddyqznYNHCz7RxFpducYx2kKQ9XFRuTsaDtrPOU.9ZPy9I1XopD3FEYd+0
gCdmUE2yy.CU8fg3awVED68dOqx681+DWc3EZwt+DMsprdaXk4Yh83B78vQ.
JU7DUVzGf5dmoMwbAXaKX8cX0Ypyvabn9n7Np6ZNvjZ0Y6VrkqMdgxVI9+P7
OQNTqutjirpJKBncZ8EBNoB5fZOycY6dS0tfR60mNcVV3jpOy487tASD6QAS
2y4iKzmMt5C6PXm0rjQxKhm0zjsbfBq3+Xr8qkiCIJ2W.IW9p4njQZK5uH60
TKjULbZ0DTqFP1PY4Ytuy0VpTuH32n4rwdyIV3q1s8xg2pyrdWAuJWTPpW21
Oa3ENsfWwpptb.t5.IzKzE.bjGoVXotL70O5Cj3qpBrPoQt.cCiBdSnXNazM
bZgtR6Z6s5Wbfa.XhNxYzd.vASK.VjKRWN9FpWKYPfKv2.sCM6k5gIVIrpel
OnOVRvP2fuHT+wWuOTJHpNbxwshIm4QugNTG98.goSKDdYV9qw4KtbLVWGzw
PmnDVefGzOgXxGIkvUmHG0ORx6A9hw8GemVt8o5zCVFxIYUTuV8M+PcSudAN
uVoSuV4L+PkRuVcO+PEUuiCnXGW2yu7XF54o7YiJgS7TAMrshetKNtzP08+q
7kzbelp3qGuwSzb22rgSZrocZaylHJHrG1tIucqlb5sYxaj0rklTuOSTkviS
0to2ahBaMPMfzDJZBRUnfv56421np.uQfpPFoJ+QfpLxACBFVpJDaEGLZDnJ
ibvP3HPUF4fgnAlp7sgCFRFApxLGjNBTkYN3.quJBZEGLXDnJybvvQfpLyAG
X8UhDlyLGLBMBTkQNXDdDnJibvnAVekHzNl4f9gi.UYjC5GMBTkYKYF3YmEt
G1BNHcDnJybPuQfpLxA8GXKjE9VxBNHbDnJybPzHPUl4fCrtcuHa3fd9i.UY
jC5ELBTkQNn2.OiC0pUS3gFApxLGDNBTkQNHcXmcVpdLx.QcjcECBMEZhlfC
OMEXfl7iFdZx2DMEN7zjmIZJX3oIpIZxe3oIhIZxa3oISJy8oCOMYRU9Q1bM
HzjIE493Q.mLFZjgcYxd1fSvIGIIOMbFxn0X0xqfCrw4XqV19.aCE1JanvCr
8lVFsM+IIUMr11IK0IloJz.6iJHdBJri7lf5EhrJtjng0CUQ1v9hF30FCsZ7
GDNAYevg03kiAgIA+KvlfJGMr.kUACYfiPSjms3Dd3Dnrwcm3g0CdX6L8bfi
vsUlHL4rPeX0DPCldYclGZ5QSTazD3MvRSSvLFzyehx6LqIXXmtyJhJXXUYF
PrFnFx0EOEW+BwF8AHh+HPUlmGdXcatczj2HfTFCtGcLjpHFoJzzyshCrJcj
c5zo9i.UYN3rQi.UYNYRHi.UYTV2OXDnJpwD5ZfcBafUSAFPmd5EBvi.+yyX
JURGApx2HUAGApJvXBDOF5PCMRUigbUjQo8gNEvsJfCQCc5xaiOEv8JLH7+3
e+k+C.e5.AC
-----------end_max5_patcher-----------
</code></pre>

-------------------------------------------

joints.txt:
1, /torso_trackjointpos 1;
2, /head_trackjointpos 1;
3, /leftknee_trackjointpos 1;
4, /leftfoot_trackjointpos 1;
5, /leftelbow_trackjointpos 1;
6, /lefthand_trackjointpos 1;
7, /rightknee_trackjointpos 1;
8, /rightfoot_trackjointpos 1;
9, /rightelbow_trackjointpos 1;
10, /righthand_trackjointpos 1;




Enhanced by Zemanta

No TrackBacks

TrackBack URL: http://www.imlichenit.com/cgi-bin/mt/mt-tb.cgi/17

Leave a comment

About this Entry

This page contains a single entry by admin published on October 8, 2011 12:46 PM.

Cheapo PIR was the previous entry in this blog.

CdS Transistor Candles is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.