Skip to content

Hacking and Headtracking

These are my notes on lecture Hacking and Headtracking by Domen Grabec that was part of Cyberpipe’s open lectures.

The big question is – how can we make 3D worlds more accessible to end users, similiar to what Johnny Lee did in his video:

So how can we achieve this without having to use WiiMote and to make it accessible to broader audience?

Headtracking 1.0

  • Wii remote library + Wii remote + headmounted LED’s
  • Developing game
  • Very robust solution
  • Sadly not everyone has a Wiimote at home as well as that games have to use that library to use these features

Headtracking 0.6 or 1.4?

Kiberpipa, Headtracking
Image by rok xxx via Flickr
  • Why do we need to have these special glasses, if we can just hack games to use our library

Arhitecture

  • Application that tracks you head 😛
  • A compontent that injects itself into the game
  • Communication library between these two things

Second big question – How can we change the viewpoint of the camera?

It turns out that the answer lies within DirectX library.

How does DirectX rending work?

  • World space – how the objects are distributed around the world
  • View space – how camera views these objects
  • Screen space – transforming this 3D space into 2D for monitor

World space moves an object in local space to world space. It transforms the objects so we have a perception of depth and we have to do this for all the objects.

View space moves world space to view space. Depending on where do we look at objects from, we need to move the camera around and change the calculations.

Screen space once again moves view space into screen space 🙂 Where we set near clip plane and far clip plane and visualize all the the objects in-between.

 

headtracking2
Image by rok xxx via Flickr

How?

Two approaches:

  • Start application in Debug mode
  • Get table of function and their location in memory
  • Override this function with our own implementation

or

  • ProxyDLL – we inject our DLL, same as original
  • We reroute all the functions to the original DLL
  • We manipulate the values of the variables

More about Proxy DLL

  • All the exports of the original DLL need to be same in the proxy one
  • Load the original DLL and reroute the functions
  • Application has to locate ProxyDLL before original DLL
  • We can spy after all the processes that use dynamic library loading

Face API is really useful for this.

 

the viewport skews
Image by teafordinner via Flickr

Interprocess communication

  • Named pipes
  • Similar usage as HTTP client/server
  • API for named pipes is the same as file system

Limitations

  • Lag of web cams
  • Doesn’t work well in dark room
  • Limitations of face recognition libraries
  • It takes quite some CPU power
  • You have to hack each game and it’s not simple if the library is not dynamically loaded

We can work around most of the limitations with custom software

 

— conferences