With the basic game mechanic selected for the re-worked scene, we were able to move onto how to technically realise this.
The task is to build a system that will sense the placement of a playing card onto the table at one of the points of the clock face, and trigger the playing of an appropriate sound file.
To tie in with the game mechanic, we only want the sounds to play when the numbered playing cards are placed in their correct position on the clock face (so a 3 of diamonds at 3 o’clock, for example), rather than just anywhere.
For the playback part of the system we are using QLab, a piece of commercial software used widely by theatre lighting designers for programming and running lighting cues for conventional theatre productions. David Haylock has found this to be a reliable off-the-shelf solution when he’s used it on other projects.
David is writing a bespoke programme to process the inputs from the sensing system and communicate them to QLab.
We haven’t yet finalised what kind of technology we will build the sensing part of the system with, but here are the two options we have explored so far.
From some preliminary research David suggested using RFID tags and readers to build the system. RFID (Radio Frequency Identification) is commonly used in the retail sector for theft prevention, and so the components are cheap and easily available. You’ll also be familiar with RFID if you use an Oyster Card on London’s transport system. Our plan is to put tags inside the playing cards, and mount small readers under the surface of our card table.
We bought a few tags and readers, and David built a small test system. In its favour, RFID is quite stable and reliable. This is an important consideration if you’re relying on it to work, because it would spoil the immersive fiction if a technician had to come and help fix something during the scene. There is one major drawback though, in terms of it working with card games. Most card games (including our collaborative solitaire) involve stacking or creating piles of playing cards as part of the game play. RFID can’t really understand stacking (there are some complex workarounds, but basically stacking is out for our purposes). The RFID reader can only read the tag that is placed immediately on top of it. All tags piled on top of the first tag are blocked from being read, and so have no effect.
We think that this is a deal-breaker, so have looked around for other solutions.
David’s current avenue of investigation is image recognition, using a camera. David has recent experience of developing this kind of system from his work on the Playable City winning project ‘Shadowing’. He proposes using a PlayStation3 camera, which we will need to mount above the table looking down (with a birds-eye view of the card table). This part is relatively easy, we will just need to design our bespoke card table with a suitable structure above it to hold the camera. It is the recognition and processing of the images where the real work comes in.
A google search turned up several Open Source projects exploring this area, but after some investigation David has begun to write his own recognition programme. The recognition is a 2-stage process: 1 – the computer needs to recognise that it is seeing a playing card (markerless object detection); 2 – once it knows its a playing card, it needs to work out what number and suit the card is, by comparing the image it can see with the ones it has in its library and finding a match (Template Matching).
The potential down-sides of image recognition is that the system is very sensitive to differing light levels, but we propose to create a reliable lighting state by mounting a downward pointing lamp next to the camera above the card table to eliminate this issue.
The positives for our purposes are that the camera and system together behave more like the human eye – the camera sees the card on the top of the pile, and will be programmed to respond to that by triggering a sound. We will also be able to use a standard cheap card deck, rather than having to make or have manufactured a special card deck with RFID tags inserted.
There’s a lot more still to discover and test before we build the final system. Then there will be lots more testing! We’ll then build all of this into a specially constructed card table, which will conceal all the technology hardware, keeping the interface as close to the experience of a ‘normal’ card game as possible.
