Interactive Schafkopf: Extending the Real Game with Virtual Players
Abstract
The staggering improvements in the quality of video cameras as well as computers have opened us a whole new world of possibilities. The raw calculating power of modern computers combined with the information contained within the megapixels of a photograph enable us to process images from the real world.
This thesis makes use of this possibility, providing a solution for replacing one human player in a real card game by an AI. The requirements for this setup are as simple as a webcam with a sufficient resolution as well as a projector. The interaction between human players and the created system in the context of card gaming is then studied.
So What's This?
In other words than the abstract, the created system enables three humans to play Schafkopf with real cards by taking over the fourth spot. The webcam provides the picture, the cards in the picture are detected and identified, the AI ponders upon those cards and the projector displays the cards that the system plays out. The main feature of the system is perhaps the fact that it can be played with real cards. This entails two facts:
- Every action that involves the handling of the real cards has to be performed by a user for the system.
- Every finished trick is one card short. The cards that the system recieves are only scanned and do not come into play at all (Note: the projections of those cards do!).
Setup
As stated above, the created system requires both a webcam and a beamer. In order to achieve a high contrast of the cards to the underlaying ground, a dark piece of paper was placed on the table.
This picture shows the setup during the user study:
Image Detection
The created system does heavy image processing work. Different approaches were implemented for two different detection of cards, namely the hand cards that the system recieves and the table cards that are placed on the dark paper during play. Many of those feats were realized with the
AForge.NET framework.
Hand Card Detection
The cards that the system recieves have to be fanned out in such a way that the groups of indices in the top left corner of all 8 cards are visible. Those 8 cards must then be held (roughly a handspan) under the webcam until all cards got detected and identified.
Finding and grouping together of white spots of similar size and in close proximity
Identification of the indices
Table Card Detection
In order for a card to get identified by the system, its entire face has to be visible and at least two of its edges must not be obscurred. This allows for the identification of the topmost card for most constellations of intersecting cards.
An input picture might look like this
After binarization
Find all white spots
Find the corners of the rectangular spots
Use the corners to extract the card image from the input picture
Binarize it & compare it with the templates
Interaction
Input to the system
The system comes with a console through which some required information can be entered. This is mostly the announced game mode. Other commands exist in order to cancel round after the dealing of the cards (
zamschmiss), to remove the last detected card (
undo) and to exit the game entirely (
quit).
Feedback from the system
The system gives out information in the following situations:
- Hand cards: During the process of the hand card detection, the system displays the amount (not the cards itself) of detected cards.
- Last card: In order to inform the players that the last played was detected and identified, the system shows a copy of the played card in front of the player who played it.
- Last trick: As every trick contains only 3 real cards, the missing card is shown in front of the player who made that last trick.
- End screen: After all cards are played, the system informs the users about their scores, highlights the winning team and displays all cards that are missing from the users trick stacks in front of those respective users.
AI
The actual game Schafkopf is highly complex and thus not easily rebuilt by the means of an AI. With the goal in mind as to recreate the basic gameplay, only a rudimentary AI was created.
It does not draw conclusions out of the played cards, but rather decides which card it will play only by the cards in the current trick.
The AI will attempt to help its own team and to hinder the opponent(s). Examples for this are the desire not to take the trick if it already belongs to an ally and at the same time to take the trick away from the enemy team.
The entire play logic of the AI is depicted in the flowcharts below.
Note that there are different decisions to be made depending on whether its the
first card of a trick or not (
Flowchart1,
Flowchart2).
User Study
A user study was conducted. The goal was to find out how the system was accepted by new users.
Two groups of testers played 3 and 4 games, respectively. Both runs took approximately 35 minutes each.
Feedback
While the idea and realization of the presented system recieved general acclaim, the resulting gameplay did less so. The main concerns were the following:
- The detection of the hand cards took too much time
- The identification of the table cards delayed the game flow
Those issues resulted in a general rejection of the presented system.
Conclusion
The feedback from the user study was both disheartening and instructive. My personal résumé was to abandon the objective of actually recreating a human player as good as possible, but to use the system for the teaching of the game. With an optimization of the AI, I can envision the system in the role of a teacher, explaining to new players which cards to play (and why!).
My conclusion from this thesis is that even if a operational substitution for a human player can be offered, there is still a lot of work to be done until the other players accept and use it without second thoughts.