In Part II of the thesis, “Search and Knowledge for Improving Human Problem Solv-ing Performance,” we addressed the followSolv-ing question: “how can we develop meth-ods based on computer heuristic search for improving human problem solving perfor-mance?” Chapters 5 and 6 answer the research question 3 (RQ3): “how can machine problem solving be used in tutoring, for teaching a human to solve problems in a given problem domain?” Chapter 7 answers the research question 4 (RQ4): “how can knowledge represented in a form suitable for the computer, be transformed into a form that can be understood and used by a human?”
In Chapter 5, we presented a novel approach for automated generation of human understandable comments on decisions in problem solving. Our approach is based on a computer program that uses heuristic search. We set a quite ambitious target domain: the complete domain of chess. The long-term goal of our research is to develop a computer system that will provide commentary of chess moves and possi-ble continuations in a comprehensipossi-ble, user-friendly and instructive way, thus using the power demonstrated by the ever stronger chess programs for the purposes of an-notating. We demonstrated the main advantages of our approach over the related proposals, namely:
• the ability to annotate chess games during all the phases of the game,
• the automatically generated commentary, aside from the ability to comment on tactical positions, also expresses a solid understanding of strategic concepts behind variations that chess programs suggest in given positions.
170
10.3. Search and Knowledge for Improving Human Problem Solving Performance
There is an array of cognitive issues still to be solved to arrive at really good annotating system, e.g., when to comment and when not, which move (the best or some similar but better) to compare the move to, and what is the most suitable depth of lookahead, to name but a few. These are obvious tasks for future work.
In order to make possible for annotating software to make more in-depth com-ments, obtaining knowledge for construction of more complex positional features is desirable. In Chapter 6 we demonstrated our novel approach, based on argument-based machine learning [MvB07], to the formalization of complex patterns for the purpose of commenting on problem solving decisions and/or intelligent tutoring. We investigated a particular aspect in the development of a chess annotating software -the ability of making intelligent comments on -the positional aspects of a chess game.
This task is made more difficult by the fact that the strength of the chess playing pro-grams mainly comes from search and not from subtle positional knowledge which is necessary for generating positional comments. Therefore, components of a chess program’s evaluation function are not sufficient for making in-depth positional com-ments. Defining deep positional patterns requires powerful knowledge-elicitation methods. Our study suggests that argument-based machine learning enables such a method.
Our future work will be associated with further improvements of our annotation software. We intend to implement several additional positional features into its eval-uation function, in order to make the commentary more instructive. In particular, the expert module of our annotation system, which provides the user with a commen-tary of chess games, based on learned or manually-crafted positional features, and possibly with more detailed explanations about particular features of chess positions, requires further attention. As part of future work, we intend to apply this knowledge-acquisition method to the formalisation of other positional concepts of fuzzy nature, such as weak or strong pawn structures, pressure on the opponent’s king, space ad-vantage, harmony among the pieces, etc.
In Chapter 7, we demonstrated a procedure for semi-automatic synthesis of knowl-edge usable for intelligent tutoring purposes. We developed an approach to deriv-ing meanderiv-ingful concepts and strategies usable for constructderiv-ing a heuristic evaluation function ready to be used for commenting on problem-solving decisions. As a case study, we semi-automatically synthesized textbook instructions for teaching the dif-ficult king, bishop, and knight versus the lone king (KBNK) endgame. Moreover, we used the obtained goal-based rules as a heuristic evaluation function to produce
ex-10. CONCLUSIONS
ample games containing automatically generated instructions. The derived strategy was found to be suitable for educational purposes at the level targeted for. Among the reasons to support this assessment was that the instructions “clearly demonstrate the intermediate subgoals of delivering checkmate.”
Our procedure for semi-automatic synthesis of knowledge combines ideas from argument-based machine learning with specialized minimax search to extract a strat-egy for solving problems that require search. Using this approach, a domain expert and a machine learning algorithm improve the model iteratively. It is particularly suitable from the expert’s point of view that argument-based machine learning offers several advantages for knowledge elicitation:
• it makes easier for the experts to articulate their knowledge,
• it facilitates the experts to adjust the level of introduced concepts to be acquired by students as dictated by the level of skills of students targeted at,
• the experts only need to provide relevant knowledge, and
• the obtained knowledge is
– consistent with expert knowledge,
– in a form suitable for use in a computer tutoring application, – in a form that can be understood and used by a human.
We also explained the guidelines for the interaction between the machine and the expert in order to obtain a human-understandable rule-based goal-oriented model for teaching how to solve problems in a particular symbolic domain, and how the in-structions, including illustrative diagrams, could be derived semi-automatically from such a model.
Deriving human-understandable concepts and strategies from chess tablebases is an actual AI challenge. The positive outcome on the human understandability of the derived concepts and strategies would represent a milestone. However, the value of the demonstrated approach yet needs to be proven. So far we only have the positive opinion of chess coaches who commented on the derived strategy. There are at least two obvious directions for future work. First, to create a computer tool for teaching the KBNK endgame. Second, to use the described procedures to synthesize instructions for another, preferably much harder endgame.
172