• Accueil
• > IEEE TF EQ

Call For papers

WCIDM (Jasná, Sep. 25-29)
CIDM (Orlando, Dec. 9-12)

IEEE Task Force on Evaluation and Quality
(in Data Mining)

There are a lot of data mining algorithms and methodologies for various fields and various problematics. Each data mining researcher/practitioner is faced with assessing the performance of his own solution(s) in order to make comparisons with state of the art approaches. He/she should also describe the intrinsic quality of the discovered patterns. Which methodology, which benchmarks, which measures of performance, which tools, which measures of interest, etc., should be used, and why? Every one should answer the previous questions, and assessing the quality and the performance is a critical issue. Therefore, IEEE has established a Task Force on Evaluation and Quality issues. This Task Force is established in the context of the Data Mining Technical Committee (DMTC) of the Computational Intelligence Society (CIS) of the Institute of Electrical and Electronic Engineers, Inc. (IEEE).

Major topics will include but are not limited to the following:

• objective measures of interest (for individual rules or rules basis, patterns, graph, data streams, clusters, etc.)
• subjective measures of interest and quality based on human knowledge, quality of ontologies, actionable rules
• algorithmic properties of measures of interest
• comparison of algorithms: issues with benchmarks, experiments and parameters tuning questioning also reproducibility of data mining results, the need of new data sets which match new problems, methodologies, statistical tests, etc.
• robustness evaluation and statistical evaluation
• graphical tools like ROC, cost curves
• special issues: imbalanced data, very large data set, very high dimensional data, changing environments, lack of training data, sample selection bias, graph data, etc.
• special issues in specialized domains: bioinformatics, security, information retrieval, sequential and times series data, social networks, geo-localized data, etc.
• etc.

From the previous list of key topics, although not exhaustive, and from recent publications and related workshops questioning the usefulness of research in machine learning and data mining, one can identify five major themes (to be extended, all propositions are welcome):

• properties of objective measures of interest (for individual rules, for rules basis) which leads to the problem on how to choose, depending of the user's goal and other factors, an appropriate interestingness measure in order to filter the huge amount of individual rules or to evaluate a set of rules.
• algorithmic properties of interestingness measures which leads to the problem of how to mine efficiently interesting patterns i.e. can we use interestingness measures as soon as possible (as well as the well-known support) to reduce both the time to mine databases and the number of founded patterns? This question has attracted a lot of works but is still a very challenging problem.
• properties of subjective measures of interest, integration of domain knowledge, quality based on human knowledge, quality of ontologies, actionable rules.
• challenges with new data and new problems: very large and very high dimensional data, imbalanced data, etc.
• evaluation and comparison of algorithms which lead to debate on how an algorithm should be evaluated, on which properties (e.g. accuracy, conciseness, specificity, sensitivity, etc.), on which trade-off between the different type of errors for multiple simultaneous hypothesis testing, on how to construct new evaluation measures?