Second International Chinese Word Segmentation Bakeoff Data

The Second International Chinese Word Segmentation Bakeoff took place over the summer of 2005 and the results were presented at the 4th SIGHAN Workshop, held at IJCNLP'05, October 14-15.

Corpora from the following organizations were used:

• CKIP, Academia Sinica, Taiwan
• City University of Hong Kong, Hong Kong SAR
• Beijing Universty, China
• Microsoft Research, China

The complete training, testing, and gold-standard data sets, as well as the scoring script, are available for research use:

• icwb2-data.rar [40 MB, md5]
• icwb2-data.zip [50 MB, md5]
• icwb2-data.tar.bz2 [37MB, md5]

The Detailed Instructions for the bakeoff are available. Please read them carefully.

Segmentation guidelines for the following corpora are available. These were supplied to SIGHAN by each data provider, and converted into PDF by the organizer:

The collected results of all participating systems are also available.

Many thanks to the data providers and the bakeoff participants!

Contact Information

The bakeoff was organized by Tom EMERSON of Basis Technology Corp.

Questions on the bakeoff should be addressed to Tom Emerson.