Group | Track | Runid | Recall | Cr | Precision | Cp | F-measure | OOV Recall | IV Recall | OOV Precision |
Microsoft Research Asia | C | | 0.940 | 0.00120696398546 | 0.926 | 0.00133038124135 | 0.933 | 0.707 | 0.963 | 0 |
National Laboratory on Machine Perception, Peking University, China | C | | 0.936 | 0.00124389198875 | 0.923 | 0.00135488043833 | 0.930 | 0.683 | 0.961 | 0 |
Natural Language Processing Lab, Northeastern University of China | C | a | 0.940 | 0.00120696398546 | 0.914 | 0.00142487603752 | 0.927 | 0.634 | 0.969 | 0 |
Institute for Infocomm Research, Singapore | C | a | 0.936 | 0.00124389198875 | 0.917 | 0.00140209833451 | 0.926 | 0.617 | 0.966 | 0 |
Institute for Infocomm Research, Singapore | C | b | 0.932 | 0.0012794318917 | 0.910 | 0.00145444301502 | 0.921 | 0.577 | 0.966 | 0 |
Simon Fraser University | C | | 0.929 | 0.00130524425624 | 0.909 | 0.00146169713989 | 0.919 | 0.628 | 0.958 | 0 |
Pattern Recognition and Intelligent System Laboratory,Beijing University of Posts and Telecommunications | C | | 0.932 | 0.0012794318917 | 0.904 | 0.00149718197943 | 0.918 | 0.546 | 0.969 | 0 |
Natural Language Processing Lab, Northeastern University of China | C | b | 0.922 | 0.00136291107707 | 0.914 | 0.00142487603752 | 0.918 | 0.637 | 0.949 | 0 |
Institue of Software, Chinese Academy of Sciences | C | | 0.922 | 0.00136291107707 | 0.912 | 0.00143977130808 | 0.917 | 0.680 | 0.945 | 0 |
National Central University at Taiwan | C | 1 | 0.917 | 0.00140209833451 | 0.904 | 0.00149718197943 | 0.910 | 0.676 | 0.940 | 0 |
Alias-i, Inc. | C | | 0.919 | 0.00138661222412 | 0.895 | 0.00155797658598 | 0.907 | 0.459 | 0.964 | 0 |
Kookmin University | C | | 0.915 | 0.00141734236736 | 0.896 | 0.00155140588269 | 0.905 | 0.565 | 0.949 | 0 |
Shenyang Institute of Aeronautical Engineering | C | | 0.902 | 0.0015110229688 | 0.887 | 0.00160899906254 | 0.895 | 0.568 | 0.934 | 0 |
Basis Technology Corp. | C | a | 0.831 | 0.00190457738677 | 0.819 | 0.00195675295219 | 0.825 | 0.487 | 0.864 | 0 |
Basis Technology Corp. | C | b | 0.809 | 0.00199777099675 | 0.827 | 0.00192234160642 | 0.818 | 0.637 | 0.825 | 0 |
Group | Track | Runid | Recall | Cr | Precision | Cp | F-measure | OOV Recall | IV Recall | OOV Precision |
Microsoft Research Asia | C | | 0.961 | 0.00128038112034 | 0.955 | 0.00137104972827 | 0.958 | 0.702 | 0.972 | 0 |
Intelligent Agent Systems Lab (IASL), Academia Sinica | C | a | 0.961 | 0.00128038112034 | 0.953 | 0.00139971830968 | 0.957 | 0.658 | 0.974 | 0 |
Intelligent Agent Systems Lab (IASL), Academia Sinica | C | b | 0.961 | 0.00128038112034 | 0.952 | 0.00141378821521 | 0.957 | 0.656 | 0.974 | 0 |
National Laboratory on Machine Perception, Peking University, China | C | | 0.958 | 0.0013266387059 | 0.948 | 0.00146842275172 | 0.953 | 0.646 | 0.972 | 0 |
Institute for Infocomm Research, Singapore | C | | 0.958 | 0.0013266387059 | 0.941 | 0.00155835386295 | 0.949 | 0.554 | 0.976 | 0 |
Natural Language Processing Lab, Northeastern University of China | C | b | 0.947 | 0.00148169286187 | 0.943 | 0.00153334025901 | 0.945 | 0.601 | 0.962 | 0 |
Natural Language Processing Lab, Northeastern University of China | C | a | 0.949 | 0.00145500155884 | 0.940 | 0.00157066952118 | 0.944 | 0.694 | 0.960 | 0 |
Alias-i, Inc. | C | | 0.951 | 0.00142768883377 | 0.935 | 0.00163045098737 | 0.943 | 0.389 | 0.976 | 0 |
Kookmin University | C | | 0.937 | 0.00160688698991 | 0.933 | 0.00165357343853 | 0.935 | 0.547 | 0.954 | 0 |
Institue of Software, Chinese Academy of Sciences | C | | 0.939 | 0.00158286172643 | 0.929 | 0.00169856548801 | 0.934 | 0.606 | 0.954 | 0 |
Basis Technology Corp. | C | a | 0.836 | 0.0024488959168 | 0.834 | 0.00246083408565 | 0.835 | 0.521 | 0.849 | 0 |
Basis Technology Corp. | C | b | 0.836 | 0.0024488959168 | 0.828 | 0.00249588548968 | 0.832 | 0.590 | 0.847 | 0 |
AI Lab., Department of Information Management, Huafan University, Taiwan. | C | 1 | 0.747 | 0.00287518521061 | 0.677 | 0.00309272183022 | 0.710 | 0.036 | 0.778 | 0 |
Group | Track | Runid | Recall | Cr | Precision | Cp | F-measure | OOV Recall | IV Recall | OOV Precision |
Microsoft Research Asia | O | | 0.964 | 0.00123206900348 | 0.955 | 0.00137104972827 | 0.959 | 0.704 | 0.975 | 0 |
University of Texas at Austin | O | | 0.959 | 0.00131143417326 | 0.949 | 0.00145500155884 | 0.954 | 0.672 | 0.972 | 0 |
National Laboratory on Machine Perception, Peking University, China | O | | 0.958 | 0.0013266387059 | 0.948 | 0.00146842275172 | 0.953 | 0.647 | 0.972 | 0 |
Language Technologies Institute, Carnegie Mellon University | O | a | 0.953 | 0.00139971830968 | 0.946 | 0.00149481591298 | 0.949 | 0.679 | 0.965 | 0 |
Language Technologies Institute, Carnegie Mellon University | O | b | 0.951 | 0.00142768883377 | 0.944 | 0.00152063602489 | 0.948 | 0.676 | 0.964 | 0 |
AI Lab., Department of Information Management, Huafan University, Taiwan. | O | 2 | 0.724 | 0.00295644012881 | 0.668 | 0.00311460185155 | 0.695 | 0.161 | 0.749 | 0 |
AI Lab., Department of Information Management, Huafan University, Taiwan. | O | 3 | 0.736 | 0.00291531934834 | 0.653 | 0.00314823100992 | 0.692 | 0.160 | 0.761 | 0 |
AI Lab., Department of Information Management, Huafan University, Taiwan. | O | 1 | 0.654 | 0.00314609757453 | 0.573 | 0.0032714215699 | 0.611 | 0.057 | 0.680 | 0 |
Group | Track | Runid | Recall | Cr | Precision | Cp | F-measure | OOV Recall | IV Recall | OOV Precision |
National Laboratory on Machine Perception, Peking University, China | C | | 0.964 | 0.00117608696533 | 0.961 | 0.00122220390419 | 0.963 | 0.612 | 0.976 | 0 |
Institute for Infocomm Research, Singapore | C | | 0.961 | 0.00122220390419 | 0.953 | 0.00133611871941 | 0.957 | 0.499 | 0.977 | 0 |
Alias-i, Inc. | C | | 0.959 | 0.00125184598648 | 0.955 | 0.0013087527644 | 0.957 | 0.494 | 0.975 | 0 |
Natural Language Processing Lab, Northeastern University of China | C | a | 0.955 | 0.0013087527644 | 0.956 | 0.00129480674688 | 0.956 | 0.650 | 0.966 | 0 |
Intelligent Agent Systems Lab (IASL), Academia Sinica | C | d | 0.953 | 0.00133611871941 | 0.956 | 0.00129480674688 | 0.955 | 0.574 | 0.966 | 0 |
France Telecom R&D Beijing | C | a | 0.955 | 0.0013087527644 | 0.953 | 0.00133611871941 | 0.954 | 0.575 | 0.969 | 0 |
Intelligent Agent Systems Lab (IASL), Academia Sinica | C | b | 0.952 | 0.00134954932471 | 0.956 | 0.00129480674688 | 0.954 | 0.575 | 0.966 | 0 |
Intelligent Agent Systems Lab (IASL), Academia Sinica | C | c | 0.949 | 0.0013888900403 | 0.957 | 0.001280677755 | 0.953 | 0.673 | 0.959 | 0 |
Intelligent Agent Systems Lab (IASL), Academia Sinica | C | a | 0.949 | 0.0013888900403 | 0.958 | 0.00126635966436 | 0.953 | 0.672 | 0.959 | 0 |
Simon Fraser University | C | | 0.952 | 0.00134954932471 | 0.954 | 0.00132252159599 | 0.953 | 0.604 | 0.964 | 0 |
France Telecom R&D Beijing | C | b | 0.950 | 0.00137593042545 | 0.954 | 0.00132252159599 | 0.952 | 0.602 | 0.962 | 0 |
Pattern Recognition and Intelligent System Laboratory,Beijing University of Posts and Telecommunications | C | | 0.956 | 0.00129480674688 | 0.947 | 0.00141436855935 | 0.951 | 0.493 | 0.972 | 0 |
Natural Language Processing Lab, Northeastern University of China | C | b | 0.946 | 0.00142689533287 | 0.952 | 0.00134954932471 | 0.949 | 0.568 | 0.959 | 0 |
Tung Nan Institute of Technology | C | c | 0.950 | 0.00137593042545 | 0.930 | 0.00161079459485 | 0.940 | 0.272 | 0.974 | 0 |
ITNLP, Harbin Intitute of Technolgy, China | C | a | 0.963 | 0.00119169104253 | 0.918 | 0.0017321198397 | 0.940 | 0.175 | 0.991 | 0 |
Tung Nan Institute of Technology | C | b | 0.954 | 0.00132252159599 | 0.921 | 0.00170291523188 | 0.937 | 0.163 | 0.981 | 0 |
Institue of Software, Chinese Academy of Sciences | C | | 0.933 | 0.00157843932587 | 0.942 | 0.00147566955573 | 0.937 | 0.640 | 0.943 | 0 |
Kookmin University | C | | 0.933 | 0.00157843932587 | 0.939 | 0.0015109405716 | 0.936 | 0.526 | 0.948 | 0 |
Shenyang Institute of Aeronautical Engineering | C | | 0.923 | 0.00168304559862 | 0.929 | 0.00162138705265 | 0.926 | 0.554 | 0.936 | 0 |
Tung Nan Institute of Technology | C | a | 0.949 | 0.0013888900403 | 0.897 | 0.00191895373047 | 0.922 | 0.022 | 0.982 | 0 |
Basis Technology Corp. | C | a | 0.830 | 0.00237144565273 | 0.832 | 0.00236029329871 | 0.831 | 0.473 | 0.842 | 0 |
Basis Technology Corp. | C | b | 0.817 | 0.00244110377585 | 0.821 | 0.0024201805334 | 0.819 | 0.491 | 0.829 | 0 |
Group | Track | Runid | Recall | Cr | Precision | Cp | F-measure | OOV Recall | IV Recall | OOV Precision |
France Telecom R&D Beijing | O | a | 0.980 | 0.000883848247265 | 0.978 | 0.000926041473091 | 0.979 | 0.839 | 0.985 | 0 |
France Telecom R&D Beijing | O | b | 0.977 | 0.000946369791984 | 0.976 | 0.00096622929134 | 0.977 | 0.840 | 0.982 | 0 |
Nanjing University, China | O | | 0.975 | 0.000985648309831 | 0.976 | 0.00096622929134 | 0.975 | 0.811 | 0.981 | 0 |
National Laboratory on Machine Perception, Peking University, China | O | | 0.977 | 0.000946369791984 | 0.971 | 0.00105939588883 | 0.974 | 0.675 | 0.988 | 0 |
Beijing University of Posts and Telecommunications | O | | 0.970 | 0.00107695154549 | 0.970 | 0.00107695154549 | 0.970 | 0.804 | 0.976 | 0 |
ITNLP, Harbin Intitute of Technolgy, China | O | a | 0.977 | 0.000946369791984 | 0.960 | 0.00123712983763 | 0.968 | 0.624 | 0.989 | 0 |
University of Texas at Austin | O | | 0.959 | 0.00125184598648 | 0.961 | 0.00122220390419 | 0.960 | 0.711 | 0.968 | 0 |
Language Technologies Institute, Carnegie Mellon University | O | | 0.949 | 0.0013888900403 | 0.954 | 0.00132252159599 | 0.952 | 0.692 | 0.958 | 0 |
The University of Tokyo | O | | 0.953 | 0.00133611871941 | 0.940 | 0.00149930234872 | 0.947 | 0.503 | 0.969 | 0 |
Shenyang Institute of Aeronautical Engineering | O | | 0.938 | 0.0015224636534 | 0.946 | 0.00142689533287 | 0.942 | 0.706 | 0.946 | 0 |
Group | Track | Runid | Recall | Cr | Precision | Cp | F-measure | OOV Recall | IV Recall | OOV Precision |
Intelligent Agent Systems Lab (IASL), Academia Sinica | C | d | 0.973 | 0.000690844573185 | 0.972 | 0.000703160062926 | 0.972 | 0.787 | 0.981 | 0 |
Intelligent Agent Systems Lab (IASL), Academia Sinica | C | b | 0.973 | 0.000690844573185 | 0.972 | 0.000703160062926 | 0.972 | 0.787 | 0.981 | 0 |
Microsoft Research Asia | C | | 0.972 | 0.000703168047021 | 0.971 | 0.000715246256277 | 0.971 | 0.792 | 0.979 | 0 |
National Laboratory on Machine Perception, Peking University, China | C | | 0.969 | 0.000738736767702 | 0.970 | 0.000727098877035 | 0.970 | 0.773 | 0.978 | 0 |
Natural Language Processing Lab, Northeastern University of China | C | a | 0.971 | 0.00071523813504 | 0.965 | 0.00078332092312 | 0.968 | 0.719 | 0.981 | 0 |
Intelligent Agent Systems Lab (IASL), Academia Sinica | C | c | 0.965 | 0.00078332092312 | 0.967 | 0.000761398917759 | 0.966 | 0.792 | 0.972 | 0 |
Intelligent Agent Systems Lab (IASL), Academia Sinica | C | a | 0.966 | 0.000772449452921 | 0.967 | 0.000761398917759 | 0.966 | 0.786 | 0.973 | 0 |
Institute for Infocomm Research, Singapore | C | | 0.968 | 0.0007501699221 | 0.961 | 0.000825165338189 | 0.965 | 0.633 | 0.983 | 0 |
France Telecom R&D Beijing | C | | 0.962 | 0.000814932020474 | 0.962 | 0.000814932020474 | 0.962 | 0.722 | 0.972 | 0 |
Simon Fraser University | C | | 0.963 | 0.000804555584739 | 0.958 | 0.000854967188547 | 0.961 | 0.689 | 0.974 | 0 |
Alias-i, Inc. | C | | 0.966 | 0.000772449452921 | 0.957 | 0.000864633871752 | 0.961 | 0.555 | 0.983 | 0 |
Natural Language Processing Lab, Northeastern University of China | C | b | 0.958 | 0.000854967188547 | 0.963 | 0.000804555584739 | 0.960 | 0.714 | 0.968 | 0 |
Institue of Software, Chinese Academy of Sciences | C | | 0.952 | 0.000911142012483 | 0.954 | 0.000892894365887 | 0.953 | 0.747 | 0.960 | 0 |
Kookmin University | C | | 0.950 | 0.00092895309117 | 0.949 | 0.000937702715464 | 0.949 | 0.638 | 0.963 | 0 |
Basis Technology Corp. | C | b | 0.845 | 0.00154253935039 | 0.844 | 0.00154659132801 | 0.844 | 0.632 | 0.854 | 0 |
Basis Technology Corp. | C | a | 0.841 | 0.00155861412689 | 0.844 | 0.00154659132801 | 0.843 | 0.506 | 0.855 | 0 |
AI Lab., Department of Information Management, Huafan University, Taiwan. | C | 1 | 0.589 | 0.00209713089212 | 0.589 | 0.00209713089212 | 0.589 | 0.022 | 0.613 | 0 |
Group | Track | Runid | Recall | Cr | Precision | Cp | F-measure | OOV Recall | IV Recall | OOV Precision |
Microsoft Research Asia | O | | 0.978 | 0.000625212636533 | 0.977 | 0.000638937207431 | 0.977 | 0.840 | 0.984 | 0 |
National Laboratory on Machine Perception, Peking University, China | O | | 0.979 | 0.000611150220341 | 0.976 | 0.000652345256976 | 0.977 | 0.813 | 0.985 | 0 |
University of Texas at Austin | O | | 0.971 | 0.00071523813504 | 0.967 | 0.000761398917759 | 0.969 | 0.795 | 0.978 | 0 |
CKIP, Academia Sinica, Taiwan | O | | 0.970 | 0.000727098877035 | 0.965 | 0.000783329817409 | 0.967 | 0.761 | 0.979 | 0 |
Language Technologies Institute, Carnegie Mellon University | O | | 0.964 | 0.000794020683485 | 0.964 | 0.000794020683485 | 0.964 | 0.787 | 0.971 | 0 |
AI Lab., Department of Information Management, Huafan University, Taiwan. | O | 2 | 0.544 | 0.0021228966445 | 0.549 | 0.00212090600906 | 0.547 | 0.194 | 0.559 | 0 |
AI Lab., Department of Information Management, Huafan University, Taiwan. | O | 3 | 0.524 | 0.00212870803409 | 0.503 | 0.00213112619011 | 0.513 | 0.195 | 0.538 | 0 |
AI Lab., Department of Information Management, Huafan University, Taiwan. | O | 1 | 0.497 | 0.00213112619011 | 0.467 | 0.0021265178092 | 0.481 | 0.057 | 0.516 | 0 |
Group | Track | Runid | Numphrases | Accuracy | Precision | Recall | F-Measure | Per-F | Loc-F | Org-F | GPE-F |
Nanjing University, China | C | | 6190 | 98.00 | 88.94 | 84.20 | 86.51 | 90.09 | 85.45 | 83.10 | |
Yahoo! | C | a | 6190 | 97.95 | 91.22 | 81.71 | 86.20 | 82.57 | 90.53 | 81.96 | |
Yahoo! | C | b | 6190 | 98.03 | 88.43 | 82.88 | 85.56 | 84.95 | 90.13 | 76.98 | |
National Institute of Information and Communications Technology, Japan | C | | 6190 | 98.22 | 88.14 | 82.34 | 85.14 | 79.38 | 90.62 | 81.50 | |
Yahoo! | C | f | 6190 | 98.02 | 88.45 | 79.31 | 83.63 | 75.68 | 90.03 | 80.71 | |
Yahoo! | C | g | 6190 | 98.07 | 86.61 | 80.32 | 83.35 | 77.53 | 89.91 | 77.42 | |
Institute of Information Science, Taiwan | C | avdic | 6190 | 97.80 | 86.37 | 77.67 | 81.79 | 81.26 | 82.33 | 81.38 | |
Institute of Information Science, Taiwan | C | dcrf | 6190 | 97.78 | 86.23 | 77.40 | 81.58 | 80.93 | 82.07 | 81.41 | |
Alias-i, Inc. | C | | 6190 | 97.57 | 81.88 | 80.97 | 81.42 | 81.96 | 85.29 | 72.95 | |
Institute of Information Science, Taiwan | C | cnoword | 6190 | 97.72 | 86.70 | 75.54 | 80.74 | 77.64 | 82.57 | 81.00 | |
Institute of Information Science, Taiwan | C | bvoting | 6190 | 97.62 | 85.83 | 76.06 | 80.65 | 78.99 | 81.33 | 81.45 | |
Institute for Infocomm Research, Singapore | C | | 6190 | 97.43 | 81.05 | 78.82 | 79.92 | 76.99 | 83.85 | 74.91 | |
Yahoo! | C | r | 6190 | 96.07 | 87.48 | 71.68 | 78.80 | 71.07 | 86.04 | 72.88 | |
Institue of Software, Chinese Academy of Sciences | C | | 6190 | 92.44 | 81.64 | 31.24 | 45.19 | 35.21 | 50.84 | 45.91 | |
Group | Track | Runid | Numphrases | Accuracy | Precision | Recall | F-Measure | Per-F | Loc-F | Org-F | GPE-F |
Beijing University of Posts and Telecommunications | O | | 6190 | 98.66 | 92.20 | 90.18 | 91.18 | 96.04 | 90.34 | 85.90 | |
Nanjing University, China | O | | 6190 | 98.48 | 90.76 | 89.22 | 89.99 | 92.61 | 90.99 | 83.97 | |
France Telecom R&D Beijing | O | b | 6190 | 98.18 | 87.67 | 87.53 | 87.60 | 92.25 | 89.76 | 76.11 | |
France Telecom R&D Beijing | O | a | 6190 | 97.42 | 86.45 | 83.99 | 85.20 | 91.99 | 87.45 | 69.45 | |
National Laboratory on Machine Perception, Peking University, China | O | | 6190 | 97.55 | 83.97 | 81.84 | 82.89 | 82.07 | 88.04 | 72.61 | |
The University of Tokyo | O | | 6190 | 97.09 | 84.68 | 78.22 | 81.32 | 82.80 | 85.52 | 69.58 | |
HKUST, Human Language Technology Center | O | | 6190 | 96.08 | 81.95 | 69.26 | 75.07 | 69.55 | 82.91 | 64.43 | |
ITNLP, Harbin Intitute of Technolgy, China | O | a | 6190 | 96.39 | 86.97 | 65.56 | 74.76 | 89.87 | 70.29 | 58.41 | |
Institue of Software, Chinese Academy of Sciences | O | | 6190 | 96.28 | 83.20 | 67.03 | 74.24 | 75.65 | 80.00 | 56.51 | |
NETEASE INFORMATION TECHNOLOGY (BEIJING) CO., LTD. | O | b | 6190 | 94.37 | 70.83 | 54.64 | 61.69 | 71.71 | 61.54 | 41.68 | |
NETEASE INFORMATION TECHNOLOGY (BEIJING) CO., LTD. | O | a | 6190 | 93.71 | 73.95 | 51.86 | 60.96 | 70.74 | 61.54 | 41.68 | |
Group | Track | Runid | Numphrases | Accuracy | Precision | Recall | F-Measure | Per-F | Loc-F | Org-F | GPE-F |
National Institute of Information and Communications Technology, Japan | C | | 16407 | 98.25 | 91.43 | 86.76 | 89.03 | 90.87 | 92.11 | 80.46 | |
Institute of Information Science, Taiwan | C | ccrf | 16407 | 98.18 | 92.01 | 85.45 | 88.61 | 88.72 | 92.51 | 80.54 | |
Yahoo! | C | a | 16407 | 98.10 | 92.66 | 84.75 | 88.53 | 89.37 | 92.32 | 79.73 | |
Yahoo! | C | b | 16407 | 98.11 | 92.42 | 84.91 | 88.50 | 89.20 | 92.36 | 79.76 | |
Institute of Information Science, Taiwan | C | avdic | 16407 | 98.16 | 90.79 | 86.26 | 88.47 | 89.14 | 92.33 | 79.84 | |
Institue of Software, Chinese Academy of Sciences | C | | 16407 | 97.80 | 92.76 | 81.81 | 86.94 | 87.69 | 91.14 | 77.07 | |
Yahoo! | C | f | 16407 | 97.81 | 91.88 | 82.31 | 86.83 | 86.52 | 91.05 | 78.52 | |
Yahoo! | C | g | 16407 | 97.82 | 91.64 | 82.46 | 86.81 | 86.36 | 91.14 | 78.42 | |
Alias-i, Inc. | C | | 16407 | 97.79 | 86.90 | 84.17 | 85.51 | 88.45 | 88.61 | 75.41 | |
Institute of Information Science, Taiwan | C | bme | 16407 | 98.01 | 87.42 | 83.07 | 85.19 | 83.95 | 90.15 | 76.67 | |
Institute for Infocomm Research, Singapore | C | | 16407 | 97.25 | 84.66 | 80.61 | 82.59 | 79.27 | 88.63 | 74.67 | |
National Central University at Taiwan | C | 1 | 16407 | 95.97 | 90.35 | 69.73 | 78.71 | 59.74 | 89.05 | 77.03 | |
National Taiwan University | C | | 16407 | 95.57 | 77.03 | 64.47 | 70.19 | 75.31 | 76.13 | 49.48 | |