/Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_c.data.new.merge.w1 s1_c.v3.truth.json 10 | grep 'Precision' [s1_c01_v01] Precision: 0.7692 Recall:0.8333 F1:0.8000 F2:0.8196 turk: 15 true: 13 match: 10 fp: 3 fn: 2 [s1_c01_v02] Precision: 0.7368 Recall:0.9333 F1:0.8235 F2:0.8860 turk: 19 true: 15 match: 14 fp: 5 fn: 1 [s1_c01_v03] Precision: 0.8182 Recall:0.8182 F1:0.8182 F2:0.8182 turk: 14 true: 12 match: 9 fp: 2 fn: 2 [s1_c01_v04] Precision: 0.7500 Recall:0.9000 F1:0.8182 F2:0.8654 turk: 12 true: 10 match: 9 fp: 3 fn: 1 [s1_c01_v05] Precision: 0.5682 Recall:1.0000 F1:0.7247 F2:0.8681 turk: 44 true: 25 match: 25 fp: 19 fn: 0 [s1_c02_v01] Precision: 0.8438 Recall:0.9643 F1:0.9000 F2:0.9375 turk: 34 true: 29 match: 27 fp: 5 fn: 1 [s1_c02_v02] Precision: 0.7407 Recall:0.7692 F1:0.7547 F2:0.7633 turk: 28 true: 26 match: 20 fp: 7 fn: 6 [s1_c02_v03] Precision: 0.6316 Recall:0.9231 F1:0.7500 F2:0.8451 turk: 21 true: 13 match: 12 fp: 7 fn: 1 [s1_c02_v04] Precision: 0.8824 Recall:0.8333 F1:0.8571 F2:0.8427 turk: 20 true: 19 match: 15 fp: 2 fn: 3 [s1_c02_v05] Precision: 0.6842 Recall:0.8125 F1:0.7429 F2:0.7831 turk: 19 true: 16 match: 13 fp: 6 fn: 3 [s1_c03_v01] Precision: 0.7222 Recall:1.0000 F1:0.8387 F2:0.9286 turk: 20 true: 15 match: 13 fp: 5 fn: 0 [s1_c03_v02] Precision: 0.5714 Recall:1.0000 F1:0.7272 F2:0.8696 turk: 21 true: 12 match: 12 fp: 9 fn: 0 [s1_c03_v03] Precision: 1.0000 Recall:1.0000 F1:1.0000 F2:1.0000 turk: 19 true: 19 match: 18 fp: 0 fn: 0 [s1_c03_v04] Precision: 0.7500 Recall:1.0000 F1:0.8571 F2:0.9375 turk: 16 true: 12 match: 12 fp: 4 fn: 0 [s1_c03_v05] Precision: 0.8000 Recall:0.8889 F1:0.8421 F2:0.8696 turk: 10 true: 9 match: 8 fp: 2 fn: 1 [s1_c04_v01] Precision: 0.9565 Recall:0.7333 F1:0.8302 F2:0.7692 turk: 23 true: 30 match: 22 fp: 1 fn: 8 [s1_c04_v02] Precision: 0.8421 Recall:0.7619 F1:0.8000 F2:0.7767 turk: 19 true: 21 match: 16 fp: 3 fn: 5 [s1_c04_v03] Precision: 0.7059 Recall:0.9231 F1:0.8000 F2:0.8696 turk: 34 true: 26 match: 24 fp: 10 fn: 2 [s1_c04_v04] Precision: 0.8095 Recall:0.8947 F1:0.8500 F2:0.8763 turk: 23 true: 21 match: 17 fp: 4 fn: 2 [s1_c04_v05] Precision: 0.6316 Recall:1.0000 F1:0.7742 F2:0.8955 turk: 38 true: 24 match: 24 fp: 14 fn: 0 [s1_c05_v01] Precision: 0.9286 Recall:1.0000 F1:0.9630 F2:0.9849 turk: 14 true: 13 match: 13 fp: 1 fn: 0 [s1_c05_v02] Precision: 0.7857 Recall:1.0000 F1:0.8800 F2:0.9483 turk: 14 true: 11 match: 11 fp: 3 fn: 0 [s1_c05_v03] Precision: 0.7857 Recall:1.0000 F1:0.8800 F2:0.9483 turk: 14 true: 11 match: 11 fp: 3 fn: 0 [s1_c05_v04] Precision: 0.7000 Recall:0.9130 F1:0.7924 F2:0.8606 turk: 32 true: 24 match: 21 fp: 9 fn: 2 [s1_c05_v05] Precision: 0.8696 Recall:0.9091 F1:0.8889 F2:0.9009 turk: 24 true: 22 match: 20 fp: 3 fn: 2 eps: 0.07 GLOBAL Precision: 0.7529 Recall:0.9041 F1:0.8216 F2:0.8692 /Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_m.data.new.merge.w1 s1_m.v3.truth.json 10 | grep 'Precision' [s1_m01_v01] Precision: 0.4444 Recall:0.8889 F1:0.5926 F2:0.7407 turk: 20 true: 9 match: 8 fp: 10 fn: 1 [s1_m01_v02] Precision: 0.7619 Recall:1.0000 F1:0.8649 F2:0.9412 turk: 22 true: 16 match: 16 fp: 5 fn: 0 [s1_m01_v03] Precision: 0.7059 Recall:0.8571 F1:0.7742 F2:0.8219 turk: 18 true: 14 match: 12 fp: 5 fn: 2 [s1_m01_v04] Precision: 0.7778 Recall:0.7778 F1:0.7778 F2:0.7778 turk: 21 true: 19 match: 14 fp: 4 fn: 4 [s1_m01_v05] Precision: 0.7826 Recall:0.7826 F1:0.7826 F2:0.7826 turk: 25 true: 25 match: 18 fp: 5 fn: 5 [s1_m02_v01] Precision: 0.6250 Recall:1.0000 F1:0.7692 F2:0.8929 turk: 8 true: 5 match: 5 fp: 3 fn: 0 [s1_m02_v02] Precision: 0.3571 Recall:0.6250 F1:0.4545 F2:0.5435 turk: 16 true: 10 match: 5 fp: 9 fn: 3 [s1_m02_v03] Precision: 0.5556 Recall:0.7143 F1:0.6250 F2:0.6757 turk: 9 true: 7 match: 5 fp: 4 fn: 2 [s1_m02_v04] Precision: 0.6000 Recall:1.0000 F1:0.7500 F2:0.8824 turk: 5 true: 3 match: 3 fp: 2 fn: 0 [s1_m02_v05] Precision: 0.3529 Recall:1.0000 F1:0.5217 F2:0.7317 turk: 17 true: 6 match: 6 fp: 11 fn: 0 [s1_m03_v01] Precision: 0.7778 Recall:0.5385 F1:0.6364 F2:0.5738 turk: 10 true: 13 match: 7 fp: 2 fn: 6 [s1_m03_v02] Precision: 0.4848 Recall:1.0000 F1:0.6530 F2:0.8247 turk: 35 true: 17 match: 16 fp: 17 fn: 0 [s1_m03_v03] Precision: 0.7143 Recall:0.7576 F1:0.7353 F2:0.7485 turk: 35 true: 33 match: 25 fp: 10 fn: 8 [s1_m03_v04] Precision: 0.7692 Recall:0.8333 F1:0.8000 F2:0.8196 turk: 13 true: 12 match: 10 fp: 3 fn: 2 [s1_m03_v05] Precision: 0.7143 Recall:1.0000 F1:0.8333 F2:0.9259 turk: 15 true: 10 match: 10 fp: 4 fn: 0 [s1_m04_v01] Precision: 0.4444 Recall:1.0000 F1:0.6153 F2:0.8000 turk: 13 true: 4 match: 4 fp: 5 fn: 0 [s1_m04_v02] Precision: 0.8750 Recall:1.0000 F1:0.9333 F2:0.9722 turk: 8 true: 7 match: 7 fp: 1 fn: 0 [s1_m04_v03] Precision: 0.9091 Recall:1.0000 F1:0.9524 F2:0.9804 turk: 23 true: 20 match: 20 fp: 2 fn: 0 [s1_m04_v04] Precision: 0.8667 Recall:0.6190 F1:0.7222 F2:0.6565 turk: 16 true: 21 match: 13 fp: 2 fn: 8 [s1_m04_v05] Precision: 0.5000 Recall:1.0000 F1:0.6667 F2:0.8333 turk: 6 true: 3 match: 3 fp: 3 fn: 0 [s1_m05_v01] Precision: 0.6875 Recall:0.8462 F1:0.7586 F2:0.8089 turk: 19 true: 14 match: 11 fp: 5 fn: 2 [s1_m05_v02] Precision: 0.4054 Recall:0.7895 F1:0.5357 F2:0.6637 turk: 38 true: 20 match: 15 fp: 22 fn: 4 [s1_m05_v03] Precision: 0.7692 Recall:0.5556 F1:0.6452 F2:0.5883 turk: 16 true: 20 match: 10 fp: 3 fn: 8 [s1_m05_v04] Precision: 0.8571 Recall:0.5000 F1:0.6316 F2:0.5455 turk: 9 true: 13 match: 6 fp: 1 fn: 6 [s1_m05_v05] Precision: 0.6667 Recall:0.7619 F1:0.7111 F2:0.7407 turk: 24 true: 21 match: 16 fp: 8 fn: 5 eps: 0.07 GLOBAL Precision: 0.6448 Recall:0.8006 F1:0.7143 F2:0.7637 /Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_p.data.new.merge.w1 s1_p.v3.truth.json 10 | grep 'Precision' [s1_p01_v01] Precision: 1.0000 Recall:1.0000 F1:1.0000 F2:1.0000 turk: 6 true: 5 match: 5 fp: 0 fn: 0 [s1_p01_v02] Precision: 0.5833 Recall:1.0000 F1:0.7368 F2:0.8750 turk: 14 true: 7 match: 7 fp: 5 fn: 0 [s1_p01_v03] Precision: 0.8125 Recall:1.0000 F1:0.8966 F2:0.9559 turk: 16 true: 13 match: 13 fp: 3 fn: 0 [s1_p01_v04] Precision: 0.8000 Recall:0.6667 F1:0.7273 F2:0.6897 turk: 11 true: 12 match: 8 fp: 2 fn: 4 [s1_p01_v05] Precision: 0.9333 Recall:1.0000 F1:0.9655 F2:0.9859 turk: 15 true: 14 match: 14 fp: 1 fn: 0 [s1_p02_v01] Precision: 0.7273 Recall:0.8889 F1:0.8000 F2:0.8511 turk: 11 true: 9 match: 8 fp: 3 fn: 1 [s1_p02_v03] Precision: 0.6842 Recall:0.9286 F1:0.7879 F2:0.8667 turk: 23 true: 15 match: 13 fp: 6 fn: 1 [s1_p02_v04] Precision: 0.6316 Recall:1.0000 F1:0.7742 F2:0.8955 turk: 21 true: 14 match: 12 fp: 7 fn: 0 [s1_p02_v05] Precision: 0.8750 Recall:1.0000 F1:0.9333 F2:0.9722 turk: 11 true: 9 match: 7 fp: 1 fn: 0 [s1_p03_v01] Precision: 0.9231 Recall:0.9231 F1:0.9231 F2:0.9231 turk: 14 true: 13 match: 12 fp: 1 fn: 1 [s1_p03_v02] Precision: 1.0000 Recall:0.8667 F1:0.9286 F2:0.8904 turk: 14 true: 15 match: 13 fp: 0 fn: 2 [s1_p03_v03] Precision: 0.7500 Recall:1.0000 F1:0.8571 F2:0.9375 turk: 8 true: 6 match: 6 fp: 2 fn: 0 [s1_p03_v04] Precision: 0.8261 Recall:0.9500 F1:0.8837 F2:0.9223 turk: 25 true: 21 match: 19 fp: 4 fn: 1 [s1_p03_v05] Precision: 0.7778 Recall:0.7778 F1:0.7778 F2:0.7778 turk: 19 true: 18 match: 14 fp: 4 fn: 4 [s1_p04_v01] Precision: 0.8750 Recall:0.5385 F1:0.6667 F2:0.5834 turk: 10 true: 13 match: 7 fp: 1 fn: 6 [s1_p04_v02] Precision: 1.0000 Recall:0.8750 F1:0.9333 F2:0.8974 turk: 16 true: 17 match: 14 fp: 0 fn: 2 [s1_p04_v03] Precision: 0.7500 Recall:0.8000 F1:0.7742 F2:0.7895 turk: 17 true: 15 match: 12 fp: 4 fn: 3 [s1_p04_v04] Precision: 0.8462 Recall:1.0000 F1:0.9167 F2:0.9649 turk: 13 true: 11 match: 11 fp: 2 fn: 0 [s1_p04_v05] Precision: 0.8125 Recall:0.8125 F1:0.8125 F2:0.8125 turk: 17 true: 17 match: 13 fp: 3 fn: 3 [s1_p05_v01] Precision: 0.8571 Recall:0.8571 F1:0.8571 F2:0.8571 turk: 7 true: 7 match: 6 fp: 1 fn: 1 [s1_p05_v02] Precision: 0.5714 Recall:0.5714 F1:0.5714 F2:0.5714 turk: 8 true: 8 match: 4 fp: 3 fn: 3 [s1_p05_v03] Precision: 0.9412 Recall:0.8421 F1:0.8889 F2:0.8602 turk: 19 true: 20 match: 16 fp: 1 fn: 3 [s1_p05_v04] Precision: 0.6364 Recall:1.0000 F1:0.7778 F2:0.8975 turk: 12 true: 8 match: 7 fp: 4 fn: 0 [s1_p05_v05] Precision: 0.8462 Recall:0.6875 F1:0.7586 F2:0.7143 turk: 16 true: 16 match: 11 fp: 2 fn: 5 eps: 0.07 GLOBAL Precision: 0.8077 Recall:0.8514 F1:0.8290 F2:0.8423