/Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_c.data.new.merge.w1 s1_c.v2.truth.json 10 | grep 'Precision' [s1_c01_v01] Precision: 0.4000 Recall:0.8571 F1:0.5454 F2:0.6977 turk: 20 true: 7 match: 6 fp: 9 fn: 1 [s1_c01_v02] Precision: 0.6842 Recall:0.9286 F1:0.7879 F2:0.8667 turk: 20 true: 14 match: 13 fp: 6 fn: 1 [s1_c01_v03] Precision: 0.6429 Recall:0.9000 F1:0.7500 F2:0.8333 turk: 18 true: 10 match: 9 fp: 5 fn: 1 [s1_c01_v04] Precision: 0.4167 Recall:0.8333 F1:0.5556 F2:0.6944 turk: 17 true: 6 match: 5 fp: 7 fn: 1 [s1_c01_v05] Precision: 0.2273 Recall:1.0000 F1:0.3704 F2:0.5953 turk: 63 true: 10 match: 10 fp: 34 fn: 0 [s1_c02_v01] Precision: 0.4412 Recall:0.9375 F1:0.6000 F2:0.7653 turk: 44 true: 16 match: 15 fp: 19 fn: 1 [s1_c02_v02] Precision: 0.6786 Recall:0.7600 F1:0.7170 F2:0.7422 turk: 32 true: 25 match: 19 fp: 9 fn: 6 [s1_c02_v03] Precision: 0.3810 Recall:0.8889 F1:0.5334 F2:0.7018 turk: 39 true: 9 match: 8 fp: 13 fn: 1 [s1_c02_v04] Precision: 0.7000 Recall:0.8235 F1:0.7567 F2:0.7954 turk: 26 true: 17 match: 14 fp: 6 fn: 3 [s1_c02_v05] Precision: 0.6316 Recall:0.8571 F1:0.7273 F2:0.8000 turk: 24 true: 14 match: 12 fp: 7 fn: 2 [s1_c03_v01] Precision: 0.7000 Recall:0.8750 F1:0.7778 F2:0.8333 turk: 24 true: 16 match: 14 fp: 6 fn: 2 [s1_c03_v02] Precision: 0.5714 Recall:1.0000 F1:0.7272 F2:0.8696 turk: 31 true: 12 match: 12 fp: 9 fn: 0 [s1_c03_v03] Precision: 0.8421 Recall:1.0000 F1:0.9143 F2:0.9639 turk: 21 true: 16 match: 16 fp: 3 fn: 0 [s1_c03_v04] Precision: 0.5625 Recall:1.0000 F1:0.7200 F2:0.8654 turk: 25 true: 9 match: 9 fp: 7 fn: 0 [s1_c03_v05] Precision: 0.7000 Recall:0.7778 F1:0.7369 F2:0.7609 turk: 13 true: 9 match: 7 fp: 3 fn: 2 [s1_c04_v01] Precision: 0.9130 Recall:0.7241 F1:0.8077 F2:0.7554 turk: 40 true: 29 match: 21 fp: 2 fn: 8 [s1_c04_v02] Precision: 0.7895 Recall:0.6818 F1:0.7317 F2:0.7009 turk: 22 true: 22 match: 15 fp: 4 fn: 7 [s1_c04_v03] Precision: 0.6765 Recall:0.6970 F1:0.6866 F2:0.6928 turk: 44 true: 33 match: 23 fp: 11 fn: 10 [s1_c04_v04] Precision: 0.7391 Recall:0.8947 F1:0.8095 F2:0.8586 turk: 38 true: 19 match: 17 fp: 6 fn: 2 [s1_c04_v05] Precision: 0.4737 Recall:0.9000 F1:0.6207 F2:0.7627 turk: 47 true: 20 match: 18 fp: 20 fn: 2 [s1_c05_v01] Precision: 0.8571 Recall:1.0000 F1:0.9231 F2:0.9677 turk: 23 true: 12 match: 12 fp: 2 fn: 0 [s1_c05_v02] Precision: 0.5714 Recall:0.8889 F1:0.6956 F2:0.8000 turk: 15 true: 9 match: 8 fp: 6 fn: 1 [s1_c05_v03] Precision: 0.7857 Recall:1.0000 F1:0.8800 F2:0.9483 turk: 24 true: 11 match: 11 fp: 3 fn: 0 [s1_c05_v04] Precision: 0.7188 Recall:0.6765 F1:0.6970 F2:0.6846 turk: 49 true: 34 match: 23 fp: 9 fn: 11 [s1_c05_v05] Precision: 0.7083 Recall:1.0000 F1:0.8292 F2:0.9239 turk: 25 true: 17 match: 17 fp: 7 fn: 0 eps: 0.07 GLOBAL Precision: 0.6106 Recall:0.8434 F1:0.7084 F2:0.7836 /Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_c.data.new.nomerge.w1 s1_c.v2.truth.json 10 | grep 'Precision' [s1_c01_v01] Precision: 0.3750 Recall:0.8571 F1:0.5217 F2:0.6818 turk: 20 true: 7 match: 6 fp: 10 fn: 1 [s1_c01_v02] Precision: 0.7000 Recall:1.0000 F1:0.8235 F2:0.9211 turk: 20 true: 14 match: 14 fp: 6 fn: 0 [s1_c01_v03] Precision: 0.6429 Recall:0.9000 F1:0.7500 F2:0.8333 turk: 18 true: 10 match: 9 fp: 5 fn: 1 [s1_c01_v04] Precision: 0.3571 Recall:0.8333 F1:0.5000 F2:0.6578 turk: 17 true: 6 match: 5 fp: 9 fn: 1 [s1_c01_v05] Precision: 0.2174 Recall:1.0000 F1:0.3572 F2:0.5814 turk: 66 true: 10 match: 10 fp: 36 fn: 0 [s1_c02_v01] Precision: 0.4286 Recall:0.9375 F1:0.5883 F2:0.7576 turk: 44 true: 16 match: 15 fp: 20 fn: 1 [s1_c02_v02] Precision: 0.6552 Recall:0.7600 F1:0.7037 F2:0.7364 turk: 34 true: 25 match: 19 fp: 10 fn: 6 [s1_c02_v03] Precision: 0.3810 Recall:0.8889 F1:0.5334 F2:0.7018 turk: 39 true: 9 match: 8 fp: 13 fn: 1 [s1_c02_v04] Precision: 0.7000 Recall:0.8235 F1:0.7567 F2:0.7954 turk: 27 true: 17 match: 14 fp: 6 fn: 3 [s1_c02_v05] Precision: 0.6000 Recall:0.8571 F1:0.7059 F2:0.7894 turk: 25 true: 14 match: 12 fp: 8 fn: 2 [s1_c03_v01] Precision: 0.7000 Recall:0.8750 F1:0.7778 F2:0.8333 turk: 26 true: 16 match: 14 fp: 6 fn: 2 [s1_c03_v02] Precision: 0.5714 Recall:1.0000 F1:0.7272 F2:0.8696 turk: 31 true: 12 match: 12 fp: 9 fn: 0 [s1_c03_v03] Precision: 0.7895 Recall:0.9375 F1:0.8572 F2:0.9036 turk: 22 true: 16 match: 15 fp: 4 fn: 1 [s1_c03_v04] Precision: 0.5294 Recall:1.0000 F1:0.6923 F2:0.8491 turk: 26 true: 9 match: 9 fp: 8 fn: 0 [s1_c03_v05] Precision: 0.8000 Recall:0.8889 F1:0.8421 F2:0.8696 turk: 13 true: 9 match: 8 fp: 2 fn: 1 [s1_c04_v01] Precision: 0.9167 Recall:0.7586 F1:0.8302 F2:0.7857 turk: 40 true: 29 match: 22 fp: 2 fn: 7 [s1_c04_v02] Precision: 0.7895 Recall:0.6818 F1:0.7317 F2:0.7009 turk: 22 true: 22 match: 15 fp: 4 fn: 7 [s1_c04_v03] Precision: 0.6471 Recall:0.6667 F1:0.6568 F2:0.6627 turk: 44 true: 33 match: 22 fp: 12 fn: 11 [s1_c04_v04] Precision: 0.7391 Recall:0.8947 F1:0.8095 F2:0.8586 turk: 38 true: 19 match: 17 fp: 6 fn: 2 [s1_c04_v05] Precision: 0.4359 Recall:0.8500 F1:0.5763 F2:0.7143 turk: 47 true: 20 match: 17 fp: 22 fn: 3 [s1_c05_v01] Precision: 0.8571 Recall:1.0000 F1:0.9231 F2:0.9677 turk: 23 true: 12 match: 12 fp: 2 fn: 0 [s1_c05_v02] Precision: 0.5333 Recall:0.8889 F1:0.6666 F2:0.7843 turk: 17 true: 9 match: 8 fp: 7 fn: 1 [s1_c05_v03] Precision: 0.7143 Recall:0.9091 F1:0.8000 F2:0.8621 turk: 26 true: 11 match: 10 fp: 4 fn: 1 [s1_c05_v04] Precision: 0.7273 Recall:0.7059 F1:0.7164 F2:0.7101 turk: 51 true: 34 match: 24 fp: 9 fn: 10 [s1_c05_v05] Precision: 0.7083 Recall:1.0000 F1:0.8292 F2:0.9239 turk: 26 true: 17 match: 17 fp: 7 fn: 0 eps: 0.07 GLOBAL Precision: 0.5954 Recall:0.8434 F1:0.6980 F2:0.7785 /Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_c.data.new.merge.w0.9 s1_c.v2.truth.json 10 | grep 'Precision' [s1_c01_v01] Precision: 0.4286 Recall:0.8571 F1:0.5714 F2:0.7143 turk: 17 true: 7 match: 6 fp: 8 fn: 1 [s1_c01_v02] Precision: 0.7000 Recall:1.0000 F1:0.8235 F2:0.9211 turk: 20 true: 14 match: 14 fp: 6 fn: 0 [s1_c01_v03] Precision: 0.7273 Recall:0.8000 F1:0.7619 F2:0.7843 turk: 16 true: 10 match: 8 fp: 3 fn: 2 [s1_c01_v04] Precision: 0.4545 Recall:0.8333 F1:0.5882 F2:0.7142 turk: 12 true: 6 match: 5 fp: 6 fn: 1 [s1_c01_v05] Precision: 0.2857 Recall:1.0000 F1:0.4444 F2:0.6667 turk: 43 true: 10 match: 10 fp: 25 fn: 0 [s1_c02_v01] Precision: 0.5000 Recall:0.9375 F1:0.6522 F2:0.7979 turk: 42 true: 16 match: 15 fp: 15 fn: 1 [s1_c02_v02] Precision: 0.6400 Recall:0.6400 F1:0.6400 F2:0.6400 turk: 25 true: 25 match: 16 fp: 9 fn: 9 [s1_c02_v03] Precision: 0.4444 Recall:0.8889 F1:0.5926 F2:0.7407 turk: 29 true: 9 match: 8 fp: 10 fn: 1 [s1_c02_v04] Precision: 0.6667 Recall:0.8235 F1:0.7369 F2:0.7865 turk: 26 true: 17 match: 14 fp: 7 fn: 3 [s1_c02_v05] Precision: 0.6111 Recall:0.7857 F1:0.6875 F2:0.7432 turk: 21 true: 14 match: 11 fp: 7 fn: 3 [s1_c03_v01] Precision: 0.8235 Recall:0.8750 F1:0.8485 F2:0.8642 turk: 23 true: 16 match: 14 fp: 3 fn: 2 [s1_c03_v02] Precision: 0.5714 Recall:1.0000 F1:0.7272 F2:0.8696 turk: 30 true: 12 match: 12 fp: 9 fn: 0 [s1_c03_v03] Precision: 0.7222 Recall:0.8125 F1:0.7647 F2:0.7927 turk: 18 true: 16 match: 13 fp: 5 fn: 3 [s1_c03_v04] Precision: 0.6429 Recall:1.0000 F1:0.7826 F2:0.9000 turk: 22 true: 9 match: 9 fp: 5 fn: 0 [s1_c03_v05] Precision: 0.8889 Recall:0.8889 F1:0.8889 F2:0.8889 turk: 11 true: 9 match: 8 fp: 1 fn: 1 [s1_c04_v01] Precision: 0.8696 Recall:0.6897 F1:0.7693 F2:0.7195 turk: 37 true: 29 match: 20 fp: 3 fn: 9 [s1_c04_v02] Precision: 0.9333 Recall:0.6364 F1:0.7568 F2:0.6796 turk: 19 true: 22 match: 14 fp: 1 fn: 8 [s1_c04_v03] Precision: 0.6176 Recall:0.6364 F1:0.6269 F2:0.6325 turk: 41 true: 33 match: 21 fp: 13 fn: 12 [s1_c04_v04] Precision: 0.6667 Recall:0.8421 F1:0.7442 F2:0.8000 turk: 37 true: 19 match: 16 fp: 8 fn: 3 [s1_c04_v05] Precision: 0.4595 Recall:0.8500 F1:0.5965 F2:0.7265 turk: 45 true: 20 match: 17 fp: 20 fn: 3 [s1_c05_v01] Precision: 0.8000 Recall:1.0000 F1:0.8889 F2:0.9524 turk: 22 true: 12 match: 12 fp: 3 fn: 0 [s1_c05_v02] Precision: 0.6923 Recall:1.0000 F1:0.8182 F2:0.9184 turk: 16 true: 9 match: 9 fp: 4 fn: 0 [s1_c05_v03] Precision: 0.8182 Recall:0.8182 F1:0.8182 F2:0.8182 turk: 22 true: 11 match: 9 fp: 2 fn: 2 [s1_c05_v04] Precision: 0.8065 Recall:0.7353 F1:0.7693 F2:0.7485 turk: 44 true: 34 match: 25 fp: 6 fn: 9 [s1_c05_v05] Precision: 0.8000 Recall:0.9412 F1:0.8649 F2:0.9091 turk: 22 true: 17 match: 16 fp: 4 fn: 1 eps: 0.07 GLOBAL Precision: 0.6376 Recall:0.8131 F1:0.7147 F2:0.7707 /Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_c.data.new.nomerge.w0.9 s1_c.v2.truth.json 10 | grep 'Precision' [s1_c01_v01] Precision: 0.4286 Recall:0.8571 F1:0.5714 F2:0.7143 turk: 17 true: 7 match: 6 fp: 8 fn: 1 [s1_c01_v02] Precision: 0.7000 Recall:1.0000 F1:0.8235 F2:0.9211 turk: 20 true: 14 match: 14 fp: 6 fn: 0 [s1_c01_v03] Precision: 0.7273 Recall:0.8000 F1:0.7619 F2:0.7843 turk: 16 true: 10 match: 8 fp: 3 fn: 2 [s1_c01_v04] Precision: 0.4545 Recall:0.8333 F1:0.5882 F2:0.7142 turk: 12 true: 6 match: 5 fp: 6 fn: 1 [s1_c01_v05] Precision: 0.2778 Recall:1.0000 F1:0.4348 F2:0.6579 turk: 44 true: 10 match: 10 fp: 26 fn: 0 [s1_c02_v01] Precision: 0.5000 Recall:0.9375 F1:0.6522 F2:0.7979 turk: 42 true: 16 match: 15 fp: 15 fn: 1 [s1_c02_v02] Precision: 0.6400 Recall:0.6400 F1:0.6400 F2:0.6400 turk: 25 true: 25 match: 16 fp: 9 fn: 9 [s1_c02_v03] Precision: 0.4444 Recall:0.8889 F1:0.5926 F2:0.7407 turk: 29 true: 9 match: 8 fp: 10 fn: 1 [s1_c02_v04] Precision: 0.6667 Recall:0.8235 F1:0.7369 F2:0.7865 turk: 26 true: 17 match: 14 fp: 7 fn: 3 [s1_c02_v05] Precision: 0.6111 Recall:0.7857 F1:0.6875 F2:0.7432 turk: 21 true: 14 match: 11 fp: 7 fn: 3 [s1_c03_v01] Precision: 0.7778 Recall:0.8750 F1:0.8235 F2:0.8537 turk: 23 true: 16 match: 14 fp: 4 fn: 2 [s1_c03_v02] Precision: 0.5714 Recall:1.0000 F1:0.7272 F2:0.8696 turk: 30 true: 12 match: 12 fp: 9 fn: 0 [s1_c03_v03] Precision: 0.7222 Recall:0.8125 F1:0.7647 F2:0.7927 turk: 18 true: 16 match: 13 fp: 5 fn: 3 [s1_c03_v04] Precision: 0.6429 Recall:1.0000 F1:0.7826 F2:0.9000 turk: 23 true: 9 match: 9 fp: 5 fn: 0 [s1_c03_v05] Precision: 0.8889 Recall:0.8889 F1:0.8889 F2:0.8889 turk: 11 true: 9 match: 8 fp: 1 fn: 1 [s1_c04_v01] Precision: 0.8696 Recall:0.6897 F1:0.7693 F2:0.7195 turk: 37 true: 29 match: 20 fp: 3 fn: 9 [s1_c04_v02] Precision: 0.9333 Recall:0.6364 F1:0.7568 F2:0.6796 turk: 19 true: 22 match: 14 fp: 1 fn: 8 [s1_c04_v03] Precision: 0.6176 Recall:0.6364 F1:0.6269 F2:0.6325 turk: 41 true: 33 match: 21 fp: 13 fn: 12 [s1_c04_v04] Precision: 0.6667 Recall:0.8421 F1:0.7442 F2:0.8000 turk: 37 true: 19 match: 16 fp: 8 fn: 3 [s1_c04_v05] Precision: 0.4595 Recall:0.8500 F1:0.5965 F2:0.7265 turk: 45 true: 20 match: 17 fp: 20 fn: 3 [s1_c05_v01] Precision: 0.8000 Recall:1.0000 F1:0.8889 F2:0.9524 turk: 22 true: 12 match: 12 fp: 3 fn: 0 [s1_c05_v02] Precision: 0.6154 Recall:0.8889 F1:0.7273 F2:0.8163 turk: 17 true: 9 match: 8 fp: 5 fn: 1 [s1_c05_v03] Precision: 0.7500 Recall:0.8182 F1:0.7826 F2:0.8036 turk: 23 true: 11 match: 9 fp: 3 fn: 2 [s1_c05_v04] Precision: 0.7742 Recall:0.7059 F1:0.7385 F2:0.7186 turk: 46 true: 34 match: 24 fp: 7 fn: 10 [s1_c05_v05] Precision: 0.8000 Recall:0.9412 F1:0.8649 F2:0.9091 turk: 22 true: 17 match: 16 fp: 4 fn: 1 eps: 0.07 GLOBAL Precision: 0.6299 Recall:0.8081 F1:0.7080 F2:0.7648