/Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_p.data.new.merge.w1 s1_p.v2.truth.json 10 | grep 'Precision' [s1_p01_v01] Precision: 0.5000 Recall:1.0000 F1:0.6667 F2:0.8333 turk: 10 true: 3 match: 3 fp: 3 fn: 0 [s1_p01_v02] Precision: 0.2857 Recall:1.0000 F1:0.4444 F2:0.6667 turk: 18 true: 4 match: 4 fp: 10 fn: 0 [s1_p01_v03] Precision: 0.5000 Recall:1.0000 F1:0.6667 F2:0.8333 turk: 23 true: 8 match: 8 fp: 8 fn: 0 [s1_p01_v04] Precision: 0.7273 Recall:0.6667 F1:0.6957 F2:0.6780 turk: 13 true: 12 match: 8 fp: 3 fn: 4 [s1_p01_v05] Precision: 0.8667 Recall:1.0000 F1:0.9286 F2:0.9702 turk: 19 true: 13 match: 13 fp: 2 fn: 0 [s1_p02_v01] Precision: 0.5455 Recall:0.8571 F1:0.6667 F2:0.7692 turk: 20 true: 7 match: 6 fp: 5 fn: 1 [s1_p02_v02] Precision: 0.6667 Recall:0.3333 F1:0.4444 F2:0.3703 turk: 4 true: 6 match: 2 fp: 1 fn: 4 [s1_p02_v03] Precision: 0.2609 Recall:0.8571 F1:0.4000 F2:0.5883 turk: 33 true: 7 match: 6 fp: 17 fn: 1 [s1_p02_v04] Precision: 0.2857 Recall:1.0000 F1:0.4444 F2:0.6667 turk: 27 true: 6 match: 6 fp: 15 fn: 0 [s1_p02_v05] Precision: 0.4545 Recall:0.8333 F1:0.5882 F2:0.7142 turk: 14 true: 6 match: 5 fp: 6 fn: 1 [s1_p03_v01] Precision: 0.5000 Recall:0.8750 F1:0.6364 F2:0.7609 turk: 18 true: 8 match: 7 fp: 7 fn: 1 [s1_p03_v02] Precision: 0.5714 Recall:0.8000 F1:0.6666 F2:0.7407 turk: 28 true: 10 match: 8 fp: 6 fn: 2 [s1_p03_v03] Precision: 0.3750 Recall:1.0000 F1:0.5455 F2:0.7500 turk: 8 true: 3 match: 3 fp: 5 fn: 0 [s1_p03_v04] Precision: 0.4400 Recall:0.9167 F1:0.5946 F2:0.7534 turk: 42 true: 12 match: 11 fp: 14 fn: 1 [s1_p03_v05] Precision: 0.5789 Recall:0.6875 F1:0.6285 F2:0.6626 turk: 26 true: 16 match: 11 fp: 8 fn: 5 [s1_p04_v01] Precision: 0.3000 Recall:0.3750 F1:0.3333 F2:0.3571 turk: 15 true: 8 match: 3 fp: 7 fn: 5 [s1_p04_v02] Precision: 0.5000 Recall:0.8000 F1:0.6154 F2:0.7143 turk: 26 true: 10 match: 8 fp: 8 fn: 2 [s1_p04_v03] Precision: 0.4118 Recall:0.7000 F1:0.5185 F2:0.6141 turk: 31 true: 10 match: 7 fp: 10 fn: 3 [s1_p04_v04] Precision: 0.6154 Recall:1.0000 F1:0.7619 F2:0.8889 turk: 27 true: 8 match: 8 fp: 5 fn: 0 [s1_p04_v05] Precision: 0.6471 Recall:0.7857 F1:0.7097 F2:0.7534 turk: 22 true: 14 match: 11 fp: 6 fn: 3 [s1_p05_v01] Precision: 0.8571 Recall:0.8571 F1:0.8571 F2:0.8571 turk: 12 true: 7 match: 6 fp: 1 fn: 1 [s1_p05_v02] Precision: 0.6250 Recall:0.6250 F1:0.6250 F2:0.6250 turk: 10 true: 8 match: 5 fp: 3 fn: 3 [s1_p05_v03] Precision: 0.8421 Recall:0.8000 F1:0.8205 F2:0.8081 turk: 20 true: 20 match: 16 fp: 3 fn: 4 [s1_p05_v04] Precision: 0.6667 Recall:1.0000 F1:0.8000 F2:0.9091 turk: 19 true: 8 match: 8 fp: 4 fn: 0 [s1_p05_v05] Precision: 0.6875 Recall:0.6875 F1:0.6875 F2:0.6875 turk: 29 true: 16 match: 11 fp: 5 fn: 5 eps: 0.07 GLOBAL Precision: 0.5318 Recall:0.8000 F1:0.6389 F2:0.7267 /Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_p.data.new.nomerge.w1 s1_p.v2.truth.json 10 | grep 'Precision' [s1_p01_v01] Precision: 0.5000 Recall:1.0000 F1:0.6667 F2:0.8333 turk: 10 true: 3 match: 3 fp: 3 fn: 0 [s1_p01_v02] Precision: 0.2857 Recall:1.0000 F1:0.4444 F2:0.6667 turk: 18 true: 4 match: 4 fp: 10 fn: 0 [s1_p01_v03] Precision: 0.4706 Recall:1.0000 F1:0.6400 F2:0.8163 turk: 24 true: 8 match: 8 fp: 9 fn: 0 [s1_p01_v04] Precision: 0.7273 Recall:0.6667 F1:0.6957 F2:0.6780 turk: 13 true: 12 match: 8 fp: 3 fn: 4 [s1_p01_v05] Precision: 0.8667 Recall:1.0000 F1:0.9286 F2:0.9702 turk: 19 true: 13 match: 13 fp: 2 fn: 0 [s1_p02_v01] Precision: 0.5455 Recall:0.8571 F1:0.6667 F2:0.7692 turk: 20 true: 7 match: 6 fp: 5 fn: 1 [s1_p02_v02] Precision: 0.6667 Recall:0.3333 F1:0.4444 F2:0.3703 turk: 4 true: 6 match: 2 fp: 1 fn: 4 [s1_p02_v03] Precision: 0.2500 Recall:0.8571 F1:0.3871 F2:0.5769 turk: 34 true: 7 match: 6 fp: 18 fn: 1 [s1_p02_v04] Precision: 0.2857 Recall:1.0000 F1:0.4444 F2:0.6667 turk: 27 true: 6 match: 6 fp: 15 fn: 0 [s1_p02_v05] Precision: 0.4545 Recall:0.8333 F1:0.5882 F2:0.7142 turk: 14 true: 6 match: 5 fp: 6 fn: 1 [s1_p03_v01] Precision: 0.5000 Recall:0.8750 F1:0.6364 F2:0.7609 turk: 18 true: 8 match: 7 fp: 7 fn: 1 [s1_p03_v02] Precision: 0.5714 Recall:0.8000 F1:0.6666 F2:0.7407 turk: 28 true: 10 match: 8 fp: 6 fn: 2 [s1_p03_v03] Precision: 0.3750 Recall:1.0000 F1:0.5455 F2:0.7500 turk: 8 true: 3 match: 3 fp: 5 fn: 0 [s1_p03_v04] Precision: 0.4400 Recall:0.9167 F1:0.5946 F2:0.7534 turk: 42 true: 12 match: 11 fp: 14 fn: 1 [s1_p03_v05] Precision: 0.5789 Recall:0.6875 F1:0.6285 F2:0.6626 turk: 26 true: 16 match: 11 fp: 8 fn: 5 [s1_p04_v01] Precision: 0.3000 Recall:0.3750 F1:0.3333 F2:0.3571 turk: 15 true: 8 match: 3 fp: 7 fn: 5 [s1_p04_v02] Precision: 0.5000 Recall:0.8000 F1:0.6154 F2:0.7143 turk: 26 true: 10 match: 8 fp: 8 fn: 2 [s1_p04_v03] Precision: 0.4118 Recall:0.7000 F1:0.5185 F2:0.6141 turk: 31 true: 10 match: 7 fp: 10 fn: 3 [s1_p04_v04] Precision: 0.5714 Recall:1.0000 F1:0.7272 F2:0.8696 turk: 27 true: 8 match: 8 fp: 6 fn: 0 [s1_p04_v05] Precision: 0.6471 Recall:0.7857 F1:0.7097 F2:0.7534 turk: 23 true: 14 match: 11 fp: 6 fn: 3 [s1_p05_v01] Precision: 0.7500 Recall:0.8571 F1:0.8000 F2:0.8333 turk: 12 true: 7 match: 6 fp: 2 fn: 1 [s1_p05_v02] Precision: 0.6250 Recall:0.6250 F1:0.6250 F2:0.6250 turk: 10 true: 8 match: 5 fp: 3 fn: 3 [s1_p05_v03] Precision: 0.8421 Recall:0.8000 F1:0.8205 F2:0.8081 turk: 20 true: 20 match: 16 fp: 3 fn: 4 [s1_p05_v04] Precision: 0.6154 Recall:1.0000 F1:0.7619 F2:0.8889 turk: 19 true: 8 match: 8 fp: 5 fn: 0 [s1_p05_v05] Precision: 0.6875 Recall:0.6875 F1:0.6875 F2:0.6875 turk: 29 true: 16 match: 11 fp: 5 fn: 5 eps: 0.07 GLOBAL Precision: 0.5242 Recall:0.8000 F1:0.6334 F2:0.7238 /Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_p.data.new.merge.w0.9 s1_p.v2.truth.json 10 | grep 'Precision' [s1_p01_v01] Precision: 0.5000 Recall:1.0000 F1:0.6667 F2:0.8333 turk: 10 true: 3 match: 3 fp: 3 fn: 0 [s1_p01_v02] Precision: 0.3636 Recall:1.0000 F1:0.5333 F2:0.7407 turk: 15 true: 4 match: 4 fp: 7 fn: 0 [s1_p01_v03] Precision: 0.5333 Recall:1.0000 F1:0.6956 F2:0.8510 turk: 18 true: 8 match: 8 fp: 7 fn: 0 [s1_p01_v04] Precision: 0.7273 Recall:0.6667 F1:0.6957 F2:0.6780 turk: 13 true: 12 match: 8 fp: 3 fn: 4 [s1_p01_v05] Precision: 0.9286 Recall:1.0000 F1:0.9630 F2:0.9849 turk: 18 true: 13 match: 13 fp: 1 fn: 0 [s1_p02_v01] Precision: 0.5556 Recall:0.7143 F1:0.6250 F2:0.6757 turk: 17 true: 7 match: 5 fp: 4 fn: 2 [s1_p02_v02] Precision: 0.6667 Recall:0.3333 F1:0.4444 F2:0.3703 turk: 4 true: 6 match: 2 fp: 1 fn: 4 [s1_p02_v03] Precision: 0.2500 Recall:0.7143 F1:0.3704 F2:0.5208 turk: 27 true: 7 match: 5 fp: 15 fn: 2 [s1_p02_v04] Precision: 0.3158 Recall:1.0000 F1:0.4800 F2:0.6977 turk: 23 true: 6 match: 6 fp: 13 fn: 0 [s1_p02_v05] Precision: 0.4545 Recall:0.8333 F1:0.5882 F2:0.7142 turk: 12 true: 6 match: 5 fp: 6 fn: 1 [s1_p03_v01] Precision: 0.5385 Recall:0.8750 F1:0.6667 F2:0.7778 turk: 17 true: 8 match: 7 fp: 6 fn: 1 [s1_p03_v02] Precision: 0.5385 Recall:0.7000 F1:0.6087 F2:0.6604 turk: 25 true: 10 match: 7 fp: 6 fn: 3 [s1_p03_v03] Precision: 0.3750 Recall:1.0000 F1:0.5455 F2:0.7500 turk: 8 true: 3 match: 3 fp: 5 fn: 0 [s1_p03_v04] Precision: 0.5000 Recall:0.9167 F1:0.6471 F2:0.7857 turk: 39 true: 12 match: 11 fp: 11 fn: 1 [s1_p03_v05] Precision: 0.7143 Recall:0.6250 F1:0.6667 F2:0.6410 turk: 19 true: 16 match: 10 fp: 4 fn: 6 [s1_p04_v01] Precision: 0.2500 Recall:0.2500 F1:0.2500 F2:0.2500 turk: 12 true: 8 match: 2 fp: 6 fn: 6 [s1_p04_v02] Precision: 0.5000 Recall:0.9000 F1:0.6429 F2:0.7759 turk: 24 true: 10 match: 9 fp: 9 fn: 1 [s1_p04_v03] Precision: 0.4375 Recall:0.7000 F1:0.5385 F2:0.6250 turk: 27 true: 10 match: 7 fp: 9 fn: 3 [s1_p04_v04] Precision: 0.5333 Recall:1.0000 F1:0.6956 F2:0.8510 turk: 25 true: 8 match: 8 fp: 7 fn: 0 [s1_p04_v05] Precision: 0.6667 Recall:0.7143 F1:0.6897 F2:0.7042 turk: 21 true: 14 match: 10 fp: 5 fn: 4 [s1_p05_v01] Precision: 0.8571 Recall:0.8571 F1:0.8571 F2:0.8571 turk: 12 true: 7 match: 6 fp: 1 fn: 1 [s1_p05_v02] Precision: 0.6250 Recall:0.6250 F1:0.6250 F2:0.6250 turk: 8 true: 8 match: 5 fp: 3 fn: 3 [s1_p05_v03] Precision: 0.8824 Recall:0.7500 F1:0.8108 F2:0.7732 turk: 17 true: 20 match: 15 fp: 2 fn: 5 [s1_p05_v04] Precision: 0.6154 Recall:1.0000 F1:0.7619 F2:0.8889 turk: 18 true: 8 match: 8 fp: 5 fn: 0 [s1_p05_v05] Precision: 0.6471 Recall:0.6875 F1:0.6667 F2:0.6790 turk: 26 true: 16 match: 11 fp: 6 fn: 5 eps: 0.07 GLOBAL Precision: 0.5511 Recall:0.7739 F1:0.6438 F2:0.7160 /Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_p.data.new.nomerge.w0.9 s1_p.v2.truth.json 10 | grep 'Precision' [s1_p01_v01] Precision: 0.5000 Recall:1.0000 F1:0.6667 F2:0.8333 turk: 10 true: 3 match: 3 fp: 3 fn: 0 [s1_p01_v02] Precision: 0.3636 Recall:1.0000 F1:0.5333 F2:0.7407 turk: 15 true: 4 match: 4 fp: 7 fn: 0 [s1_p01_v03] Precision: 0.5333 Recall:1.0000 F1:0.6956 F2:0.8510 turk: 18 true: 8 match: 8 fp: 7 fn: 0 [s1_p01_v04] Precision: 0.7273 Recall:0.6667 F1:0.6957 F2:0.6780 turk: 13 true: 12 match: 8 fp: 3 fn: 4 [s1_p01_v05] Precision: 0.9286 Recall:1.0000 F1:0.9630 F2:0.9849 turk: 18 true: 13 match: 13 fp: 1 fn: 0 [s1_p02_v01] Precision: 0.5556 Recall:0.7143 F1:0.6250 F2:0.6757 turk: 17 true: 7 match: 5 fp: 4 fn: 2 [s1_p02_v02] Precision: 0.6667 Recall:0.3333 F1:0.4444 F2:0.3703 turk: 4 true: 6 match: 2 fp: 1 fn: 4 [s1_p02_v03] Precision: 0.2381 Recall:0.7143 F1:0.3572 F2:0.5102 turk: 28 true: 7 match: 5 fp: 16 fn: 2 [s1_p02_v04] Precision: 0.3158 Recall:1.0000 F1:0.4800 F2:0.6977 turk: 23 true: 6 match: 6 fp: 13 fn: 0 [s1_p02_v05] Precision: 0.4545 Recall:0.8333 F1:0.5882 F2:0.7142 turk: 12 true: 6 match: 5 fp: 6 fn: 1 [s1_p03_v01] Precision: 0.5385 Recall:0.8750 F1:0.6667 F2:0.7778 turk: 17 true: 8 match: 7 fp: 6 fn: 1 [s1_p03_v02] Precision: 0.5385 Recall:0.7000 F1:0.6087 F2:0.6604 turk: 25 true: 10 match: 7 fp: 6 fn: 3 [s1_p03_v03] Precision: 0.3750 Recall:1.0000 F1:0.5455 F2:0.7500 turk: 8 true: 3 match: 3 fp: 5 fn: 0 [s1_p03_v04] Precision: 0.5000 Recall:0.9167 F1:0.6471 F2:0.7857 turk: 39 true: 12 match: 11 fp: 11 fn: 1 [s1_p03_v05] Precision: 0.7143 Recall:0.6250 F1:0.6667 F2:0.6410 turk: 19 true: 16 match: 10 fp: 4 fn: 6 [s1_p04_v01] Precision: 0.2500 Recall:0.2500 F1:0.2500 F2:0.2500 turk: 12 true: 8 match: 2 fp: 6 fn: 6 [s1_p04_v02] Precision: 0.5000 Recall:0.9000 F1:0.6429 F2:0.7759 turk: 24 true: 10 match: 9 fp: 9 fn: 1 [s1_p04_v03] Precision: 0.4375 Recall:0.7000 F1:0.5385 F2:0.6250 turk: 27 true: 10 match: 7 fp: 9 fn: 3 [s1_p04_v04] Precision: 0.5333 Recall:1.0000 F1:0.6956 F2:0.8510 turk: 25 true: 8 match: 8 fp: 7 fn: 0 [s1_p04_v05] Precision: 0.6667 Recall:0.7143 F1:0.6897 F2:0.7042 turk: 21 true: 14 match: 10 fp: 5 fn: 4 [s1_p05_v01] Precision: 0.8571 Recall:0.8571 F1:0.8571 F2:0.8571 turk: 12 true: 7 match: 6 fp: 1 fn: 1 [s1_p05_v02] Precision: 0.6250 Recall:0.6250 F1:0.6250 F2:0.6250 turk: 8 true: 8 match: 5 fp: 3 fn: 3 [s1_p05_v03] Precision: 0.8824 Recall:0.7500 F1:0.8108 F2:0.7732 turk: 17 true: 20 match: 15 fp: 2 fn: 5 [s1_p05_v04] Precision: 0.6154 Recall:1.0000 F1:0.7619 F2:0.8889 turk: 18 true: 8 match: 8 fp: 5 fn: 0 [s1_p05_v05] Precision: 0.6471 Recall:0.6875 F1:0.6667 F2:0.6790 turk: 26 true: 16 match: 11 fp: 6 fn: 5 eps: 0.07 GLOBAL Precision: 0.5494 Recall:0.7739 F1:0.6426 F2:0.7154