/Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_m.data.new.merge.w1 s1_m.v3.truth.json 10 | grep 'Precision' [s1_m01_v01] Precision: 0.4000 Recall:0.8889 F1:0.5517 F2:0.7143 turk: 28 true: 9 match: 8 fp: 12 fn: 1 [s1_m01_v02] Precision: 0.7273 Recall:1.0000 F1:0.8421 F2:0.9302 turk: 34 true: 16 match: 16 fp: 6 fn: 0 [s1_m01_v03] Precision: 0.6667 Recall:0.8571 F1:0.7500 F2:0.8108 turk: 26 true: 14 match: 12 fp: 6 fn: 2 [s1_m01_v04] Precision: 0.6190 Recall:0.6842 F1:0.6500 F2:0.6701 turk: 29 true: 19 match: 13 fp: 8 fn: 6 [s1_m01_v05] Precision: 0.7600 Recall:0.7600 F1:0.7600 F2:0.7600 turk: 31 true: 25 match: 19 fp: 6 fn: 6 [s1_m02_v01] Precision: 0.5000 Recall:0.8000 F1:0.6154 F2:0.7143 turk: 10 true: 5 match: 4 fp: 4 fn: 1 [s1_m02_v02] Precision: 0.4375 Recall:0.7000 F1:0.5385 F2:0.6250 turk: 24 true: 10 match: 7 fp: 9 fn: 3 [s1_m02_v03] Precision: 0.5556 Recall:0.7143 F1:0.6250 F2:0.6757 turk: 18 true: 7 match: 5 fp: 4 fn: 2 [s1_m02_v04] Precision: 0.6000 Recall:1.0000 F1:0.7500 F2:0.8824 turk: 7 true: 3 match: 3 fp: 2 fn: 0 [s1_m02_v05] Precision: 0.3529 Recall:1.0000 F1:0.5217 F2:0.7317 turk: 19 true: 6 match: 6 fp: 11 fn: 0 [s1_m03_v01] Precision: 0.7000 Recall:0.5385 F1:0.6087 F2:0.5645 turk: 16 true: 13 match: 7 fp: 3 fn: 6 [s1_m03_v02] Precision: 0.4571 Recall:0.9412 F1:0.6154 F2:0.7767 turk: 40 true: 17 match: 16 fp: 19 fn: 1 [s1_m03_v03] Precision: 0.7143 Recall:0.7576 F1:0.7353 F2:0.7485 turk: 43 true: 33 match: 25 fp: 10 fn: 8 [s1_m03_v04] Precision: 0.7692 Recall:0.8333 F1:0.8000 F2:0.8196 turk: 21 true: 12 match: 10 fp: 3 fn: 2 [s1_m03_v05] Precision: 0.6667 Recall:1.0000 F1:0.8000 F2:0.9091 turk: 18 true: 10 match: 10 fp: 5 fn: 0 [s1_m04_v01] Precision: 0.3077 Recall:1.0000 F1:0.4706 F2:0.6897 turk: 16 true: 4 match: 4 fp: 9 fn: 0 [s1_m04_v02] Precision: 0.8750 Recall:1.0000 F1:0.9333 F2:0.9722 turk: 10 true: 7 match: 7 fp: 1 fn: 0 [s1_m04_v03] Precision: 0.8261 Recall:0.9500 F1:0.8837 F2:0.9223 turk: 29 true: 20 match: 19 fp: 4 fn: 1 [s1_m04_v04] Precision: 0.8125 Recall:0.6190 F1:0.7027 F2:0.6500 turk: 25 true: 21 match: 13 fp: 3 fn: 8 [s1_m04_v05] Precision: 0.5000 Recall:1.0000 F1:0.6667 F2:0.8333 turk: 11 true: 3 match: 3 fp: 3 fn: 0 [s1_m05_v01] Precision: 0.5789 Recall:0.7857 F1:0.6666 F2:0.7333 turk: 24 true: 14 match: 11 fp: 8 fn: 3 [s1_m05_v02] Precision: 0.3947 Recall:0.7500 F1:0.5172 F2:0.6356 turk: 53 true: 20 match: 15 fp: 23 fn: 5 [s1_m05_v03] Precision: 0.7500 Recall:0.6000 F1:0.6667 F2:0.6250 turk: 19 true: 20 match: 12 fp: 4 fn: 8 [s1_m05_v04] Precision: 0.6667 Recall:0.4615 F1:0.5454 F2:0.4918 turk: 15 true: 13 match: 6 fp: 3 fn: 7 [s1_m05_v05] Precision: 0.6667 Recall:0.7619 F1:0.7111 F2:0.7407 turk: 33 true: 21 match: 16 fp: 8 fn: 5 eps: 0.07 GLOBAL Precision: 0.6054 Recall:0.7807 F1:0.6820 F2:0.7380 /Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_m.data.new.nomerge.w1 s1_m.v3.truth.json 10 | grep 'Precision' [s1_m01_v01] Precision: 0.4000 Recall:0.8889 F1:0.5517 F2:0.7143 turk: 28 true: 9 match: 8 fp: 12 fn: 1 [s1_m01_v02] Precision: 0.7273 Recall:1.0000 F1:0.8421 F2:0.9302 turk: 36 true: 16 match: 16 fp: 6 fn: 0 [s1_m01_v03] Precision: 0.6667 Recall:0.8571 F1:0.7500 F2:0.8108 turk: 26 true: 14 match: 12 fp: 6 fn: 2 [s1_m01_v04] Precision: 0.6667 Recall:0.7368 F1:0.7000 F2:0.7216 turk: 29 true: 19 match: 14 fp: 7 fn: 5 [s1_m01_v05] Precision: 0.7600 Recall:0.7600 F1:0.7600 F2:0.7600 turk: 31 true: 25 match: 19 fp: 6 fn: 6 [s1_m02_v01] Precision: 0.5000 Recall:0.8000 F1:0.6154 F2:0.7143 turk: 11 true: 5 match: 4 fp: 4 fn: 1 [s1_m02_v02] Precision: 0.4118 Recall:0.7000 F1:0.5185 F2:0.6141 turk: 25 true: 10 match: 7 fp: 10 fn: 3 [s1_m02_v03] Precision: 0.5556 Recall:0.7143 F1:0.6250 F2:0.6757 turk: 18 true: 7 match: 5 fp: 4 fn: 2 [s1_m02_v04] Precision: 0.6000 Recall:1.0000 F1:0.7500 F2:0.8824 turk: 7 true: 3 match: 3 fp: 2 fn: 0 [s1_m02_v05] Precision: 0.3529 Recall:1.0000 F1:0.5217 F2:0.7317 turk: 19 true: 6 match: 6 fp: 11 fn: 0 [s1_m03_v01] Precision: 0.7000 Recall:0.5385 F1:0.6087 F2:0.5645 turk: 16 true: 13 match: 7 fp: 3 fn: 6 [s1_m03_v02] Precision: 0.4444 Recall:0.9412 F1:0.6037 F2:0.7692 turk: 41 true: 17 match: 16 fp: 20 fn: 1 [s1_m03_v03] Precision: 0.6944 Recall:0.7576 F1:0.7246 F2:0.7441 turk: 44 true: 33 match: 25 fp: 11 fn: 8 [s1_m03_v04] Precision: 0.7692 Recall:0.8333 F1:0.8000 F2:0.8196 turk: 21 true: 12 match: 10 fp: 3 fn: 2 [s1_m03_v05] Precision: 0.6667 Recall:1.0000 F1:0.8000 F2:0.9091 turk: 18 true: 10 match: 10 fp: 5 fn: 0 [s1_m04_v01] Precision: 0.3077 Recall:1.0000 F1:0.4706 F2:0.6897 turk: 18 true: 4 match: 4 fp: 9 fn: 0 [s1_m04_v02] Precision: 0.8750 Recall:1.0000 F1:0.9333 F2:0.9722 turk: 10 true: 7 match: 7 fp: 1 fn: 0 [s1_m04_v03] Precision: 0.8333 Recall:1.0000 F1:0.9091 F2:0.9615 turk: 29 true: 20 match: 20 fp: 4 fn: 0 [s1_m04_v04] Precision: 0.8125 Recall:0.6190 F1:0.7027 F2:0.6500 turk: 25 true: 21 match: 13 fp: 3 fn: 8 [s1_m04_v05] Precision: 0.5000 Recall:1.0000 F1:0.6667 F2:0.8333 turk: 11 true: 3 match: 3 fp: 3 fn: 0 [s1_m05_v01] Precision: 0.5789 Recall:0.7857 F1:0.6666 F2:0.7333 turk: 24 true: 14 match: 11 fp: 8 fn: 3 [s1_m05_v02] Precision: 0.3902 Recall:0.8000 F1:0.5246 F2:0.6611 turk: 54 true: 20 match: 16 fp: 25 fn: 4 [s1_m05_v03] Precision: 0.7059 Recall:0.6000 F1:0.6487 F2:0.6186 turk: 19 true: 20 match: 12 fp: 5 fn: 8 [s1_m05_v04] Precision: 0.6667 Recall:0.4615 F1:0.5454 F2:0.4918 turk: 16 true: 13 match: 6 fp: 3 fn: 7 [s1_m05_v05] Precision: 0.6667 Recall:0.7619 F1:0.7111 F2:0.7407 turk: 33 true: 21 match: 16 fp: 8 fn: 5 eps: 0.07 GLOBAL Precision: 0.6013 Recall:0.7895 F1:0.6827 F2:0.7430 /Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_m.data.new.merge.w0.9 s1_m.v3.truth.json 10 | grep 'Precision' [s1_m01_v01] Precision: 0.4706 Recall:0.8889 F1:0.6154 F2:0.7547 turk: 25 true: 9 match: 8 fp: 9 fn: 1 [s1_m01_v02] Precision: 0.7222 Recall:0.8125 F1:0.7647 F2:0.7927 turk: 27 true: 16 match: 13 fp: 5 fn: 3 [s1_m01_v03] Precision: 0.7059 Recall:0.8571 F1:0.7742 F2:0.8219 turk: 24 true: 14 match: 12 fp: 5 fn: 2 [s1_m01_v04] Precision: 0.6000 Recall:0.6316 F1:0.6154 F2:0.6250 turk: 26 true: 19 match: 12 fp: 8 fn: 7 [s1_m01_v05] Precision: 0.8261 Recall:0.7600 F1:0.7917 F2:0.7724 turk: 25 true: 25 match: 19 fp: 4 fn: 6 [s1_m02_v01] Precision: 0.5000 Recall:0.8000 F1:0.6154 F2:0.7143 turk: 10 true: 5 match: 4 fp: 4 fn: 1 [s1_m02_v02] Precision: 0.4667 Recall:0.7000 F1:0.5600 F2:0.6364 turk: 20 true: 10 match: 7 fp: 8 fn: 3 [s1_m02_v03] Precision: 0.5000 Recall:0.7143 F1:0.5882 F2:0.6579 turk: 17 true: 7 match: 5 fp: 5 fn: 2 [s1_m02_v04] Precision: 0.6000 Recall:1.0000 F1:0.7500 F2:0.8824 turk: 6 true: 3 match: 3 fp: 2 fn: 0 [s1_m02_v05] Precision: 0.4000 Recall:1.0000 F1:0.5714 F2:0.7692 turk: 16 true: 6 match: 6 fp: 9 fn: 0 [s1_m03_v01] Precision: 0.7000 Recall:0.5385 F1:0.6087 F2:0.5645 turk: 16 true: 13 match: 7 fp: 3 fn: 6 [s1_m03_v02] Precision: 0.6250 Recall:0.8824 F1:0.7317 F2:0.8152 turk: 28 true: 17 match: 15 fp: 9 fn: 2 [s1_m03_v03] Precision: 0.7419 Recall:0.6970 F1:0.7187 F2:0.7055 turk: 40 true: 33 match: 23 fp: 8 fn: 10 [s1_m03_v04] Precision: 0.7692 Recall:0.8333 F1:0.8000 F2:0.8196 turk: 19 true: 12 match: 10 fp: 3 fn: 2 [s1_m03_v05] Precision: 0.6429 Recall:0.9000 F1:0.7500 F2:0.8333 turk: 17 true: 10 match: 9 fp: 5 fn: 1 [s1_m04_v01] Precision: 0.2857 Recall:1.0000 F1:0.4444 F2:0.6667 turk: 14 true: 4 match: 4 fp: 10 fn: 0 [s1_m04_v02] Precision: 0.8750 Recall:1.0000 F1:0.9333 F2:0.9722 turk: 10 true: 7 match: 7 fp: 1 fn: 0 [s1_m04_v03] Precision: 0.7273 Recall:0.8000 F1:0.7619 F2:0.7843 turk: 25 true: 20 match: 16 fp: 6 fn: 4 [s1_m04_v04] Precision: 0.8667 Recall:0.6190 F1:0.7222 F2:0.6565 turk: 22 true: 21 match: 13 fp: 2 fn: 8 [s1_m04_v05] Precision: 0.5000 Recall:1.0000 F1:0.6667 F2:0.8333 turk: 11 true: 3 match: 3 fp: 3 fn: 0 [s1_m05_v01] Precision: 0.5789 Recall:0.7857 F1:0.6666 F2:0.7333 turk: 22 true: 14 match: 11 fp: 8 fn: 3 [s1_m05_v02] Precision: 0.4839 Recall:0.7500 F1:0.5883 F2:0.6757 turk: 36 true: 20 match: 15 fp: 16 fn: 5 [s1_m05_v03] Precision: 0.8125 Recall:0.6500 F1:0.7222 F2:0.6771 turk: 17 true: 20 match: 13 fp: 3 fn: 7 [s1_m05_v04] Precision: 0.6667 Recall:0.4615 F1:0.5454 F2:0.4918 turk: 15 true: 13 match: 6 fp: 3 fn: 7 [s1_m05_v05] Precision: 0.7391 Recall:0.8095 F1:0.7727 F2:0.7944 turk: 32 true: 21 match: 17 fp: 6 fn: 4 eps: 0.07 GLOBAL Precision: 0.6402 Recall:0.7544 F1:0.6926 F2:0.7284 /Applications/MAMP/htdocs/CrowdsourcingPhotoshop/Tools > python compute-accuracy.py data/s1_m.data.new.nomerge.w0.9 s1_m.v3.truth.json 10 | grep 'Precision' [s1_m01_v01] Precision: 0.4706 Recall:0.8889 F1:0.6154 F2:0.7547 turk: 25 true: 9 match: 8 fp: 9 fn: 1 [s1_m01_v02] Precision: 0.7222 Recall:0.8125 F1:0.7647 F2:0.7927 turk: 27 true: 16 match: 13 fp: 5 fn: 3 [s1_m01_v03] Precision: 0.7059 Recall:0.8571 F1:0.7742 F2:0.8219 turk: 24 true: 14 match: 12 fp: 5 fn: 2 [s1_m01_v04] Precision: 0.6000 Recall:0.6316 F1:0.6154 F2:0.6250 turk: 26 true: 19 match: 12 fp: 8 fn: 7 [s1_m01_v05] Precision: 0.8261 Recall:0.7600 F1:0.7917 F2:0.7724 turk: 25 true: 25 match: 19 fp: 4 fn: 6 [s1_m02_v01] Precision: 0.5000 Recall:0.8000 F1:0.6154 F2:0.7143 turk: 10 true: 5 match: 4 fp: 4 fn: 1 [s1_m02_v02] Precision: 0.4667 Recall:0.7000 F1:0.5600 F2:0.6364 turk: 20 true: 10 match: 7 fp: 8 fn: 3 [s1_m02_v03] Precision: 0.5000 Recall:0.7143 F1:0.5882 F2:0.6579 turk: 17 true: 7 match: 5 fp: 5 fn: 2 [s1_m02_v04] Precision: 0.6000 Recall:1.0000 F1:0.7500 F2:0.8824 turk: 6 true: 3 match: 3 fp: 2 fn: 0 [s1_m02_v05] Precision: 0.4000 Recall:1.0000 F1:0.5714 F2:0.7692 turk: 16 true: 6 match: 6 fp: 9 fn: 0 [s1_m03_v01] Precision: 0.7000 Recall:0.5385 F1:0.6087 F2:0.5645 turk: 16 true: 13 match: 7 fp: 3 fn: 6 [s1_m03_v02] Precision: 0.6400 Recall:0.9412 F1:0.7619 F2:0.8602 turk: 28 true: 17 match: 16 fp: 9 fn: 1 [s1_m03_v03] Precision: 0.7419 Recall:0.6970 F1:0.7187 F2:0.7055 turk: 40 true: 33 match: 23 fp: 8 fn: 10 [s1_m03_v04] Precision: 0.7692 Recall:0.8333 F1:0.8000 F2:0.8196 turk: 19 true: 12 match: 10 fp: 3 fn: 2 [s1_m03_v05] Precision: 0.5714 Recall:0.8000 F1:0.6666 F2:0.7407 turk: 17 true: 10 match: 8 fp: 6 fn: 2 [s1_m04_v01] Precision: 0.2857 Recall:1.0000 F1:0.4444 F2:0.6667 turk: 14 true: 4 match: 4 fp: 10 fn: 0 [s1_m04_v02] Precision: 0.8750 Recall:1.0000 F1:0.9333 F2:0.9722 turk: 10 true: 7 match: 7 fp: 1 fn: 0 [s1_m04_v03] Precision: 0.7273 Recall:0.8000 F1:0.7619 F2:0.7843 turk: 25 true: 20 match: 16 fp: 6 fn: 4 [s1_m04_v04] Precision: 0.8667 Recall:0.6190 F1:0.7222 F2:0.6565 turk: 22 true: 21 match: 13 fp: 2 fn: 8 [s1_m04_v05] Precision: 0.5000 Recall:1.0000 F1:0.6667 F2:0.8333 turk: 11 true: 3 match: 3 fp: 3 fn: 0 [s1_m05_v01] Precision: 0.5500 Recall:0.7857 F1:0.6471 F2:0.7237 turk: 22 true: 14 match: 11 fp: 9 fn: 3 [s1_m05_v02] Precision: 0.4839 Recall:0.7500 F1:0.5883 F2:0.6757 turk: 36 true: 20 match: 15 fp: 16 fn: 5 [s1_m05_v03] Precision: 0.8125 Recall:0.6500 F1:0.7222 F2:0.6771 turk: 17 true: 20 match: 13 fp: 3 fn: 7 [s1_m05_v04] Precision: 0.6667 Recall:0.4615 F1:0.5454 F2:0.4918 turk: 15 true: 13 match: 6 fp: 3 fn: 7 [s1_m05_v05] Precision: 0.7391 Recall:0.8095 F1:0.7727 F2:0.7944 turk: 32 true: 21 match: 17 fp: 6 fn: 4 eps: 0.07 GLOBAL Precision: 0.6370 Recall:0.7544 F1:0.6907 F2:0.7276