September 9, 2019

VirusTotal APK Malware Detection Data - Week 36: 20190902-20190908

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20190902_20190908.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.52% 0.17% 92717 823 484842 443
ESET-NOD32 99.45% 0.07% 92652 357 485308 508
Trustlook 99.16% 0.13% 92374 622 485043 786
AhnLab-V3 97.98% 0.03% 91274 138 485527 1886
Fortinet 97.13% 0.01% 90485 46 485619 2675
Avast-Mobile 96.51% 0.20% 89911 970 484695 3249
Avira 96.50% 0.00% 89897 4 485661 3263
McAfee 96.28% 0.00% 89696 5 485660 3464
ZoneAlarm 95.48% 0.02% 88951 91 485574 4209
Ikarus 94.72% 0.18% 88239 882 484783 4921
Kaspersky 94.31% 0.01% 87859 38 485627 5301
DrWeb 92.73% 0.37% 86383 1775 483890 6777
Symantec 91.65% 0.02% 85381 91 485574 7779
Qihoo-360 91.02% 0.01% 84790 65 485600 8370
F-Secure 90.80% 0.00% 84586 7 485658 8574
Sophos 86.24% 0.03% 80343 139 485526 12817
NANO-Antivirus 85.66% 0.08% 79804 370 485295 13356
Tencent 83.70% 0.02% 77974 118 485547 15186
McAfee-GW-Edition 82.46% 0.00% 76816 7 485658 16344
CAT-QuickHeal 82.18% 0.19% 76556 929 484736 16604
MAX 60.31% 0.00% 56184 16 485649 36976
AVG 59.98% 0.03% 55877 159 485506 37283
Avast 58.94% 0.03% 54912 150 485515 38248
Cyren 54.14% 0.01% 50436 52 485613 42724
Rising 11.30% 0.01% 10529 33 485632 82631
Antiy-AVL 9.11% 0.02% 8486 112 485553 84674
TrendMicro-HouseCall 9.07% 0.02% 8452 86 485579 84708
BitDefender 5.43% 0.00% 5055 0 485665 88105
Baidu 0.51% 0.03% 473 150 485515 92687
Ad-Aware 0.14% 0.00% 134 0 485665 93026
Babable 0.00% 0.00% 1 1 485664 93159
Total Goodware 485665
Total Malware 93160
Total Sample 578825

Please send an email to lxu@trustlook.com if you have any comments. Thanks.