September 3, 2019

VirusTotal APK Malware Detection Data - Week 35: 20190826-20190901

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20190826_20190901.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.46% 0.16% 112272 767 478617 611
ESET-NOD32 99.37% 0.07% 112167 327 479057 716
Trustlook 99.17% 0.13% 111950 618 478766 933
Avira 98.21% 0.00% 110866 4 479380 2017
AhnLab-V3 96.60% 0.03% 109048 143 479241 3835
Avast-Mobile 96.52% 0.19% 108954 897 478487 3929
Fortinet 96.21% 0.01% 108604 52 479332 4279
ZoneAlarm 95.49% 0.01% 107792 67 479317 5091
Ikarus 94.93% 0.18% 107165 840 478544 5718
Kaspersky 93.96% 0.01% 106064 40 479344 6819
McAfee 93.95% 0.00% 106051 6 479378 6832
Qihoo-360 93.19% 0.01% 105196 66 479318 7687
DrWeb 92.94% 0.33% 104909 1570 477814 7974
Symantec 90.67% 0.02% 102350 116 479268 10533
CAT-QuickHeal 86.48% 0.18% 97621 863 478521 15262
Sophos 86.09% 0.03% 97178 161 479223 15705
Tencent 85.49% 0.02% 96499 118 479266 16384
NANO-Antivirus 84.10% 0.06% 94938 273 479111 17945
McAfee-GW-Edition 79.74% 0.00% 90016 15 479369 22867
F-Secure 77.81% 0.01% 87831 25 479359 25052
AVG 67.54% 0.03% 76242 154 479230 36641
Avast 66.53% 0.03% 75098 145 479239 37785
Cyren 56.36% 0.01% 63621 27 479357 49262
MAX 54.98% 0.00% 62058 16 479368 50825
Antiy-AVL 17.76% 0.03% 20045 130 479254 92838
Rising 12.58% 0.01% 14199 62 479322 98684
TrendMicro-HouseCall 3.74% 0.02% 4224 73 479311 108659
BitDefender 3.12% 0.00% 3524 0 479384 109359
Baidu 0.13% 0.03% 145 150 479234 112738
Ad-Aware 0.08% 0.00% 93 0 479384 112790
Babable 0.00% 0.00% 2 1 479383 112881
Total Goodware 479384
Total Malware 112883
Total Sample 592267

Please send an email to lxu@trustlook.com if you have any comments. Thanks.