VirusTotal APK Malware Detection Data - Week 20: 20200511-20200517

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200511_20200517.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.76% 0.18% 161399 183 102831 392
K7GW 99.51% 0.55% 160992 564 102450 799
Trustlook 99.44% 0.28% 160890 291 102723 901
McAfee 99.26% 0.03% 160601 34 102980 1190
ZoneAlarm 99.17% 0.09% 160445 91 102923 1346
AhnLab-V3 98.95% 0.03% 160092 36 102978 1699
Avast-Mobile 98.58% 0.33% 159501 343 102671 2290
Qihoo-360 98.24% 0.03% 158949 34 102980 2842
Kaspersky 97.99% 0.03% 158541 35 102979 3250
DrWeb 97.91% 0.25% 158414 256 102758 3377
Avira 97.82% 0.00% 158270 1 103013 3521
Ikarus 96.59% 0.31% 156274 322 102692 5517
NANO-Antivirus 96.48% 0.05% 156093 48 102966 5698
Sophos 96.13% 0.15% 155527 150 102864 6264
F-Secure 90.42% 0.02% 146293 22 102992 15498
McAfee-GW-Edition 85.99% 0.00% 139130 1 103013 22661
AVG 85.60% 0.08% 138487 86 102928 23304
Symantec 82.55% 0.13% 133551 135 102879 28240
Fortinet 79.42% 0.01% 128498 8 103006 33293
CAT-QuickHeal 57.65% 0.20% 93266 205 102809 68525
Ad-Aware 0.25% 0.00% 410 0 103014 161381
TotalGoodware 103014
TotalMalware 161791
TotalSample 264805

Please send an email to lxu@trustlook.com if you have any comments. Thanks.