April 13, 2020

VirusTotal APK Malware Detection Data - Week 15: 20200406-20200412

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200406_20200412.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.73% 0.05% 22492 41 76199 62
Trustlook 99.57% 0.30% 22457 225 76015 97
ZoneAlarm 98.85% 0.01% 22295 10 76230 259
AhnLab-V3 98.71% 0.03% 22263 26 76214 291
Kaspersky 98.28% 0.01% 22166 5 76235 388
DrWeb 97.94% 0.26% 22089 201 76039 465
K7GW 97.92% 0.13% 22084 99 76141 470
Avast-Mobile 97.41% 0.26% 21969 202 76038 585
Ikarus 96.30% 0.17% 21720 131 76109 834
Avira 96.12% 0.00% 21679 0 76240 875
Sophos 95.09% 0.03% 21446 25 76215 1108
CAT-QuickHeal 93.46% 0.05% 21080 36 76204 1474
F-Secure 92.33% 0.01% 20823 4 76236 1731
Qihoo-360 88.26% 0.03% 19907 21 76219 2647
AVG 86.68% 0.10% 19549 79 76161 3005
McAfee 85.26% 0.01% 19230 10 76230 3324
NANO-Antivirus 84.47% 0.04% 19052 34 76206 3502
Fortinet 74.04% 0.01% 16699 4 76236 5855
McAfee-GW-Edition 70.48% 0.00% 15896 1 76239 6658
Symantec 53.19% 0.03% 11996 21 76219 10558
Ad-Aware 0.37% 0.00% 84 0 76240 22470
TotalGoodware 76240
TotalMalware 22554
TotalSample 98794

Please send an email to lxu@trustlook.com if you have any comments. Thanks.