VirusTotal APK Malware Detection Data - Week 24: 20200608-20200614

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200608_20200614.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
Trustlook 98.32% 0.58% 35060 531 90507 600
ESET-NOD32 98.21% 0.03% 35022 26 91012 638
McAfee 97.43% 0.01% 34744 10 91028 916
Fortinet 96.74% 0.17% 34499 152 90886 1161
ZoneAlarm 96.65% 0.12% 34464 111 90927 1196
K7GW 96.52% 0.29% 34420 262 90776 1240
Ikarus 96.07% 0.48% 34259 433 90605 1401
CAT-QuickHeal 94.26% 2.80% 33614 2547 88491 2046
AhnLab-V3 93.89% 0.04% 33482 38 91000 2178
Kaspersky 93.54% 0.01% 33358 7 91031 2302
Avira 91.03% 0.00% 32463 0 91038 3197
NANO-Antivirus 90.86% 0.14% 32399 125 90913 3261
Symantec 90.62% 0.20% 32316 185 90853 3344
Qihoo-360 90.58% 0.03% 32302 27 91011 3358
Avast-Mobile 89.48% 0.51% 31907 463 90575 3753
DrWeb 85.39% 0.28% 30451 251 90787 5209
F-Secure 85.26% 0.01% 30403 7 91031 5257
McAfee-GW-Edition 84.34% 0.00% 30074 2 91036 5586
Sophos 76.60% 0.04% 27316 40 90998 8344
AVG 63.53% 0.06% 22655 58 90980 13005
Ad-Aware 0.57% 0.00% 204 0 91038 35456
TotalGoodware 91038
TotalMalware 35660
TotalSample 126698

Please send an email to lxu@trustlook.com if you have any comments. Thanks.