December 21, 2020

VirusTotal APK Malware Detection Data - Week 51: 20201214-20201220

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20201214_20201220.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.41% 0.01% 11220 8 56336 67
Trustlook 98.96% 0.14% 11170 78 56266 117
Avast-Mobile 98.76% 0.29% 11147 163 56181 140
Avira 98.30% 0.00% 11095 1 56343 192
DrWeb 98.05% 0.47% 11067 263 56081 220
Fortinet 98.03% 0.02% 11065 14 56330 222
K7GW 97.98% 0.20% 11059 110 56234 228
ZoneAlarm 97.37% 0.00% 10990 1 56343 297
AhnLab-V3 96.31% 0.03% 10871 18 56326 416
Kaspersky 96.06% 0.00% 10842 0 56344 445
Ikarus 95.30% 0.26% 10757 144 56200 530
CAT-QuickHeal 94.66% 0.02% 10684 14 56330 603
F-Secure 91.44% 0.01% 10321 5 56339 966
Sophos 88.60% 0.02% 10000 14 56330 1287
Qihoo-360 78.12% 0.02% 8817 13 56331 2470
McAfee 75.14% 0.00% 8481 2 56342 2806
NANO-Antivirus 73.75% 0.05% 8324 26 56318 2963
AVG 65.54% 0.03% 7398 16 56328 3889
McAfee-GW-Edition 55.06% 0.02% 6215 14 56330 5072
Symantec 41.53% 0.02% 4687 11 56333 6600
Ad-Aware 1.23% 0.00% 139 0 56344 11148
TotalGoodware 56344
TotalMalware 11287
TotalSample 67631

Please send an email to lxu@trustlook.com if you have any comments. Thanks.