July 13, 2020

VirusTotal APK Malware Detection Data - Week 28: 20200706-20200712

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200706_20200712.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.60% 0.08% 14854 44 58596 59
Fortinet 99.48% 0.04% 14835 25 58615 78
K7GW 99.35% 0.16% 14816 92 58548 97
Avira 98.57% 0.00% 14700 0 58640 213
Avast-Mobile 98.57% 0.16% 14699 95 58545 214
CAT-QuickHeal 98.40% 0.03% 14674 15 58625 239
Trustlook 97.98% 0.16% 14612 93 58547 301
AhnLab-V3 97.79% 0.05% 14583 27 58613 330
ZoneAlarm 97.75% 0.01% 14577 3 58637 336
DrWeb 97.61% 0.17% 14557 102 58538 356
Kaspersky 97.22% 0.00% 14499 2 58638 414
F-Secure 97.00% 0.01% 14466 3 58637 447
Ikarus 95.23% 0.14% 14202 83 58557 711
McAfee 94.29% 0.00% 14061 2 58638 852
Sophos 89.51% 0.03% 13349 15 58625 1564
NANO-Antivirus 84.64% 0.05% 12622 27 58613 2291
Qihoo-360 83.62% 0.02% 12470 12 58628 2443
Symantec 75.42% 0.01% 11247 8 58632 3666
AVG 69.54% 0.09% 10370 51 58589 4543
Ad-Aware 0.40% 0.00% 59 0 58640 14854
McAfee-GW-Edition 0.00% 0.00% 0 0 58640 14913
TotalGoodware 58640
TotalMalware 14913
TotalSample 73553

Please send an email to lxu@trustlook.com if you have any comments. Thanks.