November 23, 2020

VirusTotal APK Malware Detection Data - Week 47: 20201116-20201122

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20201116_20201122.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.52% 0.10% 27117 92 94596 132
ESET-NOD32 99.51% 0.00% 27115 2 94686 134
Trustlook 99.27% 0.08% 27049 75 94613 200
Fortinet 98.74% 0.01% 26906 14 94674 343
Avast-Mobile 98.63% 0.28% 26877 263 94425 372
DrWeb 98.33% 0.14% 26793 131 94557 456
Avira 98.22% 0.00% 26764 0 94688 485
ZoneAlarm 98.12% 0.00% 26736 4 94684 513
McAfee 97.96% 0.00% 26693 1 94687 556
McAfee-GW-Edition 97.91% 0.02% 26679 22 94666 570
CAT-QuickHeal 97.47% 0.00% 26559 4 94684 690
Kaspersky 97.18% 0.00% 26480 3 94685 769
Ikarus 95.49% 0.09% 26021 86 94602 1228
NANO-Antivirus 89.16% 0.02% 24296 16 94672 2953
Symantec 87.87% 0.01% 23945 8 94680 3304
F-Secure 87.82% 0.00% 23931 3 94685 3318
Sophos 87.43% 0.02% 23823 18 94670 3426
Qihoo-360 86.43% 0.01% 23550 10 94678 3699
AhnLab-V3 85.51% 0.01% 23300 11 94677 3949
AVG 72.59% 0.02% 19781 15 94673 7468
Ad-Aware 0.79% 0.00% 216 0 94688 27033
TotalGoodware 94688
TotalMalware 27249
TotalSample 121937

Please send an email to lxu@trustlook.com if you have any comments. Thanks.