VirusTotal APK Malware Detection Data - Week 21: 20200518-20200524

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200518_20200524.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.78% 0.28% 268750 209 75739 580
Trustlook 99.68% 1.64% 268468 1249 74699 862
K7GW 99.45% 0.26% 267862 201 75747 1468
ZoneAlarm 99.43% 0.07% 267798 52 75896 1532
AhnLab-V3 99.01% 0.06% 266676 46 75902 2654
Qihoo-360 98.92% 0.04% 266429 30 75918 2901
Avast-Mobile 98.65% 0.41% 265683 315 75633 3647
Kaspersky 98.60% 0.04% 265566 32 75916 3764
Avira 98.40% 0.00% 265011 0 75948 4319
DrWeb 98.15% 0.27% 264358 205 75743 4972
NANO-Antivirus 97.36% 0.06% 262213 43 75905 7117
Ikarus 96.72% 0.67% 260486 510 75438 8844
Sophos 96.62% 0.10% 260229 79 75869 9101
F-Secure 93.08% 0.03% 250688 19 75929 18642
McAfee 90.54% 0.02% 243839 16 75932 25491
AVG 89.74% 0.13% 241687 97 75851 27643
McAfee-GW-Edition 84.59% 0.13% 227817 97 75851 41513
Fortinet 80.48% 0.01% 216744 4 75944 52586
Symantec 75.83% 0.19% 204220 147 75801 65110
CAT-QuickHeal 56.80% 0.47% 152983 360 75588 116347
Ad-Aware 0.15% 0.00% 406 0 75948 268924
TotalGoodware 75948
TotalMalware 269330
TotalSample 345278

Please send an email to lxu@trustlook.com if you have any comments. Thanks.