VirusTotal APK Malware Detection Data - Week 23: 20200601-20200607

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200601_20200607.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.72% 0.03% 335199 31 116732 944
K7GW 99.44% 0.11% 334248 128 116635 1895
Trustlook 99.17% 0.23% 333369 271 116492 2774
ZoneAlarm 98.86% 0.02% 332320 28 116735 3823
Avira 98.64% 0.00% 331576 0 116763 4567
CAT-QuickHeal 98.10% 1.49% 329745 1737 115026 6398
Qihoo-360 97.84% 0.01% 328892 13 116750 7251
McAfee 97.82% 0.01% 328816 7 116756 7327
Kaspersky 97.50% 0.01% 327733 6 116757 8410
Ikarus 96.79% 0.15% 325346 171 116592 10797
AhnLab-V3 96.48% 0.03% 324304 31 116732 11839
Fortinet 95.93% 0.04% 322476 41 116722 13667
Avast-Mobile 95.02% 0.21% 319403 248 116515 16740
Symantec 94.21% 0.06% 316672 75 116688 19471
DrWeb 93.57% 0.08% 314535 98 116665 21608
F-Secure 91.60% 0.01% 307906 10 116753 28237
NANO-Antivirus 90.47% 0.03% 304108 31 116732 32035
Sophos 89.85% 0.02% 302037 25 116738 34106
McAfee-GW-Edition 81.99% 0.00% 275589 2 116761 60554
AVG 76.94% 0.05% 258636 61 116702 77507
Ad-Aware 1.66% 0.00% 5589 0 116763 330554
TotalGoodware 116763
TotalMalware 336143
TotalSample 452906

Please send an email to lxu@trustlook.com if you have any comments. Thanks.