VirusTotal APK Malware Detection Data - Week 22: 20200525-20200531

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200525_20200531.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.83% 0.04% 301669 25 60754 515
K7GW 99.45% 0.24% 300527 143 60636 1657
Trustlook 99.40% 0.67% 300375 406 60373 1809
ZoneAlarm 99.30% 0.02% 300067 14 60765 2117
Avira 98.59% 0.00% 297923 0 60779 4261
Qihoo-360 98.57% 0.03% 297851 19 60760 4333
Kaspersky 98.40% 0.01% 297339 6 60773 4845
AhnLab-V3 98.12% 0.05% 296510 29 60750 5674
Ikarus 97.42% 0.23% 294379 140 60639 7805
Avast-Mobile 96.85% 0.29% 292672 175 60604 9512
DrWeb 96.45% 0.24% 291455 147 60632 10729
McAfee 95.11% 0.01% 287406 6 60773 14778
NANO-Antivirus 94.91% 0.07% 286788 41 60738 15396
F-Secure 93.72% 0.01% 283198 6 60773 18986
Sophos 93.69% 0.03% 283116 20 60759 19068
Symantec 92.60% 0.10% 279831 61 60718 22353
CAT-QuickHeal 88.18% 0.62% 266476 375 60404 35708
McAfee-GW-Edition 85.74% 0.00% 259079 3 60776 43105
AVG 84.61% 0.08% 255673 49 60730 46511
Fortinet 79.36% 0.01% 239824 9 60770 62360
Ad-Aware 0.75% 0.00% 2255 0 60779 299929
TotalGoodware 60779
TotalMalware 302184
TotalSample 362963

Please send an email to lxu@trustlook.com if you have any comments. Thanks.