April 6, 2020

VirusTotal APK Malware Detection Data - Week 14: 20200330-20200405

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200330_20200405.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.69% 0.04% 12126 25 65133 38
Trustlook 99.05% 0.27% 12049 173 64985 115
K7GW 98.31% 0.10% 11959 67 65091 205
AhnLab-V3 98.17% 0.04% 11941 28 65130 223
ZoneAlarm 98.05% 0.00% 11927 1 65157 237
Kaspersky 97.58% 0.00% 11870 0 65158 294
Ikarus 96.93% 0.12% 11790 80 65078 374
DrWeb 96.80% 0.17% 11775 110 65048 389
Avast-Mobile 95.45% 0.29% 11611 192 64966 553
Avira 95.43% 0.00% 11608 0 65158 556
F-Secure 93.99% 0.00% 11433 1 65157 731
Sophos 92.31% 0.04% 11229 27 65131 935
CAT-QuickHeal 90.87% 0.04% 11054 24 65134 1110
McAfee 90.52% 0.01% 11011 4 65154 1153
Qihoo-360 86.69% 0.02% 10545 13 65145 1619
NANO-Antivirus 81.35% 0.04% 9895 29 65129 2269
AVG 72.28% 0.13% 8792 85 65073 3372
McAfee-GW-Edition 71.04% 0.00% 8641 1 65157 3523
Fortinet 60.36% 0.00% 7342 2 65156 4822
Symantec 58.48% 0.03% 7113 21 65137 5051
Ad-Aware 0.21% 0.00% 25 0 65158 12139
TotalGoodware 65158
TotalMalware 12164
TotalSample 77322

Please send an email to lxu@trustlook.com if you have any comments. Thanks.