VirusTotal APK Malware Detection Data - Week 19: 20200504-20200510

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200504_20200510.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.68% 0.05% 23629 44 80944 75
K7GW 99.54% 0.19% 23596 150 80838 108
Trustlook 99.50% 0.19% 23585 151 80837 119
AhnLab-V3 98.99% 0.05% 23465 42 80946 239
ZoneAlarm 98.67% 0.02% 23388 15 80973 316
Avira 98.60% 0.00% 23372 0 80988 332
DrWeb 98.45% 0.30% 23337 241 80747 367
Avast-Mobile 98.35% 0.27% 23313 215 80773 391
Kaspersky 97.57% 0.01% 23127 6 80982 577
Ikarus 96.63% 0.19% 22906 155 80833 798
Sophos 95.13% 0.03% 22550 22 80966 1154
F-Secure 93.27% 0.01% 22109 8 80980 1595
NANO-Antivirus 92.36% 0.05% 21892 42 80946 1812
Qihoo-360 92.24% 0.03% 21864 25 80963 1840
McAfee 86.62% 0.02% 20532 16 80972 3172
AVG 79.90% 0.10% 18940 79 80909 4764
Fortinet 76.65% 0.02% 18170 13 80975 5534
McAfee-GW-Edition 66.33% 0.00% 15722 0 80988 7982
CAT-QuickHeal 63.56% 0.05% 15067 39 80949 8637
Symantec 62.49% 0.01% 14812 11 80977 8892
Ad-Aware 0.56% 0.00% 133 0 80988 23571
TotalGoodware 80988
TotalMalware 23704
TotalSample 104692

Please send an email to lxu@trustlook.com if you have any comments. Thanks.