VirusTotal APK Malware Detection Data - Week 37: 20200907-20200913

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200907_20200913.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.73% 0.03% 13519 16 54412 37
K7GW 99.46% 0.17% 13483 92 54336 73
DrWeb 99.00% 0.24% 13420 128 54300 136
Fortinet 98.83% 0.02% 13398 13 54415 158
Trustlook 98.72% 0.18% 13382 97 54331 174
CAT-QuickHeal 98.71% 0.02% 13381 9 54419 175
Avira 98.28% 0.00% 13323 0 54428 233
Avast-Mobile 97.71% 0.25% 13246 134 54294 310
AhnLab-V3 97.60% 0.05% 13230 28 54400 326
McAfee 97.20% 0.00% 13176 1 54427 380
ZoneAlarm 96.86% 0.00% 13130 2 54426 426
Kaspersky 96.47% 0.00% 13078 0 54428 478
Ikarus 94.99% 0.23% 12877 125 54303 679
F-Secure 90.96% 0.01% 12331 4 54424 1225
NANO-Antivirus 83.11% 0.06% 11266 32 54396 2290
Symantec 69.21% 0.01% 9382 6 54422 4174
Sophos 68.83% 0.09% 9331 48 54380 4225
Qihoo-360 56.32% 0.03% 7635 16 54412 5921
AVG 43.83% 0.06% 5942 32 54396 7614
Ad-Aware 1.59% 0.00% 215 0 54428 13341
McAfee-GW-Edition 0.00% 0.00% 0 0 54428 13556
TotalGoodware 54428
TotalMalware 13556
TotalSample 67984

Please send an email to lxu@trustlook.com if you have any comments. Thanks.