VirusTotal APK Malware Detection Data - Week 25: 20200615-20200621

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200615_20200621.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.73% 0.05% 14997 33 62399 41
Fortinet 99.70% 0.09% 14993 55 62377 45
K7GW 99.50% 0.15% 14963 96 62336 75
Trustlook 99.26% 0.22% 14926 136 62296 112
Avira 98.74% 0.00% 14848 0 62432 190
AhnLab-V3 98.51% 0.06% 14814 36 62396 224
DrWeb 98.43% 0.22% 14802 138 62294 236
Avast-Mobile 98.24% 0.18% 14773 110 62322 265
ZoneAlarm 97.97% 0.01% 14733 4 62428 305
Kaspersky 97.61% 0.00% 14679 2 62430 359
F-Secure 97.05% 0.00% 14595 1 62431 443
Ikarus 96.74% 0.21% 14548 131 62301 490
McAfee 96.34% 0.00% 14487 1 62431 551
CAT-QuickHeal 94.91% 0.66% 14272 411 62021 766
Sophos 91.12% 0.04% 13702 22 62410 1336
McAfee-GW-Edition 86.02% 0.00% 12935 0 62432 2103
NANO-Antivirus 85.68% 0.06% 12884 37 62395 2154
Symantec 85.50% 0.04% 12857 27 62405 2181
Qihoo-360 81.16% 0.03% 12205 17 62415 2833
AVG 78.75% 0.06% 11843 37 62395 3195
Ad-Aware 0.49% 0.00% 73 0 62432 14965
TotalGoodware 62432
TotalMalware 15038
TotalSample 77470

Please send an email to lxu@trustlook.com if you have any comments. Thanks.