December 9, 2019

VirusTotal APK Malware Detection Data - Week 49: 20191202-201901208

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20191202_20191208.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.91% 0.26% 24350 205 77521 23
K7GW 99.38% 1.12% 24223 867 76859 150
Fortinet 99.02% 0.03% 24133 24 77702 240
Trustlook 98.86% 0.33% 24096 259 77467 277
AhnLab-V3 98.77% 0.19% 24073 151 77575 300
ZoneAlarm 98.01% 0.16% 23888 125 77601 485
DrWeb 97.97% 1.13% 23879 882 76844 494
Kaspersky 97.61% 0.16% 23790 121 77605 583
Avira 97.16% 0.00% 23681 0 77726 692
F-Secure 96.78% 0.10% 23589 74 77652 784
Ikarus 96.58% 1.61% 23539 1255 76471 834
McAfee 95.69% 0.01% 23323 11 77715 1050
Avast-Mobile 93.17% 0.21% 22709 163 77563 1664
Symantec 78.32% 0.08% 19088 60 77666 5285
CAT-QuickHeal 74.27% 0.33% 18102 253 77473 6271
Sophos 74.22% 0.07% 18090 57 77669 6283
McAfee-GW-Edition 69.63% 0.00% 16972 0 77726 7401
NANO-Antivirus 68.73% 0.55% 16752 426 77300 7621
Qihoo-360 63.41% 0.15% 15456 114 77612 8917
AVG 42.73% 0.08% 10415 64 77662 13958
Avast 41.89% 0.08% 10210 61 77665 14163
Cyren 34.60% 0.04% 8432 30 77696 15941
Tencent 26.56% 0.00% 6473 0 77726 17900
MAX 23.15% 0.00% 5642 3 77723 18731
Rising 6.44% 0.05% 1570 40 77686 22803
TrendMicro-HouseCall 4.31% 0.02% 1051 12 77714 23322
BitDefender 3.13% 0.00% 762 0 77726 23611
Antiy-AVL 2.01% 0.00% 491 1 77725 23882
Baidu 0.13% 0.01% 31 4 77722 24342
Ad-Aware 0.11% 0.00% 28 0 77726 24345
Babable 0.00% 0.00% 0 0 77726 24373
TotalGoodware 77726
TotalMalware 24373
TotalSample 102099

Please send an email to lxu@trustlook.com if you have any comments. Thanks.