November 25, 2019

VirusTotal APK Malware Detection Data - Week 47: 20191118-201901124

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20191118_20191124.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.59% 0.02% 34278 11 59292 141
K7GW 99.38% 0.12% 34204 71 59232 215
Trustlook 98.88% 0.35% 34032 205 59098 387
Avira 98.55% 0.00% 33920 0 59303 499
ZoneAlarm 98.43% 0.01% 33880 6 59297 539
AhnLab-V3 97.87% 0.03% 33686 18 59285 733
Ikarus 97.60% 0.22% 33593 129 59174 826
Kaspersky 96.56% 0.01% 33234 4 59299 1185
F-Secure 96.14% 0.00% 33089 2 59301 1330
McAfee 95.85% 0.02% 32991 11 59292 1428
DrWeb 95.42% 0.15% 32843 87 59216 1576
Avast-Mobile 87.61% 0.18% 30154 105 59198 4265
Fortinet 87.09% 0.01% 29976 7 59296 4443
Symantec 86.18% 0.04% 29664 24 59279 4755
NANO-Antivirus 76.37% 0.05% 26286 27 59276 8133
CAT-QuickHeal 74.79% 0.05% 25742 27 59276 8677
Qihoo-360 73.43% 0.03% 25274 20 59283 9145
Sophos 71.53% 0.02% 24621 13 59290 9798
McAfee-GW-Edition 70.87% 0.00% 24393 2 59301 10026
AVG 42.05% 0.08% 14474 48 59255 19945
Avast 41.20% 0.08% 14179 46 59257 20240
Cyren 34.59% 0.00% 11904 1 59302 22515
MAX 33.42% 0.00% 11502 0 59303 22917
Tencent 19.64% 0.00% 6760 0 59303 27659
Rising 4.34% 0.02% 1494 9 59294 32925
Antiy-AVL 4.23% 0.00% 1457 2 59301 32962
TrendMicro-HouseCall 3.33% 0.02% 1147 13 59290 33272
BitDefender 2.41% 0.00% 831 0 59303 33588
Baidu 0.12% 0.01% 40 3 59300 34379
Ad-Aware 0.05% 0.00% 18 0 59303 34401
Babable 0.00% 0.00% 0 0 59303 34419
TotalGoodware 59303
TotalMalware 34419
TotalSample 93722

Please send an email to lxu@trustlook.com if you have any comments. Thanks.