September 30, 2019

VirusTotal APK Malware Detection Data - Week 39: 20190923-201909029

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20190923_20190929.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.65% 0.03% 50266 32 110219 175
Trustlook 99.42% 0.12% 50147 136 110115 294
K7GW 99.19% 0.22% 50034 240 110011 407
Avast-Mobile 98.98% 0.17% 49928 185 110066 513
Fortinet 97.67% 0.02% 49264 21 110230 1177
ZoneAlarm 97.06% 0.02% 48960 18 110233 1481
DrWeb 96.67% 0.25% 48759 272 109979 1682
Ikarus 96.14% 0.22% 48492 246 110005 1949
Avira 96.06% 0.00% 48452 1 110250 1989
Kaspersky 96.00% 0.01% 48421 15 110236 2020
AhnLab-V3 95.07% 0.04% 47955 49 110202 2486
McAfee 93.20% 0.01% 47012 8 110243 3429
Sophos 92.30% 0.03% 46557 31 110220 3884
F-Secure 91.61% 0.01% 46207 7 110244 4234
Symantec 87.53% 0.04% 44151 41 110210 6290
NANO-Antivirus 86.53% 0.11% 43647 118 110133 6794
Qihoo-360 85.49% 0.01% 43123 15 110236 7318
CAT-QuickHeal 83.78% 0.20% 42257 221 110030 8184
Tencent 80.88% 0.07% 40796 79 110172 9645
McAfee-GW-Edition 79.46% 0.00% 40082 5 110246 10359
AVG 65.01% 0.03% 32792 32 110219 17649
Avast 64.02% 0.03% 32292 32 110219 18149
MAX 59.41% 0.01% 29965 8 110243 20476
Cyren 56.93% 0.00% 28717 4 110247 21724
Antiy-AVL 9.95% 0.01% 5019 9 110242 45422
Rising 7.42% 0.08% 3744 89 110162 46697
TrendMicro-HouseCall 6.67% 0.05% 3363 53 110198 47078
BitDefender 4.01% 0.00% 2022 0 110251 48419
Ad-Aware 0.34% 0.00% 173 0 110251 50268
Baidu 0.20% 0.01% 100 9 110242 50341
Babable 0.01% 0.00% 7 0 110251 50434
TotalGoodware 110251
TotalMalware 50441
TotalSample 16069

Please send an email to lxu@trustlook.com if you have any comments. Thanks.