May 11, 2021

VirusTotal APK Malware Detection Data - Week 19: 202100503-20210509

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210503_20210509.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
Avast-Mobile 98.94% 0.45% 9819 267 59607 105
Avira 98.77% 0.01% 9802 5 59869 122
ESET-NOD32 98.66% 0.01% 9791 8 59866 133
Ikarus 98.50% 0.04% 9775 26 59848 149
Fortinet 98.43% 0.05% 9768 30 59844 156
Trustlook 97.00% 0.06% 9626 33 59841 298
K7GW 96.95% 0.08% 9621 48 59826 303
AhnLab-V3 96.17% 0.02% 9544 13 59861 380
Kaspersky 92.92% 0.00% 9221 2 59872 703
DrWeb 92.74% 0.15% 9204 89 59785 720
McAfee 90.88% 0.01% 9019 8 59866 905
SymantecMobileInsight 89.19% 2.27% 8851 1360 58514 1073
CAT-QuickHeal 85.94% 0.02% 8529 12 59862 1395
McAfee-GW-Edition 85.80% 0.03% 8515 15 59859 1409
AegisLab 83.08% 0.13% 8245 78 59796 1679
Sophos 82.98% 0.02% 8235 10 59864 1689
NANO-Antivirus 79.64% 0.03% 7903 18 59856 2021
Symantec 79.59% 0.17% 7899 101 59773 2025
Microsoft 78.14% 0.03% 7755 16 59858 2169
Tencent 77.40% 0.07% 7681 40 59834 2243
Alibaba 73.08% 0.00% 7252 2 59872 2672
Cyren 65.21% 0.02% 6471 14 59860 3453
ZoneAlarm 55.77% 0.01% 5535 5 59869 4389
MAX 52.15% 0.00% 5175 1 59873 4749
AVG 41.55% 0.01% 4123 7 59867 5801
Avast 41.38% 0.01% 4107 7 59867 5817
Zillya 30.14% 0.09% 2991 53 59821 6933
Comodo 26.25% 0.05% 2605 27 59847 7319
Kingsoft 18.95% 0.01% 1881 8 59866 8043
Jiangmin 15.24% 0.59% 1512 351 59523 8412
GData 13.23% 0.00% 1313 0 59874 8611
ClamAV 12.76% 0.13% 1266 79 59795 8658
BitDefender 10.04% 0.00% 996 0 59874 8928
Emsisoft 9.85% 0.00% 978 0 59874 8946
Rising 9.51% 0.01% 944 3 59871 8980
Yandex 7.94% 0.00% 788 0 59874 9136
F-Secure 7.16% 0.00% 711 0 59874 9213
TrendMicro-HouseCall 7.01% 0.01% 696 4 59870 9228
TrendMicro 6.44% 0.03% 639 16 59858 9285
Zoner 5.57% 0.00% 553 2 59872 9371
MicroWorld-eScan 4.53% 0.00% 450 0 59874 9474
Arcabit 4.41% 0.00% 438 1 59873 9486
VBA32 3.47% 0.01% 344 6 59868 9580
Antiy-AVL 2.86% 0.02% 284 9 59865 9640
Ad-Aware 2.23% 0.00% 221 0 59874 9703
VIPRE 1.17% 0.00% 116 0 59874 9808
Panda 0.96% 0.00% 95 2 59872 9829
Babable 0.40% 0.06% 40 33 59841 9884
Baidu 0.28% 0.01% 28 5 59869 9896
F-Prot 0.12% 0.00% 12 0 59874 9912
ViRobot 0.11% 0.00% 11 1 59873 9913
SentinelOne 0.06% 0.00% 6 0 59874 9918
Malwarebytes 0.05% 0.00% 5 0 59874 9919
ALYac 0.03% 0.00% 3 0 59874 9921
CMC 0.01% 0.00% 1 0 59874 9923
K7AntiVirus 0.01% 0.00% 1 0 59874 9923
SUPERAntiSpyware 0.01% 0.00% 1 0 59874 9923
Bkav 0.00% 0.00% 0 1 59873 9924
TotalDefense 0.00% 0.00% 0 0 59874 9924
nProtect 0.00% 0.00% 0 0 59874 9924
CrowdStrike 0.00% 0.00% 0 0 59874 9924
TheHacker 0.00% 0.00% 0 0 59874 9924
eScan 0.00% 0.00% 0 0 59874 9924
Invincea 0.00% 0.00% 0 0 59874 9924
Endgame 0.00% 0.00% 0 0 59874 9924
Webroot 0.00% 0.00% 0 0 59874 9924
AVware 0.00% 0.00% 0 0 59874 9924
Qihoo-360 0.00% 0.00% 0 0 59874 9924
TotalGoodware 59874
TotalMalware 9924
TotalSample 69798

Please send an email to lxu@trustlook.com if you have any comments. Thanks.