VirusTotal APK Malware Detection Data - Week 18: 202100426-20210502

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210426_20210502.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
K7GW 99.79% 0.06% 14748 41 65820 31
ESET-NOD32 98.75% 0.02% 14595 12 65849 184
Avira 98.03% 0.00% 14488 1 65860 291
Fortinet 97.62% 0.00% 14427 3 65858 352
Trustlook 97.46% 0.02% 14404 14 65847 375
AhnLab-V3 97.27% 0.02% 14376 13 65848 403
McAfee 96.71% 0.02% 14293 11 65850 486
Ikarus 96.57% 0.05% 14272 33 65828 507
Avast-Mobile 95.35% 0.55% 14092 359 65502 687
Kaspersky 91.44% 0.01% 13514 4 65857 1265
DrWeb 91.20% 0.13% 13479 86 65775 1300
McAfee-GW-Edition 91.10% 0.02% 13464 15 65846 1315
Sophos 85.81% 0.01% 12682 4 65857 2097
CAT-QuickHeal 85.30% 0.14% 12606 95 65766 2173
AegisLab 85.15% 0.08% 12584 55 65806 2195
Microsoft 81.25% 0.00% 12008 2 65859 2771
SymantecMobileInsight 74.59% 2.28% 11024 1504 64357 3755
Symantec 72.68% 0.17% 10741 115 65746 4038
Alibaba 71.84% 0.01% 10617 4 65857 4162
MAX 69.94% 0.00% 10337 1 65860 4442
Tencent 68.58% 0.10% 10136 66 65795 4643
ZoneAlarm 68.39% 0.00% 10108 3 65858 4671
NANO-Antivirus 57.51% 0.04% 8499 25 65836 6280
Cyren 45.98% 0.00% 6795 2 65859 7984
AVG 40.36% 0.03% 5965 20 65841 8814
Avast 40.16% 0.03% 5935 19 65842 8844
Zillya 34.42% 0.12% 5087 77 65784 9692
Kingsoft 27.84% 0.02% 4115 11 65850 10664
GData 16.57% 0.00% 2449 0 65861 12330
BitDefender 16.32% 0.00% 2412 0 65861 12367
Comodo 16.08% 0.06% 2376 37 65824 12403
Emsisoft 16.04% 0.00% 2371 0 65861 12408
ClamAV 8.84% 0.10% 1306 63 65798 13473
Jiangmin 8.28% 0.62% 1224 411 65450 13555
Zoner 8.23% 0.02% 1217 10 65851 13562
Arcabit 6.18% 0.00% 914 0 65861 13865
F-Secure 5.83% 0.00% 861 0 65861 13918
MicroWorld-eScan 5.65% 0.00% 835 0 65861 13944
Yandex 4.24% 0.00% 626 1 65860 14153
TrendMicro-HouseCall 2.66% 0.01% 393 5 65856 14386
Rising 2.54% 0.01% 376 4 65857 14403
TrendMicro 2.43% 0.03% 359 18 65843 14420
Ad-Aware 1.38% 0.00% 204 0 65861 14575
Antiy-AVL 1.01% 0.02% 150 16 65845 14629
VBA32 0.97% 0.01% 144 4 65857 14635
Panda 0.78% 0.01% 115 6 65855 14664
Babable 0.61% 0.04% 90 26 65835 14689
Baidu 0.27% 0.00% 40 2 65859 14739
ALYac 0.20% 0.00% 29 0 65861 14750
ViRobot 0.16% 0.00% 23 0 65861 14756
Malwarebytes 0.14% 0.00% 21 0 65861 14758
SentinelOne 0.13% 0.00% 19 0 65861 14760
VIPRE 0.07% 0.00% 11 0 65861 14768
F-Prot 0.05% 0.00% 8 0 65861 14771
Qihoo-360 0.05% 0.00% 8 0 65861 14771
K7AntiVirus 0.02% 0.00% 3 0 65861 14776
Bkav 0.00% 0.00% 0 0 65861 14779
TotalDefense 0.00% 0.00% 0 0 65861 14779
nProtect 0.00% 0.00% 0 0 65861 14779
CMC 0.00% 0.00% 0 0 65861 14779
CrowdStrike 0.00% 0.00% 0 0 65861 14779
TheHacker 0.00% 0.00% 0 0 65861 14779
eScan 0.00% 0.00% 0 0 65861 14779
SUPERAntiSpyware 0.00% 0.00% 0 0 65861 14779
Invincea 0.00% 0.00% 0 0 65861 14779
Endgame 0.00% 0.00% 0 0 65861 14779
Webroot 0.00% 0.00% 0 0 65861 14779
AVware 0.00% 0.00% 0 0 65861 14779
TotalGoodware 65861
TotalMalware 14779
TotalSample 80640

Please send an email to lxu@trustlook.com if you have any comments. Thanks.