December 30, 2019

VirusTotal APK Malware Detection Data - Week 52: 20191223-201901229

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20191223_20191229.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.72% 0.14% 25715 82 59439 71
ESET-NOD32 99.71% 0.04% 25711 24 59497 75
ZoneAlarm 99.32% 0.01% 25610 6 59515 176
Fortinet 99.22% 0.01% 25584 4 59517 202
Kaspersky 99.10% 0.01% 25553 5 59516 233
DrWeb 99.00% 0.13% 25527 76 59445 259
AhnLab-V3 98.64% 0.04% 25436 25 59496 350
Ikarus 98.48% 0.16% 25393 97 59424 393
Trustlook 97.86% 0.50% 25234 295 59226 552
Avira 97.55% 0.00% 25154 1 59520 632
F-Secure 97.00% 0.01% 25013 5 59516 773
McAfee 96.96% 0.01% 25001 8 59513 785
Symantec 84.07% 0.05% 21678 27 59494 4108
Sophos 80.09% 0.02% 20652 12 59509 5134
Qihoo-360 56.83% 0.04% 14654 21 59500 11132
Avast-Mobile 52.28% 0.15% 13482 91 59430 12304
McAfee-GW-Edition 45.85% 0.00% 11824 0 59521 13962
CAT-QuickHeal 40.16% 0.05% 10356 28 59493 15430
NANO-Antivirus 38.58% 0.05% 9947 30 59491 15839
Cyren 29.06% 0.00% 7494 1 59520 18292
AVG 27.02% 0.05% 6967 27 59494 18819
Avast 26.61% 0.04% 6862 26 59495 18924
Tencent 17.06% 0.00% 4398 0 59521 21388
MAX 13.72% 0.00% 3539 0 59521 22247
Rising 4.06% 0.03% 1048 18 59503 24738
Antiy-AVL 3.13% 0.00% 806 2 59519 24980
BitDefender 2.39% 0.00% 615 0 59521 25171
TrendMicro-HouseCall 2.21% 0.04% 571 23 59498 25215
Ad-Aware 0.03% 0.00% 9 0 59521 25777
Baidu 0.03% 0.00% 8 2 59519 25778
Babable 0.00% 0.00% 0 0 59521 25786
TotalGoodware 59521
TotalMalware 25786
TotalSample 85307

Please send an email to lxu@trustlook.com if you have any comments. Thanks.