November 4, 2019

VirusTotal APK Malware Detection Data - Week 44: 20191028-201901103

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20191028_20191103.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.70% 0.04% 78341 59 147031 238
K7GW 99.53% 0.25% 78206 370 146720 373
Trustlook 99.33% 0.56% 78056 818 146272 523
McAfee 98.71% 0.01% 77562 16 147074 1017
AhnLab-V3 98.67% 0.05% 77537 68 147022 1042
Avira 97.76% 0.00% 76822 0 147090 1757
ZoneAlarm 97.63% 0.04% 76716 65 147025 1863
Avast-Mobile 96.99% 0.14% 76217 203 146887 2362
Ikarus 96.88% 0.31% 76127 460 146630 2452
DrWeb 95.80% 0.47% 75277 694 146396 3302
Kaspersky 95.57% 0.02% 75100 23 147067 3479
Qihoo-360 93.59% 0.04% 73541 64 147026 5038
Symantec 93.59% 0.08% 73540 125 146965 5039
Fortinet 92.46% 0.01% 72658 20 147070 5921
NANO-Antivirus 88.30% 0.07% 69384 110 146980 9195
Sophos 80.94% 0.05% 63599 73 147017 14980
CAT-QuickHeal 80.78% 0.22% 63474 325 146765 15105
McAfee-GW-Edition 78.55% 0.00% 61720 7 147083 16859
F-Secure 75.21% 0.01% 59096 8 147082 19483
MAX 71.01% 0.01% 55799 10 147080 22780
AVG 62.38% 0.04% 49014 53 147037 29565
Avast 61.45% 0.03% 48288 50 147040 30291
Cyren 53.56% 0.01% 42085 21 147069 36494
Tencent 38.00% 0.00% 29862 0 147090 48717
Rising 14.36% 0.03% 11282 39 147051 67297
Antiy-AVL 5.57% 0.00% 4378 4 147086 74201
BitDefender 3.56% 0.00% 2800 0 147090 75779
TrendMicro-HouseCall 2.20% 0.08% 1727 119 146971 76852
Baidu 0.05% 0.01% 37 8 147082 78542
Ad-Aware 0.05% 0.00% 36 0 147090 78543
Babable 0.00% 0.00% 2 1 147089 78577
TotalGoodware 147090
TotalMalware 78579
TotalSample 225669

Please send an email to lxu@trustlook.com if you have any comments. Thanks.