October 7, 2019

VirusTotal APK Malware Detection Data - Week 40: 20190930-201901006

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20190930_20191006.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.65% 0.14% 78251 455 325348 276
K7GW 99.45% 0.55% 78096 1776 324027 431
Avast-Mobile 99.45% 0.33% 78096 1086 324717 431
AhnLab-V3 99.24% 0.10% 77927 328 325475 600
McAfee 98.75% 0.01% 77549 17 325786 978
Trustlook 98.48% 0.36% 77332 1159 324644 1195
ZoneAlarm 98.29% 0.05% 77181 170 325633 1346
Avira 98.04% 0.00% 76991 2 325801 1536
DrWeb 97.51% 0.98% 76570 3206 322597 1957
Ikarus 96.40% 0.52% 75697 1696 324107 2830
Kaspersky 96.37% 0.02% 75678 68 325735 2849
Symantec 95.96% 0.09% 75355 304 325499 3172
Fortinet 95.28% 0.03% 74818 94 325709 3709
Sophos 92.91% 0.10% 72962 332 325471 5565
NANO-Antivirus 90.99% 0.26% 71455 842 324961 7072
Tencent 90.92% 0.07% 71396 242 325561 7131
CAT-QuickHeal 88.98% 0.41% 69874 1342 324461 8653
McAfee-GW-Edition 85.82% 0.00% 67394 14 325789 11133
Qihoo-360 82.27% 0.03% 64604 100 325703 13923
F-Secure 79.05% 0.02% 62073 65 325738 16454
AVG 68.79% 0.05% 54022 177 325626 24505
Avast 67.73% 0.05% 53189 172 325631 25338
MAX 66.64% 0.01% 52328 18 325785 26199
Cyren 61.80% 0.02% 48529 51 325752 29998
Antiy-AVL 24.93% 0.03% 19579 99 325704 58948
Rising 16.60% 0.10% 13032 319 325484 65495
TrendMicro-HouseCall 6.01% 0.39% 4717 1275 324528 73810
BitDefender 3.41% 0.00% 2680 0 325803 75847
Baidu 0.05% 0.03% 42 112 325691 78485
Ad-Aware 0.04% 0.00% 28 0 325803 78499
Babable 0.00% 0.00% 1 13 325790 78526
TotalGoodware 325803
TotalMalware 78527
TotalSample 404330

Please send an email to lxu@trustlook.com if you have any comments. Thanks.