January 13, 2020

VirusTotal APK Malware Detection Data - Week 2: 20200106-20200112

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200106_20200112.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.78% 0.11% 26869 70 63127 59
ESET-NOD32 99.71% 0.01% 26850 8 63189 78
Fortinet 99.38% 0.01% 26761 7 63190 167
ZoneAlarm 98.56% 0.01% 26539 6 63191 389
AhnLab-V3 98.36% 0.03% 26487 21 63176 441
McAfee 98.27% 0.01% 26462 8 63189 466
DrWeb 98.25% 0.14% 26457 87 63110 471
Trustlook 98.20% 0.21% 26443 133 63064 485
Kaspersky 98.16% 0.01% 26433 6 63191 495
Avira 97.94% 0.00% 26374 0 63197 554
Ikarus 97.27% 0.19% 26193 122 63075 735
F-Secure 97.14% 0.01% 26159 5 63192 769
Sophos 92.31% 0.04% 24858 27 63170 2070
Avast-Mobile 89.58% 0.20% 24121 126 63071 2807
Qihoo-360 65.07% 0.03% 17522 22 63175 9406
CAT-QuickHeal 62.99% 0.04% 16963 25 63172 9965
Symantec 54.54% 0.02% 14686 11 63186 12242
McAfee-GW-Edition 51.66% 0.00% 13912 0 63197 13016
NANO-Antivirus 45.37% 0.07% 12218 44 63153 14710
Tencent 44.96% 0.07% 12108 45 63152 14820
Cyren 41.28% 0.00% 11117 3 63194 15811
MAX 38.61% 0.00% 10396 0 63197 16532
AVG 35.88% 0.06% 9661 41 63156 17267
Avast 35.12% 0.06% 9457 39 63158 17471
Rising 5.24% 0.03% 1412 16 63181 25516
Antiy-AVL 3.81% 0.00% 1026 3 63194 25902
TrendMicro-HouseCall 3.18% 0.02% 855 12 63185 26073
BitDefender 2.90% 0.00% 780 0 63197 26148
Ad-Aware 0.09% 0.00% 24 0 63197 26904
Baidu 0.08% 0.01% 21 5 63192 26907
Babable 0.00% 0.00% 0 0 63197 26928
TotalGoodware 63197
TotalMalware 26928
TotalSample 90125

Please send an email to lxu@trustlook.com if you have any comments. Thanks.