November 2, 2020

VirusTotal APK Malware Detection Data - Week 44: 20201026-20201101

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20201026_20201101.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.31% 0.02% 14596 14 87186 102
Trustlook 99.18% 0.38% 14577 331 86869 121
Avira 98.41% 0.00% 14465 1 87199 233
K7GW 98.20% 0.15% 14434 128 87072 264
Fortinet 98.06% 0.02% 14413 16 87184 285
Avast-Mobile 97.92% 0.25% 14392 218 86982 306
DrWeb 95.09% 0.23% 13976 202 86998 722
AhnLab-V3 95.03% 0.03% 13967 27 87173 731
Ikarus 93.75% 0.14% 13780 124 87076 918
CAT-QuickHeal 93.54% 0.01% 13748 10 87190 950
ZoneAlarm 93.38% 0.01% 13725 7 87193 973
Kaspersky 92.88% 0.00% 13651 2 87198 1047
McAfee 89.08% 0.01% 13093 5 87195 1605
McAfee-GW-Edition 87.02% 0.01% 12790 10 87190 1908
F-Secure 73.66% 0.00% 10827 2 87198 3871
NANO-Antivirus 70.70% 0.03% 10392 26 87174 4306
Sophos 70.15% 0.02% 10311 21 87179 4387
Qihoo-360 68.66% 0.01% 10091 13 87187 4607
Symantec 58.96% 0.01% 8666 9 87191 6032
AVG 58.66% 0.03% 8622 23 87177 6076
Ad-Aware 1.10% 0.00% 162 0 87200 14536
TotalGoodware 87200
TotalMalware 14698
TotalSample 101898

Please send an email to lxu@trustlook.com if you have any comments. Thanks.