November 16, 2020

VirusTotal APK Malware Detection Data - Week 46: 20201109-20201115

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20201109_20201115.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.59% 0.14% 27293 95 69609 111
ESET-NOD32 99.58% 0.02% 27288 13 69691 116
Trustlook 99.49% 0.17% 27265 116 69588 139
Avast-Mobile 99.24% 0.28% 27195 192 69512 209
Fortinet 99.01% 0.02% 27132 12 69692 272
Avira 98.69% 0.00% 27046 0 69704 358
McAfee-GW-Edition 98.18% 0.01% 26906 6 69698 498
DrWeb 98.11% 0.23% 26887 162 69542 517
ZoneAlarm 98.05% 0.00% 26870 3 69701 534
McAfee 98.02% 0.01% 26862 4 69700 542
Kaspersky 97.19% 0.00% 26635 1 69703 769
CAT-QuickHeal 97.02% 0.01% 26587 6 69698 817
Ikarus 95.48% 0.20% 26164 139 69565 1240
Symantec 91.59% 0.01% 25098 5 69699 2306
Qihoo-360 89.92% 0.01% 24643 8 69696 2761
Sophos 89.47% 0.01% 24519 10 69694 2885
AhnLab-V3 89.33% 0.04% 24479 25 69679 2925
NANO-Antivirus 88.69% 0.05% 24304 33 69671 3100
F-Secure 87.49% 0.00% 23976 2 69702 3428
AVG 74.27% 0.03% 20352 22 69682 7052
Ad-Aware 0.84% 0.00% 230 0 69704 27174
TotalGoodware 69704
TotalMalware 27404
TotalSample 97108

Please send an email to lxu@trustlook.com if you have any comments. Thanks.