The verbiage of the warning may be a little misleading. It doesn't tell you how much has been scanned prior to the call into cli_scanxz(). It is likely that it scanned additional data prior to the call to cli_scanxz(), which counts towards the scan limit.
If you re-scan with the --debug flag set, it should report an additional debug-level message near the warning you saw, in the form of:
"scansize exceeded (initial: %lu, consumed: %lu, needed: %lu)"
initial:
should be the limit (3999M).
consumed:
should be the amount already scanned.
needed:
should be the amount of data that the cli_scanxz() function has on hand and would like to scan.
If you run test this, I would be curious to know what is reported. That said, your comments raises the age old question of if or how ClamAV should report that it is unable to finish scanning a file when limits are exceeded. Most clamav scanning code will not report a warning at all like you see with the xz scanner, and the only way to determine if limits have been exceeded is to run with --blockmax or --alert-exceeds-max (v0.101+).
Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.
On Dec 7, 2018, at 3:41 PM, Albert o <***@gmail.com<mailto:***@gmail.com>> wrote:
There is something I don't get
this my clamd.conf
***@debian:~/Downloads/clamav-0.101.0$ cat /etc/clamav/clamd.conf
#Automatically Generated by clamav-daemon postinst
#To reconfigure clamd run #dpkg-reconfigure clamav-daemon
#Please read /usr/share/doc/clamav-daemon/README.Debian.gz for details
LocalSocket /var/run/clamav/clamd.ctl
FixStaleSocket true
LocalSocketGroup clamav
LocalSocketMode 666
# TemporaryDirectory is not set to its default /tmp here to make overriding
# the default with environment variables TMPDIR/TMP/TEMP possible
User clamav
ScanMail true
ScanArchive true
ArchiveBlockEncrypted false
MaxDirectoryRecursion 100
FollowDirectorySymlinks true
FollowFileSymlinks true
ReadTimeout 180
MaxThreads 12
MaxConnectionQueueLength 2000
LogSyslog true
LogRotate true
LogFacility LOG_LOCAL6
LogClean false
LogVerbose false
PreludeEnable no
PreludeAnalyzerName ClamAV
DatabaseDirectory /var/lib/clamav
OfficialDatabaseOnly false
SelfCheck 3600
Foreground true
Debug 0
PidFile true
MaxEmbeddedPE 3999M
ScanOLE2 true
ScanPDF true
ScanHTML true
MaxHTMLNormalize 3999M
MaxHTMLNoTags 3999M
MaxScriptNormalize 3999M
MaxZipTypeRcg 3999M
ScanSWF true
DetectBrokenExecutables false
ExitOnOOM false
LeaveTemporaryFiles false
AlgorithmicDetection true
ScanELF true
ScanPE true
IdleTimeout 30
CrossFilesystems true
PhishingSignatures true
PhishingScanURLs true
ExtendedDetectionInfo true
PhishingAlwaysBlockSSLMismatch false
PhishingAlwaysBlockCloak false
PartitionIntersection false
DetectPUA 1
BlockMax 1
OLE2BlockMacros 1
ArchiveBlockEncrypted 1
ScanPartialMessages 1
PartitionIntersection 1
HeuristicScanPrecedence 1
StructuredDataDetection 1
CommandReadTimeout 5
SendBufTimeout 400
MaxQueue 100
ExtendedDetectionInfo 1
OLE2BlockMacros 1
ScanOnAccess true
AllowAllMatchScan true
ForceToDisk false
DisableCertCheck false
DisableCache false
MaxScanSize 3999M
MaxFileSize 3999M
MaxRecursion 160
MaxFiles 20000
MaxPartitions 10000
MaxIconsPE 10000
PCREMatchLimit 10000
PCRERecMatchLimit 10000
PCREMaxFileSize 3999M
ScanXMLDOCS true
ScanHWP3 true
MaxRecHWP3 16
StreamMaxLength 3999M
LogFile /var/log/clamav/clamav.log
LogTime true
LogFileUnlock true
Bytecode true
BytecodeSecurity TrustSigned
BytecodeTimeout 60000
BytecodeUnsigned 1
ScanOnAccess 1
OnAccessMaxFileSize 0
OnAccessMountPath /
OnAccessIncludePath /
OnAccessMaxFileSize 0
OnAccessPrevention 1
OnAccessDisableDDD 1
but I still get
LibClamAV Warning: cli_msxml_parse_document: encountered issue in
parsing xml document
LibClamAV Warning: cli_scanxz: decompress file size exceeds limits -
only scanning 27262976 bytes
What am I missing? should't all the files >4G be scanned?
On Tue, Dec 4, 2018 at 4:42 PM Ãngel <***@av.16bits.net<mailto:***@av.16bits.net>> wrote:
On 2018-12-03 at 09:58 -0800, Dennis Peterson wrote:
If it is a big concern you can use the split command to create
"splits" of the suspect file. Split accepts various size arguments
(bytes, lines...) and will create as many files as it takes to split
the entire large file. These can be scanned individually and discarded
when done. There is a risk of a split happening in the middle of a
section that might match a signature but that is small. A work around
is to split a file, scan it, delete the splits, then split it a second
time using a different split size and repeat the scan.
This is obviously tedious and works best on static files. There's
always a way if you don't mind the effort. It is easily scriptable.
dp
Splitting a file will probably make chunks other than the first to
appear as random bytes, rather than having the correct filetype, thus
making some signatures not to be applied.
(the first chunk will _probably_ be detected properly, still splitting
can make it miss what would be found on the full size, eg. splitting a
zip file will lose its central directory...)
Signatures are generally more complex than looking for a certain
substring...
Best regards
_______________________________________________
clamav-users mailing list
clamav-***@lists.clamav.net<mailto:clamav-***@lists.clamav.net>
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
_______________________________________________
clamav-users mailing list
clamav-***@lists.clamav.net<mailto:clamav-***@lists.clamav.net>
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml