Nmap 5.20 Released

[Fyodor <fyodor () insecure org>]

Happy new year, everyone.  I'm happy to announce Nmap 5.20--our first
stable Nmap release since 5.00 last July!  It offers more than 150
significant improvements, including:
 o 30+ new Nmap Scripting Engine scripts
 o enhanced performance and reduced memory consumption
 o protocol-specific payloads for more effectie UDP scanning
 o a completely rewritten traceroute engine
 o massive OS and version detection DB updates (10,000+ signatures)

The Nmap 5.00 source code and packages for Linux, Mac, and Windows are
available for download at the usual place:

http://nmap.org/download.html

Go give it a try!  And if you find any bugs, let us know on nmap-dev
(http://nmap.org/book/man-bugs.html).

Here are the CHANGELOG entries since 5.00:

o Added 31 new Nmap Scripting Engine scripts for a grand total of 80!
  They new ones are summarized in this release (tagged [NSE SCRIPT]),
  but you can learn much more about them all at
  http://nmap.org/nsedoc/.

o [Zenmap] After performing or loading a scan, you can now filter
  results to just the hosts you are interested in by pressing Ctrl+L
  (or the "Filter Hosts" button) to open the host filtering interface.
  This makes it easy to select just Linux hosts, or those running a
  certain version of Apache, or whatever interests you. You can easily
  modify the filter or remove it to see the whole scan again. See
  http://nmap.org/book/zenmap-filter.html. [Josh Marlow]

o For some UDP ports, Nmap will now send a protocol-specific payload
  that is more likely to get a response than an empty packet is. This
  improves the effectiveness of probes to those ports for host
  discovery, and also makes an open port more likely to be classified
  open rather than open|filtered. The ports and payloads are defined
  in payload.cc. The ports that have a payload are 7 (echo),
  53 (domain), 111 (rpcbind), 123 (ntp), 137 (netbios-ns), 161 (snmp),
  177 (xdmcp), 500 (isakmp), 520 (route), 1645 and 1812 (radius),
  2049 (nfs), 5353 (zeroconf), and 10080 (amanda). [David]

o Nmap's --traceroute has been rewritten for better performance.
  Probes are sent in parallel to individual hosts, not just across all
  hosts as before. Trace consolidation is more sophisticated, allowing
  common traces to be identified sooner and fewer probes to be sent.
  The older traceroute could be very slow (taking minutes per target)
  if the target did not respond to the trace probes, and this new
  traceroute avoids that. In a trace of 110 hosts in a /24 over the
  Internet, the number of probes sent dropped 50% from 1565 to 743,
  and the time taken dropped 92% from 95 seconds to 7.6
  seconds. Traceroute now uses an ICMP echo request probe if no
  working probes against the target were discovered during
  scanning. [David]

o Integrated 1,349 fingerprints (and 81 corrections) submitted by Nmap
  users! They resulted in 342 new fingerprints (a 17% increase),
  including Google's Android Linux system for smart phones, Mac OS X
  10.6 (Snow Leopard), the Chumby, and a slew number of printers, broadband
  routers, and other devices (40 new vendors). See
  http://seclists.org/nmap-dev/2009/q4/416 [David]

o Dramatically improved the version detection database, integrating
  2,596 submissions that users contributed since February 3, 2009!
  More than a thousand signatures were added, bringing the total to
  8,501. Many existing signatures were improved as well. Please keep
  those submissions and corrections coming! Nmap prints a submission
  URL and fingerprint when it receives responses it can't yet
  interpret.

o [Ncat] The --ssl, --output, and --hex-dump options now work with
  --exec and --sh-exec. Among other things, this allows you to make a
  program's I/O available over the network wrapped in SSL encryption
  for security.  It is implemented by forking a separate process to
  handle network communications and relay the data to the
  sub-process. [Venkat, David]

o [NSE SCRIPT] http-enum enumerates URLs used by popular web
  applications and servers and reports which ones exist on a target
  web server. See http://nmap.org/nsedoc/scripts/http-enum.html. [Ron,
  Andrew Orr, Rob Nicholls]

o [NSE SCRIPT] nfs-showmount displays NFS exports like "showmount -e"
  does. See http://nmap.org/nsedoc/scripts/nfs-showmount.html. [Patrik
  Karlsson]

o [NSE SCRIPT] dhcp-discover sends out DHCP probes on UDP/67 and
  displays all interesting results (or, with verbosity, all results).
  Optionally, multiple probes can be sent and the MAC address can be
  randomized in an attempt to exhaust the DHCP server's address pool
  and potentially create a denial of service condition. See
  http://nmap.org/nsedoc/scripts/dhcp-discover.html. [Ron]

o We performed a memory consumption audit and made changes to
  dramatically reduce Nmap's footprint.  This improves performance on
  all systems, but is particularly important when running Nmap on
  small embedded devices such as phones.  Our intensive UDP scan
  benchmark saw peak memory usage decrease from 34MB to 6MB, while OS
  detection consumption was reduced from 67MB to 3MB.  Read about the
  changes at http://seclists.org/nmap-dev/2009/q4/663.  Here are the
  highlights:

  * The size of the internal representation of nmap-os-db was reduced
    more than 90%. Peak memory consumption in our OS detection
    benchmark was reduced from 67MB to 3MB. [David]

  * The size of individual Port structures without service scan
    results was reduced about 70%. [Pavel Kankovsky]

  * When a port receives no response, Nmap now avoids allocating a
    Port structure at all, so scans against filtered hosts can be
    light on memory. [David]

o Ndiff now shows changes in script (NSE) output for each target
  host (in both text output format and XML). [David]

o [NSE SCRIPT] smb-psexec implements remote process execution similar
  to the Sysinternals' psexec tool (or Metasploit's psexec "exploit"),
  allowing a user to run a series of programs on a remote machine and
  read the output. This is great for gathering information about
  servers, running the same tool on a range of system, or even
  installing a backdoor on a collection of computers. It works against
  Win2K, Windows 2003, and Windows XP. See
  http://nmap.org/nsedoc/scripts/smb-psexec.html [Ron]

o [NSE SCRIPT] citrix-enum-apps and citrix-enum-apps-xml print a list
  of published applications from the Citrix ICA Browser or XML
  service, respectively. See
  http://nmap.org/nsedoc/scripts/citrix-enum-apps.html and
  http://nmap.org/nsedoc/scripts/citrix-enum-apps-xml.html. [Patrik
  Karlsson]

o [NSE SCRIPT] citrix-enum-servers and citrix-enum-servers-xml.nse
  print a list of Citrix servers from the Citrix ICA Browser or XML
  service, respectively. See
  http://nmap.org/nsedoc/scripts/citrix-enum-servers.html and
  http://nmap.org/nsedoc/scripts/citrix-enum-servers-xml.html. [Patrik
  Karlsson]

o [NSE SCRIPT] citrix-brute-xml uses the unpwdb library to guess
  credentials for the Citrix PN Web Agent Service. See
  http://nmap.org/nsedoc/scripts/citrix-brute-xml.html. [Patrik
  Karlsson]

o [NSE SCRIPT] oracle-sid-brute queries the Oracle TNS-listener for
  default instance/sid names. The SID enumeration list was prepared by
  Red Database security. See
  http://nmap.org/nsedoc/scripts/oracle-sid-brute.html. [Patrik
  Karlsson]

o [NSE SCRIPT] x11-access checks whether access to an X11 server is
  allowed (as with "xhost +" for example). See
  http://nmap.org/nsedoc/scripts/x11-access.html. [jlanthea]

o [NSE SCRIPT] db2-info enhances DB2 database instance detection. It
  provides detection when version probes fail, but will default to the
  version detection probe value if that is more precise. It also
  detects the server platform and database instance name. The DB2
  version detection port ranges were broadened to 50000-50025 and
  60000-60025 as well. See
  http://nmap.org/nsedoc/scripts/db2-info.html. [Tom]

o [NSE SCRIPT] ssl-cert retrieves and prints a target server's SSL
  certificate. It can do TLS negotiation against SMTP ports which
  support it.  See
  http://nmap.org/nsedoc/scripts/ssl-cert.html. [David]

o [Ncat] Now has configure-time ASCII art just like Nmap does:
            .       .
            \`-"'"-'/
             } 6 6 {
            ==. Y ,==
              /^^^\  .
             /     \  )  Ncat: A modern interpretation of classic Netcat
            (  )-(  )/
            -""---""---   /
           /   Ncat    \_/
          (     ____
           \_.=|____E

o [NSE] Default socket parallelism has been doubled from 10 to 20,
  which doubles speed in some situations. See
  http://seclists.org/nmap-dev/2009/q3/161. [Patrick]

o Version detection's maximum socket concurrency has been increased
  from 10-20 based on timing level to 20-40. This can dramatically
  speed up version detection when there are many open ports in a host
  group being scanned. [Fyodor]

o [Ncat] For compatibility with Hobbit's original Netcat, The -p
  option now works to set the listening port number in listen mode.
  So "ncat -l 123" can now be expressed as "ncat -l -p 123"
  too. [David]

o [NSE SCRIPT] smbv2-enabled checks if the smbv2 protocol is enabled
  on target servers. SMBv2 has already suffered from at least one
  major security vulnerability. See
  http://nmap.org/nsedoc/scripts/smbv2-enabled.html. [Ron]

o [NSE SCRIPT] http-favicon obtains the favicon file (/favicon.ico or
  whatever is specified by the HTML link tag) and tries to identify
  its source (such as a certain web application) using a database
  lookup. See
  http://nmap.org/nsedoc/scripts/http-favicon.html. [Vladz]

o [NSE SCRIPT] http-date obtains the Date: header field value from an
  HTTP server then displays it along with how much it differs from
  local time. See
  http://nmap.org/nsedoc/scripts/http-date.html. [David]

o [NSE] Added a function for scripts to format their output in a
  consistent way. See
  http://nmap.org/nsedoc/lib/stdnse.html#format_output. [Ron]

o [NSE SCRIPT] http-userdir-enum attempts to enumerate users on a
  system by trying URLs with common usernames in the Apache
  mod_userdir format (e.g. http://target-server.com/~john). See
  http://nmap.org/nsedoc/scripts/http-userdir-enum.html. [Jah]

o [NSE SCRIPT] pjl-ready-message allows viewing and setting the status
  message on printers which support the Printer Job Language (many HP
  printers do). See
  http://nmap.org/nsedoc/scripts/pjl-ready-message.html.  [Aaron
  Leininger]

o [NSE SCRIPT] http-headers performs a GET request for the root folder
  ("/") of a web server and displays the HTTP headers returned. See
  http://nmap.org/nsedoc/scripts/http-headers.html. [Ron]

o [NSE SCRIPT] http-malware-host is designed to discover hosts that
  are serving malware (perhaps because they were compromised), but so
  far it only checks for one specific attack. See
  http://nmap.org/nsedoc/scripts/http-malware-host.html. [Ron]

o [NSE SCRIPT] smb-enum-groups displays a list of groups on the remote
  system along with their membership (like enum.exe -G). See
  http://nmap.org/nsedoc/scripts/smb-enum-users.html [Ron]

o [NSE] For all the services which are commonly tunneled over SSL
  (pop3, http, imap, irc, smtp, etc.), we audited the scripts to
  ensure they can support that tunneling. The com.tryssl function
  was added for easy SSL detection. See
  http://nmap.org/nsedoc/lib/comm.html [Joao]

o [NSE SCRIPT] ntp-info prints the time and configuration variables
  provided by an NTP service. It may get such interesting information
  as the operating system, server build date, and upstream time server
  IP address. See
  http://nmap.org/nsedoc/scripts/ntp-info.html. [Richard Sammet]

o Added a service probe and match lines for the Logitech/SlimDevices
  SqueezeCenter music server. [Patrik Karlsson]

o Added service detection probe for Kerberos (udp/88) and IBM DB2
  DAS (523/UDP). [Patrik Karlsson]

o Added a UDP payload and service detection probe for Citrix
  MetaFrame, which typically runs on 1604/udp. [Thomas Buchanan]

o Added a UDP SIPOptions service detection probe corresponding to the
  TCP one. [Patrik Karlsson, Matt Selsky, David Fifield]

o Updated service detection signatures for Microsoft SQL Server 2005
  to detect recent Microsoft security update (MS09-062), and also
  updated ms-sql-info.nse to support MS SQL Server 2008
  detection. [Tom]

o Added a service probe for DNS-based service discovery (DNS-SD). See
  http://seclists.org/nmap-dev/2009/q3/0610.html. [David]

o Added Apache JServe protocol version detection probe and signatures
  and some some other nmap-service-probes patches. [Tom Sellers]

o Made RPC grinding work from service detection again by changing the
  looked-for service name from "rpc" to "rpcbind", the name it has in
  nmap-service-probes. Also removed some dead code. [David]

o A new script argument, http.useragent, lets you modify
  the User-Agent header sent by NSE from its default of "Mozilla/5.0
  (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)".
  Set it to the empty string to disable the User-Agent
  entirely. [David, Tom Sellers, Jah]

o [Zenmap] The locale setting had been taken from the Windows locale,
  which inadvertently made setting the locale with the LANG
  environment variable stop working. Now the LANG variable is examined
  first, and if that is not present, the system-wide setting is
  used. This change allows users to keep Zenmap in its original
  English (or any of Zenmap's other languages) even if their system is
  set to use a different locale.  [David]

o The Nmap source tarball (and RPMs) now included man page
  translations (16 languages so far). Nmap always installs the English
  man page, and installs the translations by default. If you only want
  some of the translations, set the LINGUAS environmental variable to
  the language codes you are interested in (e.g. "es de"). You can
  specify the configure option --disable-nls or set LINGUAS to the
  empty string to avoid installation of any man page translations. The
  RPM always installs them. [David]

o The Ndiff man page was dramatically improved with examples and
  sample output. See http://nmap.org/ndiff/man.html.
  [David]

o [NSE] Replaced our runlevel system for managing the order of script
  execution with a much more powerful dependency system. This allows
  scripts to specify which other scripts they depend on (e.g. a brute
  force authentication script might depend on username enumeration
  scripts) and NSE manages the order. Dependencies only enforce
  ordering, they cannot pull in scripts which the user didn't
  specify. See
  http://nmap.org/book/nse-script-format.html#nse-format-dependencies
  [Patrick]

o [NSE] The http-favicon script is now better at finding "link
  rel=icon" tags in pages, and uses that icon in preference to
  /favicon.ico if found. If the favicon.uri script arg is given, only
  that is tried.  Meanwhile, a giant (10 million web servers) favicon
  scan by Brandon allowed us to add about 40 more of the most popular
  icons to the DB. [David, Brandon]

o Added -Pn and -sn as aliases for -PN and -sP, respectively. They
  will eventually become the recommended and documented way to disable
  host discovery (ping scanning) and port scanning. They are more
  consistent and also match the existing -n option for disabling
  reverse DNS resolution. [David]

o Nmap script output now uses two spaces of indention rather than
  three for the first level. This better aligns with the standard set by
  the stdnse.format_output function added in the last release. Output
  now looks like:
  8082/tcp open  http        Apache httpd 2.2.13 ((Fedora))
  |_http-favicon: Apache Web Server (seen on SuSE, Linux Tux favicon)
  |_html-title: Nmap - Free Security Scanner For Network Exploration & Securit...
  ...
  Host script results:
  | smb-os-discovery:  
  |   OS: Unix (Samba 3.4.2-0.42.fc11)
  |   Name: Unknown\Unknown
  |_  System time: 2009-11-24 17:19:21 UTC-8
  |_smbv2-enabled: Server doesn't support SMBv2 protocol
  [Fyodor]

o [Ncat] Implemented basic SCTP client functionality (server already
  exists).  Only the default SCTP stream is used.  This is also called
  TCP compatible mode.  While it allows Ncat to be used for manually
  probing open SCTP ports, more complicated services making use of
  multiple streams or depending on specific message boundaries cannot
  be talked to successfully.  [Daniel Roethlisberger]

o [Ncat] Implemented SSL over SCTP in both client (connect) and server
  (listen) modes. [Daniel Roethlisberger]

o [Ncat] In verbose mode, Ncat now prints the number of bytes read and
  written after the client connection is terminated. Ncat also now
  prints elapsed time. For example, "Ncat finished: 16 bytes sent, 566
  bytes received in 8.05 seconds." [Venkat]

o [NSE] Fixed a bug which kept the nselib/data/psexec subdirectory out
  of the Windows packages. We needed to add the /s and /e options to
  xcopy in our Visual C++ project file. [David]

o [NSE] Overhauled our http library to centralize HTTP parsing and
  make it more robust. The biggest user-visible change is that
  http.request goes back to returning a parsed result table rather than raw
  HTTP data. Also the http.pipeline function no longer accepts the
  no-longer-used "raw" option. [David]

o Fixed compilation of libdnet-stripped on platforms that don't have
  socklen_t. [Michael Pattrick]

o [NSE] Our http library no longer allows cached responses from a GET
  request to be returned for a HEAD request. This could cause problems
  with at least the http-enum script. [David]

o Fixed a bug in the WinPcap installer: If the "Start the WinPcap
  service 'NPF' at startup" box was unchecked and the "Start the
  WinPcap service 'NPF' now" box was checked, the second checkbox
  would be ignored (the service would not be started now). [Rob
  Nicholls]

o Removed a limitation of snmp.lua which only allowed it to properly
  encode OID component values up to 127. The bug was reported by
  Victor Rudnev. [David]

o [NSE] Fixed (we hope) a deadlock we were seeing when doing a
  favicon.nse survey against millions of hosts. We now restore all
  threads that are waiting on a socket lock when a thread relinquishes
  its lock. We expect only one of them to be able to grab the newly
  freed lock, and the rest to go back to waiting. [David, Patrick]

o Our Windows packages are now built on Windows 7, though they are
  32-bit binaries and should continue to work on Win2K and later.

o [Zenmap] Fixed a crash when filtering with inroute: in scans without
  traceroute data. (KeyError: 'hops') [David]

o [NSE] Use a looser match pattern in auth-owners.nse for retrieving
  the owner out of an identd response. See
  http://seclists.org/nmap-dev/2009/q4/549. [Richard Sammet]

o Improved some Cyrus pop3 and Polycom SoundStation sip match
  lines. [Matt Selsky]

o Nmap now tries start the WinPcap NPF service on Windows if it is not
  already running. This is rare, since our WinPcap installer starts
  NPF running at system boot time by default. Because starting NPF
  requires administrator privileges, a UAC dialog for net.exe may
  appear on Windows Vista and Windows 7 before NPF is loaded.  Once
  NPF is loaded, it generally stays loaded until you reboot or run
  "net stop npf". [David, Michael Pattrick]

o The Nmap Windows installer and our WinPcap installer now have an
  option /NPFSTARTUP=NO, which inhibits the installer from setting the
  WinPcap NPF service to start at system startup and at install-time.
  This option only affects silent mode (/S) because existing GUI
  checkboxes allow you to configure this behavior during interactive
  installation. [David]

o [Ncat] In the Windows version of netrun, we weren't noticing when a
  command fails to be executed (when CreateProcess fails). We now see
  the return value and close the socket to disconnect the
  client. [David]

o [NSE] Updated http-iis-webdav-vuln to run against SSL-enabled
  servers [Ron]

o Nmap now prefers to display the hostname supplied by the user instead
  of the reverse-DNS name in most places. If a reverse DNS record
  exists, and it differs from the user-supplied name, it is printed
  like this:
    Nmap scan report for www.google.com (74.125.53.103)
    rDNS record for 74.125.53.103: pw-in-f103.1e100.net
  And in XML it looks like:
  <hostnames>
    <hostname name="openbsd.org" type="user"/>
    <hostname name="cvs.openbsd.org" type="PTR"/>
  </hostnames>
  Host latency is now printed more often. See
  http://seclists.org/nmap-dev/2009/q4/199 for a summary of other
  output changes. [David]

o We now print output for down hosts, even when doing scanning beyond
  just a ping scan.  This always prints to XML and grepable output,
  and is printed to normal and interactive output in verbose mode. The
  format for printing a down host has changed slightly: "Nmap scan
  report for 1.1.1.1 [host down]" [David]

o [NSE] Now supports worker threads so that a single script can
  perform multiple network operations concurrently. This patch also
  includes condition variables for synchronization. See
  http://nmap.org/nsedoc/lib/stdnse.html#new_thread,
  http://nmap.org/nsedoc/lib/nmap.html#condvar, and
  http://seclists.org/nmap-dev/2009/q4/294.

o Fixed a problem in which the Nmap installer wrongly reported that
  the Microsoft Visual C++ 2008 Redistributable Package (vcredist.exe)
  failed to install. We had to update a registry key--see
  http://seclists.org/nmap-dev/2009/q3/164. [Jah]

o Added support for connecting to nameservers over IPv6. IPv6 addresses
  can be used in /etc/resolv.conf or with the --dns-servers option. The
  parallel reverse DNS resolver still only support IPv4 addresses, but
  it can look them up over IPv6. [Ankur Nandwani]

o Zenmap now includes ports in the services view whenever Nmap found
  them "interesting," whatever their state. Previously they were only
  included if the state was "open", "filtered", or "open|filtered",
  which led to confusing behavior when a closed port showed up in the
  Services column but clicking on the service showed no ports in the
  display. [David]

o [NSE] Added HTTP pipelining support to the HTTP library and and to
  the http-enum, http-userdir-enum, and sql-injection.nse
  scripts. Pipelining can increase speed dramatically for scripts
  which make many requests.

o [NSE] The HTTP library now caches responses from http.get or
  http.head so that resources aren't requested multiple times during
  the same Nmap run even if several scripts request them. See
  http://seclists.org/nmap-dev/2009/q3/733. [Patrick]

o [Ncat, Ndiff] The exit codes of these programs now reflect whether
  they succeeded. For Ncat, 0 means the connection was successful, 1
  indicates a network error, and 2 indicates any other error. For
  Ndiff, 0 means the scans were equal, 1 means they were different,
  and 2 indicates a runtime error. [David]

o [NSE] telnet-brute.nse now uses the unpw database instead of a
  hard coded list. [Ron]

o [NSE] Scripts that are listed by name with the --script option now
  have their verbosity level automatically increased by one. Many
  will print negative results ("no infection found") at a higher
  verbosity level. The idea is that if you ask for a script
  specifically, you are more interested in such results.
  [David, Patrick]

o Added a check for a SMBv2 vulnerability (CVE-2009-3103) to
  smb-check-vulns. Due to its nature (it performs a DoS, then checks
  if the system is still online), the script isn't run by default and
  requires a special script-arg to work. See
  http://nmap.org/nsedoc/scripts/smb-check-vulns.html. [Ron]

o Upgraded our Winpcap installer to use the new WinPcap version 4.1.1.
  A bug which could prevent proper uninstallation of previous versions
  was fixed at the same time. Later we made it set some registry keys
  for compatibility with the official Winpcap project installer (see
  http://seclists.org/nmap-dev/2009/q4/237). [Rob Nicholls]

o [Ncat] Ncat now prints a message like "Connection refused." by
  default when a socket error occurs. This used to require -v, but
  printing no message at all could make a failed connection look like
  success in a case like
    ncat remote < short-file

o Zenmap no longer displays down hosts in the GUI. [Josh]

o [NSE] At debug level 2 or higher (-d2), Nmap now prints all active
  scripts (running & waiting) and a backtrace whenever a key is
  pressed. This can be quite helpful in debugging deadlocks and other
  script/NSE problems. [Patrick]

o Nmap now allows you to specify --data-length 0, and that is now the
  documented way to disable the new UDP protocol-specific probe
  payload feature. [David]

o Fixed compilation of our libdnet on Debian GNU/kFreeBSD (patch from
  Petr Salinger).

o Fixed a bug that could cause an infinite loop ("Unable to find
  listening socket in get_rpc_results") in RPC scan. The loop would
  happen when scanning a port that sent no responses, and there was at
  least one other port to scan. Thanks to Lionel Cons for reporting
  the problem. [David]

o [NSE] The dns-zone-transfer and whois script argument table syntax
  has been improved so you don't need curly braces.

o [NSE] smb-enum-shares.nse now checks whether or not a share is
  writable by attempting to write a file (and deleting it if it's
  successful).  Significantly cleaned up the code, as well. [Ron]

o The nselib/data directory is now installed. It was not installed
  before because of an error in the Makefile. The scripts that would
  not have worked after installation because they were missing data
  files are http-enum.nse, http-favicon.nse, http-iis-webdav-vuln.nse,
  http-userdir-enum.nse, smb-pwdump.nse, pop3-brute.nse,
  smb-brute.nse, and snmp-brute.nse. [David]

o Upgraded the included libpcap to 1.0.0. [David]

o Optimize MAC address prefix lookup by using an std::map rather than
  a custom hash table. This increases performance and code simplicity
  at the cost of some extra memory consumption. In one test, this
  reduced the time of a single target ARP ping scan from 0.59 seconds
  to 0.13. [David]

o Fixed an error in the handling of exclude groups that used IPv4
  ranges. Si Stransky reported the problem and provided a number of
  useful test cases in http://seclists.org/nmap-dev/2009/q4/276. The
  error caused various assertion failures along the lines of
    TargetGroup.cc:465: int
    TargetGroup::get_next_host(sockaddr_storage*, size_t*):
    Assertion `ipsleft > 1' failed.
  [David]

o [NSE] Improved the authentication used by the smb-* scripts. Instead of 
  looking in a bunch of places (registry, command-line, etc) for the
  usernames/passwords, a table is kept. This lets us store any number
  of accounts for later use, and remove them if they stop working. This
  also fixes a bug where typing in a password incorrectly would lock
  out an account (since it wouldn't stop trying the account in question). 
  [Ron]

o Removed IP ID matching in packet headers returned in ICMP errors.
  This was already the case for some operating systems that are known
  to mangle the IDs of sent IP packets. Requiring such a match could
  occasionally cause valid replies to be ignored. See
  http://seclists.org/nmap-dev/2009/q2/580 for an example of host
  order affecting scan results due to this phenomenon. [David]

o [NSE] The HTTP library now handles chunked transfer decoding more
  robustly. See http://seclists.org/nmap-dev/2009/q3/13 [David]

o [NSE] Unexpected error messages from scripts now include the target
  host and port number. [David]

o [NSE] Fixed many libraries which were inappropriately using global
  variables, meaning that multiple scripts running concurrently could
  overwrite each others values. NSE now automatically checks for this
  problem at runtime, and we have a static code checker
  (check_globals) available as well. See this whole thread
  http://seclists.org/nmap-dev/2009/q3/70. [Patrick]

o Added some additional matching rules to keep a reply to a SYN probe
  from matching an ACK probe to the same port, or vice versa, in ping
  scans that include both scan types. Such a mismatch could cause an
  ineffective timing ping or traceroute probe to be selected. [David]

o [Zenmap] There is a new command-line option, --confdir, which sets
  the per-user configuration directory. Its value defaults to
  $HOME/.zenmap. This was suggested by Jesse McCoppin. [David]

o Open bpf devices in read/write mode, not read-only, in libdnet on
  BSD. This is to work around a bug in Mac OS X 10.6 that causes
  incoming traffic to become invisible. [David]

o "make install" now removes from the Nmap script directory some
  scripts which only existed in previous versions of Nmap but weren't
  deleted during upgrades. [David]

o [NSE] Added the reconnect_ssl method for sockets. We sometimes need
  to reconnect a socket with SSL because the initial communication on
  the socket is done without SSL. See this thread for more details:
  http://seclists.org/nmap-dev/2009/q4/3 [Patrick, Tom Sellers]

o [Zenmap] Fixed a crash that could occur when entering certain
  characters in the target entry (those whose UTF-8 encoding contains
  a byte that counts as whitespace in the Windows locale):
    File "zenmapGUI\ScanNotebook.pyo", line 184, in _target_entry_changed
    File "zenmapCore\NmapOptions.pyo", line 719, in render_string
    UnicodeDecodeError: 'utf8' codec can't decode byte 0xc3 in position 1:
      unexpected end of data
  For more details on this curious problem, see
  http://seclists.org/nmap-dev/2009/q4/82 [David]

o [NSE] There is a new function, nmap.bind, to set the source address
  of a socket. [David]

o [Nsock] Made it a fatal error instead of silent memory corruption
  when an attempt is made to use a file descriptor whose number is not
  less than FD_SETSIZE. This applies only on non-Windows platforms
  where FD_SETSIZE is a limit on the value of file descriptors as well
  as a limit on the number of descriptors in the set. The error will
  look like
    nsock_core.c:186: Attempt to FD_SET fd 1024, which is not less
    than FD_SETSIZE (1024). Try using a lower parallelism.
  Thanks to Brandon Enright for discovering the problem and much help
  debugging it, and to Jay Fink for submitting an initial patch. [David]

o [Ncat] Fixed proxy connections in connect mode on Windows. Because
  the dup function does not work on Windows, an assertion failure
  would be raised reading
    (fh >= 0 && (unsigned)fd < (unsigned)_nhandle)
  [David]

o [Ncat] Fixed the combination of --max-conns and --exec on Windows.
  The count of connected clients was not decreased when the program
  spawned by --exec finished. With --max-conns 5, for example, no more
  connections would be allowed after the fifth, even if some of the
  earlier ones had ended. Jon Greaves reported the problem and Venkat
  contributed a patch.

o [Ncat] The code that manages the count of connected clients has been
  made robust with respect to signals. The code was contributed by
  Solar Designer.

o The files read by the -iL (input from file) and --excludefile
  options now support comments that start with # and go to the end of
  the line. [Tom Sellers]

o [Zenmap] On Windows, Zenmap no longer uses the cmd.exe shell to run
  Nmap sub-processes. This means that canceling a scan will kill the
  Nmap process as it does on other platforms (previously it would just
  kill the shell). It also means that that scanning will work as a
  user whose name contains characters like '&' that are significant to
  the shell.  Mike Crawford and Nick Marsh reported bugs related to
  this. [David]

o [NSE] All scripts (except for those in "version" or "demo"
  categories) are now classified in either the "safe" or "intrusive"
  categories, based on how likely they are to cause problems when run
  against other machines on the network. Those classifications already
  existed, but weren't used consistently. [Fyodor]

o Fixed an integer overflow in uptime calculation which could occur
  when a target with a low TCP timestamp clock frequency uses large
  timestamp values, such that a naive uptime calculation shows a boot
  time before the epoch. Also fixed a printf format specifier mismatch
  that was revealed by the bug. Toby Simmons reported the problem and
  helped with the fix.  [David]

o [NSE] The HTTP library now supports HTTP cookies. [Joao Correa]

o Fixed a compile error on NetBSD. It was
    tcpip.cc:2948: error: pointer of type 'void *' used in arithmetic
  Thanks to Jay Fink for reporting the problem and submitting a patch.

o [Zenmap] If you have any hosts or services selected, they will
  remain selected after aggregating another scan or running a filter
  (as long as they are still up and visible). Previously the selection
  was lost whenever the scan inventory was changed. This is
  particularly important due to the new host filter system. [David]

o [Zenmap] New translation: Russian (contributed by Alexander Khodyrev).
  Updated translations: French and German.

o Nmap now generates IP addresses without duplicates (until you cycle
  through all the allowed IPs) thanks to a new collision-free 32-bit
  number generator in nbase_rnd.c. See
  http://seclists.org/nmap-dev/2009/q3/695 [Brandon]

o There is a new OS detection pseudo-test, SCAN.DC, which records how
  the network distance in SCAN.DS was calculated. Its value can be "L"
  for localhost, "D" for a direct connection, "I" for an ICMP TTL
  calculation, and "T" for a traceroute hop count. This is mainly for
  the benefit of OS integration, when it is sometimes important to
  distinguish between DS=1%DC=I (probably the result of forged TTLs)
  and DS=1%DC=D (a true one-hop connection.) [David]

o Canonicalized the list of OS detection device types to a smaller set
  with descriptions: http://nmap.org/svn/docs/device-types.txt.
  [David, Fyodor, Doug]

o [Ncat] The --idle-timeout option now exits when *both* stdin and the
  socket have been idle for the given time. Previously it would exit
  when *either* of them had been idle, meaning that the program would
  quit contrary to your expectation when downloading a large file
  without sending anything, for example. [David]

o [Ncat] Ncat now always prefixes its own output messages with "Ncat: "
  or "NCAT DEBUG: " to make it clear that they are not coming from the
  remote host. This only matters when output goes to a terminal, where
  the standard output and standard error streams are mixed. [David]

o Nmap's Nbase library now has a new hexdump() function which produces
  output similar to Wireshark. nmap_hexdump() is a wrapper which
  prints the output using Nmap's log_write facility. The old hdump()
  and lamont_dump() functions have been removed. [Luis]

o Added explicit casts to (int)(unsigned char) for arguments to ctype function
  calls in nmap, ncat and nbase.  Thanks to Solar Designer for pointing out
  the need and fix for this. [Josh]

o Ncat now supports wildcard SSL certificates.  The wildcard character
  (*) can be in commonname field or in DNS field of Subject
  Alternative Name(SAN) Extension of SSL certificate.  Matching Rules:
  -'*' should be only on the leftmost component of FQDN.(*.example.com
     but not www.*.com or www.example*.com).
  -The leftmost component should contain only '*' and it should be
     followed by '.'(*.example.com but not *w.example.com or
     w*.example.com).
  -There should be at least three components in FQDN.(*.exmaple.com but
     not *.com or *.com.).[venkat]
  
o Nmap now handles the case when a primary network interface (venet0)
  does not have an address assigned but its aliases do (venet0:1
  etc.). This could result in the error messages
    Failed to find device venet0 which was referenced in /proc/net/route
    Failed to lookup subnet/netmask for device (venet0): venet0: no IPv4 address assigned
  This was observed under OpenVZ. [Dmitry Levin]

o [Ncat] The --ssl-cert, --ssl-key, and --ssl-trustfile options now
  automatically turn on SSL mode. Previously they were ignored if
  --ssl was not also used. [David]

o [Nsock] Now Nsock supports pure TLSv1 and SSLv3 servers in addition
  to the (already supported and far more common) SSLv2 and SSLv23
  servers.  Ncat currently never uses SSLv2 for security reasons, so
  it is unaffected by this change.

o Nmap now filters received ARP packets based on their target address
  field, not the destination address in the enclosing ethernet
  frame. Some operating systems, including Windows 7 and Solaris 10,
  are known to at least sometimes send their ARP replies to the
  broadcast address and Nmap wouldn't notice them. The symptom of this
  was that root scans wouldn't work ("Host seems down") but non-root
  scans would work. Thanks to Mike Calmus and Vijay Sankar for
  reporting the problem, and Marcus Haebler for suggesting the
  fix. [David]

o The -fno-strict-aliasing option is now used unconditionally when
  using GCC. It was already this way, in effect, because a test
  against the GCC version number was reversed: <= 4 rather than >= 4.
  Solar Designer reported the problem.

o Nmap now prints a warning instead of a fatal error when the hardware
  address of an interface can't be found. This is the case for
  FireWire interfaces, which have a hardware address format not
  supported by libdnet. Thanks to Julian Berdych for the bug report.
  [David]

o Zenmap's UI performance has improved significantly thanks to
  optimization of the update_ui() function. In particular, this speeds
  up the new host filter system. [Josh]

o Fixed a log_write call and a pfatal call to use a syntax which is
  safer from format strings bugs.  This allows Nmap to build with the
  gcc -Wformat -Werror=format-security options. [Guillaume Rousse,
  Dmitry Levin]

o A bug in Nsock was fixed: On systems where a non-blocking connect
  could succeed immediately, connections that were requested to be
  tunneled through SSL would actually be plain text. This could be
  verified with an Ncat client and server running on localhost. This
  was observed to happen with localhost connections on FreeBSD 7.2.
  Non-localhost connections were likely not affected. The bug was
  reported by Daniel Roethlisberger. [David]

o Ncat proxy now hides the proxy's response ("HTTP/1.0 200 OK" or
  whatever it may be). Before, if you retrieved a file through a
  proxy, it would have the "HTTP/1.0 200 OK" stuck to the top of
  it. For this Ncat uses blocking sockets until the proxy negotiation
  is done and once it is successful, Nsock takes over for rest of the
  connection.[Venkat]

o [NSE] socket garbage collection was rewritten for better performance
  and to ensure that socket slots are immediately available to others
  after a socket is closed.  See
  http://seclists.org/nmap-dev/2009/q2/0624.html. [Patrick]

o [NSE] Fixed a rare but possible segfault which could occur if the
  nsock binding attempted to push values on the stack of a thread
  which had already ended due to an error, and if that internal Lua
  stack was already completely full. This bug is very hard to
  reproduce with a SEGFAULT but is usually visible when Lua assertion
  checks are turned on. A socket handler routine must be called AFTER
  a thread has ended in error. [Patrick]

o [Ncat] Fixed an error that would cause Ncat to use 100% CPU in
  broker mode after a client disconnected or a read error happened.
  [Kris, David]

o [NSE] --script-args may now have whitespace in unquoted strings (but
  surrounding whitespace is ignored). For example,
  --script-args 'greeting = This is a greeting' Becomes: 
  { ["greeting"] = "This is a greeting" } [Patrick]

o [Ncat] Using --send-only in conjunction with the plain listen or
  broker modes now behaves as it should: nothing will be read from the
  network end.  Ncat previously read and discarded any data
  received. [Kris]

o [Nsock] Added a socket_count abstraction that counts the number of
  read or write events pending on a socket, for the purpose of
  maintaining an fd_set. The bit is set in the fd_set whenever the
  count is positive, and cleared when it is zero. The reason for doing
  this was that write bits were not being properly cleared when using
  Ncat with SSL in connect mode, such that a client send would cause
  Ncat to use 100% CPU until it received something from the
  server. See the thread at
  http://seclists.org/nmap-dev/2009/q2/0413.html. This change will
  also make it easier to use a different back end than select in the
  future. [David]

o [Nsock] Added compilation dependency generation (makefile.dep)
  [David]

o [Ncat] The --broker option now automatically implies --listen. [David]

o Fixed a logic error in getinterfaces_siocgifconf. The check for
  increasing the capacity of the list of interfaces was off by
  one. This caused a crash on initialization for systems with more
  than 16 network interfaces. [David]

o Fixed two memory leaks in ncat_posix.c and a bug where an open file was not
  being closed in libdnet-stripped/src/intf.c [Josh Marlow]

o [Zenmap] Added profile editor support for the Nmap SCTP options:
  -PY, -sY and -sZ. [Josh Marlow]

o Fixed a bug in --data-length parsing which in some cases could
  result in useless buffer allocations and unpredictable payload
  lengths. See http://seclists.org/nmap-dev/2009/q2/0763.html [Luis]

o The configure script now allows cross-compiling by assuming that
  libpcap is recent enough to use rather than trying to compile and
  run a test program. Libpcap will always be recent enough when Nmap's
  included copy is used. [Mike Frysinger]

o Updated the IANA assignment IP list for random IP (-iR)
  generation. The Mac OS prefix file was updated as
  well. [Kris, Fyodor]

o [Zenmap] Fix a bug which could cause a crash in the (very rare) case
  where Nmap would produce port tags in XML output without a state
  attribute. [David]

o Added a convenience top-level BSDmakefile which automatically
  redirects BSD make to GNU make on BSD systems. The Nmap Makefile
  relies on numerous GNU Make extensions. [Daniel Roethlisberger]

o Nmap now provides Christmas greetings and a reminder of Xmas scan
  (-sX) when run in verbose mode on December 25. [Fyodor]

Enjoy the new release!
-Fyodor
_______________________________________________
Sent through the nmap-hackers mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-hackers
Archived at http://seclists.org/nmap-hackers/